Perl

online regex tool (offline use regexcoach) http://regex101.com/ - its Java based but it will do in a pinch

Installing Modules

Ensure that the path that Strawberry Perl is installed in has no Spaces, else cpan/install will not work. Get around this by using old windows pathname shortcuts such as c:\progra2 etc.

Option 1 - Manually copy them to the lib directory

Option 2 - Install using cpan / cpanm - note alot of modules will not be found as you need to specify the author / package

IE try to install Log::Log4Perl initally it wont be found. look on cpan.org for the module. copy the download link

http://search.cpan.org/CPAN/authors/id/C/CT/CTILMES/Log-Log4perl-CommandLine-0.07.tar.gz

From this we want the authorname and the package

CTILMES/Log-Log4perl-CommandLine-0.07.tar.gz

To install

cpanm CTILMES/Log-Log4perl-CommandLine-0.07.tar.gz

You can also find the package name once you have the author

cpan> ls *TILMES*
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CHECKSUMS
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CT/CHECKSUMS
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CT/CTILMES/CHECKSUMS
     596 2009-09-10 CTILMES/Log-Log4perl-CommandLine-0.07.meta
   11215 2009-09-10 CTILMES/Log-Log4perl-CommandLine-0.07.tar.gz
     404 2008-08-10 CTILMES/Log-Log4perl-DataDumper-0.01.meta
    2937 2008-08-10 CTILMES/Log-Log4perl-DataDumper-0.01.tar.gz
     437 2009-05-05 CTILMES/Pod-Usage-CommandLine-0.04.meta
    2782 2009-05-05 CTILMES/Pod-Usage-CommandLine-0.04.tar.gz
     533 2011-02-16 CTILMES/Proc-Wait3-0.04.meta
    9701 2011-02-16 CTILMES/Proc-Wait3-0.04.tar.gz
     531 2015-11-03 CTILMES/Proc-Wait3-0.05.meta
   10248 2015-11-03 CTILMES/Proc-Wait3-0.05.tar.gz

Manual Install

cd Module-Name-1.23
perl Makefile.PL
dmake
dmake test
dmake install

Perl Escape

/\Q$String\E/

Autoflush

Turn on autoflush - if script is run with 'blah.pl > log.txt' log.txt will be populated as the script runs, without this setting, the log will be empty till the perl script exits.

$| = 1;

Use Strict

Ensure scripts are executed with perl -w and strict checking is enabled

#!/usr/local/bin/perl -w

use strict;

Hide the code

You can use MIME::Base64 to mask whole perl modules before distribution...

    use MIME::Base64;
    $encoded = encode_base64('Aladdin:open sesame');
    $decoded = decode_base64($encoded);

then use some tricky like the following to mask the decoding of the string / code...

require "\x{4d}\x{49}\x{4d}\x{45}\x{2f}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{2e}\x{70}\x{6d}";
eval &{"\x{4d}\x{49}\x{4d}\x{45}\x{3a}\x{3a}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{3a}\x{3a}\x{64}\x{65}\x{63}
\x{6f}\x{64}\x{65}\x{5f}\x{62}\x{61}\x{73}\x{65}\x{36}\x{34}"}($code) or die($@); 1;
## require "MIME/Base64.pm"; eval &{"MIME::Base64::decode_base64"}($code)

linky to some helpful info.. http://perldoc.perl.org/MIME/Base64.html

Find the name of the current script - good as part of a pid lock sctipt

use File::Basename;
my $script_name = basename($0);

Find installed modules

/usr/perl5/5.8.4/bin/instmodsh

and this apprently - not tested

#!/usr/bin/perl
# list installed modules
use ExtUtils::Installed;
my $instmod = ExtUtils::Installed->new();
foreach my $module ($instmod->modules()) {
my $version = $instmod->version($module) || "???";
print "$module -- $version\n";
}

Modules / Packages

if you want to put all the modules in a subdirectory (lib) from the current relevant path - put this before the use statements in the script

use FindBin;
use lib "$FindBin::Bin/lib";

In each module - add the following code after the use statements, and then you will be able to call the sub-routines without specifying the full path I.E blah instead of DIR::Module::Blah

use Exporter;

our @ISA = qw( Exporter );
our @EXPORT = qw( sub-1 sub-2 sub-3 sub-4 etc );

Using a regex, return multiple instances as an array

 @entries = ( $body =~ /(\<service.CompleteService[I|E]n?gressPacketOctetsLogRecord\>.*?\<\/service.CompleteService[I|E]n?gressPacketOctetsLogRecord\>)/gi);


 @entries = ( $body =~ /(\[openingtag\].*?\[closingtag])/gi);

Convert ugly line by line output to CSV .. might not be pretty but re-usable


use Data::Dumper;

my %header;
my %databyIP;
my @headercols;
my $blob = ":10.72.15.78:Agent SNMP Version [immediate]:2
:10.72.15.78:Average SNMP Response Time (ms) [last hour]:165.28378378
:10.72.15.78:CALC Errors (Nb) [last hour]:0
:10.72.15.78:CPU Load (Nb OIDs) [last hour]:1624
:10.72.15.78:Discovery Received PDU (Nb) [last hour]: -
:10.72.15.78:Discovery Received PDU Max Size (bytes) [last hour]: -
:10.72.15.78:Discovery Received PDU Total Size (bytes) [last hour]: -
:10.72.15.78:Discovery Sent PDU (Nb) [last hour]: -
:10.72.15.78:Discovery Sent PDU Max Size (bytes) [last hour]: -
:10.72.15.78:Discovery Sent PDU Total Size (bytes) [last hour]: -
:10.72.15.78:Expected Measures (Nb) [last hour]:1184
:10.72.15.78:Max SNMP Response Time (ms) [last hour]:760
:10.72.15.78:OID Rate (Nb/s) [last hour]:0.45111111
:10.72.15.78:Produced Measures (Nb) [last hour]:1184
:10.72.15.78:Received PDU (Nb) [last hour]:148
:10.72.15.78:Received PDU Max Size (bytes) [last hour]:527
:10.72.15.78:Received PDU Total Size (bytes) [last hour]:35812
:10.72.15.78:SNMP Availability (%) [last hour]:100
:10.72.15.78:SNMP Errors (Nb) [last hour]:0
:10.72.15.78:SNMP Requests (Nb) [last hour]:124
:10.72.15.78:Sent PDU (Nb) [last hour]:148
:10.72.15.78:Sent PDU Max Size (bytes) [last hour]:491
:10.72.15.78:Sent PDU Total Size (bytes) [last hour]:32232
:10.72.16.17:Agent SNMP Version [immediate]:2
:10.72.16.17:Average SNMP Response Time (ms) [last hour]:97.81976744
:10.72.16.17:CALC Errors (Nb) [last hour]:0
:10.72.16.17:CPU Load (Nb OIDs) [last hour]:940
:10.72.16.17:Discovery Received PDU (Nb) [last hour]: -
:10.72.16.17:Discovery Received PDU Max Size (bytes) [last hour]: -
:10.72.16.17:Discovery Received PDU Total Size (bytes) [last hour]: -
:10.72.16.17:Discovery Sent PDU (Nb) [last hour]: -
:10.72.16.17:Discovery Sent PDU Max Size (bytes) [last hour]: -
:10.72.16.17:Discovery Sent PDU Total Size (bytes) [last hour]: -
:10.72.16.17:Expected Measures (Nb) [last hour]:684
:10.72.16.17:Max SNMP Response Time (ms) [last hour]:613
:10.72.16.17:OID Rate (Nb/s) [last hour]:0.26111111
:10.72.16.17:Produced Measures (Nb) [last hour]:628
:10.72.16.17:Received PDU (Nb) [last hour]:344";

my @lines = split("\n",$blob);

foreach my $line (@lines){

 # print "I have $line\n\n";
  my @tokens = split(":",$line);
 # print "token size = " . scalar @tokens ."\n";
  if(scalar @tokens == 4){
    $header->{@tokens[2]} = @tokens[2];
   print "@tokens[2]\n";
    $databyIP->{@tokens[1]}{@tokens[2]} = @tokens[3];
  }else{
    print "DUD LINE CONTAINGING $line\n";
  }
}


print Dumper($header);


print Dumper($databyIP);



foreach my $val  (sort keys %{$header}){
  #print $val . "\n";
 push(@headercols, $val);
 print "$val,";
}
print "\n";

#print "we have " . scalar @headercols . " columns\n";


# now we have them in structures, spit em out

foreach my $ip (sort keys %{$databyIP}){
  my $colnum =0;
  while($colnum < scalar @headercols){
    if(exists($databyIP->{$ip}{@headercols[$colnum]})){
      #print "I found something\n";
      if($databyIP->{$ip}{@headercols[$colnum]} eq " -"){
      print ",";
      }else{
      print $databyIP->{$ip}{@headercols[$colnum]} . ","
      }
    }else{
      print ",";
    }

    $colnum++;
  }
  # ran out of columns, new line
  print "\n";
}

print "I am done now, can I have coffee please sir";

Perl command line to extract lines of data fields

grep MEM_STAT proviso.log | perl -ne 'print "$1 $2\n" if /(CME.*)\-.*?Total Image Size:(.*?) Used/' | sort | uniq


2014.05.06-21.10.46 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,347,098 kb Free: 220,855 kb
2014.05.06-22.02.33 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,074 kb Free: 224,879 kb
2014.05.06-22.06.14 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,184 kb Free: 224,769 kb
2014.05.06-22.10.27 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,347,083 kb Free: 220,870 kb
2014.05.06-23.15.36 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,054 kb Free: 224,899 kb
2014.05.06-23.17.48 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,152 kb Free: 224,801 kb



CME.1.1  981,993 kb
CME.1.2  513,226 kb
CME.1.3  747,609 kb
CME.1.4  1,646,081 kb
CME.1.5  1,255,441 kb
CME.1.6  1,246,201 kb
CME.2.10  474,162 kb
CME.2.11  396,034 kb
CME.2.12  1,294,505 kb
CME.2.13  1,099,185 kb
CME.2.14  1,207,137 kb
CME.2.15  903,865 kb

Net::PCAP

Centos
sudo yum install perl-Net-Pcap
Windows (10) x64

This worked using Net::Pcap 0.18 & winPCAP 4.1.0.2980 Ref: http://www.perlmonks.org/?node_id=1012521

  • Download winpcap-dev, extract to c:\ (into c:\WpdPack)
  • create x64 libraries -- IF on x64 platform, check on the existence of the x64 libraries:
dir c:\WpdPack\lib\x64\*.a

They may not exist. If not, built them:

cd c:\WpdPack\lib\x64
  pexports \Windows\system32\wpcap.dll > wpcap.def
  dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libwpcap.a --input-def wpcap.def

  pexports \Windows\system32\Packet.dll > Packet.def
  dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libpacket.a --input-def Packet.def

Download and extract Net::Pcap module into c:\temp, then create the folling 4 files:

Create 4 Patch files: https://rt.cpan.org/Public/Bug/Display.html?id=53292

0002-Add-a-sanity-self-check-for-have_functions.patch

 Makefile.PL |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index 26ab13e..b966faa 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -104,6 +104,9 @@ REASON
 # missing functions with croaking stubs.
 # We also store the list of available functions in a file for skipping the
 # corresponding tests.
+my $sanity_check = have_functions('pcap_open_live');
+die "Couldn't find pcap_open_live(). Something is very wrong."
+    unless $sanity_check;
 my @funcs = have_functions(find_functions());
 $options{DEFINE} .= cpp_defines(@funcs);
 open(FUNCS, '>funcs.txt') or warn "warning: can't write 'funcs.txt': $!\n";
--
1.6.5.1.1367.gcd48

0003-Make-have_functions-work-on-Win32-as-well.patch

 Makefile.PL |   33 ++++++++++++++++++++++++++++-----
 1 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index b966faa..ca76b8e 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -4,6 +4,7 @@ use Cwd;
 use ExtUtils::MakeMaker;
 eval "use ExtUtils::MakeMaker::Coverage";
 use File::Spec;
+use File::Basename;


 my ($DEBUG, %options, $DEVNULL, $is_Win32, $has_Win32);
@@ -533,13 +534,35 @@ sub have_functions {
     my @funcs = ();
     print "detecting available functions... ";

-    my @paths = DynaLoader::dl_findfile(qw(-lpcap));
-    my $libref = DynaLoader::dl_load_file($paths[0]);
-
+    my @paths = DynaLoader::dl_findfile(split /\s+/, $options{LIBS});
+    die "Couldn't find any library file satisfying '$options{LIBS}'"
+        unless @paths;
+    my $libfile = $paths[0];
+
+    # On Win32, we assume that the lib file will not be statically linked
+    # but will be a thin wrapper for a similarly named .dll file.
+    # This is not universal but works in many cases
+    # This assumes that a library -l$foo will map to lib$foo.a
+    # through DynaLoader. We then try to find and load $foo.dll in $ENV{PATH}
+    if ($has_Win32) {
+        (my $dll = basename $libfile) =~ s/\.\w+$//;
+        $dll =~ s/^lib//;
+        $dll .= '.dll';
+
+        ($libfile) = grep { -f } map { File::Spec->catfile($_,$dll) } File::Spec->path;
+        die "'$dll' not found in PATH"
+            unless $libfile;
+    };
+    warn "Using '$libfile' as potential symbol candidate";
+
+    my $libref = DynaLoader::dl_load_file($libfile);
+    warn "Couldn't load $libfile via DynaLoader ($^E)"
+        unless $libref;
     for my $func (@_) {
         my $symref = DynaLoader::dl_find_symbol($libref, $func);
-        push @funcs, $func if defined $symref
-    }
+        push @funcs, $func if defined $symref;
+        #print "$func", $symref ? "" : " NOT", " found\n";
+    };

     print "ok\n";
     return @funcs
--
1.6.5.1.1367.gcd48

0004-Use-the-Windows-name-for-dev-null.patch

 Makefile.PL |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index ca76b8e..c8cab5b 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -290,7 +290,7 @@ BEGIN {
     $has_Win32 = !$@;
     $is_Win32 = ($^O eq 'MSWin32');
     if ($is_Win32) {
-        $DEVNULL = 'DEVNULL';
+        $DEVNULL = 'NUL:';
     } else {
         $DEVNULL = eval { File::Spec->devnull };
         if ($@) { $DEVNULL = '/dev/null' }
--
1.6.5.1.1367.gcd48

0005-On-Win32-avoid-duplicate-HAVE_REMOVE-compilation-for.patch

 stubs.inc |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/stubs.inc b/stubs.inc
index 2cc292e..8ac5400 100644
--- a/stubs.inc
+++ b/stubs.inc
@@ -516,10 +516,13 @@ HANDLE pcap_getevent(pcap_t *p) {
 #ifdef _MSC_VER
 #pragma message( "Warning: the function pcap_setsampling() is not available" )
 #endif
+
+#ifndef HAVE_REMOTE
 struct pcap_samp {
     int method;
     int value;
 };
+#endif

 struct pcap_samp *pcap_setsampling(pcap_t *p);
 struct pcap_samp *pcap_setsampling(pcap_t *p) {
--
1.6.5.1.1367.gcd48


  • Start a Cmd prompt as ADMIN (elevated privledges), from within this start Strawberry Perl portableshell.bat
  • move to C:\Temp\Net-Pcap-0.18
  • Apply each ofthe patches using
patch -p1 < FILENAME.patch
  • run the makefile specifying the winpcap-dev x64 libraries
perl Makefile.PL INC=-IC:/WpdPack/Include "LIBS=-LC:/WpdPack/Lib/x64 -lwpcap"
}}{
  output should be
{{{
C:\Temp\Net-Pcap-0.18>perl Makefile.PL INC=-IC:/WpdPack/Include "LIBS=-LC:/WpdPack/Lib/x64 -lwpcap"
socket.h patched... ok
looking for -lwpcap... yes
checking for pcap_lib_version() in -lwpcap... yes
Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563.
detecting available functions... ok
Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563.
detecting available functions... ok
Generating a dmake-style Makefile
Writing Makefile for Net::Pcap
Writing MYMETA.yml and MYMETA.json
  • run dmake, there might be some warnings early on, but they can be ignored.
  • run dmake test , there WILL be failures, validate by testing with the following script, note this test file is different to what you may see on other threads save as test.pl
use Net::Pcap;

@devs = Net::Pcap::pcap_findalldevs(\%devinfo, \$err);
for my $dev (@devs) {
    print "$dev : $devinfo{$dev}\n"
}
  • test using the following, output should be a list of network adapters
perl -Mblib test.pl
  • If successful output of adapters, install with:
dmake install
Toolbox
Favorite Categories