Perl

From neil.tappsville.com
Jump to navigationJump to search

Useful


online regex tool (offline use regexcoach) http://regex101.com/ - its Java based but it will do in a pinch


Cssh.pl Cluster Shell

Regex.jpg

Installing Modules

Ensure that the path that Strawberry Perl is installed in has no Spaces, else cpan/install will not work. Get around this by using old windows pathname shortcuts such as c:\progra~2 etc.

Option 1 - Manually copy them to the lib directory

Option 2 - Install using cpan / cpanm - note alot of modules will not be found as you need to specify the author / package

IE try to install Log::Log4Perl initally it wont be found. look on cpan.org for the module. copy the download link

http://search.cpan.org/CPAN/authors/id/C/CT/CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz

From this we want the authorname and the package

CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz

To install

cpanm CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz

You can also find the package name once you have the author

cpan> ls '''TILMES'''
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CHECKSUMS
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CT/CHECKSUMS
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/C/CT/CTILMES/CHECKSUMS
: 596 2009-09-10 CTILMES/Log-Log4perl-[[Command Line]]-0.07.meta
: 11215 2009-09-10 CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz
: 404 2008-08-10 CTILMES/Log-Log4perl-[[Data Dumper]]-0.01.meta
: 2937 2008-08-10 CTILMES/Log-Log4perl-[[Data Dumper]]-0.01.tar.gz
: 437 2009-05-05 CTILMES/Pod-Usage-[[Command Line]]-0.04.meta
: 2782 2009-05-05 CTILMES/Pod-Usage-[[Command Line]]-0.04.tar.gz
: 533 2011-02-16 CTILMES/Proc-Wait3-0.04.meta
: 9701 2011-02-16 CTILMES/Proc-Wait3-0.04.tar.gz
: 531 2015-11-03 CTILMES/Proc-Wait3-0.05.meta
: 10248 2015-11-03 CTILMES/Proc-Wait3-0.05.tar.gz


Manual Install

cd Module-Name-1.23
perl Makefile.PL
dmake
dmake test
dmake install

Perl Escape

/\Q$String\E/

Autoflush

Turn on autoflush - if script is run with 'blah.pl > log.txt' log.txt will be populated as the script runs, without this setting, the log will be empty till the perl script exits.

$| = 1;

Use Strict

Ensure scripts are executed with perl -w and strict checking is enabled

#!/usr/local/bin/perl -w

use strict;

Hide the code

You can use MIME::Base64 to mask whole perl modules before distribution...

: use MIME::Base64;
: $encoded = encode_base64('Aladdin:open sesame');
: $decoded = decode_base64($encoded);

then use some tricky like the following to mask the decoding of the string / code...

require "\x{4d}\x{49}\x{4d}\x{45}\x{2f}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{2e}\x{70}\x{6d}";
eval &{"\x{4d}\x{49}\x{4d}\x{45}\x{3a}\x{3a}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{3a}\x{3a}\x{64}\x{65}\x{63}
\x{6f}\x{64}\x{65}\x{5f}\x{62}\x{61}\x{73}\x{65}\x{36}\x{34}"}($code) or die($@); 1;
## require "MIME/Base64.pm"; eval &{"MIME::Base64::decode_base64"}($code)

linky to some helpful info.. perldoc.perl.org/MIME/Base64.html


Find the name of the current script - good as part of a pid lock sctipt

use File::Basename;
my $script_name = basename($0);


Find installed modules

/usr/perl5/5.8.4/bin/instmodsh

and this apprently - not tested

#!/usr/bin/perl
# list installed modules
use [[Ext Utils]]::Installed;
my $instmod = [[Ext Utils]]::Installed->new();
foreach my $module ($instmod->modules()) {
my $version = $instmod->version($module) || "???";
print "$module -- $version\n";
}

Modules / Packages

if you want to put all the modules in a subdirectory (lib) from the current relevant path - put this before the use statements in the script

use [[Find Bin]];
use lib "$[[Find Bin]]::Bin/lib";

In each module - add the following code after the use statements, and then you will be able to call the sub-routines without specifying the full path I.E blah instead of DIR::Module::Blah

use Exporter;

our @ISA = qw( Exporter );
our @EXPORT = qw( sub-1 sub-2 sub-3 sub-4 etc );

Using a regex, return multiple instances as an array

 @entries = ( $body =~ /(\<service.[[E [Complete Service]][I]n?gress PacketOctetsLogRecord\>.*?\<\/service.[E [CompleteService]][I]n?gressPacketOctetsLogRecord\>)/gi);



 @entries = ( $body =~ /(\[[openingtag\]].*?\[[closingtag]])/gi);

Convert ugly line by line output to CSV .. might not be pretty but re-usable


use Data::Dumper;

my %header;
my %databyIP;
my @headercols;
my $blob = ":10.72.15.78:Agent SNMP Version [[immediate]]:2
:10.72.15.78:Average SNMP Response Time (ms) [[last hour]]:165.28378378
:10.72.15.78:CALC Errors (Nb) [[last hour]]:0
:10.72.15.78:CPU Load (Nb OIDs) [[last hour]]:1624
:10.72.15.78:Discovery Received PDU (Nb) [[last hour]]: -
:10.72.15.78:Discovery Received PDU Max Size (bytes) [[last hour]]: -
:10.72.15.78:Discovery Received PDU Total Size (bytes) [[last hour]]: -
:10.72.15.78:Discovery Sent PDU (Nb) [[last hour]]: -
:10.72.15.78:Discovery Sent PDU Max Size (bytes) [[last hour]]: -
:10.72.15.78:Discovery Sent PDU Total Size (bytes) [[last hour]]: -
:10.72.15.78:Expected Measures (Nb) [[last hour]]:1184
:10.72.15.78:Max SNMP Response Time (ms) [[last hour]]:760
:10.72.15.78:OID Rate (Nb/s) [[last hour]]:0.45111111
:10.72.15.78:Produced Measures (Nb) [[last hour]]:1184
:10.72.15.78:Received PDU (Nb) [[last hour]]:148
:10.72.15.78:Received PDU Max Size (bytes) [[last hour]]:527
:10.72.15.78:Received PDU Total Size (bytes) [[last hour]]:35812
:10.72.15.78:SNMP Availability (%) [[last hour]]:100
:10.72.15.78:SNMP Errors (Nb) [[last hour]]:0
:10.72.15.78:SNMP Requests (Nb) [[last hour]]:124
:10.72.15.78:Sent PDU (Nb) [[last hour]]:148
:10.72.15.78:Sent PDU Max Size (bytes) [[last hour]]:491
:10.72.15.78:Sent PDU Total Size (bytes) [[last hour]]:32232
:10.72.16.17:Agent SNMP Version [[immediate]]:2
:10.72.16.17:Average SNMP Response Time (ms) [[last hour]]:97.81976744
:10.72.16.17:CALC Errors (Nb) [[last hour]]:0
:10.72.16.17:CPU Load (Nb OIDs) [[last hour]]:940
:10.72.16.17:Discovery Received PDU (Nb) [[last hour]]: -
:10.72.16.17:Discovery Received PDU Max Size (bytes) [[last hour]]: -
:10.72.16.17:Discovery Received PDU Total Size (bytes) [[last hour]]: -
:10.72.16.17:Discovery Sent PDU (Nb) [[last hour]]: -
:10.72.16.17:Discovery Sent PDU Max Size (bytes) [[last hour]]: -
:10.72.16.17:Discovery Sent PDU Total Size (bytes) [[last hour]]: -
:10.72.16.17:Expected Measures (Nb) [[last hour]]:684
:10.72.16.17:Max SNMP Response Time (ms) [[last hour]]:613
:10.72.16.17:OID Rate (Nb/s) [[last hour]]:0.26111111
:10.72.16.17:Produced Measures (Nb) [[last hour]]:628
:10.72.16.17:Received PDU (Nb) [[last hour]]:344";


my @lines = split("\n",$blob);

foreach my $line (@lines){

 # print "I have $line\n\n";
: my @tokens = split(":",$line);
 # print "token size = " . scalar @tokens ."\n";
: if(scalar @tokens == 4){
: $header->{@tokens[[2]]} = @tokens[[2]];
: print "@tokens[[2]]\n";
: $databyIP->{@tokens[[1]]}{@tokens[[2]]} = @tokens[[3]];
: }else{
: print "DUD LINE CONTAINGING $line\n";
: }
}


print Dumper($header);


print Dumper($databyIP);



foreach my $val  (sort keys %{$header}){
: #print $val . "\n";
 push(@headercols, $val);
 print "$val,";
}
print "\n";

#print "we have " . scalar @headercols . " columns\n";


# now we have them in structures, spit em out

foreach my $ip (sort keys %{$databyIP}){
: my $colnum =0;
: while($colnum < scalar @headercols){
: if(exists($databyIP->{$ip}{@headercols[[$colnum]]})){
: #print "I found something\n";
: if($databyIP->{$ip}{@headercols[[$colnum]]} eq " -"){
: print ",";
: }else{
: print $databyIP->{$ip}{@headercols[[$colnum]]} . ","
: }
: }else{
: print ",";
: }

: $colnum++;
: }
: # ran out of columns, new line
: print "\n";
}

print "I am done now, can I have coffee please sir";

Perl command line to extract lines of data fields

grep MEM_STAT proviso.log | perl -ne 'print "$1 $2\n" if /(CME.''')\-.'''?Total Image Size:(.*?) Used/' | sort | uniq


2014.05.06-21.10.46 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,347,098 kb Free: 220,855 kb
2014.05.06-22.02.33 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,074 kb Free: 224,879 kb
2014.05.06-22.06.14 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,184 kb Free: 224,769 kb
2014.05.06-22.10.27 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,347,083 kb Free: 220,870 kb
2014.05.06-23.15.36 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,054 kb Free: 224,899 kb
2014.05.06-23.17.48 UTC CME.2.7-4313    2       MEM_STATS       Total Image Size: 1,567,953 kb Used 1,343,152 kb Free: 224,801 kb



CME.1.1  981,993 kb
CME.1.2  513,226 kb
CME.1.3  747,609 kb
CME.1.4  1,646,081 kb
CME.1.5  1,255,441 kb
CME.1.6  1,246,201 kb
CME.2.10  474,162 kb
CME.2.11  396,034 kb
CME.2.12  1,294,505 kb
CME.2.13  1,099,185 kb
CME.2.14  1,207,137 kb
CME.2.15  903,865 kb


Net::PCAP =

Centos
sudo yum install perl-Net-Pcap
Windows (10) x64

This worked using Net::Pcap 0.18 & winPCAP 4.1.0.2980 Ref: www.perlmonks.org/?node_id=1012521


  • Download winpcap-dev, extract to c:\ (into c:\Wpd Pack)
  • create x64 libraries -- IF on x64 platform, check on the existence of the x64 libraries:
dir c:\[[Wpd Pack]]\lib\x64\*.a

They may not exist. If not, built them:

cd c:\[[Wpd Pack]]\lib\x64
: pexports \Windows\system32\wpcap.dll > wpcap.def
: dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libwpcap.a --input-def wpcap.def

: pexports \Windows\system32\Packet.dll > Packet.def
: dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libpacket.a --input-def Packet.def

Download and extract Net::Pcap module into c:\temp, then create the folling 4 files : Create 4 Patch files: rt.cpan.org/Public/Bug/Display.html?id=53292


0002-Add-a-sanity-self-check-for-have_functions.patch

 Makefile.PL |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index 26ab13e..b966faa 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -104,6 +104,9 @@ REASON
 # missing functions with croaking stubs.
 # We also store the list of available functions in a file for skipping the
 # corresponding tests.
+my $sanity''check = have''functions('pcap''open''live');
+die "Couldn't find pcap''open''live(). Something is very wrong."
+    unless $sanity_check;
 my @funcs = have''functions(find''functions());
 $options{DEFINE} .= cpp_defines(@funcs);
 open(FUNCS, '>funcs.txt') or warn "warning: can't write 'funcs.txt': $!\n";
--
1.6.5.1.1367.gcd48


0003-Make-have_functions-work-on-Win32-as-well.patch

 Makefile.PL |   33 ++++++++++++++++++++++++++++-----
 1 files changed, 28 insertions(+), 5 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index b966faa..ca76b8e 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -4,6 +4,7 @@ use Cwd;
 use [[Ext Utils]]::[[Make Maker]];
 eval "use [[Ext Utils]]::[[Make Maker]]::Coverage";
 use File::Spec;
+use File::Basename;


 my ($DEBUG, %options, $DEVNULL, $is''Win32, $has''Win32);
@@ -533,13 +534,35 @@ sub have_functions {
: my @funcs = ();
: print "detecting available functions... ";

-    my @paths = [[Dyna Loader]]::dl_findfile(qw(-lpcap));
-    my $libref = [[Dyna Loader]]::dl''load''file($paths[[0]]);
-
+    my @paths = [[Dyna Loader]]::dl_findfile(split /\s+/, $options{LIBS});
+    die "Couldn't find any library file satisfying '$options{LIBS}'"
+        unless @paths;
+    my $libfile = $paths[[0]];
+
+    # On Win32, we assume that the lib file will not be statically linked
+    # but will be a thin wrapper for a similarly named .dll file.
+    # This is not universal but works in many cases
+    # This assumes that a library -l$foo will map to lib$foo.a
+    # through [[Dyna Loader]]. We then try to find and load $foo.dll in $ENV{PATH}
+    if ($has_Win32) {
+        (my $dll = basename $libfile) =~ s/\.\w+$//;
+        $dll =~ s/^lib//;
+        $dll .= '.dll';
+
+        ($libfile) = grep { -f } map { File::Spec->catfile($_,$dll) } File::Spec->path;
+        die "'$dll' not found in PATH"
+            unless $libfile;
+    };
+    warn "Using '$libfile' as potential symbol candidate";
+
+    my $libref = [[Dyna Loader]]::dl''load''file($libfile);
+    warn "Couldn't load $libfile via [[Dyna Loader]] ($^E)"
+        unless $libref;
: for my $func (@_) {
: my $symref = [[Dyna Loader]]::dl''find''symbol($libref, $func);
-        push @funcs, $func if defined $symref
-    }
+        push @funcs, $func if defined $symref;
+        #print "$func", $symref ? "" : " NOT", " found\n";
+    };

: print "ok\n";
: return @funcs
--
1.6.5.1.1367.gcd48


0004-Use-the-Windows-name-for-dev-null.patch

 Makefile.PL |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile.PL b/Makefile.PL
index ca76b8e..c8cab5b 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -290,7 +290,7 @@ BEGIN {
: $has_Win32 = !$@;
: $is_Win32 = ($^O eq 'MSWin32');
: if ($is_Win32) {
-        $DEVNULL = 'DEVNULL';
+        $DEVNULL = 'NUL:';
: } else {
: $DEVNULL = eval { File::Spec->devnull };
: if ($@) { $DEVNULL = '/dev/null' }
--
1.6.5.1.1367.gcd48


0005-On-Win32-avoid-duplicate-HAVE_REMOVE-compilation-for.patch

 stubs.inc |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/stubs.inc b/stubs.inc
index 2cc292e..8ac5400 100644
--- a/stubs.inc
+++ b/stubs.inc
@@ -516,10 +516,13 @@ HANDLE pcap''getevent(pcap''t *p) {
 #ifdef ''MSC''VER
 #pragma message( "Warning: the function pcap_setsampling() is not available" )
 #endif
+
+#ifndef HAVE_REMOTE
 struct pcap_samp {
: int method;
: int value;
 };
+#endif

 struct pcap''samp '''pcap''setsampling(pcap_t '''p);
 struct pcap''samp '''pcap''setsampling(pcap_t '''p) {
--
1.6.5.1.1367.gcd48



  • Start a Cmd prompt as ADMIN (elevated privledges), from within this start Strawberry Perl portableshell.bat
  • move to C:\Temp\Net-Pcap-0.18
  • Apply each ofthe patches using
patch -p1 < FILENAME.patch
  • run the makefile specifying the winpcap-dev x64 libraries
perl Makefile.PL INC=-IC:/[[Wpd Pack]]/Include "LIBS=-LC:/[[Wpd Pack]]/Lib/x64 -lwpcap"
}}{
: output should be
<pre>
C:\Temp\Net-Pcap-0.18>perl Makefile.PL INC=-IC:/[[Wpd Pack]]/Include "LIBS=-LC:/[[Wpd Pack]]/Lib/x64 -lwpcap"
socket.h patched... ok
looking for -lwpcap... yes
checking for pcap''lib''version() in -lwpcap... yes
Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563.
detecting available functions... ok
Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563.
detecting available functions... ok
Generating a dmake-style Makefile
Writing Makefile for Net::Pcap
Writing MYMETA.yml and MYMETA.json

  • run dmake, there might be some warnings early on, but they can be ignored.
  • run dmake test , there WILL be failures, validate by testing with the following script, note this test file is different to what you may see on other threads
save as test.pl
use Net::Pcap;

@devs = Net::Pcap::pcap_findalldevs(\%devinfo, \$err);
for my $dev (@devs) {
: print "$dev : $devinfo{$dev}\n"
}

  • test using the following, output should be a list of network adapters
perl -Mblib test.pl
  • If successful output of adapters, install with:
dmake install