Difference between revisions of "Perl"
m |
m |
||
Line 1: | Line 1: | ||
+ | === Useful === | ||
+ | |||
* [[perl_excel Working with Excel documents]] | * [[perl_excel Working with Excel documents]] | ||
Revision as of 02:05, 8 September 2019
Contents
Useful
online regex tool (offline use regexcoach) http://regex101.com/ - its Java based but it will do in a pinch
Cssh.pl Cluster Shell
Installing Modules
Ensure that the path that Strawberry Perl is installed in has no Spaces, else cpan/install will not work. Get around this by using old windows pathname shortcuts such as c:\progra~2 etc.
Option 1 - Manually copy them to the lib directory
Option 2 - Install using cpan / cpanm - note alot of modules will not be found as you need to specify the author / package
IE try to install Log::Log4Perl initally it wont be found. look on cpan.org for the module. copy the download link
http://search.cpan.org/CPAN/authors/id/C/CT/CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz
From this we want the authorname and the package
CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz
To install
cpanm CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz
You can also find the package name once you have the author
cpan> ls '''TILMES''' Fetching with LWP: http://cpan.strawberryperl.com/authors/id/C/CHECKSUMS Fetching with LWP: http://cpan.strawberryperl.com/authors/id/C/CT/CHECKSUMS Fetching with LWP: http://cpan.strawberryperl.com/authors/id/C/CT/CTILMES/CHECKSUMS : 596 2009-09-10 CTILMES/Log-Log4perl-[[Command Line]]-0.07.meta : 11215 2009-09-10 CTILMES/Log-Log4perl-[[Command Line]]-0.07.tar.gz : 404 2008-08-10 CTILMES/Log-Log4perl-[[Data Dumper]]-0.01.meta : 2937 2008-08-10 CTILMES/Log-Log4perl-[[Data Dumper]]-0.01.tar.gz : 437 2009-05-05 CTILMES/Pod-Usage-[[Command Line]]-0.04.meta : 2782 2009-05-05 CTILMES/Pod-Usage-[[Command Line]]-0.04.tar.gz : 533 2011-02-16 CTILMES/Proc-Wait3-0.04.meta : 9701 2011-02-16 CTILMES/Proc-Wait3-0.04.tar.gz : 531 2015-11-03 CTILMES/Proc-Wait3-0.05.meta : 10248 2015-11-03 CTILMES/Proc-Wait3-0.05.tar.gz
Manual Install
cd Module-Name-1.23 perl Makefile.PL dmake dmake test dmake install
Perl Escape
/\Q$String\E/
Autoflush
Turn on autoflush - if script is run with 'blah.pl > log.txt' log.txt will be populated as the script runs, without this setting, the log will be empty till the perl script exits.
$| = 1;
Use Strict
Ensure scripts are executed with perl -w and strict checking is enabled
#!/usr/local/bin/perl -w use strict;
Hide the code
You can use MIME::Base64 to mask whole perl modules before distribution...
: use MIME::Base64; : $encoded = encode_base64('Aladdin:open sesame'); : $decoded = decode_base64($encoded);
then use some tricky like the following to mask the decoding of the string / code...
require "\x{4d}\x{49}\x{4d}\x{45}\x{2f}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{2e}\x{70}\x{6d}"; eval &{"\x{4d}\x{49}\x{4d}\x{45}\x{3a}\x{3a}\x{42}\x{61}\x{73}\x{65}\x{36}\x{34}\x{3a}\x{3a}\x{64}\x{65}\x{63} \x{6f}\x{64}\x{65}\x{5f}\x{62}\x{61}\x{73}\x{65}\x{36}\x{34}"}($code) or die($@); 1; ## require "MIME/Base64.pm"; eval &{"MIME::Base64::decode_base64"}($code)
linky to some helpful info.. perldoc.perl.org/MIME/Base64.html
Find the name of the current script - good as part of a pid lock sctipt
use File::Basename; my $script_name = basename($0);
Find installed modules
/usr/perl5/5.8.4/bin/instmodsh
and this apprently - not tested
#!/usr/bin/perl # list installed modules use [[Ext Utils]]::Installed; my $instmod = [[Ext Utils]]::Installed->new(); foreach my $module ($instmod->modules()) { my $version = $instmod->version($module) || "???"; print "$module -- $version\n"; }
Modules / Packages
if you want to put all the modules in a subdirectory (lib) from the current relevant path - put this before the use statements in the script
use [[Find Bin]]; use lib "$[[Find Bin]]::Bin/lib";
In each module - add the following code after the use statements, and then you will be able to call the sub-routines without specifying the full path I.E blah instead of DIR::Module::Blah
use Exporter; our @ISA = qw( Exporter ); our @EXPORT = qw( sub-1 sub-2 sub-3 sub-4 etc );
Using a regex, return multiple instances as an array
@entries = ( $body =~ /(\<service.[[E [Complete Service]][I]n?gress PacketOctetsLogRecord\>.*?\<\/service.[E [CompleteService]][I]n?gressPacketOctetsLogRecord\>)/gi); @entries = ( $body =~ /(\[[openingtag\]].*?\[[closingtag]])/gi);
Convert ugly line by line output to CSV .. might not be pretty but re-usable
use Data::Dumper; my %header; my %databyIP; my @headercols; my $blob = ":10.72.15.78:Agent SNMP Version [[immediate]]:2 :10.72.15.78:Average SNMP Response Time (ms) [[last hour]]:165.28378378 :10.72.15.78:CALC Errors (Nb) [[last hour]]:0 :10.72.15.78:CPU Load (Nb OIDs) [[last hour]]:1624 :10.72.15.78:Discovery Received PDU (Nb) [[last hour]]: - :10.72.15.78:Discovery Received PDU Max Size (bytes) [[last hour]]: - :10.72.15.78:Discovery Received PDU Total Size (bytes) [[last hour]]: - :10.72.15.78:Discovery Sent PDU (Nb) [[last hour]]: - :10.72.15.78:Discovery Sent PDU Max Size (bytes) [[last hour]]: - :10.72.15.78:Discovery Sent PDU Total Size (bytes) [[last hour]]: - :10.72.15.78:Expected Measures (Nb) [[last hour]]:1184 :10.72.15.78:Max SNMP Response Time (ms) [[last hour]]:760 :10.72.15.78:OID Rate (Nb/s) [[last hour]]:0.45111111 :10.72.15.78:Produced Measures (Nb) [[last hour]]:1184 :10.72.15.78:Received PDU (Nb) [[last hour]]:148 :10.72.15.78:Received PDU Max Size (bytes) [[last hour]]:527 :10.72.15.78:Received PDU Total Size (bytes) [[last hour]]:35812 :10.72.15.78:SNMP Availability (%) [[last hour]]:100 :10.72.15.78:SNMP Errors (Nb) [[last hour]]:0 :10.72.15.78:SNMP Requests (Nb) [[last hour]]:124 :10.72.15.78:Sent PDU (Nb) [[last hour]]:148 :10.72.15.78:Sent PDU Max Size (bytes) [[last hour]]:491 :10.72.15.78:Sent PDU Total Size (bytes) [[last hour]]:32232 :10.72.16.17:Agent SNMP Version [[immediate]]:2 :10.72.16.17:Average SNMP Response Time (ms) [[last hour]]:97.81976744 :10.72.16.17:CALC Errors (Nb) [[last hour]]:0 :10.72.16.17:CPU Load (Nb OIDs) [[last hour]]:940 :10.72.16.17:Discovery Received PDU (Nb) [[last hour]]: - :10.72.16.17:Discovery Received PDU Max Size (bytes) [[last hour]]: - :10.72.16.17:Discovery Received PDU Total Size (bytes) [[last hour]]: - :10.72.16.17:Discovery Sent PDU (Nb) [[last hour]]: - :10.72.16.17:Discovery Sent PDU Max Size (bytes) [[last hour]]: - :10.72.16.17:Discovery Sent PDU Total Size (bytes) [[last hour]]: - :10.72.16.17:Expected Measures (Nb) [[last hour]]:684 :10.72.16.17:Max SNMP Response Time (ms) [[last hour]]:613 :10.72.16.17:OID Rate (Nb/s) [[last hour]]:0.26111111 :10.72.16.17:Produced Measures (Nb) [[last hour]]:628 :10.72.16.17:Received PDU (Nb) [[last hour]]:344"; my @lines = split("\n",$blob); foreach my $line (@lines){ # print "I have $line\n\n"; : my @tokens = split(":",$line); # print "token size = " . scalar @tokens ."\n"; : if(scalar @tokens == 4){ : $header->{@tokens[[2]]} = @tokens[[2]]; : print "@tokens[[2]]\n"; : $databyIP->{@tokens[[1]]}{@tokens[[2]]} = @tokens[[3]]; : }else{ : print "DUD LINE CONTAINGING $line\n"; : } } print Dumper($header); print Dumper($databyIP); foreach my $val (sort keys %{$header}){ : #print $val . "\n"; push(@headercols, $val); print "$val,"; } print "\n"; #print "we have " . scalar @headercols . " columns\n"; # now we have them in structures, spit em out foreach my $ip (sort keys %{$databyIP}){ : my $colnum =0; : while($colnum < scalar @headercols){ : if(exists($databyIP->{$ip}{@headercols[[$colnum]]})){ : #print "I found something\n"; : if($databyIP->{$ip}{@headercols[[$colnum]]} eq " -"){ : print ","; : }else{ : print $databyIP->{$ip}{@headercols[[$colnum]]} . "," : } : }else{ : print ","; : } : $colnum++; : } : # ran out of columns, new line : print "\n"; } print "I am done now, can I have coffee please sir";
Perl command line to extract lines of data fields
grep MEM_STAT proviso.log | perl -ne 'print "$1 $2\n" if /(CME.''')\-.'''?Total Image Size:(.*?) Used/' | sort | uniq 2014.05.06-21.10.46 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,347,098 kb Free: 220,855 kb 2014.05.06-22.02.33 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,343,074 kb Free: 224,879 kb 2014.05.06-22.06.14 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,343,184 kb Free: 224,769 kb 2014.05.06-22.10.27 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,347,083 kb Free: 220,870 kb 2014.05.06-23.15.36 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,343,054 kb Free: 224,899 kb 2014.05.06-23.17.48 UTC CME.2.7-4313 2 MEM_STATS Total Image Size: 1,567,953 kb Used 1,343,152 kb Free: 224,801 kb CME.1.1 981,993 kb CME.1.2 513,226 kb CME.1.3 747,609 kb CME.1.4 1,646,081 kb CME.1.5 1,255,441 kb CME.1.6 1,246,201 kb CME.2.10 474,162 kb CME.2.11 396,034 kb CME.2.12 1,294,505 kb CME.2.13 1,099,185 kb CME.2.14 1,207,137 kb CME.2.15 903,865 kb
Net::PCAP =
Centos
sudo yum install perl-Net-Pcap
Windows (10) x64
This worked using Net::Pcap 0.18 & winPCAP 4.1.0.2980 Ref: www.perlmonks.org/?node_id=1012521
- Download winpcap-dev, extract to c:\ (into c:\Wpd Pack)
- create x64 libraries -- IF on x64 platform, check on the existence of the x64 libraries:
dir c:\[[Wpd Pack]]\lib\x64\*.a
They may not exist. If not, built them:
cd c:\[[Wpd Pack]]\lib\x64 : pexports \Windows\system32\wpcap.dll > wpcap.def : dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libwpcap.a --input-def wpcap.def : pexports \Windows\system32\Packet.dll > Packet.def : dlltool --as-flags=--64 -m i386:x86-64 -k --output-lib libpacket.a --input-def Packet.def
Download and extract Net::Pcap module into c:\temp, then create the folling 4 files : Create 4 Patch files: rt.cpan.org/Public/Bug/Display.html?id=53292
0002-Add-a-sanity-self-check-for-have_functions.patch
Makefile.PL | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/Makefile.PL b/Makefile.PL index 26ab13e..b966faa 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -104,6 +104,9 @@ REASON # missing functions with croaking stubs. # We also store the list of available functions in a file for skipping the # corresponding tests. +my $sanity''check = have''functions('pcap''open''live'); +die "Couldn't find pcap''open''live(). Something is very wrong." + unless $sanity_check; my @funcs = have''functions(find''functions()); $options{DEFINE} .= cpp_defines(@funcs); open(FUNCS, '>funcs.txt') or warn "warning: can't write 'funcs.txt': $!\n"; -- 1.6.5.1.1367.gcd48
0003-Make-have_functions-work-on-Win32-as-well.patch
Makefile.PL | 33 ++++++++++++++++++++++++++++----- 1 files changed, 28 insertions(+), 5 deletions(-) diff --git a/Makefile.PL b/Makefile.PL index b966faa..ca76b8e 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -4,6 +4,7 @@ use Cwd; use [[Ext Utils]]::[[Make Maker]]; eval "use [[Ext Utils]]::[[Make Maker]]::Coverage"; use File::Spec; +use File::Basename; my ($DEBUG, %options, $DEVNULL, $is''Win32, $has''Win32); @@ -533,13 +534,35 @@ sub have_functions { : my @funcs = (); : print "detecting available functions... "; - my @paths = [[Dyna Loader]]::dl_findfile(qw(-lpcap)); - my $libref = [[Dyna Loader]]::dl''load''file($paths[[0]]); - + my @paths = [[Dyna Loader]]::dl_findfile(split /\s+/, $options{LIBS}); + die "Couldn't find any library file satisfying '$options{LIBS}'" + unless @paths; + my $libfile = $paths[[0]]; + + # On Win32, we assume that the lib file will not be statically linked + # but will be a thin wrapper for a similarly named .dll file. + # This is not universal but works in many cases + # This assumes that a library -l$foo will map to lib$foo.a + # through [[Dyna Loader]]. We then try to find and load $foo.dll in $ENV{PATH} + if ($has_Win32) { + (my $dll = basename $libfile) =~ s/\.\w+$//; + $dll =~ s/^lib//; + $dll .= '.dll'; + + ($libfile) = grep { -f } map { File::Spec->catfile($_,$dll) } File::Spec->path; + die "'$dll' not found in PATH" + unless $libfile; + }; + warn "Using '$libfile' as potential symbol candidate"; + + my $libref = [[Dyna Loader]]::dl''load''file($libfile); + warn "Couldn't load $libfile via [[Dyna Loader]] ($^E)" + unless $libref; : for my $func (@_) { : my $symref = [[Dyna Loader]]::dl''find''symbol($libref, $func); - push @funcs, $func if defined $symref - } + push @funcs, $func if defined $symref; + #print "$func", $symref ? "" : " NOT", " found\n"; + }; : print "ok\n"; : return @funcs -- 1.6.5.1.1367.gcd48
0004-Use-the-Windows-name-for-dev-null.patch
Makefile.PL | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Makefile.PL b/Makefile.PL index ca76b8e..c8cab5b 100644 --- a/Makefile.PL +++ b/Makefile.PL @@ -290,7 +290,7 @@ BEGIN { : $has_Win32 = !$@; : $is_Win32 = ($^O eq 'MSWin32'); : if ($is_Win32) { - $DEVNULL = 'DEVNULL'; + $DEVNULL = 'NUL:'; : } else { : $DEVNULL = eval { File::Spec->devnull }; : if ($@) { $DEVNULL = '/dev/null' } -- 1.6.5.1.1367.gcd48
0005-On-Win32-avoid-duplicate-HAVE_REMOVE-compilation-for.patch
stubs.inc | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/stubs.inc b/stubs.inc index 2cc292e..8ac5400 100644 --- a/stubs.inc +++ b/stubs.inc @@ -516,10 +516,13 @@ HANDLE pcap''getevent(pcap''t *p) { #ifdef ''MSC''VER #pragma message( "Warning: the function pcap_setsampling() is not available" ) #endif + +#ifndef HAVE_REMOTE struct pcap_samp { : int method; : int value; }; +#endif struct pcap''samp '''pcap''setsampling(pcap_t '''p); struct pcap''samp '''pcap''setsampling(pcap_t '''p) { -- 1.6.5.1.1367.gcd48
- Start a Cmd prompt as ADMIN (elevated privledges), from within this start Strawberry Perl portableshell.bat
- move to C:\Temp\Net-Pcap-0.18
- Apply each ofthe patches using
patch -p1 < FILENAME.patch
- run the makefile specifying the winpcap-dev x64 libraries
perl Makefile.PL INC=-IC:/[[Wpd Pack]]/Include "LIBS=-LC:/[[Wpd Pack]]/Lib/x64 -lwpcap" }}{ : output should be <pre> C:\Temp\Net-Pcap-0.18>perl Makefile.PL INC=-IC:/[[Wpd Pack]]/Include "LIBS=-LC:/[[Wpd Pack]]/Lib/x64 -lwpcap" socket.h patched... ok looking for -lwpcap... yes checking for pcap''lib''version() in -lwpcap... yes Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563. detecting available functions... ok Using 'C:\WINDOWS\system32\wpcap.dll' as potential symbol candidate at Makefile.PL line 563. detecting available functions... ok Generating a dmake-style Makefile Writing Makefile for Net::Pcap Writing MYMETA.yml and MYMETA.json
- run dmake, there might be some warnings early on, but they can be ignored.
- run dmake test , there WILL be failures, validate by testing with the following script, note this test file is different to what you may see on other threads
- save as test.pl
use Net::Pcap; @devs = Net::Pcap::pcap_findalldevs(\%devinfo, \$err); for my $dev (@devs) { : print "$dev : $devinfo{$dev}\n" }
- test using the following, output should be a list of network adapters
perl -Mblib test.pl
- If successful output of adapters, install with:
dmake install