Difference between revisions of "Unix"
m |
m |
||
Line 3: | Line 3: | ||
== Iptables== | == Iptables== | ||
[[iptables]] | [[iptables]] | ||
+ | |||
+ | == Iptables== | ||
+ | [[smtp email]] | ||
==General UNIX How-to's== | ==General UNIX How-to's== |
Revision as of 02:40, 25 November 2020
Contents
Iptables
Iptables
General UNIX How-to's
Find when the latest sub-directory was created given a path and search pattern (aka datachannel done files..)
date; bash -c 'for DIRECTORY in `find /appl/proviso/data/datachannel/ -name done -type d -user pvuser`; do echo "==";echo "$DIRECTORY"; ls -rt $DIRECTORY | tail -1 ; done'
Count the files waiting to be processed
date; find /appl/proviso/data/datachannel/BLB*/output | cut -d "/" -f 6 | sort | uniq -c
Run shell command every x seconds
bash -c 'while [[ 0 ]]; do echo "----"; date '+%H:%M:%S'; ls -l /appl/proviso/datachannel/BLB.2.251/output; echo "----"; sleep 5; done'
grep with OR
egrep -e 'JMS''PROCESS|SAM''CONNECT'
For the locker below - better to use extended ps output -incase the filename gets truncated (works on spac)
/usr/ucb/ps -xwww
how to get the full path to the executable
% ps -ef | grep [[processname]] root 24466 0.0 0.0 1476 280 ? S 2009 0:00 supervise sshd % sudo ls -l /proc/24466/exe
Bash script to force locking / only one instance of a script will run at a time
#!/bin/bash # # watchdog # # Run as a cron job to keep an eye on what_to_monitor which should always # be running. Restart what_to_monitor and send notification as needed. # NAME=dc_log_sam_blb_errors.pl ENV=/appl/proviso/datachannel/dataChannel.env APATH=/appl/proviso/Application/ExtAdapter/SAM_MONITOR PARAM=/appl/proviso/datachannel/log/proviso.log GREP=/bin/grep PS=/bin/ps $PS -ef|$GREP -v grep|$GREP $NAME >/dev/null 2>&1 case "$?" in 0) # It is running in this case so we do nothing. ;; 1) echo "$NAME is NOT RUNNING. Starting $NAME" . $ENV cd $APATH ./$NAME $PARAM 2>&1 >/dev/null & ;; esac exit
Remove files EDS put in temp and never remove (cronjob as root)
## remove old EDS files from malformed scripts that run 00 05 * * * find /tmp/CTSA* -user root -ctime +2 -type f -exec rm -f {} \; 2>&1 00 05 * * * find /tmp/disktacct* -user root -ctime +2 -type f -exec rm -f {} \; 2>&1 00 05 * * * find /tmp/locked* -user root -ctime +2 -type f -exec rm -f {} \; 2>&1
Keep a process running after you log off - with the right flags and run as root a process can never be stopped! Cli output will be written to nohup.out in the directory you run the command from
nohup <command> &
Defunct Processes - Find and map their parents
/usr/proc/bin/ptree `ps -ef | grep -v ^\ \ \ \ root | awk '/<defunct>/ {print $2}'` | less
Symbolic Link
ln -s {/path/to/file-name} {link-name} rm {link-name}
Send a text file via email from unix
cat <file> | mailx -s 'sunject' <email@dot.com>
Start Apache (if some retard didnt create the init.d file)
/usr/local/apache2/bin/httpd -k start
Using zip instead of tar (for files with REALLY long names) will output myzipfile.zip containing all data in directory (within directory folder in the zip)
zip -r myzipfile directory
Find the total size of all files in this folder like abc
du -ks /appl/proviso/dataload/log/*.pcap | awk '{t+=$1}END{printf "%.2f Mb\n", t/1024}' Remember to escape the $ " and \ ./run_command_on_dataloaders.sh " du -ks /appl/proviso/dataload/log/*.pcap | awk '{t+=\$1}END{printf \"%.2f Mb\", t/1024}' && ls -l /appl/proviso/dataload/log/*.pcap | wc -l"
Log Memory Changes for a Process run with NCO_MEM.sh PID
#! /bin/sh ############################################### # Script to log memory usage over time ############################################### export PLATFORM PLATFORM=`uname -p` ############################################### # Edit SLEEP as required ############################################### SLEEP=10 # if [ $# -eq 1 ] then PID=$1 else echo "Usage : `basename $0` [PID}" echo "SLEEP=$SLEEP" exit fi export DATE PS PID SLEEP OLD_MEM MEM OLD_VMEM VMEM OLD_RSS RSS echo "PLATFORM=$PLATFORM" ############################################### ############################################### if [ "$PLATFORM" = "" ] then echo "Unsupported platform" exit ############################################### # LINUX ############################################### elif [ "$PLATFORM" = "x86_64" ] then PS=`ps -p $PID -o "size rss vsz pmem time pcpu" | head -1` date '+%d/%m/%y %H:%M:%S' echo "Date : $PS" OLD_MEM=0 OLD_VMEM=0 OLD_RSS=0 while true do MEM=`ps -p $PID -o size | tail -1` VMEM=`ps -p $PID -o vsz | tail -1` RSS=`ps -p $PID -o rss | tail -1` if [ $MEM -ne $OLD_MEM -o $VMEM -ne $OLD_VMEM -o $RSS -ne $OLD_RSS ] then OLD_VMEM=$VMEM OLD_MEM=$MEM OLD_RSS=$RSS DATE=`date '+%y%m%d%H%M%S'` PS=`ps -p $PID -o "size rss vsz pmem time pcpu" | tail -1` echo "$DATE : $PS" fi sleep $SLEEP done ############################################### # Solaris ############################################### elif [ "$PLATFORM" = "sparc" ] then PS=`ps -p $PID -o "osz rss vsz pmem time pcpu" | head -1` date '+%d/%m/%y %H:%M:%S' echo "Date : $PS" OLD_MEM=0 OLD_VMEM=0 OLD_RSS=0 while true do MEM=`ps -p $PID -o osz | tail -1` VMEM=`ps -p $PID -o vsz | tail -1` RSS=`ps -p $PID -o rss | tail -1` if [ $MEM -ne $OLD_MEM -o $VMEM -ne $OLD_VMEM -o $RSS -ne $OLD_RSS ] then OLD_VMEM=$VMEM OLD_MEM=$MEM OLD_RSS=$RSS DATE=`date '+%y%m%d%H%M%S'` DATE=`date '+%d/%m/%y %H:%M:%S'` PS=`ps -p $PID -o "osz rss vsz pmem time pcpu" | tail -1` echo "$DATE : $PS" fi sleep $SLEEP done ############################################### # AIX ############################################### elif [ "$PLATFORM" = "powerpc" ] then #PS=`ps -p $PID -o "osz rss vsz pmem time pcpu" | head - PS=`ps -p $PID -o "rssize vsz pmem" | head -1` date '+%d/%m/%y %H:%M:%S' echo "Date : $PS" OLD_MEM=0 OLD_VMEM=0 while true do MEM=`ps -p $PID -o rssize | tail -1` VMEM=`ps -p $PID -o vsz | tail -1` if [ $MEM -ne $OLD_MEM -o $VMEM -ne $OLD_VMEM ] then OLD_VMEM=$VMEM OLD_MEM=$MEM DATE=`date '+%y%m%d%H%M%S'` DATE=`date '+%d/%m/%y %H:%M:%S'` PS=`ps -p $PID -o "rssize vsz pmem" | tail -1` echo "$DATE : $PS" fi sleep $SLEEP done ############################################### # Last FI ############################################### fi #EOF
Format the xml files in the current directory
bash -c 'for XML in `find *.xml`; do echo "$XML"; xmllint -format $XML > $XML.formatted; done'
SED and REGEX on the command line
cat blah | cut -d " " -f4 | sed 's/[0-9]*//g'
www.tutorialspoint.com/unix/unix-regular-expressions.htm
CUT and CUT -ordered
Normal cut will seperate on delimiter -d ";" and print fields -f 1,2-10 quickly but will only do them in order.
To be able to specify the output in a specific order use awk
awk -F";" '{print $9,$24,$34,$4,$11,$12,$16,$10,$13,$14,$15,$23,$33,$5,$6,$7,$19,$20,$8,$22,$36,$26,$35,$25,$1,$2,$17,$18,$27,$28,$29,$30,$32,$31,$21,$37,$38,$3}' properties_new.csv
remove a file with a strange name
$ ls -rlt total 2018 drwxrwxrwx 2 pvuser proviso 512 Jan 23 14:56 old -rw-r----- 1 pvuser proviso 1018744 Jan 23 16:03 OSS_Inventory.csv -rw-rw-rw- 1 pvuser proviso 0 Jan 23 17:04 --? -rw-rw-rw- 1 pvuser proviso 0 Jan 23 17:04 -help $ rm -- --? $ ls -l total 2018 -rw-rw-rw- 1 pvuser proviso 0 Jan 23 17:04 -help -rw-r----- 1 pvuser proviso 1018744 Jan 23 16:03 OSS_Inventory.csv drwxrwxrwx 2 pvuser proviso 512 Jan 23 14:56 old
Find open files on a partition
lsof will list open file and sockets
lsof
If lsof is not installed - or you dont have rights..
to find active processes for a filesystem do "fuser -c filesystem" [sf2395:root]:/appl/proviso/log > fuser -c /appl/proviso/log /appl/proviso/log: 25190c 21255c 24996c 18286o 6915o 5473o 4816o [sf2395:root]:/appl/proviso/log > then do ptree of the pid above to see what they're doing: [sf2395:root]:/appl/proviso/log > ptree 18286 1440 /usr/sbin/cron 18151 sh -c . /appl/proviso/DataAccessAPI/mpoi/edr/run.sh > /dev/null 2>&1 18286 /appl/proviso/DataAccessAPI/j2sdk1.4.1_06/bin/java -classpath .:/appl/proviso/D [sf2395:root]:/appl/proviso/log >
Netcat - pipe a unix input to another machines socket - or listen to a socket and pipe the output.
www.tutorialspoint.com/unix_commands/nc.htm
www.g-loaded.eu/2006/11/06/netcat-a-couple-of-useful-examples/
nc - arbitrary TCP and UDP connections and listens
* nc -words [-46Ddhklnr StUuvz] [-i interval] [-p source'port] [-s source'ip'address] [-T To S] [-w timeout] [-X proxy'protocol] [Xo -x proxy_address[: port]] [hostname] [port[s]] TCP Server / Listener nc -n <port> TCP Client nc -n <ipaddress> <port> some OS you must specify port with -p (thanks centos)
DESCRIPTION
The nc (or netcat) utility is used for just about anything under the sun involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6. Unlike telnet(1), nc scripts nicely, and separates error messages onto standard error instead of sending them to standard output, as telnet(1) does with some.
- Common uses include
- simple TCP proxies
shell-script based HTTP clients and servers network daemon testing a SOCKS or HTTP Proxy Command for ssh(1) and much, much more
Crontab Syntax
Im always forgetting the day of week part
* * * * * command to be executed - - - - - | | | | | | | | | +----- day of week (0 - 6) (Sunday=0) | | | +------- month (1 - 12) | | +--------- day of month (1 - 31) | +----------- hour (0 - 23) +------------- min (0 - 59)
Rsync Remote Backup
This page Cygwin_Rsync details how to backup a Windows XP machine to another Windows XP machine over the internet. (using SSH + Rsync + Cygwin)
Wireshark / tcpflow
Tcpflow is a good command for putting tcp sessions back together, the following command will do this and remove the report.xml
tcpflow -T%t-%A-%a -x netviz -i [interface] -S enable_report=NO -o [output_dir] [wireshark collection filter]
TCP Replay
Will resend udp and tcp packets
N.B tcpreplay must send packets out an interface (cant send to the host its running on, so two VM's required)
sudo apt install tcpreplay
tcprewrite --infile=original.cap --outfile=changed.cap --srcipmap=0.0.0.0/0:<MY HOST IP>/32 --dstipmap=0.0.0.0/0:<MY HOST IP>/32 --enet-smac=<enp0s25 mac addr> --enet-dmac=<enp0s25 mac addr> --fixcsum sudo tcpreplay --intf1=ens33 changed.cap
Control Characters
grep '[[:cntrl:]]' /tmp/file.txt
X Windows
A helpful guide www.redwireservices.com/remote-x11-for-linux-unix
RHEL 6
Unlock an account
pam_tally2 --user=username --reset