awk examples
##############################################
check for "0" (root) user id's in passwd
--> awk -F: '($3 == "0") {print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash
##############################################
check user id's and home directories
cat /etc/passwd | awk -F: '{print $3" " $6}' | sort > sorted.pass.3.6
##############################################
sort a password file in NIS
cat /var/yp/passwd | awk -F: '{print $3" "$6" "$1}' | sort
##############################################
look for a process and kill it
# looking for specific string in process, e.g. apache (web server)
ps -ef | grep apache | awk '{print "kill " $2}' > /tmp/killproc
sh /tmp/killproc
##############################################
launch xeyes, halt it, put it in the background, find the process and kill it...
--> xeyes
^Z
[1]+ Stopped xeyes
------------------------------------------------
--> bg
[1]+ xeyes &
------------------------------------------------
--> ps -ef | grep xeyes
john 9973 3421 0 23:16 pts/6 00:00:00 xeyes
john 9983 3421 0 23:16 pts/6 00:00:00 grep --colour=auto xeyes
------------------------------------------------
--> ps -ef | grep xeyes | awk '{print "kill " $2}' > /tmp/killproc ; sh /tmp/killproc ; cat /tmp/killproc
/tmp/killproc: 2: kill: No such process
kill 9973
kill 9997
[1]+ Terminated xeyes
##############################################
to change user "luser"'s group from "lgroup" to "othergrp"
-----------------------------------------------------
root@server [/home/files]
ll | grep luser | grep lgroup | awk '{print "chown luser:othergp" " " $9}' > list
chmod 700 list
./list
##############################################
test login against current userid
if [ `who am i | awk '{print $1}'`=`whoami`]
then
blah;blah
exit;exit
fi
##############################################
test to force an su login
case `/usr/bin/who am i | awk '{ print $1 }'` in
userid) if [ `/usr/bin/who am i | awk '{ print $1 }'` = `/usr/bin/whoami` ]
then
echo "################no#login#########################"
exit;exit
fi;;
esac
NOTE: userid is login name that can not login directly
##############################################
root-#->HOW-TO-CLEAR-A-USER
# handy way to clear dead sessions and ALL activities of a user
# or a process. Puts the kill on each line.
#
# ps -ef | grep userid > nuke.userid
# cat nuke.userid | awk '{ print "kill -9 " $2 }' > nuke
# more nuke
# sh nuke
# w
##############################################
to view a specific field
awk -F: '{ if ($2 == "x") print $1 }' /etc/passwd
awk -F: '{ if ($3 == 0) print $1}' /etc/passwd
##############################################
an odd ball task solved by AWK
THE PROBLEM:
I had a number of email addresses to send a message to from a recent microsoft virus problem
and I needed to sort out each one onto one line each. I tried to replace the comma separator with
a newline using sed within vi, and at the command line with little success. Didn't have my Perl notes
with me, so, I worked it this way, convoluted as it may be... ;) That's the beauty of regular expressions,
even if you have zero programming skills or a memory with as many holes as a microsoft system you can always
figure something out to solve a problem. Recommend making the command a script and making notes internally.
the solution
cat emails | awk -F , '{print $1}' > email1
cat emails | awk -F , '{print $2}' > email2
cat email2 >> email1
mv email1 list-o-emails
##############################################
using awk to center text
#!/bin/awk -f
# center.awk
# center - center lines in file(s) or standard input
# usage:center [filenames]
#
# For each line, a variable is created that holds the number of spaces it wil take to center
# the current line. It then prints out the spaces followed by the line.
{
spaces = ""
for ( i = 1 ; i < (80 - length($0)) / 2; i++ )
spaces = spaces " "
print spaces $0
}
##############################################
to view a specific field
awk -F: '{ if ($2 == "") print $1 }' /etc/passwd
awk -F: '{ if ($3 == 0) print $1}' /etc/passwd
##############################################
script to rename files on a web server
#!/bin/sh
# mv digests
# ~/web/UNIX/AWK/awk-renaming-files.txt
# GOAL: rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
# DATE: 4/27/2004 - Author John Meister
# TEST: --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}'
# PRODUCED: mv fsj-digest-v1-2123.txt fsj-2123.txt
#
# --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
# edit file to include header info & comments
#
mv fsj-digest-v1-1800.txt fsj-1800.txt
mv fsj-digest-v1-1801.txt fsj-1801.txt
mv fsj-digest-v1-1802.txt fsj-1802.txt
mv fsj-digest-v1-1803.txt fsj-1803.txt
mv fsj-digest-v1-1804.txt fsj-1804.txt
...
--> a use-of-awk.txt
##################################################################################################################################
PROBLEM: needed to remove a series of images that were defective,
they were all less than 10k in size, but only on certain days...
so a long listing was saved to a file, "fix". Then the file "fix"
was edited using vi and all entries below the last date were deleted.
The contents of fix were displayed with the cat command. One could
edit the file to remove the permissions, owner, size and date information,
or use AWK to print only what was needed.
But running awk only sends the output to stdout... does NOT execute the commands.
An intermediate file is used, and then executed as a shell script. Remember to delete
these temporary working files.
------------------------------------------------
ls -al > fix
------------------------------------------------
--> cat fix
-rw-r--r-- 1 fotos fotos 7252 Oct 6 20:40 20130102_064001_00.jpg
-rw-r--r-- 1 fotos fotos 7572 Oct 6 20:41 20130102_064501_00.jpg
....
-rw------- 1 fotos fotos 6436 Oct 9 18:32 20130105_065501_00.jpg
-rw------- 1 fotos fotos 9028 Oct 9 18:32 20130105_070001_00.jpg
------------------------------------------------
--> cat fix | awk '{print "rm -f "$9}'
rm -f 20130102_064001_00.jpg
rm -f 20130102_064501_00.jpg
...
rm -f 20130105_065501_00.jpg
rm -f 20130105_070001_00.jpg
------------------------------------------------
--> cat fix | awk '{print "rm -f "$9}' > fixnow
------------------------------------------------
--> sh ./fixnow
------------------------------------------------
--> rm -f fix fixnow
------------------------------------------------
##################################################################################################################################
this is only one of several ways of solving this problem, awk could be used with ls to bypass a step, e.g.:
-> ls -al | awk '{print $5 " "$9 }' | sort > files
vi files - remove files that you WANT, then cat files again, using awk to get the 2nd field and replace with "rm -f", e.g.
(test the process without the redirect FIRST to see if the files are listed with rm -f in front, if so, complete string...)
cat files | awk '{print "rm -f " $2}' > nukem-files ; sh ./nukem-files ; rm -f nukem-files
##################################################################################################################################
##################################################################################################################################
BASIC AWK variables:
field variables: $1, $2, $3, and so on ...
with $0 representing the entire record
NR: current count of input records.
NF: count of fields in an input record.
The last field in the input record can be designated by $NF.
so if you're looking for the last value in a string you could use $NF
FILENAME: current file name
FS: "field separator" value DEFAULT: "white space" which also includes space and tabs.
FS can be reassigned to another character FS=X
RS: Stores the current "record separator" DEFAULT: "newline"
OFS: "output field separator", DEFAULT: "space"
ORS: "output record separator", DEFAULT: "newline"
OFMT: format for numeric output. DEFAULT is "%.6g"
##############################################
experiences with awk and uniq (HP-UX - 1999)
note: most of these commands are for HP-UX
In trying to determine which users have logged into my system I attemped to use "uniq" to
break down the list from "last". It seems that uniq varies from system to system.
The goal is to reduce the output from last to a single line of user information.
So far the following is true: (on HP-UX 10.0 circa 1999)
if I type "last | wc -l", # I get 119 occurrences on a particular system.
if I type "last | grep -v root | wc -l", # I get 88 occurrences.
if I type "last | uniq -d | wc -l", # I get 0. (???)
if I type "last | awk '{print $1}' | grep -v root | wc -l", # I get 88.
and then, if type:
last | awk '{print $1}' | grep -v root | uniq -d | wc -l
I get 12... with 10 of those being duplicates, and another 2, one of those being a duplicate.
THEN, if I type:
last | awk '{print $1}' | grep -v root | grep -v reboot | uniq -d | wc -l
and didn't get the same answer!!!
so much for a "uniq" afternoon.... :) It's close enough for what I'm looking
for though... but wait... I'll try using it with sort...
last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d
AHA!, it works, while the other "uniq" without "sort" doesn't, see:
>--> last | awk '{print $1}' | grep -v root | grep -v reboot | uniq -d
jxxxxxx
jxxxxxx
-----------------------------------------------------
root@wxxxxxxx [/root]
>--> last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d
cxxxxxxx
jxxxxxxx
so, now I can take this string and run it through my for loop and remotely check each workstation for user id
so I can update my password file with only those users that have accessed my systems. :)
#!/bin/sh
# 99dec17 - get user names from last
#
echo " executing script to determine users on all HP engineering workstations "
for x in `cat /raid/02d/sa/rem/hostseng`
do
echo $x >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users
echo " ================== " >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users
echo $x
remsh $x last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d \
| tee -a /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users
echo " ================== "
echo " ================== " >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users
done
echo " completed list, results found in /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users "
########################################################
SET DISPLAY in .profile (tested on Sun and HP) ...YMMV...
who am i -R | tr -d [\(-\)] | awk '{print $NF}' | grep -v :0 | \
awk '{print "export DISPLAY="$1 ":0"}' > /tmp/setdisplay
. /tmp/setdisplay
/usr/bin/env | grep DISPLAY
##############################################
NOTE: ALWAYs look at Display setting, especially if you're moving around and logging in and
through other systems! When using xterm or other GUI tools, always launch with an ampersand
(&) at the end so you're able to kill it from jobs. (jobs lists the processes
running in the background, kill %# - #=job number)
(HP-UX 11i on an 800 series wanted env fully resolved /usr/bin/env)
##############################################
|