Linux commands, scripts, tools and systems administration

awk examples

############################################## 
 check for "0" (root) user id's in passwd 
--> awk -F: '($3 == "0") {print}' /etc/passwd
root:x:0:0:root:/root:/bin/bash 
############################################## 
 check user id's and home directories  
cat /etc/passwd | awk -F: '{print $3" " $6}' | sort > sorted.pass.3.6 
############################################## 
 sort a password file in NIS  
cat /var/yp/passwd | awk -F: '{print $3"  "$6"  "$1}' | sort
############################################## 
 look for a process and kill it 
# looking for specific string in process, e.g. apache (web server)
ps -ef | grep apache | awk '{print "kill " $2}' > /tmp/killproc
sh /tmp/killproc 
############################################## 
launch xeyes, halt it, put it in the background, find the process and kill it...
--> xeyes
^Z
[1]+  Stopped                 xeyes
------------------------------------------------
--> bg
[1]+ xeyes &
------------------------------------------------
--> ps -ef | grep xeyes
john      9973  3421  0 23:16 pts/6    00:00:00 xeyes
john      9983  3421  0 23:16 pts/6    00:00:00 grep --colour=auto xeyes
------------------------------------------------
--> ps -ef | grep xeyes | awk '{print "kill " $2}' > /tmp/killproc ; sh /tmp/killproc ; cat /tmp/killproc
/tmp/killproc: 2: kill: No such process
kill 9973
kill 9997
[1]+  Terminated              xeyes
##############################################
 to change user "luser"'s group from "lgroup" to "othergrp" 
 -----------------------------------------------------
root@server [/home/files]
ll | grep luser | grep lgroup | awk '{print "chown luser:othergp" " " $9}' > list
chmod 700 list
./list
##############################################
 test login against current userid 
if [ `who am i | awk '{print $1}'`=`whoami`] 
then 
blah;blah 
exit;exit 
fi 
##############################################
 test to force an su login 
case `/usr/bin/who am i | awk '{ print $1 }'` in
    userid)  if [ `/usr/bin/who am i | awk '{ print $1 }'` = `/usr/bin/whoami` ]
                then
                echo "################no#login#########################"
                exit;exit
                fi;;
esac

	NOTE:  userid is login name that can not login directly
##############################################
 root-#->HOW-TO-CLEAR-A-USER 
# handy way to clear dead sessions and ALL activities of a user
#  or a process.  Puts the kill on each line.
#
# ps -ef | grep userid > nuke.userid
# cat nuke.userid | awk '{ print "kill -9 " $2 }' > nuke
# more nuke
# sh nuke
# w
##############################################
 to view a specific field
awk -F: '{ if ($2 == "x") print $1 }' /etc/passwd
awk -F: '{ if ($3 == 0) print $1}' /etc/passwd

##############################################
 an odd ball task solved by AWK 
THE PROBLEM:
I had a number of email addresses to send a message to from a recent microsoft virus problem and I needed to sort out each one onto one line each. I tried to replace the comma separator with a newline using sed within vi, and at the command line with little success. Didn't have my Perl notes with me, so, I worked it this way, convoluted as it may be... ;) That's the beauty of regular expressions, even if you have zero programming skills or a memory with as many holes as a microsoft system you can always figure something out to solve a problem. Recommend making the command a script and making notes internally.
the solution
cat emails | awk -F , '{print $1}' > email1 cat emails | awk -F , '{print $2}' > email2 cat email2 >> email1 mv email1 list-o-emails ############################################## using awk to center text #!/bin/awk -f # center.awk # center - center lines in file(s) or standard input # usage:center [filenames] # # For each line, a variable is created that holds the number of spaces it wil take to center # the current line. It then prints out the spaces followed by the line. { spaces = "" for ( i = 1 ; i < (80 - length($0)) / 2; i++ ) spaces = spaces " " print spaces $0 } ############################################## to view a specific field awk -F: '{ if ($2 == "") print $1 }' /etc/passwd awk -F: '{ if ($3 == 0) print $1}' /etc/passwd ############################################## script to rename files on a web server #!/bin/sh # mv digests # ~/web/UNIX/AWK/awk-renaming-files.txt # GOAL: rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt" # DATE: 4/27/2004 - Author John Meister # TEST: --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' # PRODUCED: mv fsj-digest-v1-2123.txt fsj-2123.txt # # --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh # edit file to include header info & comments # mv fsj-digest-v1-1800.txt fsj-1800.txt mv fsj-digest-v1-1801.txt fsj-1801.txt mv fsj-digest-v1-1802.txt fsj-1802.txt mv fsj-digest-v1-1803.txt fsj-1803.txt mv fsj-digest-v1-1804.txt fsj-1804.txt ...
--> a use-of-awk.txt
##################################################################################################################################

PROBLEM:  needed to remove a series of images that were defective,
they were all less than 10k in size, but only on certain days... 
so a long listing was saved to a file, "fix".  Then the file "fix"
was edited using vi and all entries below the last date were deleted.

The contents of fix were displayed with the cat command.  One could
edit the file to remove the permissions, owner, size and date information,
or use AWK to print only what was needed.  

But running awk only sends the output to stdout... does NOT execute the commands. 
An intermediate file is used, and then executed as a shell script.  Remember to delete
these temporary working files.

------------------------------------------------
ls -al > fix
------------------------------------------------

--> cat fix
-rw-r--r--  1 fotos fotos     7252 Oct  6 20:40 20130102_064001_00.jpg
-rw-r--r--  1 fotos fotos     7572 Oct  6 20:41 20130102_064501_00.jpg
	....
-rw-------  1 fotos fotos     6436 Oct  9 18:32 20130105_065501_00.jpg
-rw-------  1 fotos fotos     9028 Oct  9 18:32 20130105_070001_00.jpg
------------------------------------------------

--> cat fix | awk '{print "rm -f "$9}'
rm -f 20130102_064001_00.jpg
rm -f 20130102_064501_00.jpg
...
rm -f 20130105_065501_00.jpg
rm -f 20130105_070001_00.jpg

------------------------------------------------
--> cat fix | awk '{print "rm -f "$9}' > fixnow
------------------------------------------------
--> sh ./fixnow
------------------------------------------------
--> rm -f fix fixnow
------------------------------------------------
##################################################################################################################################
this is only one of several ways of solving this problem, awk could be used with ls to bypass a step, e.g.:

 -> ls -al |  awk '{print $5 " "$9 }' | sort > files

vi files - remove files that you WANT, then cat files again, using awk to get the 2nd field and replace with "rm -f", e.g.
(test the process without the redirect FIRST to see if the files are listed with rm -f in front, if so, complete string...)

cat files | awk '{print "rm -f " $2}' > nukem-files ; sh ./nukem-files ; rm -f nukem-files  
##################################################################################################################################

################################################################################################################################## BASIC AWK variables: field variables: $1, $2, $3, and so on ... with $0 representing the entire record NR: current count of input records. NF: count of fields in an input record. The last field in the input record can be designated by $NF. so if you're looking for the last value in a string you could use $NF FILENAME: current file name FS: "field separator" value DEFAULT: "white space" which also includes space and tabs. FS can be reassigned to another character FS=X RS: Stores the current "record separator" DEFAULT: "newline" OFS: "output field separator", DEFAULT: "space" ORS: "output record separator", DEFAULT: "newline" OFMT: format for numeric output. DEFAULT is "%.6g"

############################################## experiences with awk and uniq (HP-UX - 1999) note: most of these commands are for HP-UX In trying to determine which users have logged into my system I attemped to use "uniq" to break down the list from "last". It seems that uniq varies from system to system. The goal is to reduce the output from last to a single line of user information. So far the following is true: (on HP-UX 10.0 circa 1999) if I type "last | wc -l", # I get 119 occurrences on a particular system. if I type "last | grep -v root | wc -l", # I get 88 occurrences. if I type "last | uniq -d | wc -l", # I get 0. (???) if I type "last | awk '{print $1}' | grep -v root | wc -l", # I get 88. and then, if type: last | awk '{print $1}' | grep -v root | uniq -d | wc -l I get 12... with 10 of those being duplicates, and another 2, one of those being a duplicate. THEN, if I type: last | awk '{print $1}' | grep -v root | grep -v reboot | uniq -d | wc -l and didn't get the same answer!!! so much for a "uniq" afternoon.... :) It's close enough for what I'm looking for though... but wait... I'll try using it with sort... last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d AHA!, it works, while the other "uniq" without "sort" doesn't, see: >--> last | awk '{print $1}' | grep -v root | grep -v reboot | uniq -d jxxxxxx jxxxxxx ----------------------------------------------------- root@wxxxxxxx [/root] >--> last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d cxxxxxxx jxxxxxxx so, now I can take this string and run it through my for loop and remotely check each workstation for user id so I can update my password file with only those users that have accessed my systems. :) #!/bin/sh # 99dec17 - get user names from last # echo " executing script to determine users on all HP engineering workstations " for x in `cat /raid/02d/sa/rem/hostseng` do echo $x >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users echo " ================== " >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users echo $x remsh $x last | awk '{print $1}' | grep -v root | grep -v reboot | sort | uniq -d \ | tee -a /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users echo " ================== " echo " ================== " >> /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users done echo " completed list, results found in /raid/02d/sa/rem/log.`/usr/bin/date +%y%b%d`.users " ######################################################## SET DISPLAY in .profile (tested on Sun and HP) ...YMMV... who am i -R | tr -d [\(-\)] | awk '{print $NF}' | grep -v :0 | \ awk '{print "export DISPLAY="$1 ":0"}' > /tmp/setdisplay . /tmp/setdisplay /usr/bin/env | grep DISPLAY ############################################## NOTE: ALWAYs look at Display setting, especially if you're moving around and logging in and through other systems! When using xterm or other GUI tools, always launch with an ampersand (&) at the end so you're able to kill it from jobs. (jobs lists the processes running in the background, kill %# - #=job number) (HP-UX 11i on an 800 series wanted env fully resolved /usr/bin/env) ##############################################

SEARCH and Navigation TOOL
Google     select a domain to search or visit.
(use back key to return )

johnmeister.com/jeep/sj
JeepMeister
"Jeep is America's
only real sports car."
-Enzo Ferrari
JohnMeister.com LinuxMeister
MeisterTech FotoMeister.us
BibleTech the rest of the web

AMSOIL product guide click
and enter your year, make and model.


assorted links
Everett weather -- Seattle - Everett traffic -- assorted News
parallel NASB/KJV -- BBC: Middle East
South East Asian Missions -- Voice of the Martyrs


Nuts-Bolts-Wrench specs

john's vehicle history since 1972