ONE HOUR Linux SESSIONS: 2014-2017
sessions in chronological order going back to 2014
Simply Linux: Basics Linux Tackles Microsoft Using BASH on Windows 10
Practical Suggestions for Microsoft Windows
 Full Size Jeep Buyer's Guide
the art of Linux sys admin
the Art of Linux SysAdmin
john's publications (click on cover for further info)

2017


Session #114 - December 8, 2017 - FRIDAY - Oracle VirtualBox and LVM
note on html: color for cell above is "yellow" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
(can use names too...)


Expanding the capactity of a dynamic vdi disk in Oracle's VirtualBox

Built a default Linux 64bit VM with Centos, however, needed more space, 
   had to increase the capacity from the default of 8GB to about 14GB. 
https://www.virtualbox.org/

http://johnmeister.com/linux/SysAdmin/Virtualization/VM_setup_SuSE_13.2/ALL.html
  
Steps to increase capacity of a vdi drive in VirtualBox (Linux or MacOSx host) at the COMMAND LINE:

-->sudo VBoxManage modifyhd /home/luser/VirtualBox_VMs/Centos-Lab/Centos_Lab.vdi --resize 14480
         0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Once that was done, checked settings in VirtualBox;  was good... but local drive in VM still not expanded.

FROM INSIDE THE VM, at the command line:  
    pvs ; pvdisplay ; vgs ; vgdisplay; lvs ; lvdisplay  # displays system values
    fdisk /dev/sda      #  create a partition in the "expanded" virtual drive
        p, n, p, 3, t, 82e, w
    pvcreate /dev/sda3    ; pvs; vgs    # create the physical volume, check it
   vgextend centos_centos7-vm /dev/sda3  # extend the existing volume group into the expande physical drive
   vgs ; pvscan ; lvdisplay ; df -h; vgdisplay
   lvextend /dev/mapper/centos_centos7--vm-root /dev/sda3 # extend the logical volume into the expanded volume group
   df -h . ; vgs ; lvs 
   resize2fs /dev/mapper/centos_centos7--vm-root  ## FAILURE - bug in RH LVM and/or XFS  (should have extended the file system)
## FAILURE - bug in RH LVM and/or XFS
--> resize2fs /dev/centos_centos7-vm/root   (FAILURE IN RH-type system and xfs)
resize2fs 1.42.9 (28-Dec-2013)
resize2fs: Bad magic number in super-block while trying to open /dev/centos_centos7-vm/root
Couldn't find valid filesystem superblock.

FIX:
--> xfs_growfs  /dev/centos_centos7-vm/root

http://johnmeister.com/linux/FileSystems/ADD-6GB-to-vdi-CentOS-VM.html

process:  Order of precendence and logic:   - PHYSICAL   - VOLUME  - LOGICAL 
PHYSICAL VOLUME(s): pvs; pvdisplay; pvresize
VOLUME GROUP(s): vgs; vgdisplay; vgextend        Add physical volumes to a volume group
LOGICAL VOLUME(s): lvs ; lvdisplay; lvextend        Add space to a logical volume
http://johnmeister.com/linux/FileSystems/setup-LVM.html
http://johnmeister.com/linux/FileSystems/lvm-commands.html


Session #113 - December 1, 2017 - FRIDAY - RECAP of 3 years of Linux at lunch!
note on html: color for cell above is "fedcba" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
(can use names too...)


recap of 3 years worth of sessions

http://johnmeister.com/linux/Intro-to-Linux/session-list-2017.html

Session #112 - November 10, 2017 - FRIDAY - KEY WORD script
note on html: color for cell above is "ABCDEF" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
(can use names too...)


 NOTE:  no session Friday 3 Nov 2017
Session #112 - Nov 10, 2017 - FRIDAY - Linux at Lunch -  KEY WORD SCRIPT walk through!
 
The Key Word Script  uses regular expressions: 
grep, sed, awk, cut, uniq, sort, perl, tr and vi to extract KEY WORDS from a text document!
Key Words are those of proper names and places; words beginning with capital letters.
 
The output of this script produces two Key-Word variants; one that shows only the line where the word appears,
and the other shows the context, grep -C 1, listing the line before and the line after. 
 
It's possible to insert HTML coding into this script to create a web page, e.g.:
with:  
with:  http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html    

There are GUI based tools like WordPress that might create nicer pages... 
but these tools have security issues, require a database, are not trivial to install, learn, use or manage. 
 
    SIMPLE ALWAYS WORKS.
 

http://johnmeister.com/bible/BibleSummary/KEY-WORDs/GET-KEY.sh.txt
(will create a version of this on johnmeister.com/linux to work with generic text files.)

------------------------------------------------------------ 
#!/bin/bash
### john meister 29 october 2017 - copyright 2017  
### script to find Key words - upper case - Who and Where
USAGE="GET-KEY.sh.txt , e.g.  GET-KEY.sh.txt 02n_Exo.txt" 
#########################################################################
cat $1 | tr -d '[{}()\;\":,\!\.\?]' > T2    
       ### remove most special characters cat T1, filter, save as T2
perl -pi -e 's/ /\n/g' T2                    
       ### put all words on one line
grep -v "^'" T2  >  T3                       
       ### remove leading ticks from list save as T3
cat T3 | sort | uniq | grep -v ^[a-z] | grep -v ^[0-9] > T4   
       ###  sort, remove lc and numbers, save as T4
rm -f T1 T2 T3                               
       ### deleting temporary files
##########################################################################
echo "edit T4 in vi to remove non-keywords; :wq will resume; hit any key to start"
read
vi T4                                   # T4 will be moved to ZLIST-names when done
######################### 
OUT="KEY-names-with-VS-$1"              # Key names with counts and verses
OUT2="TIMES-names-$1"                   # Key names with counts only
cat /dev/null > $OUT
  for Z in `cat T4`
     do
     echo "----------------------------------------------------------------------" | tee -a $OUT
     grep $Z $1 | wc -l > T1
        ####################
     echo "The word \"$Z\" in \"$1\" is found `cat T1| cut -c 5-9` times." | tee -a $OUT2
     echo "======================================================================" | tee -a $OUT2
        ####################
     echo "The word \"$Z\" in \"$1\" occurs in these verses `cat T1| cut -c 6-9` times:" | tee -a $OUT
     echo "----------------------------------------------------------------------" | tee -a $OUT
     grep -C 1 $Z $1 | tee -a $OUT
     echo "======================================================================" | tee -a $OUT
  done
###########################################################
mv T4 ZLIST-names-$1 ; rm -f T1
##########################################################
 
So, the extract files look like this for the following command(s):  
-------------------------------------------------------
GET-KEY.sh.txt 01k_Gen.txt

  KEY-names-with-VS-01k_Gen.txt 2017-10-30 00:40  1.2M 
  TIMES-names-01k_Gen.txt       2017-10-30 00:40   69K 
  ZLIST-names-01k_Gen.txt       2017-10-30 00:40  3.8K 
-------------------------------------------------------
and
-------------------------------------------------------
GET-KEY.sh.txt 01n_Gen.txt 
 
  KEY-names-with-VS-01n_Gen.txt 2017-10-30 00:40  1.2M 
  TIMES-names-01n_Gen.txt       2017-10-30 00:40   69K 
  ZLIST-names-01n_Gen.txt       2017-10-30 00:40  3.9K 
------------------------------------------------------- 
It's interesting to see the names and places listed in just ONE of the 66 books:

http://johnmeister.com/bible/BibleSummary/KEY-WORDs/01-GENESIS/ZLIST-names-01k_Gen.txt
 
In addition, I want to modify the output to only show the references: 

http://johnmeister.com/bible/BibleSummary/KEY-WORDs/01-GENESIS/KEY-names-with-VS-01k_Gen.txt 
e.g. 
----------------------------------------------------------------------
The word "Achbor" in "01k_Gen.txt" occurs 2 times: Gen 36:38; Gen 36:39 
======================================================================
or:
----------------------------------------------------------------------
The word "Achbor" in "01k_Gen.txt" occurs 2 times:
Gen 36:38; Gen 36:39 
======================================================================

SIMPLE is portable, secure and easily modified; but it may take more effort and understanding, 
and may include more "steps" and sometimes requires manual intervention; NOTICE the USE of "read" in the 
script above.  I had to manually edit the "key word" list to remove unimportant words like 
"The", "And", "Because", etc.  And a few other items that didn't filter out properly.  

It is possible that I could create a "heredoc" to filter those words, but after converting the strings 
to single words with new lines any grep -v filters would take out the rest of any words starting out 
with the same beginning (e.g And would remove Andover, etc.). 

That would push the filtering into the text prior to the new line insertion, 
and since one has no idea which words are "important" or of interest, better to remove them manually.  

One other thing, when running the script and trying to make it work, you might want to save the "key word"
list by a different name, then when editing the list you can delete the contents and read in what you
already edited.  Keep the copy file name simple and easy to remember.  It'll take a few iterations to dial it in.


Session #111 - October 27, 2017 - FRIDAY - two scripts
note on html: color for cell above is "tan" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
(can use names too...)


 NOTE:  no session NEXT week, Friday 3 Nov 2017

two scripts

work flow process for digital images and script to create web page with thumbnails workflow process:
  1. download images from SD card to computer via scripts
  2. review original images; minor edits - resize, rename with dates - keeping RAW impages untouched
  3. resized/renamed images go up one directory. review which ones will go on web page
  4. move resized/renamed images to web directory
  5. scp -r web directory to server
  6. ssh to server, run "mkpg" script http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html
  7. add "exiftool" info with extract into directories: (can be done on Mac or either Linux system) --> more exif-strip.sh #!/bin/bash # uses exiftool to extract EXIF info in original images # for x in `ls *` do exiftool $x > exifinfo-$x.txt cat exifinfo-$x.txt| grep -E -v '(Brightness|Timezone|ISO2|Zone|Bracket|Image|Artist|Crop|Sensitivity|\ Preview|Difference|Shift|MIME|Tuning|Clarity|Serial|Mode|Modify|Distance|Confusion|Gain|Tune|Phase|CFA|Depth|\ Hyper|Light|Mega|Red|Blue|GPS|Normal|TIFF|Jpg|Binary|Sub|Memory|Original|File|Directory|Sensing|Size|Power|\ Multi|Raw|Contrast|Vari|Retouch|Max|Compression|Exit|Comment|Primary|MCU|Version|Shot|Position|Picture|Black|\ Control|Points|Expansion|Zoom|Toning|Hue|Filter|Format|Min|Color|Quality|Levels|Flash|Scene|Other|Planar|Resolution|\ Copyright|Strip|Balance|Bits|Stops|Samples|Software|Orientation|Byte|Make|Savings|Scale)' > base-exif-$x.txt done mkdir ../EXIFINFO mv exifinfo-*.txt ../EXIFINFO mv base-exif-*.txt ../EXIFINFO
  8. created and add thumbnails --> convert -thumbnail 400 somepicture.jpg tn-somepicture.jpg Man page says "-thumbnail geometry" - substitute geometry for value... --> man convert | col -b > man.convert.txt http://johnmeister.com/linux/Commands/man.convert.txt
  9. Current thought is to incorporate thumbnails into the "mkpg" script,
    and also incorporate select parts from the exiftool and exif data.
  10. will likely use tables to put each thumbnail with the title/link below
    along with the exif data for exposure info.
  11. Each thumbnail would be linked to open in a new tab or window,
    an alternate would be to use javascript to do a mouse hover to show image.
It's "simple" to use a tool like WordPress or other apps to do this...
but those tools have security issues, sometimes costs and become obsolete or not installed... KNOWING HOW to do something is best, even if it doesn't look as good.
2) Creating a script to extract Proper Names and Places from text files - using sed, grep, tr, perl, sort and uniq From prior sessions there were plain text files created from two translations. The 66 books have been separated into 8 sections and have two plain text translations to work with, or 132 text files to parse. The goal is to find the proper names and places which begin with an upper case character and extract those names from the text into a file. That file of names will have to be manually edited to remove other upper case words, such as "The", "Also", "Before", and so on. I've tried to use grep to remove them. However if a word on a line contains the first letters of another word the grep -v will also remove those words. So manually editing 132 files is less trouble than working an automated solution. If this were an ongoing effort with continous data coming in from a regular test or email, then I would take the time to determine a solution. In prior sessions we've used wget to gather these files and worked them into one line per verse text files. The project continues to work with those files to extract specific information. Key names and places. current flow:
  1. Have gathered the text files for the 66 books and put them into the 8 major sections: # 1-LAW - 5 books - 10 files - identify and list proper names showing number of times and verses # 2-HISTORY - 12 books - 24 files # 3-WISDOM - 5 books - 10 files # 4-PROPHETS - 17 books - 34 files # 5-GOSPEL - 5 books - 10 files # 6-PAUL - 13 books - 26 files # 7-LTRS - 8 books - 16 files # 8-REV - 1 book - 2 files
  2. take each of the two files per book and create a copy: --> cp 02n_Exo.txt 2N ; cp 02k_Exo.txt 2K (do not edit 02?_Exo.txt)
  3. optional: count the lines in the copy: --> cat 02n_Exo.txt | wc -l ; cat 2N | wc -l ; cat 02k_Exo.txt | wc -l ; cat 2K | wc -l
  4. use perl and sed to put each word in the file on one line: --> perl -pi -e 's/ /\n/g' 2N
  5. repeat for the other file for each book, e.g. Exodus: --> perl -pi -e 's/ /\n/g' 2K
  6. clean 2N and 2K to remove blank lines, words starting with lower case, special characters and verse numbers: --> cat 2N | sort | uniq | grep -v ^[a-z] | grep -v ^[0-9] | tr -d '[{}()\;\":,\!\.\?]' | sort | uniq | grep -v ^\' > 2Na note: found that sort and uniq repeated was needed after removing special characters... will test with sorts at end.
  7. Keep 2N original to grep for the KEYWORDS with the short list found in 2Na.
  8. created script with for loop to extract keywords: #!/bin/bash OUT="V-2N.txt" # output file for x in `cat 2Na` # create variable x from the keyword list created in "2Na" do echo $x | tee -a $OUT # list keyword echo "------------------------" | tee -a $OUT grep $x 2N | tee -a $OUT # display each line with the keyword echo "------------------------" | tee -a $OUT done ls $OUT
  9. Then added the count and a better separator: --> more find-names-count-in-N2 #!/bin/bash OUT="V-2N.txt" cat /dev/null > $OUT for x in `cat 2Na` do echo $x | tee -a $OUT echo "------------------------" | tee -a $OUT echo $x | grep $x 2N | wc -l | tee -a $OUT echo "------------------------" | tee -a $OUT grep $x 2N | tee -a $OUT echo "------------------------" | tee -a $OUT echo "## KEY WORD, times found, verse(s) ##################################" | tee -a $OUT done ls $OUT --------------------------- ## KEY WORD, times found, verses ################################ Naphtali ------------------------ 1 ------------------------ Exo 1:4 Dan and Naphtali, Gad and Asher.

Session #110 - October 20, 2017 - FRIDAY - wget, advanced vi editing
note on html: color for cell above is "0F0F0F" (rrggbb), font color is "white"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


  1. we've covered .bashrc: Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
  2. we've reviewed the "five essential" commands: man, ls, cd, pwd and more
  3. we've worked our way through the SPECIAL CHARACTERS - and will continue to...
  4. will discuss some issues of interoperability with different BASH environments, e.g. MacOSx, different versions of BASH
  5. we'll work on building some web pages - include the use of sed, awk, tr and even some perl


  6. Session #109 - October 13, 2017 - FRIDAY - Adventures with Special Characters
    note on html: color for cell above is "F0F0F0" (rrggbb), font color is "black"
    (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


    1. we've been covering the elements of the .bashrc:
      Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
    2. we've been reviewing the "five essential" commands: man, ls, cd, pwd and more
    3. we've been working our way through the SPECIAL CHARACTERS - and will continue to this session...
    4. we'll discuss the scripts below - including the use of sed, awk, tr and even some perl
    5. used find, exiftool and reg exp tools to gather lens info from NEF files.
      Special characters are powerful representations of shell functions and values.





      Remove spaces and replace special characters:
      http://johnmeister.com/linux/Scripts/fix-filenames.sh.html

       
      #!/bin/bash
      ###############################################################################################
      #             removes spaces, special characters, slashes, and _-_  but NOT caps
      ###############################################################################################
      for f in *
          do
          mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
          done
      
      examples of loops:
      http://johnmeister.com/linux/Scripts/LOOPS-shell-examples.html

      Use AWK in a loop to move large numbers of files on a Linux web server: http://johnmeister.com/linux/Scripts/awk-to-rename-files.sh.html
      1. create temporary script, e.g. "vi fsj-rename"
        (if you have more than one directory create a more structured script and place it in your ~/bin).
      2. #!/bin/bash # GOAL: rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt" # DATE: 4/27/2004 - Author John Meister # TEST: --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' # PRODUCED: mv fsj-digest-v1-2123.txt fsj-2123.txt # # PROCESS: --> ls fsj-digest-v1-????.txt | \ # awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh # PROCESS: edit file to include header info & comments # mv fsj-digest-v1-1800.txt fsj-1800.txt mv fsj-digest-v1-1801.txt fsj-1801.txt ... -------------------------
      3. THEN... execute the script by typing: sh ./fsj-rename
      4. check directory and delete temporary script: ls ; rm -f fsj-rename

      Session #108 - October 6, 2017 - FRIDAY - Adventures in BASH
      note on html: color for cell above is "00FFFF" (rrggbb), font color is "black"
      (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


      Last week we discussed in detail the .bashrc file, and demonstrated some of the "five essential" commands:
      man, ls, cd, pwd and more

      This week: we will use regular expressions to replace special characters,
      deal with bad file-naming conventions, and rename long file names with AWK.

      FIRST - what are special characters?
      Special characters are powerful representations of shell functions and values.
      In the picture below there is a listing of the special characters and examples of their use.
      Special characters must be understood.




      PROBLEM: Someone created files and directories with spaces in the names, or worse, special characters.
      In UNIX-like systems spaces at the command line represent various commands or options.
      If a filename or directory has spaces,
      then those spaces must be "escaped" with a "\" or encapsulating the entire filename in quotes.
      This gets complicated in a hurry.

      With this script, we will remove spaces and replace special characters:
      http://johnmeister.com/linux/Scripts/fix-filenames.sh.html
       
      #!/bin/bash
      ###############################################################################################
      # 2014_05May_09 - john meister copyright © 2014 http://LinuxMeister
      # script to remove spaces in file names and replace with underscore
      # cleans up web pages by eliminating special characters and allows sorts to work properly
      #  to use place this script in your path and make it executable, then cd to the directory
      #    and execute by the saved name (fix-filename)  - works on Linux and MacOS
      ###############################################################################################
      #             removes spaces, special characters, slashes, and _-_  but NOT caps
      ###############################################################################################
      for f in *
          do
       mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
          done
      
      
      NOTE: we'll cover more loops as we go along, here's some examples of loops:
      http://johnmeister.com/linux/Scripts/LOOPS-shell-examples.html

      ISSUE: created a bunch of files with really long stupid names, however,
      the good news is I had a "delimiter" that would work with AWK.
      By using the "-" in the really long stupid file names I was able to use AWK
      to simplify the filenames. This was a real world problem that was fixed.

      Example of using AWK in a loop to move large numbers of files on a Linux web server: http://johnmeister.com/linux/Scripts/awk-to-rename-files.sh.html
      1. create temporary script, e.g. "vi fsj-rename"
        (if you have more than one directory create a more structured script and place it in your ~/bin).
      2.  
        #!/bin/bash
        # GOAL:  rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
        # DATE: 4/27/2004 - Author John Meister 
        #  TEST:  --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' 
        #  PRODUCED:   mv fsj-digest-v1-2123.txt fsj-2123.txt
        # 
        # --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
        #   edit file to include header info & comments
        #
        mv fsj-digest-v1-1800.txt fsj-1800.txt
        mv fsj-digest-v1-1801.txt fsj-1801.txt
        mv fsj-digest-v1-1802.txt fsj-1802.txt
        mv fsj-digest-v1-1803.txt fsj-1803.txt
        mv fsj-digest-v1-1804.txt fsj-1804.txt
        ...
        -------------------------
        
      3. THEN... execute the script by typing: sh ./fsj-rename
      4. check directory and delete temporary script: ls ; rm -f fsj-rename

If we have time we will use AWK to rename the history files discussed last week.
I noticed that I did not use the date command in a way that makes sorting work well.
Current format is: 2017_Mar_27_08:32.history
It would be better for sorting if it were 2017_03_Mar_27_08:32.history

The good news is that I used "_" in the name; thereore, I can modify the script above to change the file names.

Previously the date variable was set to:
HISTFILE="/home/luser/.History/`/bin/date '+%Y_%b_%d_%H:%M'.history`" ; export HISTFILE
Last week we modified .bashrc to fix this variable:
HISTFILE="/home/luser/.History/`/bin/date '+%Y_%m_%b_%d_%H:%M'.history`" ; export HISTFILE
THE DATE command option for alphabetical representation of the month is %b, while %m is the numeric value.

I like to see the Month in letters, but like to sort things properly.
By sorting YEAR, MONTH and then DAY it displays in chronological order.
If I wanted to keep the filename shorter I could replace the hour and minutes with just the PID,
or use the seconds, the actual time of the file is not important.

Session #107 - September 29, 2017 - FRI - Let's USE BASH on Windows 10! - Pt 3
note on html: color for cell above is "0000FF" (rrggbb), font color is "white"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


Will transition FROM: "Let's use BASH on Windows 10!" - TO: ---> "Simply Linux: Basics"
We will focus more on the actual Linux commands, less on the Win10 environment as host.  
We'll talk more about Regular Expressions and the vi editor.  The commands, the core tools.

Not everyone will be able to use Ubuntu BASH in W10, 
some will chose cygwin or other tools such as a VM or putty to a Linux system.

We will still discuss issues across platforms as we proceed. 
It's important to note differences even between UNIX, Linux and other tools.
More will be shared during the session.


finish discussion of .bashrc and .History directory.
http://johnmeister.com/linux/bashrc-basic.html and then - Discuss Chapter Two - Five Essential Commands.
Continuing in Chapter 2 of "Let's Use BASH on Windows 10", but shifting to the other book, same format... Introduction to Linux Chapter 1_Linux_Overview Chapter 2_the_5_essential_commands Chapter 3_the_vi_Editor Chapter 4_shell_config_History_script Chapter 5_File_Management:_cp_mv_rm Chapter 6_Files, Processes and Special Characters Chapter 7_Regular_Expressions_and_Filters Chapter 8_ssh_setup_and_use Chapter 9_Network_Basics Chapter 10_Files_Systems:_df_du ... Showing purpose with examples of settings in .bashrc: PATH; PROMPT; PERMISSIONS; HISTORY; EDITOR; ALIAS; SHELL OPTIONS; FUNCTIONS demonstrating "minor" issues and differences in paths, symlinks and command execution in BASH on Win10. Will show how to create a copy of files in /mnt/c/

HOMEWORK EXERCISES:
  1. RECOMMEND REVIEWING: http://johnmeister.com/linux/Notes/Philosophy_of_Linux.html - there are significant differences between Microsoft and Linux that need to be understood.
  2. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html

Session #106 - September 22, 2017 - FRI - Let's USE BASH on Windows 10! - Pt 2
note on html: color for cell above is "00ff00" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


"Let's use BASH on Windows 10!" - Part 2 - Ch1, Ch2


- Finish up Chapter One - discuss .bashrc and .History directory.  http://johnmeister.com/linux/bashrc-basic.html
- Discuss Chapter Two -  Five Essential Commands.

HOMEWORK EXERCISES:
  1. RECOMMEND REVIEWING: http://johnmeister.com/linux/Notes/Philosophy_of_Linux.html - there are significant differences between Microsoft and Linux that need to be understood.
  2. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html

TABLE OF CONTENTS for "Let's USE BASH on Windows 10 Introduction to BASH on Windows 10 How to INSTALL BASH on WIN10 How to Configure the BASH window Chapter 1_Linux_Overview Chapter 2_the_5_essential_commands Chapter 3_the_vi_Editor Chapter 4_shell_config_History_script Chapter 5_File_Management:_cp_mv_rm Chapter 6_Files, Processes and Special Characters Chapter 7_Regular_Expressions_and_Filters Chapter 8_ssh_setup_and_use - will not cover for Win10 Chapter 9_Network_Basics - will not cover for Win10 Appendix: How to INSTALL_BASH_WIN10 Appendix: The LINUX Philosophy Appendix: Exercise #1 - the 5 basic commands Appendix: Exercise #2 - chmod, chown, file permissions Appendix: Additional info on BASH Appendix: Additional info from Chapter 2 Appendix: SCRIPT: fix filenames Appendix: SCRIPT: use wget, date, and a while loop Appendix: REAL WORLD LINUX COMMANDS See session #105 for links to table of contents on smashwords.

OTHER RESOURCES:

http://johnmeister.com/linux/Overview/LinuxOverview.pdf (print out and use as a guide)
http://johnmeister.com/linux/Overview/Linux-PowerPoint-2004-overview.pdf
http://johnmeister.com/linux/Notes/Real-world-Linux-Commands.
http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide)
http://johnmeister.com/linux/bashrc-basic.html
Session #105 - September 15, 2017 - FRI - Let's USE BASH on Windows 10!
note on html: color for cell above is "1242ab" (rrggbb), font color is "white"

working through the book on using the BASH tool in Windows 10.

Follow along in Linux or MacOSX... it's mostly the same. (you can preview the book up to about page 19.)
TABLE OF CONTENTS for "Let's USE BASH on Windows 10
 
 Introduction to BASH on Windows 10 
 How to INSTALL BASH on WIN10
 How to Configure the BASH window
 Chapter 1_Linux_Overview
 Chapter 2_the_5_essential_commands
 Chapter 3_the_vi_Editor
 Chapter 4_shell_config_History_script
 Chapter 5_File_Management:_cp_mv_rm
 Chapter 6_Files, Processes and Special Characters
 Chapter 7_Regular_Expressions_and_Filters
 Chapter 8_ssh_setup_and_use - will not cover for Win10
 Chapter 9_Network_Basics - will not cover for Win10
 Appendix: How to INSTALL_BASH_WIN10 - will not address in these sessions, 
but this link might help, YMMV:   http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html
 Appendix: The LINUX Philosophy
 Appendix: Exercise #1 - the 5 basic commands
 Appendix: Exercise #2 - chmod, chown, file permissions
 Appendix: Additional info on BASH
 Appendix: Additional info from Chapter 2
 Appendix: SCRIPT: fix filenames
 Appendix: SCRIPT: use wget, date, and a while loop
 Appendix: REAL WORLD LINUX COMMANDS
  NOTE: The book "Simply Linux" follows the same format, 
but that version has more Linux and MacOSX detail, the Win10 version was scaled back slightly: 
https://www.smashwords.com/books/view/705084

Session #104 - August 4, 2017 - FRI - sshd tweaks, journalctl, Apache 2.4 details
note on html: color for cell above is "12ffab" (rrggbb), font color is "black"
 
tightening up sshd instead of using fail2ban

tweaks to /etc/ssh/sshd_config  - 
typical ssh files:  http://johnmeister.com/linux/Notes/SSH/sample-etc-ssh-config-files.html

    make sure root couldn't log in directly.:   PermitRootLogin no
    specify which users can ssh in:             AllowUsers luser
    force only one version of ssh:              Protocol 2 
    give the person ONE try:                    MaxAuthTries 1
    reduce the number of starting connections:  MaxStartups 2:50:5   
    reduce the login grace time from 2 min:     LoginGraceTime 30
    restart sshd                                # service sshd restart 
                
                (remember:    man sshd | col -b > man.sshd.txt - RTM... YMMV!) 

system logs ssh: journalctl -f | grep ssh, or journalctl -f | grep -i Error | grep ssh --> journalctl -f | grep ssh Jun 21 23:18:09 the-linux-system sshd[23682]: error: PAM: User not known to the underlying authentication module for illegal user admin from 103.79.142.31 Jun 21 23:18:09 the-linux-system sshd[23682]: error: maximum authentication attempts exceeded for invalid user admin from 103.79.142.31 port 52295 ssh2 [preauth] Jun 21 23:24:06 the-linux-system sshd[23694]: error: maximum authentication attempts exceeded for invalid user root from 190.49.255.143 port 42263 ssh2 [preauth] Jun 21 23:27:53 the-linux-system sshd[23703]: error: maximum authentication attempts exceeded for invalid user oracle from 201.20.73.3 port 56130 ssh2 [preauth]
if an IP is very persistent, use iptables to block e.g.: iptables -A INPUT -s 43.255.190.xx -j DROP
apache logs: --> ll /var/log/apache2/*error_log if an error log has content, view, and see if you can find the error.
After fixing, make a copy of the log file then empty it as follows: cp the-error_log /var/log/apache2/FIXED/the-error_log-clear-21jun2017 ; cat /dev/null > the-error_log
Session #103 - July 28, 2017 - FRI - .bashrc details
note on html: color for cell above is "12ab34" (rrggbb), font color is "black"
 

HAPPY SYS ADMIN DAY - July 28, 2017!!!
Prepping for LIVE Oreilly Media Windows 10 BASH sessions - test run...
  1. basic .bashrc: http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-bashrc.txt
  2. bashrc delta from default: http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-bashrc-DELTA.txt
  3. summary of shopt (shell options): http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-shopt-list-details.html
HAPPY SYS ADMIN DAY - July 28, 2017!!! Sys Admin song (sung to monty python's lumberjack tune): [spoken] I never really wanted to be a scientist. I wanted to be...a...a SYSADMIN! [system engineer choir and shift supervisor enter, music strikes up] Oh, I'm a sysadmin and I'm OK, I grep all night and I chown all day. [choir] He's a sysadmin and he's OK, He greps all night and he chowns all day. I ping the nodes, I do PM, I awk and perl and sed. I've got a Star Wars lunchbox, And Tron sheets on my bed! [choir] He pings the nodes, he does PM, He awks and perls and seds. He's got a Star Wars lunchbox, And Tron sheets on his bed! I ping the nodes, I change the rates, I fork the processes. I wish that all my lusers would catch some rare disease! [choir, growing slightly uncomfortable] He pings the nodes, he changes rates, He forks the processes. He wishes all his lusers would catch some rare disease! [choir brightens as they repeat chorus] I ping the nodes, I lock the /home partition and umounts.... [choir, very uncomfortable and trailing off] ...HAPPY SYS ADMIN DAY...

Session #102 - June 16, 2017 - FRI - journalctl, Apache 2.4, and guest
note on html: color for cell above is "ffffff" (rrggbb), font color is "brown"
 
system logs: dmesg and journalctl

dmesg # provides kernel information # part of util-linux package, see Linux Kernel Archive; man dmesg # sudo dmesg --clear
sudo journalctl  # sudo journalctl -f  # sudo journalctl | grep -i error # journalctl | grep -i Error | grep ssh | grep -v "Jun 0"


Migration from Apache 2.2 to 2.4 changes to apache2 involved replacing two lines regarding permissions, index directives continued to work: # #Order allow,deny# Allow from all --> Require all granted # # Order deny,allow# Deny from all --> Require all denied
--> apachectl configtest # Syntax OK --> systemctl enable apache2 --> systemctl start apache2
--> systemctl status apache2 . apache2.service - The Apache Webserver Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2017-06-07 00:34:44 PDT; 5s ago Main PID: 2422 (httpd-prefork) Status: "Processing requests..." Tasks: 6 CGroup: /system.slice/apache2.service - 2422 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2429 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2430 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2431 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2432 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2433 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... Jun 07 00:34:44 linuxsystem systemd[1]: Starting The Apache Webserver... Jun 07 00:34:44 linuxsystem systemd[1]: Started The Apache Webserver.
guest presentation: "FireJail" configuration: http://johnmeister.com/linux/SysAdmin/Contrib/FireJail-presentation/

Session #101 - June 9, 2017 - FRI - grepping along
note on html: color for cell above is "aaaaaa" (rrggbb), font color is "lime"

grep basics and using catgrep to analyze config files

http://johnmeister.com/linux/Scripts/catgrep.sh.html
    #  this is the key string:  grep -E -v '(^\s*#|^$)'

 clean copy to copy... copy to ~/bin/catgrep, chmod and update path in .bashrc

#!/bin/bash ################################################################################ # 24 jun 2016 - john meister copyright 2016 http://johnmeister.com/linux # cat's file, then greps out blank lines and comments, offers option to save # 22 jul 2016 - added sed string to strip leading slash off echo /etc | sed 's/^\///' # 6 jun 2017 - finally got the white space string working: grep -E -v '(^\s*#|^$)' # NOTE: create ~/bin directory, chmod to 755, add ~/bin to PATH - test: # ex: --> which catgrep # /home/luser/bin/catgrep ################################################################################ USAGE="catgrep filename # removes commented and blank lines, option to save" echo ' --------------------------------------------------------------------' echo " DISPLAYING file: --> $1 <-- without commented or blank lines" echo ' --------------------------------------------------------------------' cat $1 | grep -E -v '(^\s*#|^$)' # the \s is for white spaces VFL="`echo "$1" | sed 's/^\///' | tr '/' '-'`" ; export VFL echo '-----------------------------------------------------------------' echo "save info in local directory? if yes, type: y" echo '-----------------------------------------------------------------' read answer ; if [ $answer = "y" ] # note, adding brackets for upper/lower case would be [Y,y] then echo "$1" | tee ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt cat $1 | grep -E -v '(^\s*#|^$)' | tee -a ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt #### NOTE: use redirect or append on line above instead of tee -a to reduce screen clutter echo '-----------------------------------------------------------------' echo " info written to: ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt " echo '-----------------------------------------------------------------' else echo ' ### ------------------------------------' echo ' ### not written.' echo ' ### ------------------------------------' fi

additional details: http://johnmeister.com/linux/Notes/REGULAR-EXPRESSIONS-simple.html


Session #100 - May 26, 2017 - FRIDAY - Networking
note on html: color for cell above is "100000" (rrggbb)

100 lunchtime sessions since 2014!

Networking basics and testing performance

BASIC SETUP:  ifconfig lan0 192.168.11.42 netmask 255.255.255.0 broadcast 192.168.11.255 
ADD ROUTE:    route add default gw 192.168.11.1 
COMMANDS:     ifconfig -a, ip addr, ping -c 3 8.8.8.8, route, netstat -ar, traceroute 8.8.8.8, nslookup johnmeister.com/linux
(note: Microsoft commands offer fewer options and change a few names: e.g. ifconfig --> ipconfig, traceroute --> tracert.)
FILES:    /etc/hosts, /etc/resolv.conf, /etc/nsswitch.conf, /etc/hostname, /etc/sysconfig/network (YMMV)

http://johnmeister.com/linux/Notes/NETWORK/basic-networking.html

http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
examining the spreadsheet formulas (CSV)

http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-Generic-BASH.sh.html

Session #99 - May 19, 2017 - FRIDAY - Linux Commands in use.
note on html: color for cell above is "33bb44" (rrggbb)

using basic commands and the vi editor.

 After helping a few friends with Linux recently I see the need 
    for less speed and more detail... so... 
Let's review basics of .bashrc, .History, setting up .ssh 
    and converting a Windows Batch file to Linux to check network performance.
http://johnmeister.com/linux/Notes/bashrc-simple.html
http://johnmeister.com/linux/Notes/SSH/quick-ssh.html
 http://johnmeister.com/linux/Microsoft/LOADTEST-OS-COMPARISON.html 
directory with excel, batch file and large file 
http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
oh, and one slightly advanced moment:
http://johnmeister.com/linux/Notes/setup-LAMP-on-MINT.html
Session #98 - May 11, 2017 - THURSDAY - ssh, rsync, and ftp - along with WinSCP
note on html: color for cell above is "99aaff" (rrggbb)

using ssh, rsync, ftp, NFS

  1. basic: http://johnmeister.com/linux/FileSystems/man.rsync.txt
  2. config info: http://johnmeister.com/linux/Scripts/fix-ftp.sh.html
  3. config info: http://johnmeister.com/linux/Notes/ftp-fixes-for-vsftp.conf.html
  4. basics: http://johnmeister.com/linux/Notes/RSYNC-cmd.html
  5. examples: http://johnmeister.com/linux/Scripts/rsync-directories.sh.html
  6. cron job examples: http://johnmeister.com/linux/FileSystems/rsync-disks-cron.sh.html
  7. cron job examples: http://johnmeister.com/linux/SysAdmin/Cron-Rsync-sync-up-drives.sh.html
  8. using rsync to update: http://johnmeister.com/linux/Scripts/setup-diverse-systems-prototype.sh.html
  9. mention (again): http://johnmeister.com/linux/Scripts/EXAMPLE-OSYNC/
  10. NFS option: http://johnmeister.com/linux/FileSystems/NFS-Server-setup.html
  11. - if time: (demonstrate WinSCP on XP in a VM on Linux or a Mac)

Session #97 - April 28, 2017 - FRIDAY - Linux Commands in use.
note on html: color for cell above is "ff22aa" (rrggbb)

using tools to extract key information on a system

dmidecode, /proc, "stat" commands, and grep...

Using search tool and text editing

Continued discussion on using search tool:
http://johnmeister.com/linux/Scripts/SEARCH-TEXT and
examples and files to download to work with

Apache 2.4 migration

Migration from Apache 2.2 to 2.4 on SuSE.
- http://johnmeister.com/linux/SysAdmin/Apache2-setup.html
(haven't had a lot of time to sort through all the directives)
examine basic HTML structure and discuss how to migrate to 2.4 view-source:http://johnmeister.com/bible/

Session #96 - April 21, 2017 - FRIDAY - Linux Commands in use.
note on html: color for cell above is "11aa33" (rrggbb)

using search tool, catgrep, and basics on Apache web server and directives

  1. search text tool http://johnmeister.com/linux/Scripts/SEARCH-TEXT
  2. script and all files needed to use found at: http://johnmeister.com/bible/SEARCH-KJV/
  3. use catgrep to identify existing config:
    http://johnmeister.com/linux/Scripts/catgrep.sh.html - "catgrep /etc/apache2/default-server.conf"

  4. understand the basics: http://johnmeister.com/linux/SysAdmin/Apache2-setup.html 1) install apache2 - zypper in apache2 2) establish base configuration in /etc/apache2/default-server.conf 3) verify document root and log locations 4) systemctl enable apache2 (*chkconfig) * --> chkconfig -l | grep apache Note: This output shows SysV services only and does not include native systemd services. SysV configuration data might be overridden by native systemd configuration. apache2 0:off 1:off 2:off 3:off 4:off 5:on 6:off 5) systemctl start apache2
  5. examining existing directives, THESE DIRECTIVES WORK with Apache2 on SuSE 13.2, but NOT in LEAP 42.2: DocumentRoot "/srv/www/htdocs" <Directory "/srv/www/htdocs"> AllowOverride FileInfo AuthConfig Limit Indexes Options MultiViews Indexes IncludesNoExec FollowSymLinks IndexOptions IgnoreCase FancyIndexing FoldersFirst NameWidth=* DescriptionWidth=* SuppressHTMLPreamble #SymLinksIfOwnerMatch <Limit GET POST OPTIONS PROPFIND> Order allow,deny Allow from all </Limit> <LimitExcept GET POST OPTIONS PROPFIND> Order deny,allow Deny from all </LimitExcept> </Directory>
  6. review or create systemd service
    - http://johnmeister.com/linux/Scripts/Sample-systemd-script.html
  7. any changes to website can be made throughout using: find . -type f -name README.html -exec cp README.html {} \;

Session #95 - April 14, 2017 - FRIDAY - Linux Commands in use.

note on html: color for cell above is "dddddd" (rrggbb)

Notes for this session:

  1. discuss possible add ons for SuSE http://johnmeister.com/linux/SysAdmin/SuSE-add-ons.html
  2. using CSV (comma separated values) in a script to gather information (part of the Art of Linux Sys Admin series) http://johnmeister.com/linux/Scripts/collect-sys-info.sh.html - automate system documentation!
  3. CSV at the command line - using gnumeric: --> ssconvert spreadsheet.xls newfile.csv # (will this convert csv to xls? RTM anyone?) # a good use for gnumeric! --> apt-get install gnumeric or --> zypper install gnumeric or --> yum install gnumeric
  4. using the office suite at the command line (YMMV) - results on Linux with libreoffice ok... created a directory... need to try other options. CSV at the command line - using libreoffice: --> libreoffice --headless --convert-to csv $filename --outdir $outdir # this worked, created a directory CSV at the command line - using OpenOffice: --> soffice --headless --convert-to csv test-file.xls test.csv # not recommended on a mac, haven't tried on Linux or Win! # tried this with "advice" from a search - it created a ".lock" file owned by root in: (following advice on the internet is NOT recommended) # further advice says delete .lock... # BETTER - chown!!! (I should never do what I find on the internet! (without thinking)... and not on a mac) # /Users/luser/Library/Application Support/OpenOffice/4 removing it removed all of my configuration and history - thinking gnumeric on linux best...

prep material to put into CSV to import into spreadsheet

  1. collect chapters into one book for Proof of Concept (POC): --> cat 11n_1Ki-* > 1Kings.txt ; perl -pi -e 's/1 Ki/1Ki/g' 1Kings.txt # fix space in naming --> cat 1Kings.txt | tr " " "\n" | grep ^[A-Z] | sort | uniq -d > 1Kings-KEYWORDS.txt # vi to clean up sp ch --> cat 1Kings-NAS-KEYWORDS.txt | wc -l 119
  2. create script to take KEYWORDS and list times found and references: #!/bin/bash # Find-Keywords-1Kings.sh.txt 13 Apr 2017 jm ########################################## # list of names: 1Kings-KEYWORDS.txt # complete book: 1Kings.txt ########################################## FILE="Names-found-in-1st-Kings.txt" # output, names to references echo `date +'%Y_%m_%d-%H%M'` > $FILE # way to reset the file and / or keep track echo "======================" >> $FILE # eye candy separator for x in `cat 1Kings-NAS-KEYWORDS.txt` # x is the name to search on do echo $x >> $FILE # list the name echo "----------" >> $FILE # eye candy grep $x 1Kings.txt | wc -l >> $FILE # number of times name is found in book echo "----------" >> $FILE # eye candy grep $x 1Kings.txt | awk '{print $1" "$2}' >> $FILE # print book and reference echo "======================" >> $FILE # eye candy done ########################################## OUTPUT: ########################################## # 2017_04_13-2209 # ====================== # Abel-meholah, # ---------- # 2 # ---------- # 1Ki 4:12 # 1Ki 19:16 # ======================
  3. NEXT CSV - modify script to insert commas for CSV formatting... (not complete, need to wrap lines) ########################################## #!/bin/bash # Find-Keywords-SET-AS-CSV-1Kings.sh.txt 13 Apr 2017 jm ########################################## # 1Kings-NAS-KEYWORDS.txt - grep'd Names, manually edited file # 1Kings.txt # NAS 1 Kings text ########################################## FILE="Names-found-in-1st-Kings-CSV-FORMAT.csv.txt" for x in `cat 1Kings-NAS-KEYWORDS.txt` do echo $x >> $FILE # no eye candy, name, followed by a comma, number of times, references echo ", ">> $FILE grep $x 1Kings.txt | wc -l >> $FILE echo ", " >> $FILE grep $x 1Kings.txt | awk '{print $1" "$2", "}' >> $FILE echo "====================== \n" >> $FILE done ########################################## # OUTPUT: ########################################## Abel-meholah , 2 , 1Ki 4:12, 1Ki 19:16, ====================== Abiathar , 9 , 1Ki 1:7, 1Ki 1:19, 1Ki 1:25,

using grep to get the number of times a word appears in the NAS, and getting the references for study 694 grep -i love NAS.txt | wc -l # 573 695 grep -i hate NAS.txt | wc -l # 312 696 grep -i Lord NAS.txt | wc -l # 6707 697 grep -i sin NAS.txt | wc -l # 1566 698 grep John NAS.txt | wc -l # 136 699 grep Jesus NAS.txt | wc -l # 946 700 grep Joseph NAS.txt | wc -l # 233 701 grep Rebecca NAS.txt | wc -l # 0 702 grep Rebeka NAS.txt | wc -l # 31 703 grep Paul NAS.txt | wc -l # 161 704 grep -i faith NAS.txt | wc -l # 404 705 grep -i faithful NAS.txt | wc -l # 163 --> grep -i faith NAS.txt | awk '{print $1" "$2}' Gen 32:10 Gen 47:29 Lev 5:15 ... Rev 19:11 Rev 21:5 Rev 22:6

Session #94 - April 7, 2017 - FRIDAY - Linux Commands in use.

note on html: color for cell above is "abdddf" (rrggbb)

Notes for this session:

 
  • win10 freebie: ALL COMMANDS from Win explorer: mkdir "c:\ALL_CMDS.{ED7BA470-8E54-465E-825C-99712043E01C}" http://johnmeister.com/linux/Microsoft/SHOW_ALL_COMMANDS-win7-8-10.html
  • got MATE working on SuSE: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
    1. DISPLAYMANAGER_XSERVER="Xorg"
    2. DISPLAYMANAGER="sddm"
    3. DISPLAYMANAGER_REMOTE_ACCESS="yes"
    4. DISPLAYMANAGER_ROOT_LOGIN_REMOTE="no"
    5. DISPLAYMANAGER_STARTS_XSERVER="yes"
    6. DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
    7. DISPLAYMANAGER_AUTOLOGIN="john"
    8. Default: "yes"
    9. DISPLAYMANAGER_PASSWORD_LESS_LOGIN="yes"
    10. DISPLAYMANAGER_AD_INTEGRATION="no"
    11. DISPLAYMANAGER_SHUTDOWN="auto"
  • this is SuSE still runnig xcfe, will fix...: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
    1. DISPLAYMANAGER_XSERVER="mate-desktop"
    2. DISPLAYMANAGER="lightdm"
    3. DISPLAYMANAGER_REMOTE_ACCESS="yes"
    4. DISPLAYMANAGER_ROOT_LOGIN_REMOTE="yes"
    5. DISPLAYMANAGER_STARTS_XSERVER="yes"
    6. DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
    7. DISPLAYMANAGER_PASSWORD_LESS_LOGIN="no"
    8. DISPLAYMANAGER_AD_INTEGRATION="no"
    9. DISPLAYMANAGER_SHUTDOWN="auto"
    10. DISPLAYMANAGER_AUTOLOGIN=""

  • further refinement of process to build study tools, worked through comparing line by line, editing and checking: --> more mergefiles.sh.txt #!/bin/bash # john 30 Mar 2017 merge-files-project.sh.txt ################################################ cp KJV.txt KJVb.txt cp NAS.txt NASb.txt perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt perl -pi -e 's/(.*)/$1(nas)/' NASb.txt paste -d '\n' KJVb.txt NASb.txt > KNb.txt #######################
  • To test I'd view the combined file, if I found an error,
    I'd edit the NAS version and repeat, put it all on one command line: --> vi NAS.txt ; sh ./merge-files-project.sh.txt ; more KNb.txt The first books I configured had more issues, but once my updated scripts had been used it went quickly; took about 3 evenings to finish. This also found the one discrepancy between the translations, as noted in 3Jn 1:15 , where I added a line to the kjv to match: --> grep "3Jn 1:1" KNb.txt # note: if you forget the quotes you get everything that starts out 3Jn and the name of the file as well... # note: the indent on the second line for nas - this is found in the NAS.txt file, one space leading ... 3Jn 1:13 I had many things to write, but I will not with ink and pen write unto thee: (kjv) 3Jn 1:13 I had many things to write to you, \ but I am not willing to write them to you with pen and ink; (nas) 3Jn 1:14 But I trust I shall shortly see thee, and we shall speak face to face. \ Peace be to thee. Our friends salute thee. Greet the friends by name.(kjv) 3Jn 1:14 but I hope to see you shortly, and we will speak face to face. (nas) 3Jn 1:15 THIS VERSE DOES NOT APPEAR IN THE KJV BUT DOES IN THE NASB.(kjv) 3Jn 1:15 Peace be to you. The friends greet you. Greet the friends by name. (nas)
  • So, now I have several versions to work with for searching key words and phrases. The kjv with a verse on each line with and without (kjv), and the same for nas. The combined version has (kjv) and (nas) for clarity. The NAS.txt and KJV.txt are the base copies, but I have both translations by chapter and by book. Using grep I have control for searches on the combined file showing any differences in the translations. Looking for a common phrase: ------------------------------------------------ --> grep -i "Jesus wept" KNb.txt Jn 11:35 Jesus wept. (kjv) Jn 11:35 Jesus wept. (nas) ------------------------------------------------ When I check a phrase initially I'll often have the screen flash by with lots of info, so this helps: --> grep -i "know that I am the Lord." KNb.txt | wc -l 154 ------------------------------------------------ So, then I use a pipe and grep another key word to narrow the results... But sometimes the phrase or keys aren't the same in both and I'll only see one version: ------------------------------------------------ --> grep -i "you will know that I am the Lord." KNb.txt | grep Zion Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ------------------------------------------------ By reducing the phrase and dropping "you will" from the string, then both versions are shown. ------------------------------------------------ --> grep -i "know that I am the Lord." KNb.txt | grep Zion Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain: then shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv) Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ----------------------------------------------- another variation that may work: --> grep -i "[shall ye|will] know that I am the Lord." KNb.txt | grep Zion Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain: then shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv) Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ----------------------------------------------- When I know there is a difference in the translations, or suspect it,
    and I have the reference, I can grep on that: --> grep "Jn 8:58" KNb.txt Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv) Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas)
  • now I can use the file KNb.txt, or KJV.txt, NAS.txt to search out phrases or key words ------------------------------------------------ --> grep -i "know that I am the Lord." KNb.txt | more # both translations with (kjv|nas) at the end --> grep -i "know that I am the Lord." KJV.txt | more # KJV without (kjv) at the end --> grep -i "know that I am the Lord." KJVb.txt | more # KJV with (kjv) at the end --> grep -i "know that I am the Lord." NAS.txt | more # NAS without (nas) at the end --> grep -i "know that I am the Lord." NASb.txt | more # NAS with (nas) at the end
  • an example of a passage to search out is one using "I am", a statement that declared diety. In the Torah when Moses asked who he should say told him, the response was "I AM that I AM". ------------------------------------------------ So, I search: --> grep -i "I am that I am" KNb.txt Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said, Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv) ------------------------------------------------ ok... but where's the nas version? No problem, simply grep for the passage: --> grep "Exo 3:14" KNb.txt Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said, Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv) Exo 3:14 God said to Moses, "I A M W H O I A M"; and He said, "Thus you shall say to the sons of Israel, 'I A M has sent me to you.'"(nas) ------------------------------------------------ AND GUESS WHAT? I just found another edit I need to fix in NAS.txt... :) That's why it didn't show up. Perhaps I should write a script to update all my computers where I have these files now... ------------------------------------------------ But, continuing the search for the other statement of diety... ------------------------------------------------ --> grep -i "I am" KNb.txt | wc -l 1641 ------------------------------------------------ well, need to narrow that down a bit ------------------------------------------------ --gt; grep -i "I am" KNb.txt | grep Jesus | wc -l 66 ------------------------------------------------ still too many verses... going to guess the passage is in the Gospels ------------------------------------------------ --> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | wc -l 33 ------------------------------------------------ Ah, just remembered, Abraham was mentioned in the context... adding a grep: ------------------------------------------------ --> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | grep Abraham Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv) Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas) ------------------------------------------------ Of course I could just memorize the verse with the passage reference... well, I did, but I forgot it... ------------------------------------------------ So... now I've got a tool that allows me to search on key words, phrases and known references! ------------------------------------------------ I'm a happy student... and I need to fix /etc/sysconfig/displaymanager and test, and find a way of checking and fixing NAS.txt in Exo 3:14, and coming up with a way of copying or updating all my systems where I have these notes... ------------------------------------------------ Ok, if we make it this far and have time... fixing NAS.txt - LET'S use grep and awk to find any other instances like Exo 3:14 and create a script or command that lists those passages for us to review, and then possibly create a script using perl or sed to fix those known instances specifically so we don't create other problems in the file. We will need to review those passages to make sure they are the same as the source document. It's tedious and laborius, but important, and using these powerful regex tools in Linux at the command line makes the creation and editing of this tool possible. And if we still have time we can generate an rsync script to update other directories and systems...

Session #93 - March 30, 2017 - THURSDAY - Linux Commands and Scripts

note on html: color for cell above is "abcdef" (rrggbb)
MET on Thursday this week... normally on Friday... 

Notes and the script from the session:

--> more merge-files-project.sh.txt #!/bin/bash # john 30 Mar 2017 merge-files-project.sh.txt # three objectives: # 1) file 1 first, then file 2 (add new line in process to create combined file) - maybe paste isn't the right tool? # 2) copy files, then annotate each line with designator - file1X.txt file2Y.txt # 3) preserve original, annotated and combined in plain text - file1, file1X, file2, file2X AND: BOTHfiles.txt # 4) create file with html table to show cells for each line, two cells in one row... 4.3M files! - BOTHfiles.html # paste -d '\n' file1.txt file2.txt # perl -pi -e 's/(.*)/<img src=\"$1\"><br>$1<hr>/' # KJV.txt NAS.txt ################################################ cp KJV.txt K1.txt cp NAS.txt N1.txt paste K1.txt N1.txt > KN-test1.txt # file with one long line for both files paste -d '\n' K1.txt N1.txt > KN-test2.txt # file with two separate lines in sequence and in order ################################################ # sed 's/$/ [KJV]/' <KJV.txt >KVJb.txt # perl -pe 's/$/ (kjv)/' <KJV.txt >KVJb.txt # need to find global replacement, and also how to insert into a string, if possible ################################################ cp KJV.txt KJVb.txt cp NAS.txt NASb.txt perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt perl -pi -e 's/(.*)/$1(nas)/' NASb.txt paste -d '\n' KJVb.txt NASb.txt > KNb.txt # file with two separate lines in sequence and in order head KNb.txt # put a head command in script to test read -p 'hit enter' ################################################ sed -i 's$.*$<tr><td>&<p></td>$g' K1.txt # using sed to add html tags on each line sed -i 's$.*$<td>&<p></td></tr>$g' N1.txt # using sed to add html tags on each line head -n 3 K1.txt head -n 3 N1.txt read -p 'hit enter' ####################### echo "<html><title>TEST</title><center><table border='10' \ width='95%'>" > KN1.txt-testX.html paste K1.txt N1.txt >> KN1.txt-testX.html # the paste command reads one line from each file echo "</table></center> <a href='copyright-info.txt'>copyright info</a></html>" \ >> KN1.txt-testX.html # adding closing table and link ####################### # perl -pi -e 's$Dan 9:$<b><em>Dan 9:</em></b>$g' KN27-Daniel-Ch9.html # decided to emphasis the book and chapter # head KN27-Daniel-Ch9.html # put a head command in script to test ####################### head KN1.txt-testX.html # put a head command in script to test #######################

Session #92 - March 24, 2017 - FRIDAY - Linux Commands and Scripts

discussion of perl, sed, and editing files using vi and scripts.
COMPLETED the filtering of information from the wget files and fixed special characters and formatting.

1) Will review the entire process and scripts used to extract the information and format it,
2)and, also discuss using rsync between systems.

3) Next step, merge two plain text files one line at a time using a loop to create a table in html.
IF YOU HAVE A SCRIPT that can read one line from two files at a time, please share... KISS... SAW...
KISS - keep it simple sir; ... SAW - simple always works...
the text project:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/the-wget-perl-sed-vi-grep-rsync-plain-text-project.txt

Session #91 - March 17, 2017 - FRIDAY - Linux Commands and Scripts

  1. A quick comparison of commands for MS use:
    http://johnmeister.com/linux/Microsoft/PowerShell-vs-BASH-commands-compared.html
  2. http://johnmeister.com/linux/Scripts/wget-perl-WX-details.html
  3. http://johnmeister.com/linux/Scritps/perl-fix-spaces-remove-brackets.sh.html
  4. the art of Linux Sys Admin outline, sample vi video
  5. SHELL - history, process, commands
  6. discussion on ASCII vs. UTF-8, etc.

Then, continued discussion on using grep, perl, vi and sed to
reformat my favorite translation of the Bible for searching of key words:
we started out using: http://johnmeister.com/linux/Scripts/remove-brackets-w-perl.sh.html to fix brackets,
#!/bin/bash
echo "remove bracketed footnotes"
perl -pi -e 's/\[.\]//g' *.txt
perl -pi -e 's/\[..\]//g' *.txt
after we ran the script in last week's session, see Session #90 notes. So after several books I've
added more filters: http://johnmeister.com/linux/Scripts/perl-fix-spaces-remove-brackets.sh.html
###############################################################################################

#!/bin/bash
echo "remove bracketed footnotes, fixed commas and semicolon spacing"
perl -pi -e 's/\[.\]//g' *.txt  # remove bracketed text such as a footnote
perl -pi -e 's/\[..\]//g' *.txt # remove bracketed text such as a footnote
perl -pi -e 's/,/, /g' *.txt    # adds a space after a comma
perl -pi -e 's/,  /, /g' *.txt  # removes any extra spaces after a comma after adding a space
perl -pi -e 's/;/; /g' *.txt    # adds a space after a semicolon
perl -pi -e 's/:/: /g' *.txt    # adds a space after a colon
perl -pi -e 's$\.$. $g' *.txt   # adds a space after a period
perl -pi -e 's/?/? /g' *.txt    # adds a space after a question mark
###############################################################################################
#  the following sed and perl string replace all the lines below, used brackets for upper-case
###############################################################################################
sed -i 's/[A-Z]/ /g             # add space before uppercase letter
perl -pi -e 's/  / /g           # change two spaces to one 
###############################################################################################
# perl -pi -e 's/A/A /g' *.txt    # adds a space after an uppercase A
# perl -pi -e 's/A  /A /g' *.txt  # removes any extra spaces before an uppercase A
# perl -pi -e 's/T/T /g' *.txt    # adds a space after an uppercase T
# perl -pi -e 's/T  /T /g' *.txt  # removes any extra spaces before an uppercase T
# perl -pi -e 's/W/W /g' *.txt    # adds a space after an uppercase W
# perl -pi -e 's/W  /W /g' *.txt  # removes any extra spaces before an uppercase W
# perl -pi -e 's/Y/Y /g' *.txt    # adds a space after an uppercase Y
# perl -pi -e 's/Y  /Y /g' *.txt  # removes any extra spaces before an uppercase Y
###############################################################################################
#   need to figure out bracket in perl:  perl -pi -e  's/(.*)/<img src=\"$1\"><br>$1<hr>/'
###############################################################################################

Would like to discuss getting the parameters for using perl
instead of sed for character replacement, as I did here:
http://johnmeister.com/linux/Scripts/wget-perl-script-WX-details.html
that uses $1 for bringing buffer content over.
I would also like to refine my notes on moving numbers to the beginning of the line
as I did several sessions ago.

   The filter for the numbers doesn't always get it right, so I've added the filters above. 
##################################### cat $x | grep "text $x" | sed 's/New American/ New American/g' | \ sed 's/[^<]*<\/span>/ /g' | \ sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt #####################################
The goal is to further refine the process of filtering the text data to carefully remove
issues that I have been manually editing.
Also: would like to be able to replace the " and ' and -
symbols that are NOT standard, need the info to replace those "special" characters.

Session #90 - March 10, 2017 - FRIDAY - Linux Commands and Scripts

Discussion on use of wget, grep, vi and sed parameters: http://johnmeister.com/linux/Notes/MANAGE-TEXT/
(discovered that there are issues with Regular Expressions as part of Posix 2 standards!
complicating the use of special characters! see:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/man.regex-7.html)

  1. process: download files (script: base-get-book.sh ) ##################################### wget -O INFO-1 "http://www...page 1 wget -O INFO-2 "http://www...page 2 wget -O INFO-3 "http://www...page 3 #####################################
  2. process: strip out formatting to get specific content (script: extract-text-book.sh) ##################################### #!/bin/bash ##################################### mkdir INFO for x in `ls INFO-*` do ##################################### cat $x | grep "text $x" | sed 's/New American/ New American/g' | \ sed 's/[^<]*<\/span>/ /g' | \ sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt ##################################### mv $x INFO done #####################################
  3. process: make sure all files have proper names for sorting ls > fix ##################################### vi fix remove lines that have digits that will sort: e.g. filename-10.txt leave lines that need to have a zero added, then: :%s/.*/mv & &/g results: mv filename-1.txt filename-1.txt then cursor to the number on the right, and insert a 0 before 1, to get: filename-01.txt repeat (usually 1-9) until done. :wq ##################################### sh ./fix ; rm -f fix ##################################### ls -al (should show all files sorted nicely) #####################################
  4. process: remove remaining internal brackets use script to remove brackets: sh ./remove-brackets.sh * ##################################### script: remove-brackets.sh #!/bin/bash echo "remove bracketed footnotes" perl -pi -e 's/\[.\]//g' *.txt perl -pi -e 's/\[..\]//g' *.txt #####################################
  5. process: edit all the files to remove errors, line up on one line vi INFO-* after making sure there is a number on the left and all the text on one line, then :%s/.*/INFO 9:&/g :wn :%s/.*/INFO 10:&/g :wn :%s/.*/INFO 11:&/g :wn # where INFO is the book or file name, and 9 is the chapter. # The ampersand places the text string with line number to the right repeat until done editing all files #####################################
  6. process: create unique summary file and then copy to other locations cat INFO-* > INFO_number.txt copy files to proper directories and other servers: cp INFO-*.txt ../SORTED-INFO scp -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO or rsync -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO #####################################
  7. repeat until all 1,189 chapters are cleaned up and sorted... then test to see that there will be 31,102 lines, and based on the version, 781,621 words cat INFO-* > total-info.txt ; cat total-info.txt | wc (note: as of 3/7/2017 I'm on 14/66)

Session #89 - March 3, 2017 - FRIDAY - Linux Commands and Scripts

Discussion on use of wget, grep, vi and sed parameters: http://johnmeister.com/linux/Notes/MANAGE-TEXT/
Worked with sed strings to extract specific details from a downloaded file:
cat $x | grep "text $x" | sed 's/New American/ New American/g' |\ sed 's/[^<]*<\/span>/ /g' | \
sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt

As we approached the close of the session we ran out of time sorting out how to remove content between brackets... My subsequent RESEARCH
discovered that there are "some" issues with Regular Expressions as part of Posix 2 standards! Thereby complicating the use of special characters! see:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/man.regex-7.html) for the notes, which confused me for quite some time... reverted to brute force method as shown for session #90.

Session #88 - February 24, 2017 - FRIDAY - Linux Commands and Scripts

http://johnmeister.com/linux/Overview/LinuxOverview.pdf

last weekend: Released Simply Linux: Basics
https://www.smashwords.com/books/view/705084
weekend before: "Let's BASH Win10!" https://www.smashwords.com/books/view/703463
(I skipped watching Doctor Who for TWO weeks now to find time to get these done!!! Glad I have TIVO!)
This week: command line searching for KEY WORDS and organizing results.
Will be using grep, awk, cut, wc, script, nl, and a few others to organize the material. Command line fun!
Parsing through text files for KeyWords and organizing them through REGULAR EXPRESSIONS.

the current list of books and videos - 22 February 2017


click for pdf

Session #87 - February 17, 2017 - FRIDAY - Linux Commands and Scripts

http://johnmeister.com/linux/Overview/
http://johnmeister.com/linux/Overview/LinuxOverview.pdf

NEWS ITEM: Finished the BASH on Win10 book:

https://www.smashwords.com/books/view/703463


Windows 10 and BASH security issues:
https://msdn.microsoft.com/en-us/commandline/wsl/release_notes
https://www.youtube.com/watch?v=_p3RtkwstNk&feature=youtu.be
https://github.com/ionescu007/lxss/blob/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf


In other news, info on Linux, USA based distros:
http://distrowatch.com/table.php?distribution=centos -- https://www.centos.org/ -- https://fedoramagazine.org/
https://en.wikipedia.org/wiki/Fedora_%28operating_system%29
https://distrowatch.com/table.php?distribution=elementary
All USA distributions:
http://distrowatch.com/search.php?ostype=All&category=All&origin=USA&basedon=All¬basedon=None&desktop=\ All&architecture=All&package=All&rolling=All&isosize=All&netinstall=All&language=All&status=Active#simple


Session #86 - February 10, 2017 - FRIDAY - Linux Commands and Scripts

...back to the basics... LPIC 1, BASH Win10, etc...
http://johnmeister.com/linux/Overview/
http://johnmeister.com/linux/Overview/LinuxOverview.pdf
http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html


Session #85 - January 27, 2017 - FRIDAY - Let's BASH Windows 10!

Discuss commands in Windows 10 BASH shell, 148 commands in /bin
- how they compare to Mac OS X and Linux (YMMV)
prior week install notes:
discuss steps to configure Ubuntu BASH in Windows 10.


Session #84 - January 20, 2017 - FRI - improve performance of Linux and setup BASH in WIN10

Finish discussion from Session #83 below,
then discuss steps to configure Ubuntu BASH in Windows 10.

cool items found in today's session:
--> rpm -qf /usr/bin/pdftotext poppler-tools-0.24.3-15.1.x86_64 --> /etc/icewm "MENU" - edit capability plain text!!!! :)


Session #83 - January 13, 2017 - FRI - improving performance of your Linux (desktop/laptop) experience


# quick note on Friday the 13th... the superstition likely started in Europe
before Gutenberg and Luther made Bibles available widely across Europe...
the idea that "Good Friday" was some how "unlucky" because Christ was crucified and
that there were 13 people in the room... (assuming 12 disciples and Christ) apparently came from this belief.
Two problems with this.
1) This happened on the 14th of Nisan, or our Wednesday night (Last Supper/Passover), and
2) if Christ hadn't given His life on the cross we'd be completely without hope, or "luck".
If you have another explanation for Friday the 13th, please share.
For more information on this time line and issue, see: http://johnmeister.com/bible/Crucifixion/THURSDAY-Crucifixion-3days-3nights.jpg

IMPROVING desktop/laptop/server performance of Linux by monitoring, removing or tweaking.

Situation: Running XFCE https://en.wikipedia.org/wiki/Xfce on SuSE Linux (either Leap 42.1, or 13.2)
and noticed CPU consumption by a bunch of applications that are NOT used,
NOT wanted, and shouldn't be there... of course YMMV...
I don't need "orage" (calendar tool), Thunar (file manager -
GUI... if you're going to use the GUI don't remove this),
Clipman or Parcellite (GUI clipboard). I remove gvfs-fuse. because, in part, it produces this error:
find: ‘/run/user/1000/gvfs’: Permission denied
(I don't like the "convenience" of this "feature", it reminds me of Stuxnet,
and the error as root annoys me greatly...)
--------------------------------------------------------------------------
UNNEEDED APPLICATIONS or SERVICES: Process to remove an unneeded/unwanted application: top # observe any heavy hitters or unneeded tools, e.g. parcellite, ModemManager, fuse-gvfs, etc. ps -ef | sort | uniq | grep -v ... those things you know are system related or desired... ps -e | awk '{print $4}' | sort | uniq -c | grep -Ev '(ssh|bash|firefox)' | sort -n ps -ef | awk '{print $8}' | sort | uniq | grep -Ev '(ssh|bash|firefox)' ps -ef | awk '{print $8}' | sort | uniq > processes-Jan-4.txt # examine the output: cat processes-Jan-4.txt FOUND: ModemManager (on a desktop without a phone line) # there are some other items that could be removed, gnome-key-ring... sudo lsof /usr/sbin/ModemManager # this gives info on the application sudo kill -9 743 # the pid for ModemManager or better, use service ModemManager status service ModemManager stop sudo zypper remove ModemManager ...(1 package to remove: ModemManager (frees up 2.3MB) ...removed ModemManager-1.4.10-3.2.x86_64 sudo zypper -ps -s # to check services using deleted files --> sudo zypper ps -s The following running processes use deleted files: PID | PPID | UID | User | Command | Service -----+------+------+---------+----------------------------+---------------- 831 | 1 | 0 | root | mcelog | mcelog 861 | 1 | 497 | polkitd | polkitd | polkit 1223 | 1 | 0 | root | cupsd | cups 1241 | 1 | 0 | root | sshd | sshd 1398 | 1396 | 0 | root | Xorg | display-manager 1411 | 1396 | 0 | root | xdm | 1499 | 1435 | 1000 | luser | ssh-agent | 1501 | 1 | 1000 | luser | xfconfd | 1504 | 1435 | 1000 | luser | xfce4-session | 1511 | 1 | 1000 | luser | kdeinit4 | 1513 | 1511 | 1000 | luser | kdeinit4 | 1515 | 1 | 1000 | luser | kdeinit4 | 1517 | 1 | 1000 | luser | gconfd-2 | 1522 | 1 | 1000 | luser | xfwm4 | 1526 | 1 | 1000 | luser | xfce4-panel | 1528 | 1 | 1000 | luser | thunar | 1530 | 1 | 1000 | luser | xfdesktop | 1533 | 1 | 1000 | luser | xfsettingsd | 1534 | 1504 | 1000 | luser | mate-volume-control-applet | 1536 | 1504 | 1000 | luser | python2.7 | 1541 | 1504 | 1000 | luser | python3.4 | 1542 | 1 | 1000 | luser | xfce4-power-manager | 1543 | 1504 | 1000 | luser | nm-applet | 1544 | 1504 | 1000 | luser | kdeconnectd | 1550 | 1504 | 1000 | luser | pk-update-icon | 1556 | 1 | 0 | root | upowerd | upower 1565 | 1 | 1000 | luser | pulseaudio | 1595 | 1 | 1000 | luser | at-spi-bus-launcher | 1610 | 1 | 1000 | luser | at-spi2-registryd | 1615 | 1536 | 1000 | luser | python2.7 | 1616 | 1615 | 1000 | luser | python2.7 | 1624 | 1565 | 1000 | luser | gconf-helper | 1630 | 1 | 1000 | luser | python2.7 | 1649 | 1526 | 1000 | luser | wrapper-1.0 | 1650 | 1526 | 1000 | luser | wrapper-1.0 | 1657 | 1526 | 1000 | luser | xfce4-terminal | 1698 | 1526 | 1000 | luser | VirtualBox | 1718 | 1 | 1000 | luser | VBoxXPCOMIPCD | 1723 | 1 | 1000 | luser | VBoxSVC | 1744 | 1723 | 1000 | luser | VirtualBox | You may wish to restart these processes. See 'man zypper' for information about the meaning of values in the above table. alternate, disable service sudo systemctl disable ModemManager.service sudo zypper remove tracker-extract To find out what packages are installed: --> zypper wp > zypper-wp-SuSE.txt # NOTE: wp stands for "what-provides" --> zypper wp > cat zypper-wp-SuSE.txt # if there is an "i" in front, it's installed. # there wasn't an "i" in front of zile, so I tried to remove it: --> zypper remove zile Root privileges are required for installing or uninstalling packages. ------------------------------------------------ --> sudo zypper remove zile Loading repository data... Reading installed packages... 'zile' not found in package names. Trying capabilities. No provider of 'zile' found. Resolving package dependencies... Nothing to do. ------------------------------------------------ # THIS PERPLEXED ME... so... I checked the man page: --> man zypper | col -b > man.zypper.txt # found several directories, and then noticed the log and history file. --> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}' --> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}' | grep Modem install libKF5ModemManagerQt6 install ModemManager so.... one could: sudo zypper remove ModemManager # already did... ------------------------------------------------ see also: Zypper cheat sheet see also: assorted tools (apt) cheat sheet see also: Yum cheat sheet NOTE: If you're using MUD then it would be: sudo apt-get remove ModemManager IF you realize that you really needed the app, you could reinstall: sudo zypper install ModemManager (or apt-get install...) If you're using Centos or RH, sudo yum remove / install .... If you're using RPM, see man pages for options. -------------------------------------------------------------------------- KERNEL TWEAKS: ADJUST SYSTEM TO REDUCE SWAP: sysctl -w vm.swappiness=1 When swappiness is reduced the kernel doesn't attempt to increase cache by paging out. Paging is not good... To change permanently, write vm.swappiness=1 into your /etc/sysctl.conf file. (NOTE: do NOT do this is you are memory constrained on your system, typical default setting is around 60) -------------------------------------------------------------------------- APPLICATION TWEAKS: As far as "recommended" applications (Firefox, Thunderbird and OpenOffice (LibreOffice) 1) set cache to 0 in firefox. (go to the hamburger thing on the far right side, select preferences, advanced, Cached Web Content) More tweaks for the mighty trio to come... (feel free to share...)

Session #82 - January 6, 2017 - FRIDAY - managing file archives at the command line

using Linux and vi at the command line to manage files and directories
 PROBLEM: have a large server with duplicate directories and files, 
                need to organize by date, compare and consolidate
       OBJECTIVE:  Standardize DIRECTORY names for proper sorting:   YYYY-MM-DD-filename.txt
                   Individual file names can have the date as a suffix before the file type, 
                            e.g.  filename-YYYY-MM-DD.txt
                    Files will sort by number, then letter, starting from the left to the right.  

step 1: change into directory, for example, the year 2000, June... cd 06_June-2000 step 2: list the files and create a file to edit (see details below) step 3: use vi to create a quick script to rename files, when you exit vi, it executes the script, then removes the script step 4: then you examine the directory to make sure the files were named correctly and mv them up to the parent directory step 5: once you're in the parent directory, check the old directory using a recursive listing: "ls -AlR 06_June-2000" step 6: If the directory is empty and didn't contain any hidden directories (e.g. .profile ), then: "rm -rf 06_June-2000" NOTE: if you're comfortable with this process, then you can add: " mv * .. ; cd .. ; rm -rf 06_June-2000 " to the script. DETAILED STEPS: --> cd /home/luser/files/2000/06_June-2000 ------------------------------------------------ luser@penguin [/home/luser/files/2000/06_June-2000] ------------------------------------------------ --> ls > fix ; vi fix ; sh ./fix ; rm -f fix
# NOTE: be sure NOT to use a keyword or command, fix might not be the best choice... to check the word used: --> which fix which: no fix in (/home/.... # we're ok... if it was a keyword or command, just use some other combination, like mymess2...
# while in vi: # note: you will be in vi immediately, when you quit out it'll attempt to execute the file, make sure it's right... :%s/.*/mv & 2000-06--&/g # global copy and paste with a prefix of information for another editing step (note: where 06 = Jun) :%s/2000-06--06-Jun-/2000-06-/g # another global replace to modify the file, putting the date in the preferred order # while in vi you'll need to edit out the REDUNDANT -2000 (the original file had 06-June-15-2000 as the file format) # there might be a better way of creating the FROM TO names using perl, sed, and awk, but this is a quick and dirty script... # use /-2000 to find the 2nd instance, then delete TWO words using vi/command mode: "2dw" - if you goof, <esc>u and try again. # the desired line in the "fix" script mv 06-Jun-15-2000-SJ-egr-tube 2000-06-15-SJ-egr-tube # Once the directories are named correctly, move them up to the main directory and remove the old directory. --> mv * .. ; cd .. ; ls -AlR 06_June-2000 # if it's all clean, then nuke it... --> rm -rf 06_June-2000

updating directories for web presentation

After the directories for each year are organized by YYYY, then MM, then DD (e.g. 2000 06 15) you can add any particular web content into each directory via a quick and dirty script. I add a README.html, HEADER.html and for images I add an ALL.html. The manual process is something like this: --> cd 1999-12-Marks-J3000 ; mk-webpage "1999-12-Marks-J3000" ; ls -al ; cd .. # the "mk-webpage" is a custom script to create localized web content - we've discussed this script in prior sessions see: http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html # So, the process we need to "script" is: cd 1999-12-Superdawg ; mk-webpage "1999-12-Superdawg" ; ls -al ; cd .. So, for every directory we need to cd into it, run the mk-webpage script. PROCESS: --> cd 2000 --> ls > addall --> vi addall # while in vi the process is simple and quick: :%s/.*/cd & ; mk-webpage "&" ; cd ../g # the lines should look like: cd 2000-01-01-BEST ; mk-webpage "2000-01-01-BEST" ; ls ; cd .. cd 2000-01-01-Superdawg ; mk-webpage "2000-01-01-Superdawg" ; ls ; cd .. ... then simply escape, write, quit: "<esc>:wq" then execute the quick and dirty script to update the entire year's worth of directories for your website: --> sh ./addall then copy the updated directory, to your web server (unless you edited in place), and then check one of the directories and try an ALL.html file: http://johnmeister.com/fotos/2002/2002-02-05-Sun-Solaris-Install/ALL.html This is an updated version of a Sun Solaris install from February 2002. ------------------------------------------------


Session #81 - December 16, 2016 - FRIDAY - Linux text management

use sed, vi and perl to combine lines in large text files
http://johnmeister.com/linux/Notes/MANAGE-TEXT/
using sed, vi, and perl will demonstrate how to combine lines in a large block of text
in order to use grep to locate key words and display entire content of line.

Session #80 - December 9, 2016 - FRIDAY - Linux and troubleshooting

troubleshooting, diagnosis, monitoring
Have been setting up some systems and having challenges with BIOS, proprietary RAID and hard drive failures.
some issues during setup of dell towers in bios random bios screens


"Bob" found some interesting links on SMART:
https://en.wikipedia.org/wiki/Comparison_of_S.M.A.R.T._tools - https://www.smartmontools.org/
https://sourceforge.net/projects/smartmontools/ and https://sourceforge.net/projects/smartmontools/files/smartmontools/6.5/

file system recovery
http://johnmeister.com/linux/FileSystems - recovery of failing drives - tune2fs - grub and UUID
http://johnmeister.com/linux/FileSystems/rescuing-failing-drive-w-dd.html

fresh file system setup
http://johnmeister.com/linux/Notes/KERNEL-BOOTING/grub.cfg-example.html
http://johnmeister.com/linux/FileSystems/UUID-details.html
http://johnmeister.com/linux/FileSystems/newdisk-setup.html
http://johnmeister.com/linux/FileSystems/mount-external-drive-rysnc.txt
http://johnmeister.com/linux/FileSystems/setup-LVM.html

in other use: http://johnmeister.com/linux/Notes/using-find-grep-xargs.html


Session #79- December 2, 2016 - FRIDAY - Simply Linux and mysql

open disucssion: current issue: metadata from RAID on Dell
fdisk -l details
SuSE update screen
updated h/w, dell tower T5600 --> T5810 - Linux not able to boot... metadata based on proprietary consumer grade h/w,
also using LVM... this is why SAW (simple always works... and using "new" features often doesn't)

Continued discussion on command line mysql on Linux.
steps to install, activate, and create simple databases as the command line in Linux.
still working on importing CSV, in the meantime, found the commands to make the table...
will capture working commands and populate... will incorporate commands into BASH scripts and continue developing
tools at the command line to manipulate data.
• sudo zypper install mysql (or sudo apt-get install mysql)
• sudo services mysql start
• mysql -u root -p -h 127.0.0.1
• create DATABASE firsttry;
• show databases;
• show tables;
• create table equipment (hostname varchar(20), location varchar(20), OS varchar(10));
• describe equipment;
• insert into equipment(hostname,location,OS) values ('lab1', 'bench 1', 'Mint');
• select * from equipment;
• status;
• help;
Alternate paths for links: http://LinuxMeister
http://shop.oreilly.com/product/0636920050209.do?cmp=ex-prog-books-videos-product-na_moo_card_for_john_e._meister,_jr.\
_for_study_guide_for_lpic_2_exams_201_and_202
http://johnmeister.com/linux/Intro-to-Linux/One-Hour-Linux-Sessions-2016.html
(not updated as frequently - mirrored on different server)


Session #78- November 18, 2016 - FRIDAY - Simply Linux and mysql

  • sudo zypper install mysql
  • sudo services mysql start
  • mysql -u root -p -h 127.0.0.1
  • create DATABASE firsttry;
  • show databases;
  • status;
  • help;

Session #77- November 11, 2016 - FRIDAY - Simply Linux: book outline

  1. Using Linux
  2. Linux Summary
  3. Linux Overview and Setup
  4. Five (5) Commands you need to master
  5. Shell configuration, History, and "script"
  6. File Management
  7. Files and Processes: Special Characters
  8. Regular Expressions and Filtering Data
  9. SSH setup and use
  10. Network Basics
  11. File Systems - Create, Monitor, Manage
  12. User Management (System Administration tasks)
  13. Package Management - Upgrading and Installing
  14. Appendix A: The LINUX Philosophy
  15. Appendix B: Exercise #1 - basic commands

Session #76- November 4, 2016 - FRIDAY - Linux: command line

Regular expressions experimenting with "cal" in a loop: using a loop to determine the day of the week for the 15th of July over various years...
a discussion on ffmpeg with a cool timelapse, and look at x2go.

copy and paste links below, not live

Video created by Uncle Mike: Facebook post Public time lapse video:
https://www.facebook.com/m.d.clay/videos/vb.703246157/10154101081271158/?type=2&theater
A look at another NX technology tool for remote access of Linux systems:
http://wiki.x2go.org/doku.php/doc:newtox2go and https://en.wikipedia.org/wiki/X2Go

Session #75- October 20, 2016 - THURSDAY - Linux: open discussion and follow up

Discuss follow up on ffmpeg, video formats, and Apache configuration and services. apache2-service-commands.html

Session #74- October 14, 2016 - Friday - Linux: trace a script to encode video

https://gist.github.com/donmelton/7734177 - found this script to encode video, looks interesting... still don't have ffmpeg working

Session #73- October 7, 2016 - Friday - Linux: fixing crontab email and examining ffmpeg, vlc and mencoder

http://johnmeister.com/linux/Notes/crontab-email-setup.html
Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series

Session #72- September 28, 2016 - Wednesday - Linux: Using grep, awk, cat, diff, wc, and comm

NOTE: ### WEDNESDAY THIS WEEK ###.... WEDNESDAY - COMMAND LINE POWER
using cat, grep, awk, sort, redirection, comm, more, perl (sed), tr
and wc to solve a cool problem
There were multiple files with common values and lots of data.
We needed to find the common values among several files.

Took real world info and completely buggered it up to show ONLY how to find
certain things in a large file.
The content of the files is irrelevant, so it was obscured to help us find only
what we were seeking.

# this is the process used, but was NOT the strings used to corrupt the data...
cat original.file.1.txt | tr c-z a-x | tr 2-8 3-9 | tr A-Y B-Z > 1.RAW
# after completely messing up the content, look for the one string,
"Messedup", and change it to our primary search value:
perl -pi -e 's/Messedup/KEY-VALUE/g' *.RAW
Then we use grep to find that KEY-VALUE among thousands of lines in the file
and look for the info labeled as "Key-Value":

cat .RAW | grep KEY-VALUE | grep - | awk '{print $4}' | sort > 1.rpt
...
comm -12 1.rpt 2.rpt | comm -12 - 3.rpt > common-values.txt
...
we'll test our results with cat and grep...
time permitting we'll convert this into a script... and enhance it...

Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series

Session #71- September 22, 2016 - Friday - Linux: file systems, shell scripts and configurations

The discussion on file systems most likely include Clay's presentation from last week:
(we've been having an email discussion on ext4 and journaling...)
http://johnmeister.com/linux/SysAdmin/Contrib/lunch-n-learn_2016-09/
Clay has done more research on journaled (ext4) file systems along with Mark, the two of them will want to share.
script in work: http://johnmeister.com/linux/Scripts/setup-diverse-systems-prototype.sh.html
have been working on shell script to gather key info from a variety of linux versions and distros,
seeking to gather config data and either automate a standard configuration,
or at least have an on-screen checklist with commands spelled out that can be used with tweaks.
There are certain environments where managing Linux installs is a challenge because of networking related issues,
primarily those lacking name services, so one must use the IP address or use /etc/hosts to make things work.
This can be a problem as these environments use either DHCP or DDNS. Automation in an uncertain environment is risky...
Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series

Session #70 - Sep 16, 2016 - Friday - Linux command line - Clay's persistent USB 3.0 Portable Linux

Clay will present "Portable Linux" with the details and shell script to turn a USB drive into a persistent Linux instance.
http://johnmeister.com/linux/SysAdmin/Contrib/lunch-n-learn_2016-09/
His recommendation for host device is the USB 3.0 San Disk ultra,
linked below is the 128GB drive, Clay suggests a minimum of 32GB:
https://www.amazon.com/SanDisk-Ultra-SDCZ43-128G-GAM46-Newest-Version/dp/B01BGTG2A0/ref=dp_ob_title_ce
While other devices will work Clay recommends this type because it is small, USB 3.0,
and doesn't stick out of the USB slot so it doesn't risk damage to your USB port.
He will focus on minimizing writes to the device to prolong its life.
A link to my server with the script and details will be provided later in the week.

In the past we've discussed using Gparted booted from a USB device to recover a system, Clay's presentation is a
significant advancement on that idea, allowing someone to maintain a complete Linux system with persistent data and
configuration and use whatever computer is available.
The perfect PARASITE that doesn't leave any itchy red spots when finished.
The beauty of using the USB device is nothing on the local hard drive is touched,
so you can use any computer anywhere with this setup and not risk infection
or exposing data on the computer or to the computer.

Links to prior discussions as a baseline:
http://johnmeister.com/linux/Notes/USB-live-Gparted-setup.txt
http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.html
http://johnmeister.com/linux/Notes/failing-drive-system-rescue.html

Session #69 - Sep 9, 2016 - Friday - Linux command line - Discuss Oracle's Virtual Box and Centos Install

INSTALLING CENTOS 7 on a VM...
(distracted: found the limit when we opened 4 VM's (Win10, WinXP, Solaris and Centos)
on a MacBook Air: 2 browsers, Webex, and Thunderbird)
- rather than wait for it to recover, powered down... it did NOT crash though!
continued discussion on using Windows 10 (v1607) BASH (Ubuntu) REGULAR EXPRESSIONS
http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/
http://www.tenforums.com/tutorials/46769-bash-ubuntu-windows-10-enable-disable.html
http://www.omgubuntu.co.uk/2016/08/windows-10-anniversary-update-delete-partition

Session #68 - Aug 26, 2016 - Friday - Linux command line - Digital photography and web pages

continue discussion on setting up pages using vi, html and regular expression,
and also discuss Windows 10 BASH in v.1607.
Creating new script to add exiftool data to pages on web server.
Exiftool extracts exif info from raw and jpg files.
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
http://johnmeister.com/linux/Scripts/exiftool.sh.txt


Session #67 - Aug 19, 2016 - Friday - Linux command line - Digital photography and web pages

Creating new script to add exiftool data to pages on web server. Exiftool extracts exif info from raw and jpg files.
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
http://johnmeister.com/linux/Scripts/exiftool.sh.txt


Session #66 - Aug 10, 2016 - Wednesday - Linux command line discussion

Demonstrate the use of rsync and scp in scripts and setting up cron to automate.
http://johnmeister.com/linux/Scripts/rsync-directories.sh.html
http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html

Session #65 - Aug 2, 2016 - Wednesday - Linux command line discussion

Discussion showing differences between rsync and scp and the effects of placing in the background:
      --> rsync -rp --delete linuxbox:/home/luser/files/METRICS/ linuxbox-files/METRICS
IF THE slash on the source is left off... then the entire directory is copied over to the destination directory.
IF the slash is present on the source, then the files are moved into the destination directory. 
    (the idea is to copy the entire directory as part of the path, replicated to the remote box... if you don't
    get the slashes correct the files will be placed in a directory one level up, creating a mess...)
------------------------------------------------
--> echo "THIS WORKS for SCP, need slash, no dest DIR: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/"
        scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/ &
THIS WORKS for SCP, need slash, no dest DIR is required: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/
The -q is for quiet, keeps the pipe buffer from filling and stopping the process if scp is placed in bg. 


Session #64 - July 29, 2016 - Friday - Linux command line discussion

SYSTEM ADMINISTRATOR APPRECIATION DAY!!! http://sysadminday.com
Continue discussion of command line configurations and scripting the setup of printers and workstations.
Review of SuSE LEAP distribution as based on SLE.


Session #63 - July 22, 2016 - Linux discussion

The Art of Linux System Administration - LPIC-2 Study Guide

Session #62 - July 15, 2016 - Linux Course released by O'Reilly! Free videos - bring popcorn

The Art of Linux System Administration - LPIC-2 Study Guide Will go over the course outline and maybe show one or two of the videos.
Will also review "shell" options: shopt (which works in any BASH environment, Linux, UNIX or MacOSX
-----------------------------------------------
--> shopt -s histverify
-----------------------------------------------
--> shopt
...
histverify on
...
------------------------------------------------


Session #61 - June 24, 2016 - Linux Overview - continued

see Session #60 below for content links - continued command line demonstrations and explanations.

Session #60 - June 17, 2016 - Linux Overview - continued

Linux Overview on one page
...we left off discussing Special Characters in preparation to show the use of Sed, Grep and Awk in a shell...
...of course we also mentioned col -b and creating copies of text files of man pages, see your homework:
http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html man 'command' | col -b > man.command.txt
Then we'll talk about using sort... and other filters... with whatever chunks of data I have at hand...

Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk


Session #59 - June 9, 2016 - Linux Overview

Linux Overview
Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk


Session #58 - June 2, 2016 - Linux Tuning Overview

Linux Tuning

Session #57 - April 8, 2016 - using a script to extract system info and create a CSV file

creating a script to create CSV

Session #56 - April 1, 2016 - Debunking Calendar Myths with Calendar

calendar myths
example... 5 Sundays and 5 Saturdays... an email might claim it only happens once every X years,
with X being some massive number... wrong... but who's going to flip through a century's worth of calendars?
Well, Linux will:   (adjust Y=2009 to something like Y=1900 to see much more)

--> for ((Y=2009; Y<=2016; Y++)); do for M in 1 3 5 7 8 10 12;\
 do cal $M $Y | grep -q ^31 && printf '%02d/%d\n' $M $Y; done; done
05/2009
01/2010
10/2010
07/2011
03/2013
08/2014
05/2015
01/2016
07/2016


Session #55 - March 25, 2016 - ssh configuration files and tunneling

http://johnmeister.com/linux/Scripts/extract-SSH-config-info.sh.html http://johnmeister.com/linux/Notes/SSH/sample-etc-ssh-config-files.html http://johnmeister.com/linux/Notes/SSH/ssh-base.html http://johnmeister.com/linux/Notes/SSH/typical-etc-ssh-conf-files.html http://johnmeister.com/linux/Notes/SSH/quick-ssh.html http://johnmeister.com/linux/Notes/SSH/ssh-full-duplex-process.html http://johnmeister.com/linux/Notes/SSH/ssh-timeout-tweaks.html

http://johnmeister.com/linux/Notes/SSH/reverse-tunnel.html http://johnmeister.com/linux/Notes/SSH/ssh-tunnel.jpg

Session #54 - March 18, 2016 - Compile the Kernel and recovery


http://johnmeister.com/linux/Notes/Compile-a-Kernel.html 

http://johnmeister.com/linux/Notes/Dracut-recovery-shell.html 



Session #53 - March 11, 2016 - Kernel, booting and GRUB

Discussion on Kernels, booting, and GRUB - Grand Unified Boot Loader

http://www.ibm.com/developerworks/library/l-linuxboot/ 

http://johnmeister.com/linux/Notes/grub.cfg-example.html 


Session #52 - March 4, 2016 - NFS basics and RAID

Discussion on Network File Systems and RAID

http://johnmeister.com/linux/FileSystems/NFS-info.html

http://johnmeister.com/linux/FileSystems/RAID-info.html


Session #51 - February 26, 2016 - The use of "script" and col -b

  The use of "script", as root user, from /root, type:

	script setup-new-user-home-luser.RAW
optional ex: (add user to /etc/passwd, /etc/shadow, /etc/group - edit files manually or use "useradd -m username")	
        mkdir -p /home/luser/.History ; mkdir -p /home/lusers/.ssh ; mkdir -p /home/lusers/bin
	cp /root/users.bashrc /home/luser/.bashrc
	man bash | col -b > /home/luser/bin/man.bash.txt
	ssh --help | col -b | tee -a /home/luser/bin/man.ssh.txt
	chown -R luser:users /home/lusers
	exit
	cat setup-new-user-home-luser.RAW | col -b > setup-user-home.txt ; rm -f setup-new-user-home-luser.RAW
	ls -alR /home/luser | more
	vi setup-user-home.txt  (replace "luser" with variable for user, and modify as a wrapper script to use with tool)


---------------------------------------------------------------------------------------------------------------------------- Notes, general (SuSE and MINT): --> which useradd: /usr/sbin/useradd --> useradd --help: Options: -m, --create-home create the user's home directory SYNOPSIS useradd [options] LOGIN useradd -D useradd -D [options] ---------------------------------------------------------------------------------------------------------------------------- Notes, MINT (MUD): SYNOPSIS adduser [options] [--home DIR] [--shell SHELL] [--no-create-home] [--uid ID] [--firstuid ID] [--lastuid ID] [--ingroup GROUP | --gid ID] [--disabled-password] [--disabled-login] [--gecos GECOS] [--add_extra_groups] [--encrypt-home] user DESCRIPTION useradd is a low level utility for adding users. On Debian, administrators should usually use adduser(8) instead. When invoked without the -D option, the useradd command creates a new user account using the values specified on the command line plus the default values from the system. Depending on command line options, the useradd command will update system files and may also create the new user's home directory and copy initial files. By default, a group will also be created for the new user (see -g, -N, -U, and USERGROUPS_ENAB). ----------------------------------------------------------------------------------------------------------------------------


Session #50 - February 19, 2016 - Script Templates

Script template basics: 
  1. #!/bin/bash (shebang: #!) - shell directive ALTERNATE: #!/usr/bin/env bash ANOTHER ALTERNATIVE: #!/bin/bash -e
  2. Author
  3. Date and version info
  4. comments about changes, requirements, etc.
  5. Title or purpose of script
  6. USAGE variable - or comments on how to use the script
  7. Variable declaration.
DISCUSSION on "fix-filename": (without appropriate comments, authorship and dates - just the basics) #!/bin/bash for f in * do mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/_-_/_/g'` done http://stackoverflow.com/questions/14008125/shell-script-common-template https://gist.github.com/KylePDavis/3901321 http://linuxcommand.org/lc3_new_script.php


Session #49 - February 12, 2016 - Syncronizing files between systems

Osync script example: http://johnmeister.com/linux/Scripts/EXAMPLE-OSYNC/
discussion will include a review of this very complex BASH script, and how it can be done
with less code using rsync, scp and ssh commands...  
http://www.netpower.fr/projects/osync/Osync_v1.00a.html#toc-Subsection-1.1

Session #48 - February 5, 2016 - The Art of Linux Systems Administration

Continued Topics for discussion:

- Understanding the Linux philosophy and how it affects commands and tools 

- Systems Administration overview: what is a sys admin and what do they do? 
- Developing the Art of Systems Administrator

- master the vi editor...  
- Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
(any artists?  would like to draw a snail... purchase and install will be the tail then heading around to deal
with the other parts that are an iterative process, not unlike the SDLC.


Session #47 - January 29, 2016 - The Art of Linux Systems Administration

Topics for discussion:
- Understanding the Linux philosophy and how it affects commands and tools 
- Systems Administration overview: what is a sys admin and what do they do? 
- Developing the Art of Systems Administrator
- master the vi editor...  
- Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
(any artists?  would like to draw a snail... purchase and install will be the tail then heading around to deal
with the other parts that are an iterative process, not unlike the SDLC.


Session #46 - January 22, 2016 - FIRST Session 2016 - LPIC - Administering Small to Medium sized Linux Networks

LPIC-2 - administering small to medium-sized mixed Linux networks.

2015 sessions and links below


Session #45 - December 16, 2015 - LAST Session 2015 - LPIC - Administering Small to Medium sized Linux Networks

LPIC-2 - administering small to medium-sized mixed Linux networks.
FREE BOOK:
  • FREE BOOK on Microsoft Windows


    Session #44 - December 9, 2015 (Wednesday) - NESTED FOR loop in a Batch file and BASH script

  • BATCH file to setup a workstation - load registry hacks
  • BATCH FILE explicitly coded for each box - "before" the FOR loop
  • BATCH using FOR LOOPS - significant savings in coding! FREE BOOK:
  • FREE BOOK (includes background on the TEMP issue)
  • could be used to automate cleanup LINUX SCRIPTS USING FOR LOOP:
  • variable definition less complex in BASH
  • using for loops to gather info quickly - less stringent syntax than MS
  • this wouldn't be possible in an MS Batch file


    Session #43 - December 2, 2015 (Wednesday) - VirtualBox Strategy, Plan and Goals

    http://johnmeister.com/linux/Notes/VirtualBox-VM-Plan


    Session #42 - November 25, 2015 (Wednesday) - update on Performance Tests; VirtualBox: SuSE on SuSE

    Main pages for #42 presentation: http://johnmeister.com/linux/Microsoft/LoadTest-MacOSx.sh.txt update on performance testing
    http://johnmeister.com/linux/Microsoft/LOADTEST-OS-COMPARISON.html
    http://johnmeister.com/linux/Notes/VM_setup_SuSE_13.2/ALL.html
     Step-by-step screen shots of install of SuSE 13.2 Linux on VirtualBox Virtual Machine on SuSE Linux
    exercises:  (use VirtualBox with two VMs, or Cygwin, or a Linux system) (continuing Q and A and review)
            the goals:   
        1) work with the command line and SSH between systems – learn basic Linux functions and shell commands
        2) later, build VMs for Windows testing and interoperability 
            - demonstrate the use of Linux as a SECURE and reliable host for Microsoft
            - will build Win7 VM's for testing browsers and java by test team 
                    - showing stability of Linux as VM host/cost saver
            - will build various NON-networked Windows Server versions for testing (w2k8r2, w2k12 and w2k16)
        3) networking between 2 VM's but isolated from local network 
            - Allow active testing of unapproved Operating Systems (win and linux)
        4) remote access from Microsoft to Linux system running a Microsoft system to permit active development and testing
    REFERENCE notes and exercises FOR VirtualBox:
    1. http://johnmeister.com/linux/Notes/VirtualBox-notes.html
    2. http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
    3. http://johnmeister.com/linux/Notes/bashrc-simple.html
    4. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
    5. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
    6. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
    7. http://johnmeister.com/linux/Scripts/chksys.sh.html
    8. http://johnmeister.com/linux/Scripts/while-loop.sh.html
    9. http://johnmeister.com/linux/Notes/VM_setup_CENTOS_7/ALL.html - coming soon
    10. http://johnmeister.com/linux/Notes/Networking-btwn-VBox-VMs.html - coming later

    note: there was no session November 18, 2015

    Session #41 - November 11, 2015 - SHORT SESSION: VirtualBox - discussion continues

    Short session today... starting 11:15, ending before noon... Volunteered for a Veterans' Day activity...
    1. http://johnmeister.com/linux/Notes/VirtualBox-notes.html
    2. http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
    3. http://johnmeister.com/linux/Notes/bashrc-simple.html
    4. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
    5. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
    6. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
    7. http://johnmeister.com/linux/Scripts/chksys.sh.html
    8. http://johnmeister.com/linux/Scripts/while-loop.sh.html


    Session #40 - November 4 2015 - Q and A - VirtualBox - lab exercise

    QandA - VirtualBox - lab exercises - review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
    (continuing QandA and review) working with the command line once you have a VirtualBox Linux shell.
    Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
    http://johnmeister.com/linux/Notes/zypper-update-ps.html
    http://johnmeister.com/linux/Notes/apt-get-update-upgrade.html

    1. http://johnmeister.com/linux/Notes/VirtualBox-notes.html
    2. http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
    3. http://johnmeister.com/linux/Notes/bashrc-simple.html
    4. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
    5. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
    6. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
    7. http://johnmeister.com/linux/Scripts/chksys.sh.html
    8. http://johnmeister.com/linux/Scripts/while-loop.sh.html


    Session #39 - October 28, 2015 - using the VirtualBox - lab exercise

    Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
    1. http://johnmeister.com/linux/Notes/VirtualBox-notes.html
    2. http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
    3. http://johnmeister.com/linux/Notes/bashrc-simple.html
    4. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
    5. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
    6. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
    7. http://johnmeister.com/linux/Scripts/chksys.sh.html
    8. http://johnmeister.com/linux/Scripts/while-loop.sh.html


    Session #38 - October 21, 2015 - SSH One Command Setup, C programming Overview & CENTOS VirtualBox install

    1. ONE LINE SSH setup! http://johnmeister.com/linux/Notes/Setup-SSH-Key-on-ONE-line.html
    2. brief C programming overview: http://johnmeister.com/linux/Notes/C.html
    3. live SETUP CENTOS VM: http://johnmeister.com/linux/Notes/VirtualBox-notes.html


    Session #37 - 14 October 2015 - Troubleshooting Overview

    How do you solve system problems? What tools do you use? Is there a simple way of determining the problem?
    http://johnmeister.com/linux/Notes/Systems-Adminstration-Overview.html

    Session #36 - 7 October 2015 - VirtualBox Overview

    VirtualBox basics - setup and uses http://johnmeister.com/linux/Notes/VirtualBox-notes.html

    Session #35 - 30 September 2015 - Systems Administration Overview

    so you think you want to be a Sys Admin? http://johnmeister.com/linux/Notes/Systems-Adminstration-Overview.html

    Session #34 - 9 September 2015 - System Configuration Files

    What files and directories are important? http://johnmeister.com/linux/Notes/System-Configuration-Details.html

    Session #33 - 2 September 2015 - GRUB and Wrapper Scripts

    1. http://johnmeister.com/linux/Notes/date-suffix.html
    2. http://johnmeister.com/linux/Scripts/wrapper-script.html
    3. http://johnmeister.com/linux/Notes/grub.cfg-example.html
    4. http://johnmeister.com/linux/Notes/grub2-notes.html


    Session #32 - 19 August 2015 - How to Dual Boot Microsoft Windows 10 and SUSE Linux 13.2

    1. http://johnmeister.com/linux/Microsoft/Dual-Boot-Win10-SUSE13.2/


    Session #31 - 12 August 2015 - Security Basics

    The vulnerabilities, exploits, attack vectors, actual risks and payoffs
    1. http://johnmeister.com/linux/Notes/SecurityOverview/


    Session #30 August 5, 2015 - CSV, XML, Spreadsheets and the use of AWK to filter output

    1. https://tools.ietf.org/rfc/rfc4180.txt
    2. http://www.w3schools.com/xml/
    3. https://msdn.microsoft.com/en-us/library/bb387090.aspx
    4. http://johnmeister.com/linux/Commands/awk-example.html


    Session #29 July 8, 2015 - performance analysis tools

    http://johnmeister.com/linux/Notes/Tuning-Linux.html


    Session #28 June 24, 2015 - as requested: REAL WORLD SCRIPTING

    practical scripts: creating, dating, renaming, numbering and managing files
    1. http://johnmeister.com/linux/Scripts/Lab-exercise-create-text-man-pages-for-vi-practice/
      - a basic loop to create man pages for plain text reading
    2. http://johnmeister.com/linux/Scripts/man-page-create-textfiles.sh.txt
      - the script for the exercise above
    3. http://johnmeister.com/linux/Scripts/replace-spaces-plus.txt
      - using tr, sed and mv to change case and/or remove spaces in file names
    4. http://johnmeister.com/linux/Scripts/removing-spaces-and-special-characters.html
      - expansion of prior script, includes dealing with special characters
    5. http://johnmeister.com/linux/Scripts/filedate.sh.html
      - using grep, sort, awk, sed to create a list of files to be "mv"'d (renamed) with a date PREFIX for sorting
    6. http://johnmeister.com/linux/Scripts/number-file-names.sh.html
      - create a number prefix for files to aid in sorting, used for mp3 files.
    7. http://johnmeister.com/linux/Scripts/mail-list.sh.html
      - use for loop to send mail/files to several friends one by one
    8. http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
      - create a web page in a directory of pictures
    9. http://johnmeister.com/linux/Scripts/mk-timelapse-avi-on-2-sites-2015-05-08.txt
      - one of the scripts to create time lapse videos from pictures
    10. http://johnmeister.com/linux/Scripts/chksys.sh.html
      - check on the health of your system, including disk usages (drives fill up fast with images)



    Session #27 June 17, 2015 - server name link changes, drive fail/system rescue

    1. http://johnmeister.com/linux/Notes/Update-amazon-ad-links.html
    2. http://johnmeister.com/linux/Notes/failing-drive-system-rescue.html


    Session #26 June 10, 2015 - Networking basics, analysis/tools and differences

    1. http://johnmeister.com/linux/Notes/basic-networking.html
    2. http://johnmeister.com/linux/Notes/networking-analysis-and-tools.html
    3. http://johnmeister.com/linux/Notes/networking-differences-by-distros.html
      differences in network and basic commands between SuSE, MUD and RedCents) (MUD=Mint, Ubunutu, and Debian)


    Session #25 June 3, 2015 - /var/log/messages - errors, fixing... attacks, stopping... good networks...

    1. http://johnmeister.com/linux/Notes/slapd-unsupported-operation.html
    2. http://johnmeister.com/linux/Notes/using-iptables-to-block-spammers-or-attackers.html
    3. http://johnmeister.com/linux/Notes/Network-Analysis-and-Observations.html


    NOTE: NO CLASS on May 27, 2015

    insomnia prevention classes resumes on 3 June 2015 with Session #25!

    Session #24 May 20, 2015 - NOTE: (no class on the 27th)

    conclusion: Detailed analysis of TimeLapse scripts, page creation and crontab

    1. http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html - what failed...
    2. describe the troubleshooting process used to learn why the cron job failed and
      how a "hunch" solved a "newly created" problem.
    3. http://johnmeister.com/linux/Notes/ManagingFiles-ssh-rsync.html setup of ssh, using rsync... or scp... and
    4. http://johnmeister.com/linux/Notes/wget-n-crontab.html when to use wget...
    5. http://johnmeister.com/linux/Notes/ftp-fixes-for-vsftp.conf.html and when to use ftp.
    6. http://johnmeister.com/linux/Microsoft/Setup-TaskManager-AutoReboot.html
      Comparing a Microsoft captive user interface (GUI) "cron job" to cron, at and nohup.
    7. in other news: headhunter emailed, looking for:
      Intermediate to Expert (command line heavy) knowledge of
      one or more of the following Unix operating systems:
      70%: AIX v.5.3 and/or 7.1 - 25%: SUSE (v.11 Enterprise) - 5%:
      Red Hat
      Enterprise Linux (v.11 (ed.???) Working with Core Technologies (Overall 10%):
      Working familiarity with: Apache Server and/or Tomcat - OpenSSH (Unix) and SSL
      (Windows + Unix; configuration + troubleshooting)
      - S/FTP (Unix-based configuration + troubleshooting) -
      Moderate shell scripting ability (Perl considered a plus)
    8. references: http://johnmeister.com/linux/Commands/MAN/man.chmod.txt,
      http://johnmeister.com/linux/Commands/MAN/man.at.txt, http://johnmeister.com/linux/Commands/MAN/man.nohup.txt,
      http://johnmeister.com/linux/Commands/MAN/man.ftp.txt, http://johnmeister.com/linux/Commands/MAN/man.ssh.txt,
      http://johnmeister.com/linux/Commands/MAN/man.scp.txt, http://johnmeister.com/linux/Commands/MAN/man.ssh.txt,
      http://johnmeister.com/linux/Commands/MAN/man.wget.txt, http://johnmeister.com/linux/Commands/MAN/man.rsync.txt,
      http://johnmeister.com/linux/Commands/MAN/man.bash.txt, etc...

    Session #23 May 13, 2015 - Detailed analysis of TimeLapse scripts, page creation and crontab

    http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html

    Session #22 May 6, 2015 - follow up of LVM, discussion of scripts

    http://johnmeister.com/linux/FileSystems/setup-LVM.html

    Session #21 Apr 29, 2015 - resize2fs and LVM

    http://johnmeister.com/linux/FileSystems/setup-LVM.html

    Session #20 Apr 22, 2015 - LVM configuration

    http://johnmeister.com/linux/FileSystems/setup-LVM.html

    Session #19 Apr 15, 2015 - ftp, vsftp, scp, ssh


    Session #18 - Apr 8, 2015 - Systemd

    http://johnmeister.com/linux/Notes/systemd-info.html systemd and
    the general boot up process of Linux systems. (comparing /etc/init.d/)
    for additional reading: https://www.novell.com/docrep/2015/01/systemd_in_suse_linux_enterprise_12_white_paper.pdf

    Session #17 - Apr 1, 2015 - proxy/vnc

    http://johnmeister.com/linux/Notes/vnc-setup.html
    http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
    http://johnmeister.com/linux/Scripts/run-now-on-all-hosts.sh.html

    Session #16 - Mar 25, 2015 - ssh, scp, and rsync

    use of ssh to examine, monitor and scp to copy files, create a script and use rsync
    to manage several systems from one.
    Walk through the process and establish a wrapper script to make archive copies on each system.
    http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
    http://johnmeister.com/linux/Scripts/run-now-on-all-hosts.sh.html

    Session #15 - Mar 18, 2015 - systemd, tree, ssh and scp

    discussion on systemd, tree command, use ssh to examine and scp to copy
    files, create a script and use rsync to manage several systems from one.

    Session #14 - Mar 11, 2015 - general discussion on Linux shells, commands - open discussion with group


    Session #13 - Mar 4, 2015 - 1) general Linux info, 2) basics, and 3) advanced topics


    Session #12 - Feb 25, 2015 - The power of the command line: grep

    Discussion of GREP: example: grep -v ^[[:space:]]$ and grep -v [[:space:]]

    Session #11 - Feb 18, 2015 -The basics of Apache2 configuration, sed examples and use.

    1) http://johnmeister.com/linux/Notes/?C=M;O=D
    (general updates can be seen near the top of this page)
    2) http://LinuxMeister/Notes/basic-Apache-setup.
    (basic install and config, and major config files)
    3) http://LinuxMeister/Notes/apache-tricks.
    (virtual server configurations and settings)
    4) http://johnmeister.com/linux/Notes/apache-tomcat-topic-list.
    (the list of modules used for development purposes)
    5) http://johnmeister.com/linux/Notes/setup-LAMP-on-MINT.
    (adding apache to Ubuntu, Debian or Mint - not there by default)

    Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic

    Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic-
    wget, crontab, ls , vi, mencoder, sed, grep, cat, tac, lsof, netstat.

    1) http://johnmeister.com/linux/Notes/?C=M;O=D
    2) http://johnmeister.com/tech/Traffic/TUES-PM/00_Tues-PM.html
    3) http://johnmeister.com/linux/Notes/wget-n-crontab.html
    4) http://johnmeister.com/tech/Traffic/
    5) http://johnmeister.com/tech/Traffic/files.txt
    6) http://johnmeister.com/tech/Traffic/Make-Commute-pages.txt
    7) http://johnmeister.com/linux/Notes/setup-in-irfanview.jpg
    8) http://johnmeister.com/linux/Notes/mencoder-info.html
    9) http://johnmeister.com/linux/Notes/zypper-install-mencoder.html
    10) http://johnmeister.com/linux/Notes/cat-tac-and-sed.html
    11) http://johnmeister.com/linux/Notes/Network-Analysis-and-Observations.html

    Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.

    Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.

    1) http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
    2) http://johnmeister.com/linux/Notes/RSYNC-cmd.html
    3) http://LinuxMeister/Notes/ManagingFiles-ssh-rsync.html
    Will discuss details on how to configure directory structure on Windows and Linux systems
    to synchronize a set of Files on several computers.
    Will discuss setup of bi-directional ssh passwordless connection between two Linux systems,
    and RSYNC configuration.
    Will discuss wrapper scripts to make backup copies for files that will be rotated and replaced,
    preventing redundant or corrupt files.
    http://johnmeister.com/linux/Intro-to-Linux/Notes/FileMgtPlan.txt

    If we have time we'll also discuss WinSCP and how it can be used from Microsoft systems. Emphasis will be on the Linux aspect of

    ssh and rsync, but also on general file management and config mgt for end users.

    If time, will discuss SSH tunneling through firewalls/VPNs/NATs: http://johnmeister.com/linux/Notes/ssh-tunnel.jpg


    NOTE: updated Gparted page from last week:
    http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.html


    Session #8 - 28 January 2015 - discuss the use of cal, grep, tee and the use of a simple loop in a shell

    Work with cal, grep, and a do-while loop in a BASH script to determine if it there is truth to the internet claim that:
    December 2012 having 5 Saturdays, 5 Sundays and 5 Mondays only happened once every 823 years!
    Commands covered: cal, cat, man, more, ls, grep, tee and basic do while loops and tests in BASH
    http://LinuxMeister/Intro-to-Linux/28jan2015/grep.help.txt
    http://johnmeister.com/linux/Scripts/BUST-MYTHS-WITH-CAL-GREP/ http://johnmeister.com/linux/Commands/grep-awk-uniq.
    http://johnmeister.com/linux/Overview/LinuxOverview.pdf
    Brief discussion on recovery of /home on a failing or corrupted system:
    http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.
    http://LinuxMeister/Intro-to-Linux/28jan2015/how-to-preserve-home.txt
    tar cvfz home.tar /home(note: will need to be root or use sudo, make sure you have enough space to create the file)

    Session #7 - 21 January 2015

      
    http://opensuse-guide.org/installation.php 
    https://software.opensuse.org/132/en 
    https://en.opensuse.org/Portal:Installation
    https://activedoc.opensuse.org/book/opensuse-start-up
    https://activedoc.opensuse.org/book/opensuse-start-up/chapter-1-installation-quick-start
    http://johnmeister.com/linux/http://johnmeister.com/linux/Overview/LinuxOverview.pdf
    https://www.smashwords.com/profile/view/johnmeister  
    https://www.smashwords.com/books/view/505731

    Session #6 - 14 January 2015

    http://LinuxMeister/Intro-to-Linux/ misc notes and screenshots found here… not organized.
    Files created during session found here.
    In particular http://LinuxMeister/Intro-to-Linux/14jan2015/reg-exp-users.txt
    demonstrated the use of script and the creation of the
    http://LinuxMeister/Intro-to-Linux/14jan2015/reg-exp-users.RAW file,
    which was then filtered using cat {filename}.RAW | col -b > {filename}.txt
    Regular Expressions and the power of the command line.Continued demonstrating filtering.
    http://LinuxMeister/Intro-to-Linux/
    Demonstration of:
    grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo, man, --help, ifconfig, dmesg and which

    Session #5 - 7 January 2015 http://LinuxMeister/Intro-to-Linux/

    Demonstration of:
    grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo,
    man, --help, ifconfig, dmesg and which

    SESSION #4 - December 17, 2014: pkg mgt, rsync, wine, mount

    http://johnmeister.com/linux/PDF-info/PackageManagementCheatSheet.pdf
    http://johnmeister.com/linux/FileSystems/newdisk-setup.txt
    http://johnmeister.com/linux/Notes/USB-live-Gparted-setup.txt
    http://johnmeister.com/linux/Notes/Setup-SuSE-13.2-select-screens/ALL.
    http://johnmeister.com/linux/Notes/zypper-installs.
    http://johnmeister.com/linux/FileSystems/setup-LVM.txt
    http://johnmeister.com/linux/Commands/rsync-example-man. RSYNC basics
    http://johnmeister.com/linux/FileSystems/mount-external-drive-rysnc.txt
    http://johnmeister.com/linux/FileSystems/rsync-disks-cron.sh.txt
    http://johnmeister.com/linux/Notes/Firefox-about-support-Tools.
    Firefox about features showing add-ons, in the URL type: about:support
    http://johnmeister.com/linux/Notes/wine-examples-manpage. using wine - some examples and the man page

    SESSION #3 - December 10, 2014 - basic commands

    discuss ssh files, show how to use Gparted,
    exercise on real world Linux commands, discuss VNC setup
    continue discussion on regularly used commands,
    e.g. grep, sed, awk, sort, uniq, file permissions and chmod, chown
    exercise #2 - real world examples of commands and file permissions.
    ---------------------------------------------------------------------------- Links:
    1. http://johnmeister.com/linux/Overview/
    2. http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf
    3. http://johnmeister.com/linux/Notes/bashrc-the-dotfile.html
    4. http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
    5. http://johnmeister.com/linux/Notes/ssh-authorized-keys-setup-example.html
    6. http://johnmeister.com/linux/Notes/Gparted-for-Setup-or-Rescue.html
    7. http://johnmeister.com/linux/Notes/vnc-setup.html
    8. http://johnmeister.com/linux/Notes/Real-world-Linux-Commands.html
    9. http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html


    SESSION #2 - December 3, 2014 - SAW:"simple always works"

    provided link to exercise #1, discussed environment (.bashrc), path, chmod, discussed "SAW" -
    SAW:"simple always works",
    History files viewed with sort, uniq, and how to create notes for reference: ls, sort, grep, uniq, wc -l
    (after using a command, use history recall, add echo and quotes around it,
    and then append to ~/bin/cool-commands.txt e.g.

    echo "ls *.jpg > ALL.html ; perl -pi -e 's/(.*)/<img src="$1"><BR>$1<HR>/g' ALL.html ; \
    perl -pi -e 's/<img src=""><BR><HR>//g' ALL.html ; cat ALL.html" >> ~/bin/cool-commands.txt

    ----------------------------------------------------------------------------
    Links: http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
    (do this on your system, use "script exercise-1.raw",
    when finished type exit, then cat exercise-1.raw | col -b > exercise-1.txt - then edit in vi and save)
    http://johnmeister.com/linux/Scripts/chksys.sh.html (use the vi editor to create and run on your system)
    http://johnmeister.com/linux/Scripts/man-page-create-textfiles.sh.txt
    (mkdir and cd LAB, then create this script in vi and run)
    http://johnmeister.com/linux/Commands/grep-awk-uniq.html
    (the output of the script above is needed to use the commands, YMMV, counts may be different)


    SESSION #1 - November 26, 2014 - OVERVIEW

    	 provided overview of Linux using the pdf: http://johnmeister.com/linux/Overview/LinuxOverview.pdf  
    	5 basic commands: man, ls (ls -al, ls -Al), cd, pwd, more 
    	discussed .bashrc and showed a few script examples, talked about "script"
    ----------------------------------------------------------------------------
    Links:	http://johnmeister.com/linux/Overview/LinuxOverview.pdf    (print out and use as a guide)
    	http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and keep as a guide)
    	http://johnmeister.com/linux/Notes/bashrc-the-dotfile.html  (copy and build your own .basrhc for Cygwin or Linux)
    	http://johnmeister.com/linux/Scripts/chksys.sh.html  
    	http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html  (use the commands and the vi editor)
    

    THE CORE MATERIAL:The power of the command line - Simply Linux: Basics will work our way through the Linux book (Simply Linux: Basics) under construction based on http://johnmeister.com/linux/Overview/ http://johnmeister.com/linux/Overview/LinuxOverview.pdf (print out and use as a guide) http://johnmeister.com/linux/Overview/Linux-PowerPoint-2004-overview.pdf http://johnmeister.com/linux/Notes/Real-world-Linux-Commands. http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide) http://johnmeister.com/linux/Notes/bashrc-the-dotfile. (copy and build your own .basrhc for Cygwin or Linux)

    ONE HOUR Linux SESSIONS



  • Simply Linux: Basics  Full Size Jeep Buyer's Guide Using BASH on Windows 10
    Practical Suggestions for Microsoft Windows
    Linux Tackles Microsoft
    12 hour Video Course by john:
    The Art of Linux System Administration
    published by O'Reilly Media
    Study Guide for the LPIC-2 Certification Exams
    search for:
    on the internet, or:
    JohnMeister.com-fotos
    LinuxMeister-Linux
    BibleTech- Bible overview

    overview of mankind's history
    Biblical history:
    "Promises and Prophets"

    Wagoneers

    FULL SIZE JEEPS

    JeepMeister
    "Jeep is America's
    only real sports car."
    -Enzo Ferrari


    MeisterTech
    Diesels +

    One Page Overview of Linux Commands

    click for an image of the 5 essential Linux commands

    An Intro to Linux
    AMSOIL product guide,
    or, AMSOIL web, or 1-800-956-5695,
    use customer #283461

    Amsoil dealer since 1983
    purchase AMSOIL
    at Midway Auto on SR9 in Snohomish,
    or at Northland Diesel in Bellingham, WA


    SJ - 1962-1991

    XJ - 1984-2001

    WJ - 1999-2004

    KJ - 2002-2007

    WK - 2005-2010

    Find the recommended
    AMSOIL synthetics
    for your Jeep

    CJ-10A - 1984-1986

    Jeepsters

    MJ - 1984-1992

    Willys - 1946-1965

    Other Jeeps (FC)