ONE HOUR Linux SESSIONS: 2014-2017
sessions in chronological order going back to 2014
Simply Linux: Basics Linux Tackles Microsoft Using BASH on Windows 10
Practical Suggestions for Microsoft Windows
 Full Size Jeep Buyer's Guide
the art of Linux sys admin
the Art of Linux SysAdmin
john's publications (click on cover for further info)

2017


Session #109 - October 20, 2017 - FRIDAY - wget, advanced vi editing
note on html: color for cell above is "0F0F0F" (rrggbb), font color is "white"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


  1. we've covered .bashrc: Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
  2. we've reviewed the "five essential" commands: man, ls, cd, pwd and more
  3. we've worked our way through the SPECIAL CHARACTERS - and will continue to...
  4. will discuss some issues of interoperability with different BASH environments, e.g. MacOSx, different versions of BASH
  5. we'll work on building some web pages - include the use of sed, awk, tr and even some perl


  6. Session #109 - October 13, 2017 - FRIDAY - Adventures with Special Characters
    note on html: color for cell above is "F0F0F0" (rrggbb), font color is "black"
    (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


    1. we've been covering the elements of the .bashrc: Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
    2. we've been reviewing the "five essential" commands: man, ls, cd, pwd and more
    3. we've been working our way through the SPECIAL CHARACTERS - and will continue to this session...
    4. we'll discuss the scripts below - including the use of sed, awk, tr and even some perl
    5. used find, exiftool and other reg exp tools to gather lens info from NEF files, if we have time, otherwise a future session.
      Special characters are powerful representations of shell functions and values.





      Remove spaces and replace special characters: http://linuxmeister.net/Scripts/fix-filenames.sh.html
       
      #!/bin/bash
      ###############################################################################################
      #             removes spaces, special characters, slashes, and _-_  but NOT caps
      ###############################################################################################
      for f in *
          do
          mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
          done
      
      examples of loops: http://linuxmeister.net/Scripts/LOOPS-shell-examples.html

      Use AWK in a loop to move large numbers of files on a Linux web server: http://linuxmeister.net/Scripts/awk-to-rename-files.sh.html
      1. create temporary script, e.g. "vi fsj-rename" (if you have more than one directory create a more structured script and place it in your ~/bin).
      2.  
        #!/bin/bash
        # GOAL:  rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
        # DATE: 4/27/2004 - Author John Meister 
        #  TEST:  --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' 
        #  PRODUCED:   mv fsj-digest-v1-2123.txt fsj-2123.txt
        # 
        # PROCESS:  --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
        # PROCESS:    edit file to include header info & comments
        #
        mv fsj-digest-v1-1800.txt fsj-1800.txt
        mv fsj-digest-v1-1801.txt fsj-1801.txt
        ...
        -------------------------
        
      3. THEN... execute the script by typing: sh ./fsj-rename
      4. check directory and delete temporary script: ls ; rm -f fsj-rename

      Session #108 - October 6, 2017 - FRIDAY - Adventures in BASH
      note on html: color for cell above is "00FFFF" (rrggbb), font color is "black"
      (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


      Last week we discussed in detail the .bashrc file, and demonstrated some of the "five essential" commands: man, ls, cd, pwd and more

      This week: we will use regular expressions to replace special characters, deal with bad file-naming conventions, and rename long file names with AWK.

      FIRST - what are special characters?
      Special characters are powerful representations of shell functions and values.
      In the picture below there is a listing of the special characters and examples of their use. Special characters must be understood.




      PROBLEM: Someone created files and directories with spaces in the names, or worse, special characters.
      In UNIX-like systems spaces at the command line represent various commands or options. If a filename or directory has spaces,
      then those spaces must be "escaped" with a "\" or encapsulating the entire filename in quotes. This gets complicated in a hurry.

      With this script, we will remove spaces and replace special characters: http://linuxmeister.net/Scripts/fix-filenames.sh.html
       
      #!/bin/bash
      ###############################################################################################
      # 2014_05May_09 - john meister copyright © 2014 http://LinuxMeister.net
      # script to remove spaces in file names and replace with underscore
      # cleans up web pages by eliminating special characters and allows sorts to work properly
      #  to use place this script in your path and make it executable, then cd to the directory
      #    and execute by the saved name (fix-filename)  - works on Linux and MacOS
      ###############################################################################################
      #             removes spaces, special characters, slashes, and _-_  but NOT caps
      ###############################################################################################
      for f in *
          do
          mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
          done
      
      
      NOTE: we'll cover more loops as we go along, here's some examples of loops: http://linuxmeister.net/Scripts/LOOPS-shell-examples.html

      ISSUE: created a bunch of files with really long stupid names, however, the good news is I had a "delimiter" that would work with AWK.
      By using the "-" in the really long stupid file names I was able to use AWK to simplify the filenames. This was a real world problem that was fixed.

      Example of using AWK in a loop to move large numbers of files on a Linux web server: http://linuxmeister.net/Scripts/awk-to-rename-files.sh.html
      1. create temporary script, e.g. "vi fsj-rename" (if you have more than one directory create a more structured script and place it in your ~/bin).
      2.  
        #!/bin/bash
        # GOAL:  rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
        # DATE: 4/27/2004 - Author John Meister 
        #  TEST:  --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' 
        #  PRODUCED:   mv fsj-digest-v1-2123.txt fsj-2123.txt
        # 
        # --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
        #   edit file to include header info & comments
        #
        mv fsj-digest-v1-1800.txt fsj-1800.txt
        mv fsj-digest-v1-1801.txt fsj-1801.txt
        mv fsj-digest-v1-1802.txt fsj-1802.txt
        mv fsj-digest-v1-1803.txt fsj-1803.txt
        mv fsj-digest-v1-1804.txt fsj-1804.txt
        ...
        -------------------------
        
      3. THEN... execute the script by typing: sh ./fsj-rename
      4. check directory and delete temporary script: ls ; rm -f fsj-rename

      If we have time we will use AWK to rename the history files discussed last week.
      I noticed that I did not use the date command in a way that makes sorting work well.
      Current format is: 2017_Mar_27_08:32.history It would be better for sorting if it were 2017_03_Mar_27_08:32.history

      The good news is that I used "_" in the name; thereore, I can modify the script above to change the file names.

      Previously the date variable was set to: HISTFILE="/home/luser/.History/`/bin/date '+%Y_%b_%d_%H:%M'.history`" ; export HISTFILE
      Last week we modified .bashrc to fix this variable: HISTFILE="/home/luser/.History/`/bin/date '+%Y_%m_%b_%d_%H:%M'.history`" ; export HISTFILE
      THE DATE command option for alphabetical representation of the month is %b, while %m is the numeric value.

      I like to see the Month in letters, but like to sort things properly. By sorting YEAR, MONTH and then DAY it displays in chronological order.
      If I wanted to keep the filename shorter I could replace the hour and minutes with just the PID, or use the seconds, the actual time of the file is not important.

      Session #107 - September 29, 2017 - FRIDAY - Let's USE BASH on Windows 10! - Part 3
      note on html: color for cell above is "0000FF" (rrggbb), font color is "white"
      (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


      Will transition FROM: "Let's use BASH on Windows 10!" - TO: ---> "Simply Linux: Basics"
      We will focus more on the actual Linux commands, less on the Win10 environment as host.  
      We'll talk more about Regular Expressions and the vi editor.  The commands, the core tools.
      
      Not everyone will be able to use Ubuntu BASH in W10, some will chose cygwin or other tools such as a VM or putty to a Linux system.
      
      We will still discuss issues across platforms as we proceed. It's important to note differences even between UNIX, Linux and other tools.
      More will be shared during the session.
      
      
      finish discussion of .bashrc and .History directory. http://linuxmeister.net/bashrc-basic.html and then - Discuss Chapter Two - Five Essential Commands.
      Continuing in Chapter 2 of "Let's Use BASH on Windows 10", but shifting to the other book, same format... Introduction to Linux Chapter 1_Linux_Overview Chapter 2_the_5_essential_commands Chapter 3_the_vi_Editor Chapter 4_shell_config_History_script Chapter 5_File_Management:_cp_mv_rm Chapter 6_Files, Processes and Special Characters Chapter 7_Regular_Expressions_and_Filters Chapter 8_ssh_setup_and_use Chapter 9_Network_Basics Chapter 10_Files_Systems:_df_du ... Showing purpose with examples of settings in .bashrc: PATH; PROMPT; PERMISSIONS; HISTORY; EDITOR; ALIAS; SHELL OPTIONS; FUNCTIONS demonstrating "minor" issues and differences in paths, symlinks and command execution in BASH on Win10. Will show how to create a copy of files in /mnt/c/

      HOMEWORK EXERCISES:
      1. RECOMMEND REVIEWING: http://linuxmeister.net/Notes/Philosophy_of_Linux.html - there are significant differences between Microsoft and Linux that need to be understood.
      2. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html

      Session #106 - September 22, 2017 - FRIDAY - Let's USE BASH on Windows 10! - Part 2
      note on html: color for cell above is "00ff00" (rrggbb), font color is "black"
      (red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue


      "Let's use BASH on Windows 10!" - Part 2 - Ch1, Ch2


      - Finish up Chapter One - discuss .bashrc and .History directory.  http://linuxmeister.net/bashrc-basic.html
      - Discuss Chapter Two -  Five Essential Commands.
      

      HOMEWORK EXERCISES:
      1. RECOMMEND REVIEWING: http://linuxmeister.net/Notes/Philosophy_of_Linux.html - there are significant differences between Microsoft and Linux that need to be understood.
      2. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html

      TABLE OF CONTENTS for "Let's USE BASH on Windows 10 Introduction to BASH on Windows 10 How to INSTALL BASH on WIN10 How to Configure the BASH window Chapter 1_Linux_Overview Chapter 2_the_5_essential_commands Chapter 3_the_vi_Editor Chapter 4_shell_config_History_script Chapter 5_File_Management:_cp_mv_rm Chapter 6_Files, Processes and Special Characters Chapter 7_Regular_Expressions_and_Filters Chapter 8_ssh_setup_and_use - will not cover for Win10 Chapter 9_Network_Basics - will not cover for Win10 Appendix: How to INSTALL_BASH_WIN10 Appendix: The LINUX Philosophy Appendix: Exercise #1 - the 5 basic commands Appendix: Exercise #2 - chmod, chown, file permissions Appendix: Additional info on BASH Appendix: Additional info from Chapter 2 Appendix: SCRIPT: fix filenames Appendix: SCRIPT: use wget, date, and a while loop Appendix: REAL WORLD LINUX COMMANDS See session #105 for links to table of contents on smashwords.

      OTHER RESOURCES:

      http://linuxmeister.net/Overview/LinuxOverview.pdf (print out and use as a guide)
      http://linuxmeister.net/Overview/Linux-PowerPoint-2004-overview.pdf
      http://linuxmeister.net/Notes/Real-world-Linux-Commands.
      http://linuxmeister.net/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide)
      http://linuxmeister.net/bashrc-basic.html
      Session #105 - September 15, 2017 - FRIDAY - Let's USE BASH on Windows 10!
      note on html: color for cell above is "1242ab" (rrggbb), font color is "white"

      working through the book on using the BASH tool in Windows 10.

      Follow along in Linux or MacOSX... it's mostly the same. (you can preview the book up to about page 19.)
      TABLE OF CONTENTS for "Let's USE BASH on Windows 10
       
       Introduction to BASH on Windows 10 
       How to INSTALL BASH on WIN10
       How to Configure the BASH window
       Chapter 1_Linux_Overview
       Chapter 2_the_5_essential_commands
       Chapter 3_the_vi_Editor
       Chapter 4_shell_config_History_script
       Chapter 5_File_Management:_cp_mv_rm
       Chapter 6_Files, Processes and Special Characters
       Chapter 7_Regular_Expressions_and_Filters
       Chapter 8_ssh_setup_and_use - will not cover for Win10
       Chapter 9_Network_Basics - will not cover for Win10
       Appendix: How to INSTALL_BASH_WIN10 - will not address in these sessions, 
      but this link might help, YMMV:   http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html
       Appendix: The LINUX Philosophy
       Appendix: Exercise #1 - the 5 basic commands
       Appendix: Exercise #2 - chmod, chown, file permissions
       Appendix: Additional info on BASH
       Appendix: Additional info from Chapter 2
       Appendix: SCRIPT: fix filenames
       Appendix: SCRIPT: use wget, date, and a while loop
       Appendix: REAL WORLD LINUX COMMANDS
        NOTE: The book "Simply Linux" follows the same format, but that version has more Linux and MacOSX detail, 
      the Win10 version was scaled back slightly: https://www.smashwords.com/books/view/705084
      

      Session #104 - August 4, 2017 - FRIDAY - sshd tweaks, journalctl, Apache 2.4 details
      note on html: color for cell above is "12ffab" (rrggbb), font color is "black"
       
      tightening up sshd instead of using fail2ban
      
      tweaks to /etc/ssh/sshd_config  - typical ssh files:  http://linuxmeister.net/Notes/SSH/sample-etc-ssh-config-files.html
      
          make sure root couldn't log in directly.:   PermitRootLogin no
          specify which users can ssh in:             AllowUsers luser
          force only one version of ssh:              Protocol 2 
          give the person ONE try:                    MaxAuthTries 1
          reduce the number of starting connections:  MaxStartups 2:50:5   
          reduce the login grace time from 2 min:     LoginGraceTime 30
          restart sshd                                # service sshd restart 
                      (remember:    man sshd | col -b > man.sshd.txt - RTM... YMMV!) 
      
      system logs ssh: journalctl -f | grep ssh, or journalctl -f | grep -i Error | grep ssh --> journalctl -f | grep ssh Jun 21 23:18:09 the-linux-system sshd[23682]: error: PAM: User not known to the underlying authentication module for illegal user admin from 103.79.142.31 Jun 21 23:18:09 the-linux-system sshd[23682]: error: maximum authentication attempts exceeded for invalid user admin from 103.79.142.31 port 52295 ssh2 [preauth] Jun 21 23:24:06 the-linux-system sshd[23694]: error: maximum authentication attempts exceeded for invalid user root from 190.49.255.143 port 42263 ssh2 [preauth] Jun 21 23:27:53 the-linux-system sshd[23703]: error: maximum authentication attempts exceeded for invalid user oracle from 201.20.73.3 port 56130 ssh2 [preauth]
      if an IP is very persistent, use iptables to block e.g.: iptables -A INPUT -s 43.255.190.xx -j DROP
      apache logs: --> ll /var/log/apache2/*error_log if an error log has content, view, and see if you can find the error. After fixing, make a copy of the log file then empty it as follows: cp the-error_log /var/log/apache2/FIXED/the-error_log-clear-21jun2017 ; cat /dev/null > the-error_log
      Session #103 - July 28, 2017 - FRIDAY - .bashrc details
      note on html: color for cell above is "12ab34" (rrggbb), font color is "black"
       
      
      HAPPY SYS ADMIN DAY - July 28, 2017!!!
      Prepping for LIVE Oreilly Media Windows 10 BASH sessions - test run...
      1. basic .bashrc: http://linuxmeister.net/Microsoft/Win10-BASH/W10-bashrc.txt
      2. bashrc delta from default: http://linuxmeister.net/Microsoft/Win10-BASH/W10-bashrc-DELTA.txt
      3. summary of shopt (shell options): http://linuxmeister.net/Microsoft/Win10-BASH/W10-shopt-list-details.html
      HAPPY SYS ADMIN DAY - July 28, 2017!!! Sys Admin song (sung to monty python's lumberjack tune): [spoken] I never really wanted to be a scientist. I wanted to be...a...a SYSADMIN! [system engineer choir and shift supervisor enter, music strikes up] Oh, I'm a sysadmin and I'm OK, I grep all night and I chown all day. [choir] He's a sysadmin and he's OK, He greps all night and he chowns all day. I ping the nodes, I do PM, I awk and perl and sed. I've got a Star Wars lunchbox, And Tron sheets on my bed! [choir] He pings the nodes, he does PM, He awks and perls and seds. He's got a Star Wars lunchbox, And Tron sheets on his bed! I ping the nodes, I change the rates, I fork the processes. I wish that all my lusers would catch some rare disease! [choir, growing slightly uncomfortable] He pings the nodes, he changes rates, He forks the processes. He wishes all his lusers would catch some rare disease! [choir brightens as they repeat chorus] I ping the nodes, I lock the /home partition and umounts.... [choir, very uncomfortable and trailing off] ...HAPPY SYS ADMIN DAY...

      Session #102 - June 16, 2017 - FRIDAY - journalctl, Apache 2.4, and guest
      note on html: color for cell above is "ffffff" (rrggbb), font color is "brown"
       
      system logs: dmesg and journalctl
      dmesg # provides kernel information # part of util-linux package, see Linux Kernel Archive; man dmesg # sudo dmesg --clear
      sudo journalctl  # sudo journalctl -f  # sudo journalctl | grep -i error # journalctl | grep -i Error | grep ssh | grep -v "Jun 0"
      
      Migration from Apache 2.2 to 2.4 changes to apache2 involved replacing two lines regarding permissions, index directives continued to work: # #Order allow,deny# Allow from all --> Require all granted # # Order deny,allow# Deny from all --> Require all denied
      --> apachectl configtest # Syntax OK --> systemctl enable apache2 --> systemctl start apache2
      --> systemctl status apache2 . apache2.service - The Apache Webserver Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2017-06-07 00:34:44 PDT; 5s ago Main PID: 2422 (httpd-prefork) Status: "Processing requests..." Tasks: 6 CGroup: /system.slice/apache2.service - 2422 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2429 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2430 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2431 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2432 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... - 2433 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD... Jun 07 00:34:44 linuxsystem systemd[1]: Starting The Apache Webserver... Jun 07 00:34:44 linuxsystem systemd[1]: Started The Apache Webserver.
      guest presentation: "FireJail" configuration: http://linuxmeister.net/SysAdmin/Contrib/FireJail-presentation/

      Session #101 - June 9, 2017 - FRIDAY - grepping along
      note on html: color for cell above is "aaaaaa" (rrggbb), font color is "lime"

      grep basics and using catgrep to analyze config files

      http://linuxmeister.net/Scripts/catgrep.sh.html
          #  this is the key string:  grep -E -v '(^\s*#|^$)'
      
       clean copy to copy... copy to ~/bin/catgrep, chmod and update path in .bashrc
      

      #!/bin/bash ################################################################################ # 24 jun 2016 - john meister copyright 2016 http://linuxmeister.net # cat's file, then greps out blank lines and comments, offers option to save # 22 jul 2016 - added sed string to strip leading slash off echo /etc | sed 's/^\///' # 6 jun 2017 - finally got the white space string working: grep -E -v '(^\s*#|^$)' # NOTE: create ~/bin directory, chmod to 755, add ~/bin to PATH - test: # ex: --> which catgrep # /home/luser/bin/catgrep ################################################################################ USAGE="catgrep filename # removes commented and blank lines, option to save" echo ' --------------------------------------------------------------------' echo " DISPLAYING file: --> $1 <-- without commented or blank lines" echo ' --------------------------------------------------------------------' cat $1 | grep -E -v '(^\s*#|^$)' # the \s is for white spaces VFL="`echo "$1" | sed 's/^\///' | tr '/' '-'`" ; export VFL echo '-----------------------------------------------------------------' echo "save info in local directory? if yes, type: y" echo '-----------------------------------------------------------------' read answer ; if [ $answer = "y" ] # note, adding brackets for upper/lower case would be [Y,y] then echo "$1" | tee ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt cat $1 | grep -E -v '(^\s*#|^$)' | tee -a ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt #### NOTE: use redirect or append on line above instead of tee -a to reduce screen clutter echo '-----------------------------------------------------------------' echo " info written to: ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt " echo '-----------------------------------------------------------------' else echo ' ### ------------------------------------' echo ' ### not written.' echo ' ### ------------------------------------' fi

      additional details: http://linuxmeister.net/Notes/REGULAR-EXPRESSIONS-simple.html


      Session #100 - May 26, 2017 - FRIDAY - Networking
      note on html: color for cell above is "100000" (rrggbb)

      100 lunchtime sessions since 2014!

      Networking basics and testing performance

      BASIC SETUP:  ifconfig lan0 192.168.11.42 netmask 255.255.255.0 broadcast 192.168.11.255 
      ADD ROUTE:    route add default gw 192.168.11.1 
      COMMANDS:     ifconfig -a, ip addr, ping -c 3 8.8.8.8, route, netstat -ar, traceroute 8.8.8.8, nslookup linuxmeister.net
      (note: Microsoft commands offer fewer options and change a few names: e.g. ifconfig --> ipconfig, traceroute --> tracert.)
      FILES:    /etc/hosts, /etc/resolv.conf, /etc/nsswitch.conf, /etc/hostname, /etc/sysconfig/network (YMMV)
      
      http://johnmeister.com/linux/Notes/NETWORK/basic-networking.html

      http://linuxmeister.net/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
      examining the spreadsheet formulas (CSV)

      http://linuxmeister.net/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-Generic-BASH.sh.html

      Session #99 - May 19, 2017 - FRIDAY - Linux Commands in use.
      note on html: color for cell above is "33bb44" (rrggbb)

      using basic commands and the vi editor.

       After helping a few friends with Linux recently I see the need for less speed and more detail... so... 
      Let's review basics of .bashrc, .History, setting up .ssh and converting a Windows Batch file to Linux to check network performance.
      http://linuxmeister.net/Notes/bashrc-simple.html
      http://linuxmeister.net/Notes/SSH/quick-ssh.html
       http://linuxmeister.net/Microsoft/LOADTEST-OS-COMPARISON.html 
      directory with excel, batch file and large file 
      http://linuxmeister.net/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
      oh, and one slightly advanced moment:
      http://linuxmeister.net/Notes/setup-LAMP-on-MINT.html
      
      Session #98 - May 11, 2017 - THURSDAY - ssh, rsync, and ftp - along with WinSCP
      note on html: color for cell above is "99aaff" (rrggbb)

      using ssh, rsync, ftp, NFS

      1. basic: http://linuxmeister.net/FileSystems/man.rsync.txt
      2. config info: http://linuxmeister.net/Scripts/fix-ftp.sh.html
      3. config info: http://linuxmeister.net/Notes/ftp-fixes-for-vsftp.conf.html
      4. basics: http://linuxmeister.net/Notes/RSYNC-cmd.html
      5. examples: http://linuxmeister.net/Scripts/rsync-directories.sh.html
      6. cron job examples: http://linuxmeister.net/FileSystems/rsync-disks-cron.sh.html
      7. cron job examples: http://linuxmeister.net/SysAdmin/Cron-Rsync-sync-up-drives.sh.html
      8. using rsync to update: http://linuxmeister.net/Scripts/setup-diverse-systems-prototype.sh.html
      9. mention (again): http://linuxmeister.net/Scripts/EXAMPLE-OSYNC/
      10. NFS option: http://linuxmeister.net/FileSystems/NFS-Server-setup.html
      11. - if time: (demonstrate WinSCP on XP in a VM on Linux or a Mac)

      Session #97 - April 28, 2017 - FRIDAY - Linux Commands in use.
      note on html: color for cell above is "ff22aa" (rrggbb)

      using tools to extract key information on a system

      dmidecode, /proc, "stat" commands, and grep...

      Using search tool and text editing

      Continued discussion on using search tool:
      http://linuxmeister.net/Scripts/SEARCH-TEXT and
      examples and files to download to work with

      Apache 2.4 migration

      Migration from Apache 2.2 to 2.4 on SuSE. - http://linuxmeister.net/SysAdmin/Apache2-setup.html (haven't had a lot of time to sort through all the directives)
      examine basic HTML structure and discuss how to migrate to 2.4 view-source:http://bibletech.net/

      Session #96 - April 21, 2017 - FRIDAY - Linux Commands in use.
      note on html: color for cell above is "11aa33" (rrggbb)

      using search tool, catgrep, and basics on Apache web server and directives

      1. search text tool http://linuxmeister.net/Scripts/SEARCH-TEXT
      2. script and all files needed to use found at: http://bibletech.net/SEARCH-KJV/
      3. use catgrep to identify existing config: http://linuxmeister.net/Scripts/catgrep.sh.html - "catgrep /etc/apache2/default-server.conf"
      4. understand the basics: http://linuxmeister.net/SysAdmin/Apache2-setup.html 1) install apache2 - zypper in apache2 2) establish base configuration in /etc/apache2/default-server.conf 3) verify document root and log locations 4) systemctl enable apache2 (*chkconfig) * --> chkconfig -l | grep apache Note: This output shows SysV services only and does not include native systemd services. SysV configuration data might be overridden by native systemd configuration. apache2 0:off 1:off 2:off 3:off 4:off 5:on 6:off 5) systemctl start apache2
      5. examining existing directives, THESE DIRECTIVES WORK with Apache2 on SuSE 13.2, but NOT in LEAP 42.2: DocumentRoot "/srv/www/htdocs" <Directory "/srv/www/htdocs"> AllowOverride FileInfo AuthConfig Limit Indexes Options MultiViews Indexes IncludesNoExec FollowSymLinks IndexOptions IgnoreCase FancyIndexing FoldersFirst NameWidth=* DescriptionWidth=* SuppressHTMLPreamble #SymLinksIfOwnerMatch <Limit GET POST OPTIONS PROPFIND> Order allow,deny Allow from all </Limit> <LimitExcept GET POST OPTIONS PROPFIND> Order deny,allow Deny from all </LimitExcept> </Directory>
      6. review or create systemd service - http://linuxmeister.net/Scripts/Sample-systemd-script.html
      7. any changes to website can be made throughout using: find . -type f -name README.html -exec cp README.html {} \;

      Session #95 - April 14, 2017 - FRIDAY - Linux Commands in use.

      note on html: color for cell above is "dddddd" (rrggbb)

      Notes for this session:

      1. discuss possible add ons for SuSE http://linuxmeister.net/SysAdmin/SuSE-add-ons.html
      2. using CSV (comma separated values) in a script to gather information (part of the Art of Linux Sys Admin series) http://linuxmeister.net/Scripts/collect-sys-info.sh.html - automate system documentation!
      3. CSV at the command line - using gnumeric: --> ssconvert spreadsheet.xls newfile.csv # (will this convert csv to xls? RTM anyone?) # a good use for gnumeric! --> apt-get install gnumeric or --> zypper install gnumeric or --> yum install gnumeric
      4. using the office suite at the command line (YMMV) - results on Linux with libreoffice ok... created a directory... need to try other options. CSV at the command line - using libreoffice: --> libreoffice --headless --convert-to csv $filename --outdir $outdir # this worked, created a directory CSV at the command line - using OpenOffice: --> soffice --headless --convert-to csv test-file.xls test.csv # not recommended on a mac, haven't tried on Linux or Win! # tried this with "advice" from a search - it created a ".lock" file owned by root in: (following advice on the internet is NOT recommended) # further advice says delete .lock... # BETTER - chown!!! (I should never do what I find on the internet! (without thinking)... and not on a mac) # /Users/luser/Library/Application Support/OpenOffice/4 removing it removed all of my configuration and history - thinking gnumeric on linux best...

      prep material to put into CSV to import into spreadsheet

      1. collect chapters into one book for Proof of Concept (POC): --> cat 11n_1Ki-* > 1Kings.txt ; perl -pi -e 's/1 Ki/1Ki/g' 1Kings.txt # fix space in naming --> cat 1Kings.txt | tr " " "\n" | grep ^[A-Z] | sort | uniq -d > 1Kings-KEYWORDS.txt # vi to clean up sp ch --> cat 1Kings-NAS-KEYWORDS.txt | wc -l 119
      2. create script to take KEYWORDS and list times found and references: #!/bin/bash # Find-Keywords-1Kings.sh.txt 13 Apr 2017 jm ########################################## # list of names: 1Kings-KEYWORDS.txt # complete book: 1Kings.txt ########################################## FILE="Names-found-in-1st-Kings.txt" # output, names to references echo `date +'%Y_%m_%d-%H%M'` > $FILE # way to reset the file and / or keep track echo "======================" >> $FILE # eye candy separator for x in `cat 1Kings-NAS-KEYWORDS.txt` # x is the name to search on do echo $x >> $FILE # list the name echo "----------" >> $FILE # eye candy grep $x 1Kings.txt | wc -l >> $FILE # number of times name is found in book echo "----------" >> $FILE # eye candy grep $x 1Kings.txt | awk '{print $1" "$2}' >> $FILE # print book and reference echo "======================" >> $FILE # eye candy done ########################################## OUTPUT: ########################################## # 2017_04_13-2209 # ====================== # Abel-meholah, # ---------- # 2 # ---------- # 1Ki 4:12 # 1Ki 19:16 # ======================
      3. NEXT CSV - modify script to insert commas for CSV formatting... (not complete, need to wrap lines) ########################################## #!/bin/bash # Find-Keywords-SET-AS-CSV-1Kings.sh.txt 13 Apr 2017 jm ########################################## # 1Kings-NAS-KEYWORDS.txt - grep'd Names, manually edited file # 1Kings.txt # NAS 1 Kings text ########################################## FILE="Names-found-in-1st-Kings-CSV-FORMAT.csv.txt" for x in `cat 1Kings-NAS-KEYWORDS.txt` do echo $x >> $FILE # no eye candy, name, followed by a comma, number of times, references echo ", ">> $FILE grep $x 1Kings.txt | wc -l >> $FILE echo ", " >> $FILE grep $x 1Kings.txt | awk '{print $1" "$2", "}' >> $FILE echo "====================== \n" >> $FILE done ########################################## # OUTPUT: ########################################## Abel-meholah , 2 , 1Ki 4:12, 1Ki 19:16, ====================== Abiathar , 9 , 1Ki 1:7, 1Ki 1:19, 1Ki 1:25,

      using grep to get the number of times a word appears in the NAS, and getting the references for study 694 grep -i love NAS.txt | wc -l # 573 695 grep -i hate NAS.txt | wc -l # 312 696 grep -i Lord NAS.txt | wc -l # 6707 697 grep -i sin NAS.txt | wc -l # 1566 698 grep John NAS.txt | wc -l # 136 699 grep Jesus NAS.txt | wc -l # 946 700 grep Joseph NAS.txt | wc -l # 233 701 grep Rebecca NAS.txt | wc -l # 0 702 grep Rebeka NAS.txt | wc -l # 31 703 grep Paul NAS.txt | wc -l # 161 704 grep -i faith NAS.txt | wc -l # 404 705 grep -i faithful NAS.txt | wc -l # 163 --> grep -i faith NAS.txt | awk '{print $1" "$2}' Gen 32:10 Gen 47:29 Lev 5:15 ... Rev 19:11 Rev 21:5 Rev 22:6

      Session #94 - April 7, 2017 - FRIDAY - Linux Commands in use.

      note on html: color for cell above is "abdddf" (rrggbb)

      Notes for this session:

       
      • win10 freebie: ALL COMMANDS from Win explorer: mkdir "c:\ALL_CMDS.{ED7BA470-8E54-465E-825C-99712043E01C}" http://linuxmeister.net/Microsoft/SHOW_ALL_COMMANDS-win7-8-10.html
      • got MATE working on SuSE: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
        1. DISPLAYMANAGER_XSERVER="Xorg"
        2. DISPLAYMANAGER="sddm"
        3. DISPLAYMANAGER_REMOTE_ACCESS="yes"
        4. DISPLAYMANAGER_ROOT_LOGIN_REMOTE="no"
        5. DISPLAYMANAGER_STARTS_XSERVER="yes"
        6. DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
        7. DISPLAYMANAGER_AUTOLOGIN="john"
        8. Default: "yes"
        9. DISPLAYMANAGER_PASSWORD_LESS_LOGIN="yes"
        10. DISPLAYMANAGER_AD_INTEGRATION="no"
        11. DISPLAYMANAGER_SHUTDOWN="auto"
      • this is SuSE still runnig xcfe, will fix...: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
        1. DISPLAYMANAGER_XSERVER="mate-desktop"
        2. DISPLAYMANAGER="lightdm"
        3. DISPLAYMANAGER_REMOTE_ACCESS="yes"
        4. DISPLAYMANAGER_ROOT_LOGIN_REMOTE="yes"
        5. DISPLAYMANAGER_STARTS_XSERVER="yes"
        6. DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
        7. DISPLAYMANAGER_PASSWORD_LESS_LOGIN="no"
        8. DISPLAYMANAGER_AD_INTEGRATION="no"
        9. DISPLAYMANAGER_SHUTDOWN="auto"
        10. DISPLAYMANAGER_AUTOLOGIN=""

      • further refinement of process to build study tools, worked through comparing line by line, editing and checking: --> more mergefiles.sh.txt #!/bin/bash # john 30 Mar 2017 merge-files-project.sh.txt ################################################ cp KJV.txt KJVb.txt cp NAS.txt NASb.txt perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt perl -pi -e 's/(.*)/$1(nas)/' NASb.txt paste -d '\n' KJVb.txt NASb.txt > KNb.txt #######################
      • To test I'd view the combined file, if I found an error, I'd edit the NAS version and repeat, put it all on one command line: --> vi NAS.txt ; sh ./merge-files-project.sh.txt ; more KNb.txt The first books I configured had more issues, but once my updated scripts had been used it went quickly; took about 3 evenings to finish. This also found the one discrepancy between the translations, as noted in 3Jn 1:15, where I added a line to the kjv to match: --> grep "3Jn 1:1" KNb.txt # note: if you forget the quotes you get everything that starts out 3Jn and the name of the file as well... # note: the indent on the second line for nas - this is found in the NAS.txt file, one space leading ... 3Jn 1:13 I had many things to write, but I will not with ink and pen write unto thee: (kjv) 3Jn 1:13 I had many things to write to you, but I am not willing to write them to you with pen and ink; (nas) 3Jn 1:14 But I trust I shall shortly see thee, and we shall speak face to face. Peace be to thee. Our friends salute thee. Greet the friends by name.(kjv) 3Jn 1:14 but I hope to see you shortly, and we will speak face to face. (nas) 3Jn 1:15 THIS VERSE DOES NOT APPEAR IN THE KJV BUT DOES IN THE NASB.(kjv) 3Jn 1:15 Peace be to you. The friends greet you. Greet the friends by name. (nas)
      • So, now I have several versions to work with for searching key words and phrases. The kjv with a verse on each line with and without (kjv), and the same for nas. The combined version has (kjv) and (nas) for clarity. The NAS.txt and KJV.txt are the base copies, but I have both translations by chapter and by book. Using grep I have control for searches on the combined file showing any differences in the translations. Looking for a common phrase: ------------------------------------------------ --> grep -i "Jesus wept" KNb.txt Jn 11:35 Jesus wept. (kjv) Jn 11:35 Jesus wept. (nas) ------------------------------------------------ When I check a phrase initially I'll often have the screen flash by with lots of info, so this helps: --> grep -i "know that I am the Lord." KNb.txt | wc -l 154 ------------------------------------------------ So, then I use a pipe and grep another key word to narrow the results... But sometimes the phrase or keys aren't the same in both and I'll only see one version: ------------------------------------------------ --> grep -i "you will know that I am the Lord." KNb.txt | grep Zion Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ------------------------------------------------ By reducing the phrase and dropping "you will" from the string, then both versions are shown. ------------------------------------------------ --> grep -i "know that I am the Lord." KNb.txt | grep Zion Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain: then shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv) Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ----------------------------------------------- another variation that may work: --> grep -i "[shall ye|will] know that I am the Lord." KNb.txt | grep Zion Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain: then shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv) Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain. So Jerusalem will be holy, And strangers will pass through it no more. (nas) ----------------------------------------------- When I know there is a difference in the translations, or suspect it, and I have the reference, I can grep on that: --> grep "Jn 8:58" KNb.txt Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv) Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas)
      • now I can use the file KNb.txt, or KJV.txt, NAS.txt to search out phrases or key words ------------------------------------------------ --> grep -i "know that I am the Lord." KNb.txt | more # both translations with (kjv|nas) at the end --> grep -i "know that I am the Lord." KJV.txt | more # KJV without (kjv) at the end --> grep -i "know that I am the Lord." KJVb.txt | more # KJV with (kjv) at the end --> grep -i "know that I am the Lord." NAS.txt | more # NAS without (nas) at the end --> grep -i "know that I am the Lord." NASb.txt | more # NAS with (nas) at the end
      • an example of a passage to search out is one using "I am", a statement that declared diety. In the Torah when Moses asked who he should say told him, the response was "I AM that I AM". ------------------------------------------------ So, I search: --> grep -i "I am that I am" KNb.txt Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said, Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv) ------------------------------------------------ ok... but where's the nas version? No problem, simply grep for the passage: --> grep "Exo 3:14" KNb.txt Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said, Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv) Exo 3:14 God said to Moses, "I A M W H O I A M"; and He said, "Thus you shall say to the sons of Israel, 'I A M has sent me to you.'"(nas) ------------------------------------------------ AND GUESS WHAT? I just found another edit I need to fix in NAS.txt... :) That's why it didn't show up. Perhaps I should write a script to update all my computers where I have these files now... ------------------------------------------------ But, continuing the search for the other statement of diety... ------------------------------------------------ --> grep -i "I am" KNb.txt | wc -l 1641 ------------------------------------------------ well, need to narrow that down a bit ------------------------------------------------ --gt; grep -i "I am" KNb.txt | grep Jesus | wc -l 66 ------------------------------------------------ still too many verses... going to guess the passage is in the Gospels ------------------------------------------------ --> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | wc -l 33 ------------------------------------------------ Ah, just remembered, Abraham was mentioned in the context... adding a grep: ------------------------------------------------ --> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | grep Abraham Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv) Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas) ------------------------------------------------ Of course I could just memorize the verse with the passage reference... well, I did, but I forgot it... ------------------------------------------------ So... now I've got a tool that allows me to search on key words, phrases and known references! ------------------------------------------------ I'm a happy student... and I need to fix /etc/sysconfig/displaymanager and test, and find a way of checking and fixing NAS.txt in Exo 3:14, and coming up with a way of copying or updating all my systems where I have these notes... ------------------------------------------------ Ok, if we make it this far and have time... fixing NAS.txt - LET'S use grep and awk to find any other instances like Exo 3:14 and create a script or command that lists those passages for us to review, and then possibly create a script using perl or sed to fix those known instances specifically so we don't create other problems in the file. We will need to review those passages to make sure they are the same as the source document. It's tedious and laborius, but important, and using these powerful regex tools in Linux at the command line makes the creation and editing of this tool possible. And if we still have time we can generate an rsync script to update other directories and systems...

      Session #93 - March 30, 2017 - THURSDAY - Linux Commands and Scripts

      note on html: color for cell above is "abcdef" (rrggbb)
      MET on Thursday this week... normally on Friday... 
      
      

      Notes and the script from the session:

      --> more merge-files-project.sh.txt #!/bin/bash # john 30 Mar 2017 merge-files-project.sh.txt # three objectives: # 1) file 1 first, then file 2 (add new line in process to create combined file) - maybe paste isn't the right tool? # 2) copy files, then annotate each line with designator - file1X.txt file2Y.txt # 3) preserve original, annotated and combined in plain text - file1, file1X, file2, file2X AND: BOTHfiles.txt # 4) create file with html table to show cells for each line, two cells in one row... 4.3M files! - BOTHfiles.html # paste -d '\n' file1.txt file2.txt # perl -pi -e 's/(.*)/<img src=\"$1\"><br>$1<hr>/' # KJV.txt NAS.txt ################################################ cp KJV.txt K1.txt cp NAS.txt N1.txt paste K1.txt N1.txt > KN-test1.txt # file with one long line for both files paste -d '\n' K1.txt N1.txt > KN-test2.txt # file with two separate lines in sequence and in order ################################################ # sed 's/$/ [KJV]/' <KJV.txt >KVJb.txt # perl -pe 's/$/ (kjv)/' <KJV.txt >KVJb.txt # need to find global replacement, and also how to insert into a string, if possible ################################################ cp KJV.txt KJVb.txt cp NAS.txt NASb.txt perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt perl -pi -e 's/(.*)/$1(nas)/' NASb.txt paste -d '\n' KJVb.txt NASb.txt > KNb.txt # file with two separate lines in sequence and in order head KNb.txt # put a head command in script to test read -p 'hit enter' ################################################ sed -i 's$.*$<tr><td>&<p></td>$g' K1.txt # using sed to add html tags on each line sed -i 's$.*$<td>&<p></td></tr>$g' N1.txt # using sed to add html tags on each line head -n 3 K1.txt head -n 3 N1.txt read -p 'hit enter' ####################### echo "<html><title>TEST</title><center><table border='10' width='95%'>" > KN1.txt-testX.html paste K1.txt N1.txt >> KN1.txt-testX.html # the paste command reads one line from each file echo "</table></center> <a href='copyright-info.txt'>copyright info</a></html>" >> KN1.txt-testX.html # adding closing table and link ####################### # perl -pi -e 's$Dan 9:$<b><em>Dan 9:</em></b>$g' KN27-Daniel-Ch9.html # decided to emphasis the book and chapter # head KN27-Daniel-Ch9.html # put a head command in script to test ####################### head KN1.txt-testX.html # put a head command in script to test #######################

      Session #92 - March 24, 2017 - FRIDAY - Linux Commands and Scripts

      discussion of perl, sed, and editing files using vi and scripts.
      COMPLETED the filtering of information from the wget files and fixed special characters and formatting.

      1) Will review the entire process and scripts used to extract the information and format it,
      2)and, also discuss using rsync between systems.

      3) Next step, merge two plain text files one line at a time using a loop to create a table in html.
      IF YOU HAVE A SCRIPT that can read one line from two files at a time, please share... KISS... SAW...
      KISS - keep it simple sir; ... SAW - simple always works...
      the text project:
      http://johnmeister.com/linux/Notes/MANAGE-TEXT/the-wget-perl-sed-vi-grep-rsync-plain-text-project.txt

      Session #91 - March 17, 2017 - FRIDAY - Linux Commands and Scripts

      1. A quick comparison of commands for MS use: http://linuxmeister.net/Microsoft/PowerShell-vs-BASH-commands-compared.html
      2. http://linuxmeister.net/Scripts/wget-perl-WX-details.html
      3. http://linuxmeister.net/Scritps/perl-fix-spaces-remove-brackets.sh.html
      4. the art of Linux Sys Admin outline, sample vi video
      5. SHELL - history, process, commands
      6. discussion on ASCII vs. UTF-8, etc.

      Then, continued discussion on using grep, perl, vi and sed to reformat my favorite translation of the Bible for searching of key words:
      we started out using: http://linuxmeister.net/Scripts/remove-brackets-w-perl.sh.html to fix brackets,
      #!/bin/bash
      echo "remove bracketed footnotes"
      perl -pi -e 's/\[.\]//g' *.txt
      perl -pi -e 's/\[..\]//g' *.txt
      
      after we ran the script in last week's session, see Session #90 notes. So after several books I've
      added more filters: http://linuxmeister.net/Scripts/perl-fix-spaces-remove-brackets.sh.html
      ###############################################################################################
      
      #!/bin/bash
      echo "remove bracketed footnotes, fixed commas and semicolon spacing"
      perl -pi -e 's/\[.\]//g' *.txt  # remove bracketed text such as a footnote
      perl -pi -e 's/\[..\]//g' *.txt # remove bracketed text such as a footnote
      perl -pi -e 's/,/, /g' *.txt    # adds a space after a comma
      perl -pi -e 's/,  /, /g' *.txt  # removes any extra spaces after a comma after adding a space
      perl -pi -e 's/;/; /g' *.txt    # adds a space after a semicolon
      perl -pi -e 's/:/: /g' *.txt    # adds a space after a colon
      perl -pi -e 's$\.$. $g' *.txt   # adds a space after a period
      perl -pi -e 's/?/? /g' *.txt    # adds a space after a question mark
      ###############################################################################################
      #  the following sed and perl string replace all the lines below, used brackets for upper-case
      ###############################################################################################
      sed -i 's/[A-Z]/ /g             # add space before uppercase letter
      perl -pi -e 's/  / /g           # change two spaces to one 
      ###############################################################################################
      # perl -pi -e 's/A/A /g' *.txt    # adds a space after an uppercase A
      # perl -pi -e 's/A  /A /g' *.txt  # removes any extra spaces before an uppercase A
      # perl -pi -e 's/T/T /g' *.txt    # adds a space after an uppercase T
      # perl -pi -e 's/T  /T /g' *.txt  # removes any extra spaces before an uppercase T
      # perl -pi -e 's/W/W /g' *.txt    # adds a space after an uppercase W
      # perl -pi -e 's/W  /W /g' *.txt  # removes any extra spaces before an uppercase W
      # perl -pi -e 's/Y/Y /g' *.txt    # adds a space after an uppercase Y
      # perl -pi -e 's/Y  /Y /g' *.txt  # removes any extra spaces before an uppercase Y
      ###############################################################################################
      #   need to figure out bracket in perl:  perl -pi -e  's/(.*)/<img src=\"$1\"><br>$1<hr>/'
      ###############################################################################################
      

      Would like to discuss getting the parameters for using perl instead of sed for character replacement,
      as I did here: http://linuxmeister.net/Scripts/wget-perl-script-WX-details.html , uses $1 for bringing buffer content over.
      I would also like to refine my notes on moving numbers to the beginning of the line as I did several sessions ago.

         The filter for the numbers doesn't always get it right, so I've added the filters above. 
      ##################################### cat $x | grep "text $x" | sed 's/New American/ New American/g' | \ sed 's/[^<]*<\/span>/ /g' | \ sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt #####################################
      The goal is to further refine the process of filtering the text data to carefully remove issues that I have been manually editing.
      Also: would like to be able to replace the " and ' and - symbols that are NOT standard, need the info to replace those "special" characters.

      Session #90 - March 10, 2017 - FRIDAY - Linux Commands and Scripts

      Discussion on use of wget, grep, vi and sed parameters: http://linuxmeister.net/Notes/MANAGE-TEXT/
      (discovered that there are issues with Regular Expressions as part of Posix 2 standards! complicating the use of special characters! see:
      http://linuxmeister.net/Notes/MANAGE-TEXT/man.regex-7.html)

      1. process: download files (script: base-get-book.sh ) ##################################### wget -O INFO-1 "http://www...page 1 wget -O INFO-2 "http://www...page 2 wget -O INFO-3 "http://www...page 3 #####################################
      2. process: strip out formatting to get specific content (script: extract-text-book.sh) ##################################### #!/bin/bash ##################################### mkdir INFO for x in `ls INFO-*` do ##################################### cat $x | grep "text $x" | sed 's/New American/ New American/g' | \ sed 's/[^<]*<\/span>/ /g' | \ sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt ##################################### mv $x INFO done #####################################
      3. process: make sure all files have proper names for sorting ls > fix ##################################### vi fix remove lines that have digits that will sort: e.g. filename-10.txt leave lines that need to have a zero added, then: :%s/.*/mv & &/g results: mv filename-1.txt filename-1.txt then cursor to the number on the right, and insert a 0 before 1, to get: filename-01.txt repeat (usually 1-9) until done. :wq ##################################### sh ./fix ; rm -f fix ##################################### ls -al (should show all files sorted nicely) #####################################
      4. process: remove remaining internal brackets use script to remove brackets: sh ./remove-brackets.sh * ##################################### script: remove-brackets.sh #!/bin/bash echo "remove bracketed footnotes" perl -pi -e 's/\[.\]//g' *.txt perl -pi -e 's/\[..\]//g' *.txt #####################################
      5. process: edit all the files to remove errors, line up on one line vi INFO-* after making sure there is a number on the left and all the text on one line, then :%s/.*/INFO 9:&/g :wn :%s/.*/INFO 10:&/g :wn :%s/.*/INFO 11:&/g :wn # where INFO is the book or file name, and 9 is the chapter. # The ampersand places the text string with line number to the right repeat until done editing all files #####################################
      6. process: create unique summary file and then copy to other locations cat INFO-* > INFO_number.txt copy files to proper directories and other servers: cp INFO-*.txt ../SORTED-INFO scp -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO or rsync -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO #####################################
      7. repeat until all 1,189 chapters are cleaned up and sorted... then test to see that there will be 31,102 lines, and based on the version, 781,621 words cat INFO-* > total-info.txt ; cat total-info.txt | wc (note: as of 3/7/2017 I'm on 14/66)

      Session #89 - March 3, 2017 - FRIDAY - Linux Commands and Scripts

      Discussion on use of wget, grep, vi and sed parameters: http://linuxmeister.net/Notes/MANAGE-TEXT/
      Worked with sed strings to extract specific details from a downloaded file:
      cat $x | grep "text $x" | sed 's/New American/ New American/g' |\ sed 's/[^<]*<\/span>/ /g' | \
      sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt

      As we approached the close of the session we ran out of time sorting out how to remove content between brackets... My subsequent RESEARCH
      discovered that there are "some" issues with Regular Expressions as part of Posix 2 standards! Thereby complicating the use of special characters! see:
      http://linuxmeister.net/Notes/MANAGE-TEXT/man.regex-7.html) for the notes, which confused me for quite some time... reverted to brute force method as shown for session #90.

      Session #88 - February 24, 2017 - FRIDAY - Linux Commands and Scripts

      http://linuxmeister.net/Overview/LinuxOverview.pdf

      last weekend: Released Simply Linux: Basics
      https://www.smashwords.com/books/view/705084
      weekend before: "Let's BASH Win10!" https://www.smashwords.com/books/view/703463
      (I skipped watching Doctor Who for TWO weeks now to find time to get these done!!! Glad I have TIVO!)
      This week: command line searching for KEY WORDS and organizing results.
      Will be using grep, awk, cut, wc, script, nl, and a few others to organize the material. Command line fun!
      Parsing through text files for KeyWords and organizing them through REGULAR EXPRESSIONS.

      the current list of books and videos - 22 February 2017

      click for pdf

      Session #87 - February 17, 2017 - FRIDAY - Linux Commands and Scripts

      http://linuxmeister.net/Overview/
      http://linuxmeister.net/Overview/LinuxOverview.pdf

      NEWS ITEM: Finished the BASH on Win10 book:

      https://www.smashwords.com/books/view/703463


      Windows 10 and BASH security issues:
      https://msdn.microsoft.com/en-us/commandline/wsl/release_notes
      https://www.youtube.com/watch?v=_p3RtkwstNk&feature=youtu.be
      https://github.com/ionescu007/lxss/blob/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf


      In other news, info on Linux, USA based distros:
      http://distrowatch.com/table.php?distribution=centos -- https://www.centos.org/ -- https://fedoramagazine.org/
      https://en.wikipedia.org/wiki/Fedora_%28operating_system%29
      https://distrowatch.com/table.php?distribution=elementary
      All USA distributions:
      http://distrowatch.com/search.php?ostype=All&category=All&origin=USA&basedon=All¬basedon=None&desktop=All&architecture=All&package=All&rolling=All&isosize=All&netinstall=All&language=All&status=Active#simple


      Session #86 - February 10, 2017 - FRIDAY - Linux Commands and Scripts

      ...back to the basics... LPIC 1, BASH Win10, etc...
      http://linuxmeister.net/Overview/
      http://linuxmeister.net/Overview/LinuxOverview.pdf
      http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html


      Session #85 - January 27, 2017 - FRIDAY - Let's BASH Windows 10!

      Discuss commands in Windows 10 BASH shell, 148 commands in /bin - how they compare to Mac OS X and Linux (YMMV)
      prior week install notes:
      discuss steps to configure Ubuntu BASH in Windows 10.


      Session #84 - January 20, 2017 - FRIDAY - improving performance of Linux conclusion and setup BASH in WIN10

      Finish discussion from Session #83 below, then discuss steps to configure Ubuntu BASH in Windows 10.

      cool items found in today's session:
      --> rpm -qf /usr/bin/pdftotext poppler-tools-0.24.3-15.1.x86_64 --> /etc/icewm "MENU" - edit capability plain text!!!! :)


      Session #83 - January 13, 2017 - FRIDAY - improving performance of your Linux (desktop/laptop) experience


      # quick note on Friday the 13th... the superstition likely started in Europe before Gutenberg and Luther made Bibles available widely across Europe... the idea that "Good Friday" was some how "unlucky" because Christ was crucified and that there were 13 people in the room... (assuming 12 disciples and Christ) apparently came from this belief. Two problems with this. 1) This happened on the 14th of Nisan, or our Wednesday night (Last Supper/Passover), and 2) if Christ hadn't given His life on the cross we'd be completely without hope, or "luck". If you have another explanation for Friday the 13th, please share. For more information on this time line and issue, see: http://bibletech.net/Crucifixion/THURSDAY-Crucifixion-3days-3nights.jpg
      IMPROVING desktop/laptop/server performance of Linux by monitoring, removing or tweaking. Situation: Running XFCE https://en.wikipedia.org/wiki/Xfce on SuSE Linux (either Leap 42.1, or 13.2) and noticed CPU consumption by a bunch of applications that are NOT used, NOT wanted, and shouldn't be there... of course YMMV... I don't need "orage" (calendar tool), Thunar (file manager - GUI... if you're going to use the GUI don't remove this), Clipman or Parcellite (GUI clipboard). I remove gvfs-fuse. because, in part, it produces this error: find: ‘/run/user/1000/gvfs’: Permission denied (I don't like the "convenience" of this "feature", it reminds me of Stuxnet, and the error as root annoys me greatly...) -------------------------------------------------------------------------- UNNEEDED APPLICATIONS or SERVICES: Process to remove an unneeded/unwanted application: top # observe any heavy hitters or unneeded tools, e.g. parcellite, ModemManager, fuse-gvfs, etc. ps -ef | sort | uniq | grep -v ...those things you know are system related or desired... ps -e | awk '{print $4}' | sort | uniq -c | grep -Ev '(ssh|bash|firefox)' | sort -n ps -ef | awk '{print $8}' | sort | uniq | grep -Ev '(ssh|bash|firefox)' ps -ef | awk '{print $8}' | sort | uniq > processes-Jan-4.txt # examine the output: cat processes-Jan-4.txt FOUND: ModemManager (on a desktop without a phone line) # there are some other items that could be removed, gnome-key-ring... sudo lsof /usr/sbin/ModemManager # this gives info on the application sudo kill -9 743 # the pid for ModemManager or better, use service ModemManager status service ModemManager stop sudo zypper remove ModemManager ...(1 package to remove: ModemManager (frees up 2.3MB) ...removed ModemManager-1.4.10-3.2.x86_64 sudo zypper -ps -s # to check services using deleted files --> sudo zypper ps -s The following running processes use deleted files: PID | PPID | UID | User | Command | Service -----+------+------+---------+----------------------------+---------------- 831 | 1 | 0 | root | mcelog | mcelog 861 | 1 | 497 | polkitd | polkitd | polkit 1223 | 1 | 0 | root | cupsd | cups 1241 | 1 | 0 | root | sshd | sshd 1398 | 1396 | 0 | root | Xorg | display-manager 1411 | 1396 | 0 | root | xdm | 1499 | 1435 | 1000 | luser | ssh-agent | 1501 | 1 | 1000 | luser | xfconfd | 1504 | 1435 | 1000 | luser | xfce4-session | 1511 | 1 | 1000 | luser | kdeinit4 | 1513 | 1511 | 1000 | luser | kdeinit4 | 1515 | 1 | 1000 | luser | kdeinit4 | 1517 | 1 | 1000 | luser | gconfd-2 | 1522 | 1 | 1000 | luser | xfwm4 | 1526 | 1 | 1000 | luser | xfce4-panel | 1528 | 1 | 1000 | luser | thunar | 1530 | 1 | 1000 | luser | xfdesktop | 1533 | 1 | 1000 | luser | xfsettingsd | 1534 | 1504 | 1000 | luser | mate-volume-control-applet | 1536 | 1504 | 1000 | luser | python2.7 | 1541 | 1504 | 1000 | luser | python3.4 | 1542 | 1 | 1000 | luser | xfce4-power-manager | 1543 | 1504 | 1000 | luser | nm-applet | 1544 | 1504 | 1000 | luser | kdeconnectd | 1550 | 1504 | 1000 | luser | pk-update-icon | 1556 | 1 | 0 | root | upowerd | upower 1565 | 1 | 1000 | luser | pulseaudio | 1595 | 1 | 1000 | luser | at-spi-bus-launcher | 1610 | 1 | 1000 | luser | at-spi2-registryd | 1615 | 1536 | 1000 | luser | python2.7 | 1616 | 1615 | 1000 | luser | python2.7 | 1624 | 1565 | 1000 | luser | gconf-helper | 1630 | 1 | 1000 | luser | python2.7 | 1649 | 1526 | 1000 | luser | wrapper-1.0 | 1650 | 1526 | 1000 | luser | wrapper-1.0 | 1657 | 1526 | 1000 | luser | xfce4-terminal | 1698 | 1526 | 1000 | luser | VirtualBox | 1718 | 1 | 1000 | luser | VBoxXPCOMIPCD | 1723 | 1 | 1000 | luser | VBoxSVC | 1744 | 1723 | 1000 | luser | VirtualBox | You may wish to restart these processes. See 'man zypper' for information about the meaning of values in the above table. alternate, disable service sudo systemctl disable ModemManager.service sudo zypper remove tracker-extract To find out what packages are installed: --> zypper wp > zypper-wp-SuSE.txt # NOTE: wp stands for "what-provides" --> zypper wp > cat zypper-wp-SuSE.txt # if there is an "i" in front, it's installed. # there wasn't an "i" in front of zile, so I tried to remove it: --> zypper remove zile Root privileges are required for installing or uninstalling packages. ------------------------------------------------ --> sudo zypper remove zile Loading repository data... Reading installed packages... 'zile' not found in package names. Trying capabilities. No provider of 'zile' found. Resolving package dependencies... Nothing to do. ------------------------------------------------ # THIS PERPLEXED ME... so... I checked the man page: --> man zypper | col -b > man.zypper.txt # found several directories, and then noticed the log and history file. --> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}' --> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}' | grep Modem install libKF5ModemManagerQt6 install ModemManager so.... one could: sudo zypper remove ModemManager # already did... ------------------------------------------------ see also: Zypper cheat sheet see also: assorted tools (apt) cheat sheet see also: Yum cheat sheet NOTE: If you're using MUD then it would be: sudo apt-get remove ModemManager IF you realize that you really needed the app, you could reinstall: sudo zypper install ModemManager (or apt-get install...) If you're using Centos or RH, sudo yum remove / install .... If you're using RPM, see man pages for options. -------------------------------------------------------------------------- KERNEL TWEAKS: ADJUST SYSTEM TO REDUCE SWAP: sysctl -w vm.swappiness=1 When swappiness is reduced the kernel doesn't attempt to increase cache by paging out. Paging is not good... To change permanently, write vm.swappiness=1 into your /etc/sysctl.conf file. (NOTE: do NOT do this is you are memory constrained on your system, typical default setting is around 60) -------------------------------------------------------------------------- APPLICATION TWEAKS: As far as "recommended" applications (Firefox, Thunderbird and OpenOffice (LibreOffice) 1) set cache to 0 in firefox. (go to the hamburger thing on the far right side, select preferences, advanced, Cached Web Content) More tweaks for the mighty trio to come... (feel free to share...)

      Session #82 - January 6, 2017 - FRIDAY - managing file archives at the command line

      using Linux and vi at the command line to manage files and directories
       PROBLEM: have a large server with duplicate directories and files, need to organize by date, compare and consolidate
             OBJECTIVE:  Standardize DIRECTORY names for proper sorting:   YYYY-MM-DD-filename.txt
                         Individual file names can have the date as a suffix before the file type, e.g.  filename-YYYY-MM-DD.txt
                          Files will sort by number, then letter, starting from the left to the right.  
      
      step 1: change into directory, for example, the year 2000, June... cd 06_June-2000 step 2: list the files and create a file to edit (see details below) step 3: use vi to create a quick script to rename files, when you exit vi, it executes the script, then removes the script step 4: then you examine the directory to make sure the files were named correctly and mv them up to the parent directory step 5: once you're in the parent directory, check the old directory using a recursive listing: "ls -AlR 06_June-2000" step 6: If the directory is empty and didn't contain any hidden directories (e.g. .profile ), then: "rm -rf 06_June-2000" NOTE: if you're comfortable with this process, then you can add: " mv * .. ; cd .. ; rm -rf 06_June-2000 " to the script. DETAILED STEPS: --> cd /home/luser/files/2000/06_June-2000 ------------------------------------------------ luser@penguin [/home/luser/files/2000/06_June-2000] ------------------------------------------------ --> ls > fix ; vi fix ; sh ./fix ; rm -f fix
      # NOTE: be sure NOT to use a keyword or command, fix might not be the best choice... to check the word used: --> which fix which: no fix in (/home/.... # we're ok... if it was a keyword or command, just use some other combination, like mymess2...
      # while in vi: # note: you will be in vi immediately, when you quit out it'll attempt to execute the file, make sure it's right... :%s/.*/mv & 2000-06--&/g # global copy and paste with a prefix of information for another editing step (note: where 06 = Jun) :%s/2000-06--06-Jun-/2000-06-/g # another global replace to modify the file, putting the date in the preferred order # while in vi you'll need to edit out the REDUNDANT -2000 (the original file had 06-June-15-2000 as the file format) # there might be a better way of creating the FROM TO names using perl, sed, and awk, but this is a quick and dirty script... # use /-2000 to find the 2nd instance, then delete TWO words using vi/command mode: "2dw" - if you goof, <esc>u and try again. # the desired line in the "fix" script mv 06-Jun-15-2000-SJ-egr-tube 2000-06-15-SJ-egr-tube # Once the directories are named correctly, move them up to the main directory and remove the old directory. --> mv * .. ; cd .. ; ls -AlR 06_June-2000 # if it's all clean, then nuke it... --> rm -rf 06_June-2000

      updating directories for web presentation

      After the directories for each year are organized by YYYY, then MM, then DD (e.g. 2000 06 15) you can add any particular web content into each directory via a quick and dirty script. I add a README.html, HEADER.html and for images I add an ALL.html. The manual process is something like this: --> cd 1999-12-Marks-J3000 ; mk-webpage "1999-12-Marks-J3000" ; ls -al ; cd .. # the "mk-webpage" is a custom script to create localized web content - we've discussed this script in prior sessions see: http://linuxmeister.net/Scripts/mk-webpage-2015-05-08.html # So, the process we need to "script" is: cd 1999-12-Superdawg ; mk-webpage "1999-12-Superdawg" ; ls -al ; cd .. So, for every directory we need to cd into it, run the mk-webpage script. PROCESS: --> cd 2000 --> ls > addall --> vi addall # while in vi the process is simple and quick: :%s/.*/cd & ; mk-webpage "&" ; cd ../g # the lines should look like: cd 2000-01-01-BEST ; mk-webpage "2000-01-01-BEST" ; ls ; cd .. cd 2000-01-01-Superdawg ; mk-webpage "2000-01-01-Superdawg" ; ls ; cd .. ... then simply escape, write, quit: "<esc>:wq" then execute the quick and dirty script to update the entire year's worth of directories for your website: --> sh ./addall then copy the updated directory, to your web server (unless you edited in place), and then check one of the directories and try an ALL.html file: http://johnmeister.com/fotos/2002/2002-02-05-Sun-Solaris-Install/ALL.html This is an updated version of a Sun Solaris install from February 2002. ------------------------------------------------


      Session #81 - December 16, 2016 - FRIDAY - Linux text management

      use sed, vi and perl to combine lines in large text files
      http://johnmeister.com/linux/Notes/MANAGE-TEXT/
      using sed, vi, and perl will demonstrate how to combine lines in a large block of text
      in order to use grep to locate key words and display entire content of line.

      Session #80 - December 9, 2016 - FRIDAY - Linux and troubleshooting

      troubleshooting, diagnosis, monitoring
      Have been setting up some systems and having challenges with BIOS, proprietary RAID and hard drive failures.
      some issues during setup of dell towers in bios random bios screens


      "Bob" found some interesting links on SMART:
      https://en.wikipedia.org/wiki/Comparison_of_S.M.A.R.T._tools - https://www.smartmontools.org/
      https://sourceforge.net/projects/smartmontools/ and https://sourceforge.net/projects/smartmontools/files/smartmontools/6.5/

      file system recovery
      http://johnmeister.com/linux/FileSystems - recovery of failing drives - tune2fs - grub and UUID
      http://johnmeister.com/linux/FileSystems/rescuing-failing-drive-w-dd.html

      fresh file system setup
      http://johnmeister.com/linux/Notes/KERNEL-BOOTING/grub.cfg-example.html
      http://johnmeister.com/linux/FileSystems/UUID-details.html
      http://johnmeister.com/linux/FileSystems/newdisk-setup.html
      http://johnmeister.com/linux/FileSystems/mount-external-drive-rysnc.txt
      http://johnmeister.com/linux/FileSystems/setup-LVM.html

      in other use: http://johnmeister.com/linux/Notes/using-find-grep-xargs.html


      Session #79- December 2, 2016 - FRIDAY - Simply Linux and mysql

      open disucssion: current issue: metadata from RAID on Dell
      fdisk -l details
      SuSE update screen
      updated h/w, dell tower T5600 --> T5810 - Linux not able to boot... metadata based on proprietary consumer grade h/w, also using LVM... this is why SAW (simple always works... and using "new" features often doesn't)

      Continued discussion on command line mysql on Linux.
      steps to install, activate, and create simple databases as the command line in Linux.
      still working on importing CSV, in the meantime, found the commands to make the table...
      will capture working commands and populate... will incorporate commands into BASH scripts and continue developing
      tools at the command line to manipulate data.
      • sudo zypper install mysql (or sudo apt-get install mysql)
      • sudo services mysql start
      • mysql -u root -p -h 127.0.0.1
      • create DATABASE firsttry;
      • show databases;
      • show tables;
      • create table equipment (hostname varchar(20), location varchar(20), OS varchar(10));
      • describe equipment;
      • insert into equipment(hostname,location,OS) values ('lab1', 'bench 1', 'Mint');
      • select * from equipment;
      • status;
      • help;
      Alternate paths for links: http://LinuxMeister.net
      http://shop.oreilly.com/product/0636920050209.do?cmp=ex-prog-books-videos-product-na_moo_card_for_john_e._meister,_jr._for_study_guide_for_lpic_2_exams_201_and_202
      http://linuxmeister.net/Intro-to-Linux/One-Hour-Linux-Sessions-2016.html (not updated as frequently - mirrored on different server)


      Session #78- November 18, 2016 - FRIDAY - Simply Linux and mysql

      • sudo zypper install mysql
      • sudo services mysql start
      • mysql -u root -p -h 127.0.0.1
      • create DATABASE firsttry;
      • show databases;
      • status;
      • help;

      Session #77- November 11, 2016 - FRIDAY - Simply Linux: book outline

      1. Using Linux
      2. Linux Summary
      3. Linux Overview and Setup
      4. Five (5) Commands you need to master
      5. Shell configuration, History, and "script"
      6. File Management
      7. Files and Processes: Special Characters
      8. Regular Expressions and Filtering Data
      9. SSH setup and use
      10. Network Basics
      11. File Systems - Create, Monitor, Manage
      12. User Management (System Administration tasks)
      13. Package Management - Upgrading and Installing
      14. Appendix A: The LINUX Philosophy
      15. Appendix B: Exercise #1 - basic commands

      Session #76- November 4, 2016 - FRIDAY - Linux: command line

      Regular expressions experimenting with "cal" in a loop: using a loop to determine the day of the week for the 15th of July over various years...
      a discussion on ffmpeg with a cool timelapse, and look at x2go.

      copy and paste links below, not live

      Video created by Uncle Mike: Facebook post Public time lapse video:
      https://www.facebook.com/m.d.clay/videos/vb.703246157/10154101081271158/?type=2&theater
      A look at another NX technology tool for remote access of Linux systems:
      http://wiki.x2go.org/doku.php/doc:newtox2go and https://en.wikipedia.org/wiki/X2Go

      Session #75- October 20, 2016 - THURSDAY - Linux: open discussion and follow up

      Discuss follow up on ffmpeg, video formats, and Apache configuration and services. apache2-service-commands.html

      Session #74- October 14, 2016 - Friday - Linux: trace a script to encode video

      https://gist.github.com/donmelton/7734177 - found this script to encode video, looks interesting... still don't have ffmpeg working

      Session #73- October 7, 2016 - Friday - Linux: fixing crontab email and examining ffmpeg, vlc and mencoder

      http://johnmeister.com/linux/Notes/crontab-email-setup.html
      Alternate paths for links: http://LinuxMeister.net
      O'Reilly Media Linux Sys Admin video series

      Session #72- September 28, 2016 - Wednesday - Linux: Using grep, awk, cat, diff, wc, and comm

      NOTE: ### WEDNESDAY THIS WEEK ###.... WEDNESDAY - COMMAND LINE POWER
      using cat, grep, awk, sort, redirection, comm, more, perl (sed), tr and wc to solve a cool problem
      There were multiple files with common values and lots of data. We needed to find the common values among several files.

      Took real world info and completely buggered it up to show ONLY how to find certain things in a large file.
      The content of the files is irrelevant, so it was obscured to help us find only what we were seeking.

      # this is the process used, but was NOT the strings used to corrupt the data...
      cat original.file.1.txt | tr c-z a-x | tr 2-8 3-9 | tr A-Y B-Z > 1.RAW
      # after completely messing up the content, look for the one string, "Messedup", and change it to our primary search value:
      perl -pi -e 's/Messedup/KEY-VALUE/g' *.RAW
      Then we use grep to find that KEY-VALUE among thousands of lines in the file and look for the info labeled as "Key-Value":

      cat .RAW | grep KEY-VALUE | grep - | awk '{print $4}' | sort > 1.rpt
      ...
      comm -12 1.rpt 2.rpt | comm -12 - 3.rpt > common-values.txt
      ...
      we'll test our results with cat and grep...
      time permitting we'll convert this into a script... and enhance it...

      Alternate paths for links: http://LinuxMeister.net
      O'Reilly Media Linux Sys Admin video series

      Session #71- September 22, 2016 - Friday - Linux: file systems, shell scripts and configurations

      The discussion on file systems most likely include Clay's presentation from last week:
      (we've been having an email discussion on ext4 and journaling...)
      http://johnmeister.com/linux/SysAdmin/Contrib/lunch-n-learn_2016-09/
      Clay has done more research on journaled (ext4) file systems along with Mark, the two of them will want to share.
      script in work: http://johnmeister.com/linux/Scripts/setup-diverse-systems-prototype.sh.html
      have been working on shell script to gather key info from a variety of linux versions and distros, seeking to gather config data
      and either automate a standard configuration, or at least have an on-screen checklist with commands spelled out that can be used with tweaks.
      There are certain environments where managing Linux installs is a challenge because of networking related issues,
      primarily those lacking name services, so one must use the IP address or use /etc/hosts to make things work.
      This can be a problem as these environments use either DHCP or DDNS. Automation in an uncertain environment is risky...
      Alternate paths for links: http://LinuxMeister.net
      O'Reilly Media Linux Sys Admin video series

      Session #70 - Sep 16, 2016 - Friday - Linux command line - Clay's persistent USB 3.0 Portable Linux

      Clay will present "Portable Linux" with the details and shell script to turn a USB drive into a persistent Linux instance.
      http://linuxmeister.net/SysAdmin/Contrib/lunch-n-learn_2016-09/
      His recommendation for host device is the USB 3.0 San Disk ultra, linked below is the 128GB drive, Clay suggests a minimum of 32GB:
      https://www.amazon.com/SanDisk-Ultra-SDCZ43-128G-GAM46-Newest-Version/dp/B01BGTG2A0/ref=dp_ob_title_ce
      While other devices will work Clay recommends this type because it is small, USB 3.0, and doesn't stick out of the USB slot
      so it doesn't risk damage to your USB port. He will focus on minimizing writes to the device to prolong its life.
      A link to my server with the script and details will be provided later in the week.

      In the past we've discussed using Gparted booted from a USB device to recover a system, Clay's presentation is a
      significant advancement on that idea, allowing someone to maintain a complete Linux system with persistent data and
      configuration and use whatever computer is available. The perfect PARASITE that doesn't leave any itchy red spots when finished.
      The beauty of using the USB device is nothing on the local hard drive is touched, so you can use any computer anywhere
      with this setup and not risk infection or exposing data on the computer or to the computer.

      Links to prior discussions as a baseline:
      http://linuxmeister.net/Notes/USB-live-Gparted-setup.txt
      http://linuxmeister.net/Notes/Gparted-for-Recovery/ALL.html
      http://linuxmeister.net/Notes/failing-drive-system-rescue.html

      Session #69 - Sep 9, 2016 - Friday - Linux command line - Discuss Oracle's Virtual Box and Centos Install

      INSTALLING CENTOS 7 on a VM... (distracted: found the limit when we opened 4 VM's (Win10, WinXP, Solaris and Centos)
      on a MacBook Air: 2 browsers, Webex, and Thunderbird) - rather than wait for it to recover, powered down... it did NOT crash though!
      continued discussion on using Windows 10 (v1607) BASH (Ubuntu) REGULAR EXPRESSIONS
      http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/
      http://www.tenforums.com/tutorials/46769-bash-ubuntu-windows-10-enable-disable.html
      http://www.omgubuntu.co.uk/2016/08/windows-10-anniversary-update-delete-partition

      Session #68 - Aug 26, 2016 - Friday - Linux command line - Digital photography and web pages

      continue discussion on setting up pages using vi, html and regular expression, and also discuss Windows 10 BASH in v.1607.
      Creating new script to add exiftool data to pages on web server. Exiftool extracts exif info from raw and jpg files.
      http://linuxmeister.net/Scripts/mk-webpage-2015-05-08.txt
      http://linuxmeister.net/Scripts/exiftool.sh.txt


      Session #67 - Aug 19, 2016 - Friday - Linux command line - Digital photography and web pages

      Creating new script to add exiftool data to pages on web server. Exiftool extracts exif info from raw and jpg files.
      http://linuxmeister.net/Scripts/mk-webpage-2015-05-08.txt
      http://linuxmeister.net/Scripts/exiftool.sh.txt


      Session #66 - Aug 10, 2016 - Wednesday - Linux command line discussion

      Demonstrate the use of rsync and scp in scripts and setting up cron to automate.
      http://linuxmeister.net/Scripts/rsync-directories.sh.html
      http://linuxmeister.net/Notes/Setup-TimeLapse-Creation.html

      Session #65 - Aug 2, 2016 - Wednesday - Linux command line discussion

      Discussion showing differences between rsync and scp and the effects of placing in the background:
            --> rsync -rp --delete linuxbox:/home/luser/files/METRICS/ linuxbox-files/METRICS
      IF THE slash on the source is left off... then the entire directory is copied over to the destination directory.
      IF the slash is present on the source, then the files are moved into the destination directory. 
          (the idea is to copy the entire directory as part of the path, replicated to the remote box... if you don't
          get the slashes correct the files will be placed in a directory one level up, creating a mess...)
      ------------------------------------------------
      --> echo "THIS WORKS for SCP, need slash, no dest DIR: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/"
              scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/ &
      THIS WORKS for SCP, need slash, no dest DIR is required: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/
      The -q is for quiet, keeps the pipe buffer from filling and stopping the process if scp is placed in bg. 


      Session #64 - July 29, 2016 - Friday - Linux command line discussion

      SYSTEM ADMINISTRATOR APPRECIATION DAY!!! http://sysadminday.com
      Continue discussion of command line configurations and scripting the setup of printers and workstations.
      Review of SuSE LEAP distribution as based on SLE.


      Session #63 - July 22, 2016 - Linux discussion

      The Art of Linux System Administration - LPIC-2 Study Guide

      Session #62 - July 15, 2016 - Linux Course released by O'Reilly! Free videos - bring popcorn

      The Art of Linux System Administration - LPIC-2 Study Guide Will go over the course outline and maybe show one or two of the videos.
      Will also review "shell" options: shopt (which works in any BASH environment, Linux, UNIX or MacOSX
      -----------------------------------------------
      --> shopt -s histverify
      -----------------------------------------------
      --> shopt
      ...
      histverify on
      ...
      ------------------------------------------------


      Session #61 - June 24, 2016 - Linux Overview - continued

      see Session #60 below for content links - continued command line demonstrations and explanations.

      Session #60 - June 17, 2016 - Linux Overview - continued

      Linux Overview on one page
      ...we left off discussing Special Characters in preparation to show the use of Sed, Grep and Awk in a shell...
      ...of course we also mentioned col -b and creating copies of text files of man pages, see your homework:
      http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html man 'command' | col -b > man.command.txt
      Then we'll talk about using sort... and other filters... with whatever chunks of data I have at hand...

      Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk


      Session #59 - June 9, 2016 - Linux Overview

      Linux Overview
      Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk


      Session #58 - June 2, 2016 - Linux Tuning Overview

      Linux Tuning

      Session #57 - April 8, 2016 - using a script to extract system info and create a CSV file

      creating a script to create CSV

      Session #56 - April 1, 2016 - Debunking Calendar Myths with Calendar

      calendar myths
      example... 5 Sundays and 5 Saturdays... an email might claim it only happens once every X years,
      with X being some massive number... wrong... but who's going to flip through a century's worth of calendars?
      Well, Linux will:   (adjust Y=2009 to something like Y=1900 to see much more)
      
      --> for ((Y=2009; Y<=2016; Y++)); do for M in 1 3 5 7 8 10 12;\
       do cal $M $Y | grep -q ^31 && printf '%02d/%d\n' $M $Y; done; done
      05/2009
      01/2010
      10/2010
      07/2011
      03/2013
      08/2014
      05/2015
      01/2016
      07/2016
      


      Session #55 - March 25, 2016 - ssh configuration files and tunneling

      http://linuxmeister.net/Scripts/extract-SSH-config-info.sh.html http://linuxmeister.net/Notes/SSH/sample-etc-ssh-config-files.html http://linuxmeister.net/Notes/SSH/ssh-base.html http://linuxmeister.net/Notes/SSH/typical-etc-ssh-conf-files.html http://linuxmeister.net/Notes/SSH/quick-ssh.html http://linuxmeister.net/Notes/SSH/ssh-full-duplex-process.html http://linuxmeister.net/Notes/SSH/ssh-timeout-tweaks.html

      http://linuxmeister.net/Notes/SSH/reverse-tunnel.html http://linuxmeister.net/Notes/SSH/ssh-tunnel.jpg

      Session #54 - March 18, 2016 - Compile the Kernel and recovery

      
      http://linuxmeister.net/Notes/Compile-a-Kernel.html 
      
      http://linuxmeister.net/Notes/Dracut-recovery-shell.html 
      



      Session #53 - March 11, 2016 - Kernel, booting and GRUB

      Discussion on Kernels, booting, and GRUB - Grand Unified Boot Loader
      
      http://www.ibm.com/developerworks/library/l-linuxboot/ 
      
      http://linuxmeister.net/Notes/grub.cfg-example.html 
      


      Session #52 - March 4, 2016 - NFS basics and RAID

      Discussion on Network File Systems and RAID
      
      http://linuxmeister.net/FileSystems/NFS-info.html
      
      http://linuxmeister.net/FileSystems/RAID-info.html
      


      Session #51 - February 26, 2016 - The use of "script" and col -b

        The use of "script", as root user, from /root, type:
      
      	script setup-new-user-home-luser.RAW
      optional ex: (add user to /etc/passwd, /etc/shadow, /etc/group - edit files manually or use "useradd -m username")	
              mkdir -p /home/luser/.History ; mkdir -p /home/lusers/.ssh ; mkdir -p /home/lusers/bin
      	cp /root/users.bashrc /home/luser/.bashrc
      	man bash | col -b > /home/luser/bin/man.bash.txt
      	ssh --help | col -b | tee -a /home/luser/bin/man.ssh.txt
      	chown -R luser:users /home/lusers
      	exit
      	cat setup-new-user-home-luser.RAW | col -b > setup-user-home.txt ; rm -f setup-new-user-home-luser.RAW
      	ls -alR /home/luser | more
      	vi setup-user-home.txt  (replace "luser" with variable for user, and modify as a wrapper script to use with tool)
      
      
      ---------------------------------------------------------------------------------------------------------------------------- Notes, general (SuSE and MINT): --> which useradd: /usr/sbin/useradd --> useradd --help: Options: -m, --create-home create the user's home directory SYNOPSIS useradd [options] LOGIN useradd -D useradd -D [options] ---------------------------------------------------------------------------------------------------------------------------- Notes, MINT (MUD): SYNOPSIS adduser [options] [--home DIR] [--shell SHELL] [--no-create-home] [--uid ID] [--firstuid ID] [--lastuid ID] [--ingroup GROUP | --gid ID] [--disabled-password] [--disabled-login] [--gecos GECOS] [--add_extra_groups] [--encrypt-home] user DESCRIPTION useradd is a low level utility for adding users. On Debian, administrators should usually use adduser(8) instead. When invoked without the -D option, the useradd command creates a new user account using the values specified on the command line plus the default values from the system. Depending on command line options, the useradd command will update system files and may also create the new user's home directory and copy initial files. By default, a group will also be created for the new user (see -g, -N, -U, and USERGROUPS_ENAB). ----------------------------------------------------------------------------------------------------------------------------


      Session #50 - February 19, 2016 - Script Templates

      Script template basics: 
      
      1. #!/bin/bash (shebang: #!) - shell directive ALTERNATE: #!/usr/bin/env bash ANOTHER ALTERNATIVE: #!/bin/bash -e
      2. Author
      3. Date and version info
      4. comments about changes, requirements, etc.
      5. Title or purpose of script
      6. USAGE variable - or comments on how to use the script
      7. Variable declaration.
      DISCUSSION on "fix-filename": (without appropriate comments, authorship and dates - just the basics) #!/bin/bash for f in * do mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/_-_/_/g'` done http://stackoverflow.com/questions/14008125/shell-script-common-template https://gist.github.com/KylePDavis/3901321 http://linuxcommand.org/lc3_new_script.php


      Session #49 - February 12, 2016 - Syncronizing files between systems

      Osync script example: http://johnmeister.com/linux/Scripts/EXAMPLE-OSYNC/
      discussion will include a review of this very complex BASH script, and how it can be done
      with less code using rsync, scp and ssh commands...  
      
      http://www.netpower.fr/projects/osync/Osync_v1.00a.html#toc-Subsection-1.1

      Session #48 - February 5, 2016 - The Art of Linux Systems Administration

      Continued Topics for discussion:
      
      - Understanding the Linux philosophy and how it affects commands and tools 
      
      - Systems Administration overview: what is a sys admin and what do they do? 
      - Developing the Art of Systems Administrator
      
      - master the vi editor...  
      - Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
      (any artists?  would like to draw a snail... purchase and install will be the tail then heading around to deal
      with the other parts that are an iterative process, not unlike the SDLC.
      


      Session #47 - January 29, 2016 - The Art of Linux Systems Administration

      Topics for discussion:
      - Understanding the Linux philosophy and how it affects commands and tools 
      - Systems Administration overview: what is a sys admin and what do they do? 
      - Developing the Art of Systems Administrator
      - master the vi editor...  
      - Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
      (any artists?  would like to draw a snail... purchase and install will be the tail then heading around to deal
      with the other parts that are an iterative process, not unlike the SDLC.
      


      Session #46 - January 22, 2016 - FIRST Session 2016 - LPIC - Administering Small to Medium sized Linux Networks

      LPIC-2 - administering small to medium-sized mixed Linux networks.

      2015 sessions and links below


      Session #45 - December 16, 2015 - LAST Session 2015 - LPIC - Administering Small to Medium sized Linux Networks

      LPIC-2 - administering small to medium-sized mixed Linux networks.
      FREE BOOK:
    6. FREE BOOK on Microsoft Windows


      Session #44 - December 9, 2015 (Wednesday) - NESTED FOR loop in a Batch file and BASH script

    7. BATCH file to setup a workstation - load registry hacks
    8. BATCH FILE explicitly coded for each box - "before" the FOR loop
    9. BATCH using FOR LOOPS - significant savings in coding! FREE BOOK:
    10. FREE BOOK (includes background on the TEMP issue)
    11. could be used to automate cleanup LINUX SCRIPTS USING FOR LOOP:
    12. variable definition less complex in BASH
    13. using for loops to gather info quickly - less stringent syntax than MS
    14. this wouldn't be possible in an MS Batch file


      Session #43 - December 2, 2015 (Wednesday) - VirtualBox Strategy, Plan and Goals

      http://linuxmeister.net/Notes/VirtualBox-VM-Plan


      Session #42 - November 25, 2015 (Wednesday) - update on Performance Tests; VirtualBox: SuSE on SuSE

      Main pages for #42 presentation: http://linuxmeister.net/Microsoft/LoadTest-MacOSx.sh.txt update on performance testing
      http://linuxmeister.net/Microsoft/LOADTEST-OS-COMPARISON.html
      http://linuxmeister.net/Notes/VM_setup_SuSE_13.2/ALL.html
       Step-by-step screen shots of install of SuSE 13.2 Linux on VirtualBox Virtual Machine on SuSE Linux
      exercises:  (use VirtualBox with two VMs, or Cygwin, or a Linux system) (continuing Q and A and review)
              the goals:   
          1) work with the command line and SSH between systems – learn basic Linux functions and shell commands
          2) later, build VMs for Windows testing and interoperability - demonstrate the use of Linux as a SECURE and reliable host for Microsoft
              - will build Win7 VM's for testing browsers and java by test team - showing stability of Linux as VM host/cost saver
              - will build various NON-networked Windows Server versions for testing (w2k8r2, w2k12 and w2k16)
          3) networking between 2 VM's but isolated from local network - Allow active testing of unapproved Operating Systems (win and linux)
          4) remote access from Microsoft to Linux system running a Microsoft system to permit active development and testing
      REFERENCE notes and exercises FOR VirtualBox:
      1. http://linuxmeister.net/Notes/VirtualBox-notes.html
      2. http://linuxmeister.net/Intro-to-Linux/Slides/ALL.html
      3. http://linuxmeister.net/Notes/bashrc-simple.html
      4. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html
      5. http://linuxmeister.net/Intro-to-Linux/lab-exercise-2.html
      6. http://linuxmeister.net/Intro-to-Linux/lab-exercise-with-VM-servers.html
      7. http://linuxmeister.net/Scripts/chksys.sh.html
      8. http://linuxmeister.net/Scripts/while-loop.sh.html
      9. http://linuxmeister.net/Notes/VM_setup_CENTOS_7/ALL.html - coming soon
      10. http://linuxmeister.net/Notes/Networking-btwn-VBox-VMs.html - coming later

      note: there was no session November 18, 2015

      Session #41 - November 11, 2015 - SHORT SESSION: VirtualBox - discussion continues

      Short session today... starting 11:15, ending before noon... Volunteered for a Veterans' Day activity...
      1. http://linuxmeister.net/Notes/VirtualBox-notes.html
      2. http://linuxmeister.net/Intro-to-Linux/Slides/ALL.html
      3. http://linuxmeister.net/Notes/bashrc-simple.html
      4. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html
      5. http://linuxmeister.net/Intro-to-Linux/lab-exercise-2.html
      6. http://linuxmeister.net/Intro-to-Linux/lab-exercise-with-VM-servers.html
      7. http://linuxmeister.net/Scripts/chksys.sh.html
      8. http://linuxmeister.net/Scripts/while-loop.sh.html


      Session #40 - November 4 2015 - Q and A - VirtualBox - lab exercise

      QandA - VirtualBox - lab exercises - review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
      (continuing QandA and review) working with the command line once you have a VirtualBox Linux shell.
      Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
      http://linuxmeister.net/Notes/zypper-update-ps.html
      http://linuxmeister.net/Notes/apt-get-update-upgrade.html

      1. http://linuxmeister.net/Notes/VirtualBox-notes.html
      2. http://linuxmeister.net/Intro-to-Linux/Slides/ALL.html
      3. http://linuxmeister.net/Notes/bashrc-simple.html
      4. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html
      5. http://linuxmeister.net/Intro-to-Linux/lab-exercise-2.html
      6. http://linuxmeister.net/Intro-to-Linux/lab-exercise-with-VM-servers.html
      7. http://linuxmeister.net/Scripts/chksys.sh.html
      8. http://linuxmeister.net/Scripts/while-loop.sh.html


      Session #39 - October 28, 2015 - using the VirtualBox - lab exercise

      Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
      1. http://linuxmeister.net/Notes/VirtualBox-notes.html
      2. http://linuxmeister.net/Intro-to-Linux/Slides/ALL.html
      3. http://linuxmeister.net/Notes/bashrc-simple.html
      4. http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html
      5. http://linuxmeister.net/Intro-to-Linux/lab-exercise-2.html
      6. http://linuxmeister.net/Intro-to-Linux/lab-exercise-with-VM-servers.html
      7. http://linuxmeister.net/Scripts/chksys.sh.html
      8. http://linuxmeister.net/Scripts/while-loop.sh.html


      Session #38 - October 21, 2015 - SSH One Command Setup, C programming Overview & CENTOS VirtualBox install

      1. ONE LINE SSH setup! http://linuxmeister.net/Notes/Setup-SSH-Key-on-ONE-line.html
      2. brief C programming overview: http://linuxmeister.net/Notes/C.html
      3. live SETUP CENTOS VM: http://linuxmeister.net/Notes/VirtualBox-notes.html


      Session #37 - 14 October 2015 - Troubleshooting Overview

      How do you solve system problems? What tools do you use? Is there a simple way of determining the problem?
      http://linuxmeister.net/Notes/Systems-Adminstration-Overview.html

      Session #36 - 7 October 2015 - VirtualBox Overview

      VirtualBox basics - setup and uses http://linuxmeister.net/Notes/VirtualBox-notes.html

      Session #35 - 30 September 2015 - Systems Administration Overview

      so you think you want to be a Sys Admin? http://linuxmeister.net/Notes/Systems-Adminstration-Overview.html

      Session #34 - 9 September 2015 - System Configuration Files

      What files and directories are important? http://linuxmeister.net/Notes/System-Configuration-Details.html

      Session #33 - 2 September 2015 - GRUB and Wrapper Scripts

      1. http://linuxmeister.net/Notes/date-suffix.html
      2. http://linuxmeister.net/Scripts/wrapper-script.html
      3. http://linuxmeister.net/Notes/grub.cfg-example.html
      4. http://linuxmeister.net/Notes/grub2-notes.html


      Session #32 - 19 August 2015 - How to Dual Boot Microsoft Windows 10 and SUSE Linux 13.2

      1. http://linuxmeister.net/Microsoft/Dual-Boot-Win10-SUSE13.2/


      Session #31 - 12 August 2015 - Security Basics

      The vulnerabilities, exploits, attack vectors, actual risks and payoffs
      1. http://linuxmeister.net/Notes/SecurityOverview/


      Session #30 August 5, 2015 - CSV, XML, Spreadsheets and the use of AWK to filter output

      1. https://tools.ietf.org/rfc/rfc4180.txt
      2. http://www.w3schools.com/xml/
      3. https://msdn.microsoft.com/en-us/library/bb387090.aspx
      4. http://linuxmeister.net/Commands/awk-example.html


      Session #29 July 8, 2015 - performance analysis tools

      http://linuxmeister.net/Notes/Tuning-Linux.html


      Session #28 June 24, 2015 - as requested: REAL WORLD SCRIPTING

      practical scripts: creating, dating, renaming, numbering and managing files
      1. http://linuxmeister.net/Scripts/Lab-exercise-create-text-man-pages-for-vi-practice/ - a basic loop to create man pages for plain text reading
      2. http://linuxmeister.net/Scripts/man-page-create-textfiles.sh.txt - the script for the exercise above
      3. http://linuxmeister.net/Scripts/replace-spaces-plus.txt - using tr, sed and mv to change case and/or remove spaces in file names
      4. http://linuxmeister.net/Scripts/removing-spaces-and-special-characters.html - expansion of prior script, includes dealing with special characters
      5. http://linuxmeister.net/Scripts/filedate.sh.html - using grep, sort, awk, sed to create a list of files to be "mv"'d (renamed) with a date PREFIX for sorting
      6. http://linuxmeister.net/Scripts/number-file-names.sh.html - create a number prefix for files to aid in sorting, used for mp3 files.
      7. http://linuxmeister.net/Scripts/mail-list.sh.html - use for loop to send mail/files to several friends one by one
      8. http://linuxmeister.net/Scripts/mk-webpage-2015-05-08.txt - create a web page in a directory of pictures
      9. http://linuxmeister.net/Scripts/mk-timelapse-avi-on-2-sites-2015-05-08.txt - one of the scripts to create time lapse videos from pictures
      10. http://linuxmeister.net/Scripts/chksys.sh.html - check on the health of your system, including disk usages (drives fill up fast with images)



      Session #27 June 17, 2015 - server name link changes, drive fail/system rescue

      1. http://linuxmeister.net/Notes/Update-amazon-ad-links.html
      2. http://linuxmeister.net/Notes/failing-drive-system-rescue.html


      Session #26 June 10, 2015 - Networking basics, analysis/tools and differences

      1. http://linuxmeister.net/Notes/basic-networking.html
      2. http://linuxmeister.net/Notes/networking-analysis-and-tools.html
      3. http://linuxmeister.net/Notes/networking-differences-by-distros.html
        differences in network and basic commands between SuSE, MUD and RedCents) (MUD=Mint, Ubunutu, and Debian)


      Session #25 June 3, 2015 - /var/log/messages - errors, fixing... attacks, stopping... good networks...

      1. http://linuxmeister.net/Notes/slapd-unsupported-operation.html
      2. http://linuxmeister.net/Notes/using-iptables-to-block-spammers-or-attackers.html
      3. http://linuxmeister.net/Notes/Network-Analysis-and-Observations.html


      NOTE: NO CLASS on May 27, 2015

      insomnia prevention classes resumes on 3 June 2015 with Session #25!

      Session #24 May 20, 2015 - NOTE: (no class on the 27th)

      conclusion: Detailed analysis of TimeLapse scripts, page creation and crontab

      1. http://linuxmeister.net/Notes/Setup-TimeLapse-Creation.html - what failed...
      2. describe the troubleshooting process used to learn why the cron job failed and how a "hunch" solved a "newly created" problem.
      3. http://linuxmeister.net/Notes/ManagingFiles-ssh-rsync.html setup of ssh, using rsync... or scp... and
      4. http://linuxmeister.net/Notes/wget-n-crontab.html when to use wget...
      5. http://linuxmeister.net/Notes/ftp-fixes-for-vsftp.conf.html and when to use ftp.
      6. http://linuxmeister.net/Microsoft/Setup-TaskManager-AutoReboot.html Comparing a Microsoft captive user interface (GUI) "cron job" to cron, at and nohup.
      7. in other news: headhunter emailed, looking for: Intermediate to Expert (command line heavy) knowledge of one or more of the following Unix operating systems:
        70%: AIX v.5.3 and/or 7.1 - 25%: SUSE (v.11 Enterprise) - 5%: Red Hat Enterprise Linux (v.11 (ed.???) Working with Core Technologies (Overall 10%):
        Working familiarity with: Apache Server and/or Tomcat - OpenSSH (Unix) and SSL (Windows + Unix; configuration + troubleshooting)
        - S/FTP (Unix-based configuration + troubleshooting) - Moderate shell scripting ability (Perl considered a plus)
      8. references: http://linuxmeister.net/Commands/MAN/man.chmod.txt, http://linuxmeister.net/Commands/MAN/man.at.txt, http://linuxmeister.net/Commands/MAN/man.nohup.txt,
        http://linuxmeister.net/Commands/MAN/man.ftp.txt, http://linuxmeister.net/Commands/MAN/man.ssh.txt, http://linuxmeister.net/Commands/MAN/man.scp.txt,
        http://linuxmeister.net/Commands/MAN/man.ssh.txt, http://linuxmeister.net/Commands/MAN/man.wget.txt, http://linuxmeister.net/Commands/MAN/man.rsync.txt,
        http://linuxmeister.net/Commands/MAN/man.bash.txt, etc...

      Session #23 May 13, 2015 - Detailed analysis of TimeLapse scripts, page creation and crontab

      http://linuxmeister.net/Notes/Setup-TimeLapse-Creation.html

      Session #22 May 6, 2015 - follow up of LVM, discussion of scripts

      http://linuxmeister.net/FileSystems/setup-LVM.html

      Session #21 Apr 29, 2015 - resize2fs and LVM

      http://linuxmeister.net/FileSystems/setup-LVM.html

      Session #20 Apr 22, 2015 - LVM configuration

      http://linuxmeister.net/FileSystems/setup-LVM.html

      Session #19 Apr 15, 2015 - ftp, vsftp, scp, ssh


      Session #18 - Apr 8, 2015 - Systemd

      http://linuxmeister.net/Notes/systemd-info.html systemd and
      the general boot up process of Linux systems. (comparing /etc/init.d/)
      for additional reading: https://www.novell.com/docrep/2015/01/systemd_in_suse_linux_enterprise_12_white_paper.pdf

      Session #17 - Apr 1, 2015 - proxy/vnc

      http://linuxmeister.net/Notes/vnc-setup.html
      http://linuxmeister.net/Notes/SSH-setup-and-details.html
      http://linuxmeister.net/Scripts/run-now-on-all-hosts.sh.html

      Session #16 - Mar 25, 2015 - ssh, scp, and rsync

      use of ssh to examine, monitor and scp to copy files, create a script and use rsync
      to manage several systems from one. Walk through the process and establish a wrapper script to make archive copies on each system.
      http://linuxmeister.net/Notes/SSH-setup-and-details.html
      http://linuxmeister.net/Scripts/run-now-on-all-hosts.sh.html

      Session #15 - Mar 18, 2015 - systemd, tree, ssh and scp

      discussion on systemd, tree command, use ssh to examine and scp to copy
      files, create a script and use rsync to manage several systems from one.

      Session #14 - Mar 11, 2015 - general discussion on Linux shells, commands - open discussion with group


      Session #13 - Mar 4, 2015 - 1) general Linux info, 2) basics, and 3) advanced topics


      Session #12 - Feb 25, 2015 - The power of the command line: grep

      Discussion of GREP: example: grep -v ^[[:space:]]$ and grep -v [[:space:]]

      Session #11 - Feb 18, 2015 -The basics of Apache2 configuration, sed examples and use.

      1) http://linuxmeister.net/Notes/?C=M;O=D (general updates can be seen near the top of this page)
      2) http://LinuxMeister.net/Notes/basic-Apache-setup.(basic install and config, and major config files)
      3) http://LinuxMeister.net/Notes/apache-tricks.(virtual server configurations and settings)
      4) http://linuxmeister.net/Notes/apache-tomcat-topic-list.(the list of modules used for development purposes)
      5) http://linuxmeister.net/Notes/setup-LAMP-on-MINT.(adding apache to Ubuntu, Debian or Mint - not there by default)

      Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic

      Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic- wget, crontab, ls , vi, mencoder, sed, grep, cat, tac, lsof, netstat.

      1) http://linuxmeister.net/Notes/?C=M;O=D
      2) http://meistertech.net/Traffic/TUES-PM/00_Tues-PM.html
      3) http://linuxmeister.net/Notes/wget-n-crontab.html
      4) http://meistertech.net/Traffic/
      5) http://meistertech.net/Traffic/files.txt
      6) http://meistertech.net/Traffic/Make-Commute-pages.txt
      7) http://linuxmeister.net/Notes/setup-in-irfanview.jpg
      8) http://linuxmeister.net/Notes/mencoder-info.html
      9) http://linuxmeister.net/Notes/zypper-install-mencoder.html
      10) http://linuxmeister.net/Notes/cat-tac-and-sed.html
      11) http://linuxmeister.net/Notes/Network-Analysis-and-Observations.html

      Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.

      Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.

      1) http://linuxmeister.net/Notes/SSH-setup-and-details.html
      2) http://linuxmeister.net/Notes/RSYNC-cmd.html
      3) http://LinuxMeister.net/Notes/ManagingFiles-ssh-rsync.html
      Will discuss details on how to configure directory structure on Windows and Linux systems to synchronize a set of Files on several computers.
      Will discuss setup of bi-directional ssh passwordless connection between two Linux systems, and RSYNC configuration.
      Will discuss wrapper scripts to make backup copies for files that will be rotated and replaced, preventing redundant or corrupt files.
      http://linuxmeister.net/Intro-to-Linux/Notes/FileMgtPlan.txt

      If we have time we'll also discuss WinSCP and how it can be used from Microsoft systems. Emphasis will be on the Linux aspect of
      ssh and rsync, but also on general file management and config mgt for end users.
      If time, will discuss SSH tunneling through firewalls/VPNs/NATs: http://linuxmeister.net/Notes/ssh-tunnel.jpg

      NOTE: updated Gparted page from last week: http://linuxmeister.net/Notes/Gparted-for-Recovery/ALL.html


      Session #8 - 28 January 2015 - discuss the use of cal, grep, tee and the use of a simple loop in a shell

      Work with cal, grep, and a do-while loop in a BASH script to determine if it there is truth to the internet claim that:
      December 2012 having 5 Saturdays, 5 Sundays and 5 Mondays only happened once every 823 years!
      Commands covered: cal, cat, man, more, ls, grep, tee and basic do while loops and tests in BASH
      http://LinuxMeister.net/Intro-to-Linux/28jan2015/grep.help.txt
      http://linuxmeister.net/Scripts/BUST-MYTHS-WITH-CAL-GREP/ http://linuxmeister.net/Commands/grep-awk-uniq.
      http://linuxmeister.net/Overview/LinuxOverview.pdf
      Brief discussion on recovery of /home on a failing or corrupted system:
      http://linuxmeister.net/Notes/Gparted-for-Recovery/ALL.
      http://LinuxMeister.net/Intro-to-Linux/28jan2015/how-to-preserve-home.txt
      tar cvfz home.tar /home(note: will need to be root or use sudo, make sure you have enough space to create the file)

      Session #7 - 21 January 2015

      http://opensuse-guide.org/installation.php https://software.opensuse.org/132/en https://en.opensuse.org/Portal:Installation
      https://activedoc.opensuse.org/book/opensuse-start-uphttps://activedoc.opensuse.org/book/opensuse-start-up/chapter-1-installation-quick-start
      http://linuxmeister.net/http://linuxmeister.net/Overview/LinuxOverview.pdf
      https://www.smashwords.com/profile/view/johnmeister https://www.smashwords.com/books/view/505731

      Session #6 - 14 January 2015

      http://LinuxMeister.net/Intro-to-Linux/ misc notes and screenshots found here… not organized.
      Files created during session found here. In particular http://LinuxMeister.net/Intro-to-Linux/14jan2015/reg-exp-users.txt
      demonstrated the use of “script� and the creation of the http://LinuxMeister.net/Intro-to-Linux/14jan2015/reg-exp-users.RAW file,
      which was then filtered using cat {filename}.RAW | col -b > {filename}.txt
      Regular Expressions and the power of the command line.Continued demonstrating filtering. http://LinuxMeister.net/Intro-to-Linux/
      Demonstration of: grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo, man, --help, ifconfig, dmesg and which

      Session #5 - 7 January 2015 http://LinuxMeister.net/Intro-to-Linux/

      Demonstration of: grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo, man, --help, ifconfig, dmesg and which

      SESSION #4 - December 17, 2014: pkg mgt, rsync, wine, mount

      http://linuxmeister.net/PDF-info/PackageManagementCheatSheet.pdf
      http://linuxmeister.net/FileSystems/newdisk-setup.txt http://linuxmeister.net/Notes/USB-live-Gparted-setup.txt
      http://linuxmeister.net/Notes/Setup-SuSE-13.2-select-screens/ALL.
      http://linuxmeister.net/Notes/zypper-installs. http://linuxmeister.net/FileSystems/setup-LVM.txt
      http://linuxmeister.net/Commands/rsync-example-man. RSYNC basics
      http://linuxmeister.net/FileSystems/mount-external-drive-rysnc.txt
      http://linuxmeister.net/FileSystems/rsync-disks-cron.sh.txt
      http://linuxmeister.net/Notes/Firefox-about-support-Tools. Firefox about features showing add-ons, in the URL type: about:support
      http://linuxmeister.net/Notes/wine-examples-manpage. using wine - some examples and the man page

      SESSION #3 - December 10, 2014 - basic commands

      discuss ssh files, show how to use Gparted, exercise on real world Linux commands, discuss VNC setup
      continue discussion on regularly used commands, e.g. grep, sed, awk, sort, uniq, file permissions and chmod, chown
      exercise #2 - real world examples of commands and file permissions.
      ---------------------------------------------------------------------------- Links:
      1. http://linuxmeister.net/Overview/
      2. http://linuxmeister.net/Intro-to-Linux/Special-Characters.pdf
      3. http://linuxmeister.net/Notes/bashrc-the-dotfile.html
      4. http://linuxmeister.net/Notes/SSH-setup-and-details.html
      5. http://linuxmeister.net/Notes/ssh-authorized-keys-setup-example.html
      6. http://linuxmeister.net/Notes/Gparted-for-Setup-or-Rescue.html
      7. http://linuxmeister.net/Notes/vnc-setup.html
      8. http://linuxmeister.net/Notes/Real-world-Linux-Commands.html
      9. http://linuxmeister.net/Intro-to-Linux/lab-exercise-2.html


      SESSION #2 - December 3, 2014 - SAW:"simple always works"

      provided link to exercise #1, discussed environment (.bashrc), path, chmod, discussed "SAW" -
      SAW:"simple always works", History files viewed with sort, uniq, and how to create notes for reference: ls, sort, grep, uniq, wc -l
      (after using a command, use history recall, add echo and quotes around it, and then append to ~/bin/cool-commands.txt e.g.
      echo "ls *.jpg > ALL.html ; perl -pi -e 's/(.*)/<img src="$1"><BR>$1<HR>/g' ALL.html ; perl -pi -e 's/<img src=""><BR><HR>//g' ALL.html ; cat ALL.html" >> ~/bin/cool-commands.txt

      ----------------------------------------------------------------------------
      Links: http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html
      (do this on your system, use "script exercise-1.raw",
      when finished type exit, then cat exercise-1.raw | col -b > exercise-1.txt - then edit in vi and save)
      http://linuxmeister.net/Scripts/chksys.sh.html (use the vi editor to create and run on your system)
      http://linuxmeister.net/Scripts/man-page-create-textfiles.sh.txt (mkdir and cd LAB, then create this script in vi and run)
      http://linuxmeister.net/Commands/grep-awk-uniq.html (the output of the script above is needed to use the commands, YMMV, counts may be different)


      SESSION #1 - November 26, 2014 - OVERVIEW

      	 provided overview of Linux using the pdf: http://linuxmeister.net/Overview/LinuxOverview.pdf  
      	5 basic commands: man, ls (ls -al, ls -Al), cd, pwd, more 
      	discussed .bashrc and showed a few script examples, talked about "script"
      ----------------------------------------------------------------------------
      Links:	http://linuxmeister.net/Overview/LinuxOverview.pdf    (print out and use as a guide)
      	http://linuxmeister.net/Intro-to-Linux/Special-Characters.pdf (print out and keep as a guide)
      	http://linuxmeister.net/Notes/bashrc-the-dotfile.html  (copy and build your own .basrhc for Cygwin or Linux)
      	http://linuxmeister.net/Scripts/chksys.sh.html  
      	http://linuxmeister.net/Intro-to-Linux/lab-exercise-1.html  (use the commands and the vi editor)
      

      THE CORE MATERIAL:The power of the command line - Simply Linux: Basics will work our way through the Linux book (Simply Linux: Basics) under construction based on http://linuxmeister.net/Overview/ http://linuxmeister.net/Overview/LinuxOverview.pdf (print out and use as a guide) http://linuxmeister.net/Overview/Linux-PowerPoint-2004-overview.pdf http://linuxmeister.net/Notes/Real-world-Linux-Commands. http://linuxmeister.net/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide) http://linuxmeister.net/Notes/bashrc-the-dotfile. (copy and build your own .basrhc for Cygwin or Linux)

      ONE HOUR Linux SESSIONS



Simply Linux: Basics  Full Size Jeep Buyer's Guide Using BASH on Windows 10
Practical Suggestions for Microsoft Windows
Linux Tackles Microsoft
12 hour Video Course by john:
The Art of Linux System Administration
published by O'Reilly Media
Study Guide for the LPIC-2 Certification Exams
search for:
on the internet, or:
JohnMeister.com-fotos
LinuxMeister.net-Linux
BibleTech.net- Bible overview

overview of mankind's history
Biblical history:
"Promises and Prophets"

Wagoneers

FULL SIZE JEEPS

JeepMeister.com
"Jeep is America's
only real sports car."
-Enzo Ferrari


MeisterTech.net
Diesels +

One Page Overview of Linux Commands

click for an image of the 5 essential Linux commands

An Intro to Linux
AMSOIL product guide,
or, AMSOIL web, or 1-800-956-5695,
use customer #283461

Amsoil dealer since 1983
purchase AMSOIL
at Midway Auto on SR9 in Snohomish,
or at Northland Diesel in Bellingham, WA


SJ - 1962-1991

XJ - 1984-2001

WJ - 1999-2004

KJ - 2002-2007

WK - 2005-2010

Find the recommended
AMSOIL synthetics
for your Jeep

CJ-10A - 1984-1986

Jeepsters

MJ - 1984-1992

Willys - 1946-1965

Other Jeeps (FC)