|
Session #115 - December 15, 2017 - FRIDAY - using find and perl
|
note on html: color for cell above is "ivory" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue (can use names too...)
| NOTE: LAST SESSION OF 2017 |
using find and perl to update HTML pages for URL correction and consolidation of domains
http://johnmeister.com/linux/SysAdmin/Using-find-n-perl-to-manage-URLs.html
also: time, grep, col, and Apache: /etc/apache2/errors.conf and creating a missing.html page
|
Session #114 - December 8, 2017 - FRIDAY - Oracle VirtualBox and LVM
|
note on html: color for cell above is "yellow" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue (can use names too...)
Expanding the capactity of a dynamic vdi disk in Oracle's VirtualBox
Built a default Linux 64bit VM with Centos, however, needed more space,
had to increase the capacity from the default of 8GB to about 14GB.
https://www.virtualbox.org/
http://johnmeister.com/linux/SysAdmin/Virtualization/VM_setup_SuSE_13.2/ALL.html
Steps to increase capacity of a vdi drive in VirtualBox (Linux or MacOSx host) at the COMMAND LINE:
-->sudo VBoxManage modifyhd /home/luser/VirtualBox_VMs/Centos-Lab/Centos_Lab.vdi --resize 14480
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Once that was done, checked settings in VirtualBox; was good... but local drive in VM still not expanded.
FROM INSIDE THE VM, at the command line:
pvs ; pvdisplay ; vgs ; vgdisplay; lvs ; lvdisplay # displays system values
fdisk /dev/sda # create a partition in the "expanded" virtual drive
p, n, p, 3, t, 82e, w
pvcreate /dev/sda3 ; pvs; vgs # create the physical volume, check it
vgextend centos_centos7-vm /dev/sda3 # extend the existing volume group into the expande physical drive
vgs ; pvscan ; lvdisplay ; df -h; vgdisplay
lvextend /dev/mapper/centos_centos7--vm-root /dev/sda3 # extend the logical volume into the expanded volume group
df -h . ; vgs ; lvs
resize2fs /dev/mapper/centos_centos7--vm-root ## FAILURE - bug in RH LVM and/or XFS (should have extended the file system)
## FAILURE - bug in RH LVM and/or XFS
--> resize2fs /dev/centos_centos7-vm/root (FAILURE IN RH-type system and xfs)
resize2fs 1.42.9 (28-Dec-2013)
resize2fs: Bad magic number in super-block while trying to open /dev/centos_centos7-vm/root
Couldn't find valid filesystem superblock.
FIX:
--> xfs_growfs /dev/centos_centos7-vm/root
http://johnmeister.com/linux/FileSystems/ADD-6GB-to-vdi-CentOS-VM.html
process: Order of precendence and logic: - PHYSICAL - VOLUME - LOGICAL
PHYSICAL VOLUME(s): pvs; pvdisplay; pvresize
VOLUME GROUP(s): vgs; vgdisplay; vgextend Add physical volumes to a volume group
LOGICAL VOLUME(s): lvs ; lvdisplay; lvextend Add space to a logical volume
http://johnmeister.com/linux/FileSystems/setup-LVM.html
http://johnmeister.com/linux/FileSystems/lvm-commands.html
|
Session #113 - December 1, 2017 - FRIDAY - RECAP of 3 years of Linux at lunch!
|
note on html: color for cell above is "fedcba" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue (can use names too...)
recap of 3 years worth of sessions
http://johnmeister.com/linux/Intro-to-Linux/session-list-2017.html
|
Session #112 - November 10, 2017 - FRIDAY - KEY WORD script
|
note on html: color for cell above is "ABCDEF" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue (can use names too...)
NOTE: no session Friday 3 Nov 2017
Session #112 - Nov 10, 2017 - FRIDAY - Linux at Lunch - KEY WORD SCRIPT walk through!
The Key Word Script uses regular expressions:
grep, sed, awk, cut, uniq, sort, perl, tr and vi to extract KEY WORDS from a text document!
Key Words are those of proper names and places; words beginning with capital letters.
The output of this script produces two Key-Word variants; one that shows only the line where the word appears,
and the other shows the context, grep -C 1, listing the line before and the line after.
It's possible to insert HTML coding into this script to create a web page, e.g.:
with:
with: http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html
There are GUI based tools like WordPress that might create nicer pages...
but these tools have security issues, require a database, are not trivial to install, learn, use or manage.
SIMPLE ALWAYS WORKS.
http://johnmeister.com/bible/BibleSummary/KEY-WORDs/GET-KEY.sh.txt
(will create a version of this on johnmeister.com/linux to work with generic text files.)
------------------------------------------------------------
#!/bin/bash
### john meister 29 october 2017 - copyright 2017
### script to find Key words - upper case - Who and Where
USAGE="GET-KEY.sh.txt , e.g. GET-KEY.sh.txt 02n_Exo.txt"
#########################################################################
cat $1 | tr -d '[{}()\;\":,\!\.\?]' > T2
### remove most special characters cat T1, filter, save as T2
perl -pi -e 's/ /\n/g' T2
### put all words on one line
grep -v "^'" T2 > T3
### remove leading ticks from list save as T3
cat T3 | sort | uniq | grep -v ^[a-z] | grep -v ^[0-9] > T4
### sort, remove lc and numbers, save as T4
rm -f T1 T2 T3
### deleting temporary files
##########################################################################
echo "edit T4 in vi to remove non-keywords; :wq will resume; hit any key to start"
read
vi T4 # T4 will be moved to ZLIST-names when done
#########################
OUT="KEY-names-with-VS-$1" # Key names with counts and verses
OUT2="TIMES-names-$1" # Key names with counts only
cat /dev/null > $OUT
for Z in `cat T4`
do
echo "----------------------------------------------------------------------" | tee -a $OUT
grep $Z $1 | wc -l > T1
####################
echo "The word \"$Z\" in \"$1\" is found `cat T1| cut -c 5-9` times." | tee -a $OUT2
echo "======================================================================" | tee -a $OUT2
####################
echo "The word \"$Z\" in \"$1\" occurs in these verses `cat T1| cut -c 6-9` times:" | tee -a $OUT
echo "----------------------------------------------------------------------" | tee -a $OUT
grep -C 1 $Z $1 | tee -a $OUT
echo "======================================================================" | tee -a $OUT
done
###########################################################
mv T4 ZLIST-names-$1 ; rm -f T1
##########################################################
So, the extract files look like this for the following command(s):
-------------------------------------------------------
GET-KEY.sh.txt 01k_Gen.txt
KEY-names-with-VS-01k_Gen.txt 2017-10-30 00:40 1.2M
TIMES-names-01k_Gen.txt 2017-10-30 00:40 69K
ZLIST-names-01k_Gen.txt 2017-10-30 00:40 3.8K
-------------------------------------------------------
and
-------------------------------------------------------
GET-KEY.sh.txt 01n_Gen.txt
KEY-names-with-VS-01n_Gen.txt 2017-10-30 00:40 1.2M
TIMES-names-01n_Gen.txt 2017-10-30 00:40 69K
ZLIST-names-01n_Gen.txt 2017-10-30 00:40 3.9K
-------------------------------------------------------
It's interesting to see the names and places listed in just ONE of the 66 books:
http://johnmeister.com/bible/BibleSummary/KEY-WORDs/01-GENESIS/ZLIST-names-01k_Gen.txt
In addition, I want to modify the output to only show the references:
http://johnmeister.com/bible/BibleSummary/KEY-WORDs/01-GENESIS/KEY-names-with-VS-01k_Gen.txt
e.g.
----------------------------------------------------------------------
The word "Achbor" in "01k_Gen.txt" occurs 2 times: Gen 36:38; Gen 36:39
======================================================================
or:
----------------------------------------------------------------------
The word "Achbor" in "01k_Gen.txt" occurs 2 times:
Gen 36:38; Gen 36:39
======================================================================
SIMPLE is portable, secure and easily modified; but it may take more effort and understanding,
and may include more "steps" and sometimes requires manual intervention; NOTICE the USE of "read" in the
script above. I had to manually edit the "key word" list to remove unimportant words like
"The", "And", "Because", etc. And a few other items that didn't filter out properly.
It is possible that I could create a "heredoc" to filter those words, but after converting the strings
to single words with new lines any grep -v filters would take out the rest of any words starting out
with the same beginning (e.g And would remove Andover, etc.).
That would push the filtering into the text prior to the new line insertion,
and since one has no idea which words are "important" or of interest, better to remove them manually.
One other thing, when running the script and trying to make it work, you might want to save the "key word"
list by a different name, then when editing the list you can delete the contents and read in what you
already edited. Keep the copy file name simple and easy to remember. It'll take a few iterations to dial it in.
|
Session #111 - October 27, 2017 - FRIDAY - two scripts
|
note on html: color for cell above is "tan" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue (can use names too...)
NOTE: no session NEXT week, Friday 3 Nov 2017
two scripts
work flow process for digital images and script to create web page with thumbnails
workflow process:
- download images from SD card to computer via scripts
- review original images; minor edits - resize, rename with dates - keeping RAW impages untouched
- resized/renamed images go up one directory. review which ones will go on web page
- move resized/renamed images to web directory
- scp -r web directory to server
- ssh to server, run "mkpg" script http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html
- add "exiftool" info with extract into directories: (can be done on Mac or either Linux system)
--> more exif-strip.sh
#!/bin/bash
# uses exiftool to extract EXIF info in original images
#
for x in `ls *`
do
exiftool $x > exifinfo-$x.txt
cat exifinfo-$x.txt| grep -E -v '(Brightness|Timezone|ISO2|Zone|Bracket|Image|Artist|Crop|Sensitivity|\
Preview|Difference|Shift|MIME|Tuning|Clarity|Serial|Mode|Modify|Distance|Confusion|Gain|Tune|Phase|CFA|Depth|\
Hyper|Light|Mega|Red|Blue|GPS|Normal|TIFF|Jpg|Binary|Sub|Memory|Original|File|Directory|Sensing|Size|Power|\
Multi|Raw|Contrast|Vari|Retouch|Max|Compression|Exit|Comment|Primary|MCU|Version|Shot|Position|Picture|Black|\
Control|Points|Expansion|Zoom|Toning|Hue|Filter|Format|Min|Color|Quality|Levels|Flash|Scene|Other|Planar|Resolution|\
Copyright|Strip|Balance|Bits|Stops|Samples|Software|Orientation|Byte|Make|Savings|Scale)' > base-exif-$x.txt
done
mkdir ../EXIFINFO
mv exifinfo-*.txt ../EXIFINFO
mv base-exif-*.txt ../EXIFINFO
- created and add thumbnails
--> convert -thumbnail 400 somepicture.jpg tn-somepicture.jpg
Man page says "-thumbnail geometry" - substitute geometry for value...
--> man convert | col -b > man.convert.txt
http://johnmeister.com/linux/Commands/man.convert.txt
- Current thought is to incorporate thumbnails into the "mkpg" script,
and also incorporate select parts from the exiftool and exif data.
- will likely use tables to put each thumbnail with the title/link below
along with the exif data for exposure info.
- Each thumbnail would be linked to open in a new tab or window,
an alternate would be to use javascript to do a mouse hover to show image.
It's "simple" to use a tool like WordPress or other apps to do this...
but those tools have security issues, sometimes costs and become
obsolete or not installed...
KNOWING HOW to do something is best, even if it doesn't look as good.
2) Creating a script to extract Proper Names and Places from text files
- using sed, grep, tr, perl, sort and uniq
From prior sessions there were plain text files created from two translations.
The 66 books have been separated into 8 sections and have two plain text translations
to work with, or 132 text files to parse.
The goal is to find the proper names and places which begin with an upper case character
and extract those names from the text into a file.
That file of names will have to be manually edited to remove other upper case words,
such as "The", "Also", "Before", and so on.
I've tried to use grep to remove them. However if a word on a line contains the
first letters of another word the grep -v will also remove those words.
So manually editing 132 files is less trouble than working an automated solution.
If this were an ongoing effort with continous data coming in from a regular test or email,
then I would take the time to determine a solution.
In prior sessions we've used wget to gather these files and worked them into one line per verse text files.
The project continues to work with those files to extract specific information. Key names and places.
current flow:
- Have gathered the text files for the 66 books and put them into the 8 major sections:
# 1-LAW - 5 books - 10 files - identify and list proper names showing number of times and verses
# 2-HISTORY - 12 books - 24 files
# 3-WISDOM - 5 books - 10 files
# 4-PROPHETS - 17 books - 34 files
# 5-GOSPEL - 5 books - 10 files
# 6-PAUL - 13 books - 26 files
# 7-LTRS - 8 books - 16 files
# 8-REV - 1 book - 2 files
- take each of the two files per book and create a copy: --> cp 02n_Exo.txt 2N ; cp 02k_Exo.txt 2K (do not edit 02?_Exo.txt)
- optional: count the lines in the copy: --> cat 02n_Exo.txt | wc -l ; cat 2N | wc -l ; cat 02k_Exo.txt | wc -l ; cat 2K | wc -l
- use perl and sed to put each word in the file on one line: --> perl -pi -e 's/ /\n/g' 2N
- repeat for the other file for each book, e.g. Exodus: --> perl -pi -e 's/ /\n/g' 2K
- clean 2N and 2K to remove blank lines, words starting with lower case, special characters and verse numbers:
--> cat 2N | sort | uniq | grep -v ^[a-z] | grep -v ^[0-9] | tr -d '[{}()\;\":,\!\.\?]' | sort | uniq | grep -v ^\' > 2Na
note: found that sort and uniq repeated was needed after removing special characters... will test with sorts at end.
- Keep 2N original to grep for the KEYWORDS with the short list found in 2Na.
- created script with for loop to extract keywords:
#!/bin/bash
OUT="V-2N.txt" # output file
for x in `cat 2Na` # create variable x from the keyword list created in "2Na"
do
echo $x | tee -a $OUT # list keyword
echo "------------------------" | tee -a $OUT
grep $x 2N | tee -a $OUT # display each line with the keyword
echo "------------------------" | tee -a $OUT
done
ls $OUT
- Then added the count and a better separator:
--> more find-names-count-in-N2
#!/bin/bash
OUT="V-2N.txt"
cat /dev/null > $OUT
for x in `cat 2Na`
do
echo $x | tee -a $OUT
echo "------------------------" | tee -a $OUT
echo $x | grep $x 2N | wc -l | tee -a $OUT
echo "------------------------" | tee -a $OUT
grep $x 2N | tee -a $OUT
echo "------------------------" | tee -a $OUT
echo "## KEY WORD, times found, verse(s) ##################################" | tee -a $OUT
done
ls $OUT
---------------------------
## KEY WORD, times found, verses ################################
Naphtali
------------------------
1
------------------------
Exo 1:4 Dan and Naphtali, Gad and Asher.
|
Session #110 - October 20, 2017 - FRIDAY - wget, advanced vi editing
|
note on html: color for cell above is "0F0F0F" (rrggbb), font color is "white"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
- we've covered .bashrc:
Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
- we've reviewed the "five essential" commands: man, ls, cd, pwd and more
- we've worked our way through the SPECIAL CHARACTERS - and will continue to...
- will discuss some issues of interoperability with different BASH environments,
e.g. MacOSx, different versions of BASH
- we'll work on building some web pages - include the use of sed, awk, tr and even some perl
|
Session #109 - October 13, 2017 - FRIDAY - Adventures with Special Characters
|
note on html: color for cell above is "F0F0F0" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
- we've been covering the elements of the .bashrc:
Path, Prompt, Permissions, History, Editor, Alias, Shell options and Functions
- we've been reviewing the "five essential" commands: man, ls, cd, pwd and more
- we've been working our way through the SPECIAL CHARACTERS - and will continue to this session...
- we'll discuss the scripts below - including the use of sed, awk, tr and even some perl
- used find, exiftool and reg exp tools to gather lens info from NEF files.
Special characters are powerful representations of shell functions and values.
Remove spaces and replace special characters:
http://johnmeister.com/linux/Scripts/fix-filenames.sh.html
#!/bin/bash
###############################################################################################
# removes spaces, special characters, slashes, and _-_ but NOT caps
###############################################################################################
for f in *
do
mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
done
examples of loops:
http://johnmeister.com/linux/Scripts/LOOPS-shell-examples.html
Use AWK in a loop to move large numbers of files on a Linux web server:
http://johnmeister.com/linux/Scripts/awk-to-rename-files.sh.html
- create temporary script, e.g. "vi fsj-rename"
(if you have more than one directory create a more structured script and place it in your ~/bin).
-
#!/bin/bash
# GOAL: rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
# DATE: 4/27/2004 - Author John Meister
# TEST: --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}'
# PRODUCED: mv fsj-digest-v1-2123.txt fsj-2123.txt
#
# PROCESS: --> ls fsj-digest-v1-????.txt | \
# awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
# PROCESS: edit file to include header info & comments
#
mv fsj-digest-v1-1800.txt fsj-1800.txt
mv fsj-digest-v1-1801.txt fsj-1801.txt
...
-------------------------
- THEN... execute the script by typing: sh ./fsj-rename
- check directory and delete temporary script: ls ; rm -f fsj-rename
|
Session #108 - October 6, 2017 - FRIDAY - Adventures in BASH
|
note on html: color for cell above is "00FFFF" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
Last week we discussed in detail the .bashrc file, and demonstrated some of the "five essential" commands:
man, ls, cd, pwd and more
This week: we will use regular expressions to replace special characters,
deal with bad file-naming conventions, and rename long file names with AWK.
FIRST - what are special characters?
Special characters are powerful representations of shell functions and values.
In the picture below there is a listing of the special characters and examples of their use.
Special characters must be understood.
PROBLEM: Someone created files and directories with spaces in the names, or worse, special characters.
In UNIX-like systems spaces at the command line represent various commands or options.
If a filename or directory has spaces,
then those spaces must be "escaped" with a "\" or encapsulating the entire filename in quotes.
This gets complicated in a hurry.
With this script, we will remove spaces and replace special characters:
http://johnmeister.com/linux/Scripts/fix-filenames.sh.html
#!/bin/bash
###############################################################################################
# 2014_05May_09 - john meister copyright © 2014 http://LinuxMeister
# script to remove spaces in file names and replace with underscore
# cleans up web pages by eliminating special characters and allows sorts to work properly
# to use place this script in your path and make it executable, then cd to the directory
# and execute by the saved name (fix-filename) - works on Linux and MacOS
###############################################################################################
# removes spaces, special characters, slashes, and _-_ but NOT caps
###############################################################################################
for f in *
do
mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/&/-n-/g' | sed 's/_-_/_/g'`
done
NOTE: we'll cover more loops as we go along, here's some examples of loops:
http://johnmeister.com/linux/Scripts/LOOPS-shell-examples.html
ISSUE: created a bunch of files with really long stupid names, however,
the good news is I had a "delimiter" that would work with AWK.
By using the "-" in the really long stupid file names I was able to use AWK
to simplify the filenames. This was a real world problem that was fixed.
Example of using AWK in a loop to move large numbers of files on a Linux web server:
http://johnmeister.com/linux/Scripts/awk-to-rename-files.sh.html
- create temporary script, e.g. "vi fsj-rename"
(if you have more than one directory create a more structured script and place it in your ~/bin).
-
#!/bin/bash
# GOAL: rename all files in a directory from "fsj-digest-v1-????.txt" to "fsj-????.txt"
# DATE: 4/27/2004 - Author John Meister
# TEST: --> ls fsj-digest-v1-2123.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}'
# PRODUCED: mv fsj-digest-v1-2123.txt fsj-2123.txt
#
# --> ls fsj-digest-v1-????.txt | awk -F - '{print "mv "$1"-"$2"-"$3"-"$4" " $1"-"$4}' > mv-digests.sh
# edit file to include header info & comments
#
mv fsj-digest-v1-1800.txt fsj-1800.txt
mv fsj-digest-v1-1801.txt fsj-1801.txt
mv fsj-digest-v1-1802.txt fsj-1802.txt
mv fsj-digest-v1-1803.txt fsj-1803.txt
mv fsj-digest-v1-1804.txt fsj-1804.txt
...
-------------------------
- THEN... execute the script by typing: sh ./fsj-rename
- check directory and delete temporary script: ls ; rm -f fsj-rename
If we have time we will use AWK to rename the history files discussed last week.
I noticed that I did not use the date command in a way that makes sorting work well.
Current format is: 2017_Mar_27_08:32.history
It would be better for sorting if it were 2017_03_Mar_27_08:32.history
The good news is that I used "_" in the name; thereore, I can modify the script above to change the file names.
Previously the date variable was set to:
HISTFILE="/home/luser/.History/`/bin/date '+%Y_%b_%d_%H:%M'.history`" ; export HISTFILE
Last week we modified .bashrc to fix this variable:
HISTFILE="/home/luser/.History/`/bin/date '+%Y_%m_%b_%d_%H:%M'.history`" ; export HISTFILE
THE DATE command option for alphabetical representation of the month is %b, while %m is the numeric value.
I like to see the Month in letters, but like to sort things properly.
By sorting YEAR, MONTH and then DAY it displays in chronological order.
If I wanted to keep the filename shorter I could replace the hour and minutes with just the PID,
or use the seconds, the actual time of the file is not important.
|
Session #107 - September 29, 2017 - FRI - Let's USE BASH on Windows 10! - Pt 3
|
note on html: color for cell above is "0000FF" (rrggbb), font color is "white"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
 Will transition FROM: "Let's use BASH on Windows 10!" - TO: --->  "Simply Linux: Basics"
We will focus more on the actual Linux commands, less on the Win10 environment as host.
We'll talk more about Regular Expressions and the vi editor. The commands, the core tools.
Not everyone will be able to use Ubuntu BASH in W10,
some will chose cygwin or other tools such as a VM or putty to a Linux system.
We will still discuss issues across platforms as we proceed.
It's important to note differences even between UNIX, Linux and other tools.
More will be shared during the session.
finish discussion of .bashrc and .History directory.
http://johnmeister.com/linux/bashrc-basic.html
and then - Discuss Chapter Two - Five Essential Commands.
Continuing in Chapter 2 of "Let's Use BASH on Windows 10", but shifting to the other book, same format...
Introduction to Linux
Chapter 1_Linux_Overview
Chapter 2_the_5_essential_commands
Chapter 3_the_vi_Editor
Chapter 4_shell_config_History_script
Chapter 5_File_Management:_cp_mv_rm
Chapter 6_Files, Processes and Special Characters
Chapter 7_Regular_Expressions_and_Filters
Chapter 8_ssh_setup_and_use
Chapter 9_Network_Basics
Chapter 10_Files_Systems:_df_du
...
Showing purpose with examples of settings in .bashrc:
PATH; PROMPT; PERMISSIONS; HISTORY; EDITOR; ALIAS; SHELL OPTIONS; FUNCTIONS
demonstrating "minor" issues and differences in paths, symlinks and command execution in BASH on Win10.
Will show how to create a copy of files in /mnt/c/
HOMEWORK EXERCISES:
- RECOMMEND REVIEWING:
http://johnmeister.com/linux/Notes/Philosophy_of_Linux.html
- there are significant differences between Microsoft and Linux that need to be understood.
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
|
Session #106 - September 22, 2017 - FRI - Let's USE BASH on Windows 10! - Pt 2
|
note on html: color for cell above is "00ff00" (rrggbb), font color is "black"
(red, green, and blue): colors between 0 and 255 (hex values= ff). 255,0,0 = red ; 0,255,0 = green ; 0,0,255 = blue
"Let's use BASH on Windows 10!" - Part 2 - Ch1, Ch2
- Finish up Chapter One - discuss .bashrc and .History directory. http://johnmeister.com/linux/bashrc-basic.html
- Discuss Chapter Two - Five Essential Commands.
HOMEWORK EXERCISES:
- RECOMMEND REVIEWING:
http://johnmeister.com/linux/Notes/Philosophy_of_Linux.html
- there are significant differences between Microsoft and Linux that need to be understood.
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
TABLE OF CONTENTS for "Let's USE BASH on Windows 10
Introduction to BASH on Windows 10
How to INSTALL BASH on WIN10
How to Configure the BASH window
Chapter 1_Linux_Overview
Chapter 2_the_5_essential_commands
Chapter 3_the_vi_Editor
Chapter 4_shell_config_History_script
Chapter 5_File_Management:_cp_mv_rm
Chapter 6_Files, Processes and Special Characters
Chapter 7_Regular_Expressions_and_Filters
Chapter 8_ssh_setup_and_use - will not cover for Win10
Chapter 9_Network_Basics - will not cover for Win10
Appendix: How to INSTALL_BASH_WIN10
Appendix: The LINUX Philosophy
Appendix: Exercise #1 - the 5 basic commands
Appendix: Exercise #2 - chmod, chown, file permissions
Appendix: Additional info on BASH
Appendix: Additional info from Chapter 2
Appendix: SCRIPT: fix filenames
Appendix: SCRIPT: use wget, date, and a while loop
Appendix: REAL WORLD LINUX COMMANDS
See session #105 for links to table of contents on smashwords.
OTHER RESOURCES:
http://johnmeister.com/linux/Overview/LinuxOverview.pdf (print out and use as a guide)
http://johnmeister.com/linux/Overview/Linux-PowerPoint-2004-overview.pdf
http://johnmeister.com/linux/Notes/Real-world-Linux-Commands.
http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide)
http://johnmeister.com/linux/bashrc-basic.html
|
Session #105 - September 15, 2017 - FRI - Let's USE BASH on Windows 10!
|
note on html: color for cell above is "1242ab" (rrggbb), font color is "white"
working through the book on using the BASH tool in Windows 10.
Follow along in Linux or MacOSX... it's mostly the same. (you can preview the book up to about page 19.)
TABLE OF CONTENTS for "Let's USE BASH on Windows 10
Introduction to BASH on Windows 10
How to INSTALL BASH on WIN10
How to Configure the BASH window
Chapter 1_Linux_Overview
Chapter 2_the_5_essential_commands
Chapter 3_the_vi_Editor
Chapter 4_shell_config_History_script
Chapter 5_File_Management:_cp_mv_rm
Chapter 6_Files, Processes and Special Characters
Chapter 7_Regular_Expressions_and_Filters
Chapter 8_ssh_setup_and_use - will not cover for Win10
Chapter 9_Network_Basics - will not cover for Win10
Appendix: How to INSTALL_BASH_WIN10 - will not address in these sessions,
but this link might help, YMMV: http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html
Appendix: The LINUX Philosophy
Appendix: Exercise #1 - the 5 basic commands
Appendix: Exercise #2 - chmod, chown, file permissions
Appendix: Additional info on BASH
Appendix: Additional info from Chapter 2
Appendix: SCRIPT: fix filenames
Appendix: SCRIPT: use wget, date, and a while loop
Appendix: REAL WORLD LINUX COMMANDS
NOTE: The book "Simply Linux" follows the same format,
but that version has more Linux and MacOSX detail, the Win10 version was scaled back slightly:
https://www.smashwords.com/books/view/705084
|
Session #104 - August 4, 2017 - FRI - sshd tweaks, journalctl, Apache 2.4 details
|
note on html: color for cell above is "12ffab" (rrggbb), font color is "black"
tightening up sshd instead of using fail2ban
tweaks to /etc/ssh/sshd_config -
typical ssh files: http://johnmeister.com/linux/Notes/SSH/sample-etc-ssh-config-files.html
make sure root couldn't log in directly.: PermitRootLogin no
specify which users can ssh in: AllowUsers luser
force only one version of ssh: Protocol 2
give the person ONE try: MaxAuthTries 1
reduce the number of starting connections: MaxStartups 2:50:5
reduce the login grace time from 2 min: LoginGraceTime 30
restart sshd # service sshd restart
(remember: man sshd | col -b > man.sshd.txt - RTM... YMMV!)
system logs ssh: journalctl -f | grep ssh, or journalctl -f | grep -i Error | grep ssh
--> journalctl -f | grep ssh
Jun 21 23:18:09 the-linux-system sshd[23682]: error: PAM: User not known to the underlying authentication module for illegal user admin from 103.79.142.31
Jun 21 23:18:09 the-linux-system sshd[23682]: error: maximum authentication attempts exceeded for invalid user admin from 103.79.142.31 port 52295 ssh2 [preauth]
Jun 21 23:24:06 the-linux-system sshd[23694]: error: maximum authentication attempts exceeded for invalid user root from 190.49.255.143 port 42263 ssh2 [preauth]
Jun 21 23:27:53 the-linux-system sshd[23703]: error: maximum authentication attempts exceeded for invalid user oracle from 201.20.73.3 port 56130 ssh2 [preauth]
if an IP is very persistent, use iptables to block
e.g.: iptables -A INPUT -s 43.255.190.xx -j DROP
apache logs: --> ll /var/log/apache2/*error_log
if an error log has content, view, and see if you can find the error.
After fixing, make a copy of the log file then empty it as follows:
cp the-error_log /var/log/apache2/FIXED/the-error_log-clear-21jun2017 ; cat /dev/null > the-error_log
|
Session #103 - July 28, 2017 - FRI - .bashrc details
|
note on html: color for cell above is "12ab34" (rrggbb), font color is "black"
HAPPY SYS ADMIN DAY - July 28, 2017!!!
Prepping for LIVE Oreilly Media Windows 10 BASH sessions - test run...
- basic .bashrc: http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-bashrc.txt
- bashrc delta from default: http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-bashrc-DELTA.txt
- summary of shopt (shell options): http://johnmeister.com/linux/Microsoft/Win10-BASH/W10-shopt-list-details.html
HAPPY SYS ADMIN DAY - July 28, 2017!!!
Sys Admin song (sung to monty python's lumberjack tune):
[spoken]
I never really wanted to be a scientist.
I wanted to be...a...a SYSADMIN!
[system engineer choir and shift supervisor enter, music strikes up]
Oh, I'm a sysadmin and I'm OK,
I grep all night and I chown all day.
[choir]
He's a sysadmin and he's OK,
He greps all night and he chowns all day.
I ping the nodes, I do PM,
I awk and perl and sed.
I've got a Star Wars lunchbox,
And Tron sheets on my bed!
[choir]
He pings the nodes, he does PM,
He awks and perls and seds.
He's got a Star Wars lunchbox,
And Tron sheets on his bed!
I ping the nodes, I change the rates,
I fork the processes.
I wish that all my lusers
would catch some rare disease!
[choir, growing slightly uncomfortable]
He pings the nodes, he changes rates,
He forks the processes.
He wishes all his lusers
would catch some rare disease!
[choir brightens as they repeat chorus]
I ping the nodes, I lock the /home partition and umounts....
[choir, very uncomfortable and trailing off]
...HAPPY SYS ADMIN DAY...
|
Session #102 - June 16, 2017 - FRI - journalctl, Apache 2.4, and guest
|
note on html: color for cell above is "ffffff" (rrggbb), font color is "brown"
system logs: dmesg and journalctl
dmesg # provides kernel information # part of util-linux package, see Linux Kernel Archive; man dmesg # sudo dmesg --clear
sudo journalctl # sudo journalctl -f # sudo journalctl | grep -i error # journalctl | grep -i Error | grep ssh | grep -v "Jun 0"
Migration from Apache 2.2 to 2.4
changes to apache2 involved replacing two lines regarding permissions, index directives continued to work:
# #Order allow,deny# Allow from all --> Require all granted
# # Order deny,allow# Deny from all --> Require all denied
--> apachectl configtest # Syntax OK
--> systemctl enable apache2
--> systemctl start apache2
--> systemctl status apache2
. apache2.service - The Apache Webserver
Loaded: loaded (/usr/lib/systemd/system/apache2.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2017-06-07 00:34:44 PDT; 5s ago
Main PID: 2422 (httpd-prefork)
Status: "Processing requests..."
Tasks: 6
CGroup: /system.slice/apache2.service
- 2422 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
- 2429 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
- 2430 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
- 2431 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
- 2432 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
- 2433 /usr/sbin/httpd-prefork -f /etc/apache2/httpd.conf -DSYSCONFIG -C PidFile /var/run/httpd.pid -C Include /etc/apache2/sysconfig.d/ -DSYSTEMD...
Jun 07 00:34:44 linuxsystem systemd[1]: Starting The Apache Webserver...
Jun 07 00:34:44 linuxsystem systemd[1]: Started The Apache Webserver.
|
Session #101 - June 9, 2017 - FRI - grepping along
|
note on html: color for cell above is "aaaaaa" (rrggbb), font color is "lime"
grep basics and using catgrep to analyze config files
http://johnmeister.com/linux/Scripts/catgrep.sh.html
# this is the key string: grep -E -v '(^\s*#|^$)'
clean copy to copy... copy to ~/bin/catgrep, chmod and update path in .bashrc
#!/bin/bash
################################################################################
# 24 jun 2016 - john meister copyright 2016 http://johnmeister.com/linux
# cat's file, then greps out blank lines and comments, offers option to save
# 22 jul 2016 - added sed string to strip leading slash off echo /etc | sed 's/^\///'
# 6 jun 2017 - finally got the white space string working: grep -E -v '(^\s*#|^$)'
# NOTE: create ~/bin directory, chmod to 755, add ~/bin to PATH - test:
# ex: --> which catgrep
# /home/luser/bin/catgrep
################################################################################
USAGE="catgrep filename # removes commented and blank lines, option to save"
echo ' --------------------------------------------------------------------'
echo " DISPLAYING file: --> $1 <-- without commented or blank lines"
echo ' --------------------------------------------------------------------'
cat $1 | grep -E -v '(^\s*#|^$)' # the \s is for white spaces
VFL="`echo "$1" | sed 's/^\///' | tr '/' '-'`" ; export VFL
echo '-----------------------------------------------------------------'
echo "save info in local directory? if yes, type: y"
echo '-----------------------------------------------------------------'
read answer ; if [ $answer = "y" ] # note, adding brackets for upper/lower case would be [Y,y]
then
echo "$1" | tee ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt
cat $1 | grep -E -v '(^\s*#|^$)' | tee -a ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt
#### NOTE: use redirect or append on line above instead of tee -a to reduce screen clutter
echo '-----------------------------------------------------------------'
echo " info written to: ~/FYI-$VFL-content-`date +%Y%b%d-%H%M`.txt "
echo '-----------------------------------------------------------------'
else
echo ' ### ------------------------------------'
echo ' ### not written.'
echo ' ### ------------------------------------'
fi
additional details:
http://johnmeister.com/linux/Notes/REGULAR-EXPRESSIONS-simple.html
|
Session #100 - May 26, 2017 - FRIDAY - Networking
|
note on html: color for cell above is "100000" (rrggbb)
100 lunchtime sessions since 2014!
Networking basics and testing performance
BASIC SETUP: ifconfig lan0 192.168.11.42 netmask 255.255.255.0 broadcast 192.168.11.255
ADD ROUTE: route add default gw 192.168.11.1
COMMANDS: ifconfig -a, ip addr, ping -c 3 8.8.8.8, route, netstat -ar, traceroute 8.8.8.8, nslookup johnmeister.com/linux
(note: Microsoft commands offer fewer options and change a few names: e.g. ifconfig --> ipconfig, traceroute --> tracert.)
FILES: /etc/hosts, /etc/resolv.conf, /etc/nsswitch.conf, /etc/hostname, /etc/sysconfig/network (YMMV)
http://johnmeister.com/linux/Notes/NETWORK/basic-networking.html
http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
examining the spreadsheet formulas (CSV)
http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-Generic-BASH.sh.html
|
Session #99 - May 19, 2017 - FRIDAY - Linux Commands in use.
|
note on html: color for cell above is "33bb44" (rrggbb)
using basic commands and the vi editor.
After helping a few friends with Linux recently I see the need
for less speed and more detail... so...
Let's review basics of .bashrc, .History, setting up .ssh
and converting a Windows Batch file to Linux to check network performance.
http://johnmeister.com/linux/Notes/bashrc-simple.html
http://johnmeister.com/linux/Notes/SSH/quick-ssh.html
http://johnmeister.com/linux/Microsoft/LOADTEST-OS-COMPARISON.html
directory with excel, batch file and large file
http://johnmeister.com/linux/Microsoft/NETWORK-PERFORMANCE/Network-Time-Test-GENERIC.bat.html
oh, and one slightly advanced moment:
http://johnmeister.com/linux/Notes/setup-LAMP-on-MINT.html
|
Session #98 - May 11, 2017 - THURSDAY - ssh, rsync, and ftp - along with WinSCP
|
note on html: color for cell above is "99aaff" (rrggbb)
using ssh, rsync, ftp, NFS
-
basic: http://johnmeister.com/linux/FileSystems/man.rsync.txt
-
config info: http://johnmeister.com/linux/Scripts/fix-ftp.sh.html
-
config info: http://johnmeister.com/linux/Notes/ftp-fixes-for-vsftp.conf.html
-
basics: http://johnmeister.com/linux/Notes/RSYNC-cmd.html
-
examples: http://johnmeister.com/linux/Scripts/rsync-directories.sh.html
-
cron job examples: http://johnmeister.com/linux/FileSystems/rsync-disks-cron.sh.html
-
cron job examples: http://johnmeister.com/linux/SysAdmin/Cron-Rsync-sync-up-drives.sh.html
-
using rsync to update: http://johnmeister.com/linux/Scripts/setup-diverse-systems-prototype.sh.html
- mention (again): http://johnmeister.com/linux/Scripts/EXAMPLE-OSYNC/
-
NFS option: http://johnmeister.com/linux/FileSystems/NFS-Server-setup.html
- - if time: (demonstrate WinSCP on XP in a VM on Linux or a Mac)
|
Session #97 - April 28, 2017 - FRIDAY - Linux Commands in use.
|
note on html: color for cell above is "ff22aa" (rrggbb)
using tools to extract key information on a system
dmidecode, /proc, "stat" commands, and grep...
Using search tool and text editing
Continued discussion on using search tool:
http://johnmeister.com/linux/Scripts/SEARCH-TEXT and
examples and files to download to work with
Apache 2.4 migration
Migration from Apache 2.2 to 2.4 on SuSE.
- http://johnmeister.com/linux/SysAdmin/Apache2-setup.html (haven't had a lot of time to sort through all the directives)
examine basic HTML structure and discuss how to migrate to 2.4 view-source:http://johnmeister.com/bible/
|
Session #96 - April 21, 2017 - FRIDAY - Linux Commands in use.
|
note on html: color for cell above is "11aa33" (rrggbb)
using search tool, catgrep, and basics on Apache web server and directives
- search text tool http://johnmeister.com/linux/Scripts/SEARCH-TEXT
- script and all files needed to use found at: http://johnmeister.com/bible/SEARCH-KJV/
- use catgrep to identify existing config:
http://johnmeister.com/linux/Scripts/catgrep.sh.html - "catgrep /etc/apache2/default-server.conf"
- understand the basics: http://johnmeister.com/linux/SysAdmin/Apache2-setup.html
1) install apache2 - zypper in apache2
2) establish base configuration in /etc/apache2/default-server.conf
3) verify document root and log locations
4) systemctl enable apache2 (*chkconfig)
* --> chkconfig -l | grep apache
Note: This output shows SysV services only and does not include native
systemd services. SysV configuration data might be overridden by native systemd configuration.
apache2 0:off 1:off 2:off 3:off 4:off 5:on 6:off
5) systemctl start apache2
- examining existing directives, THESE DIRECTIVES WORK with Apache2 on SuSE 13.2, but NOT in LEAP 42.2:
DocumentRoot "/srv/www/htdocs"
<Directory "/srv/www/htdocs">
AllowOverride FileInfo AuthConfig Limit Indexes Options MultiViews Indexes IncludesNoExec FollowSymLinks
IndexOptions IgnoreCase FancyIndexing FoldersFirst NameWidth=* DescriptionWidth=* SuppressHTMLPreamble
#SymLinksIfOwnerMatch
<Limit GET POST OPTIONS PROPFIND>
Order allow,deny
Allow from all
</Limit>
<LimitExcept GET POST OPTIONS PROPFIND>
Order deny,allow Deny from all
</LimitExcept>
</Directory>
- review or create systemd service
- http://johnmeister.com/linux/Scripts/Sample-systemd-script.html
- any changes to website can be made throughout using:
find . -type f -name README.html -exec cp README.html {} \;
Session #95 - April 14, 2017 - FRIDAY - Linux Commands in use.
|
note on html: color for cell above is "dddddd" (rrggbb)
Notes for this session:
- discuss possible add ons for SuSE
http://johnmeister.com/linux/SysAdmin/SuSE-add-ons.html
- using CSV (comma separated values) in a script to gather information (part of the Art of Linux Sys Admin series)
http://johnmeister.com/linux/Scripts/collect-sys-info.sh.html
- automate system documentation!
- CSV at the command line - using gnumeric:
--> ssconvert spreadsheet.xls newfile.csv # (will this convert csv to xls? RTM anyone?)
# a good use for gnumeric!
--> apt-get install gnumeric or --> zypper install gnumeric or --> yum install gnumeric
- using the office suite at the command line (YMMV) - results on Linux with libreoffice ok...
created a directory... need to try other options.
CSV at the command line - using libreoffice:
--> libreoffice --headless --convert-to csv $filename --outdir $outdir
# this worked, created a directory
CSV at the command line - using OpenOffice:
--> soffice --headless --convert-to csv test-file.xls test.csv
# not recommended on a mac, haven't tried on Linux or Win!
# tried this with "advice" from a search - it created a ".lock" file owned by root in:
(following advice on the internet is NOT recommended)
# further advice says delete .lock... # BETTER - chown!!!
(I should never do what I find on the internet! (without thinking)... and not on a mac)
# /Users/luser/Library/Application Support/OpenOffice/4 removing it removed all of my
configuration and history - thinking gnumeric on linux best...
prep material to put into CSV to import into spreadsheet
- collect chapters into one book for Proof of Concept (POC):
--> cat 11n_1Ki-* > 1Kings.txt ; perl -pi -e 's/1 Ki/1Ki/g' 1Kings.txt
# fix space in naming
--> cat 1Kings.txt | tr " " "\n" | grep ^[A-Z] | sort | uniq -d > 1Kings-KEYWORDS.txt
# vi to clean up sp ch
--> cat 1Kings-NAS-KEYWORDS.txt | wc -l
119
- create script to take KEYWORDS and list times found and references:
#!/bin/bash
# Find-Keywords-1Kings.sh.txt 13 Apr 2017 jm
##########################################
# list of names: 1Kings-KEYWORDS.txt # complete book: 1Kings.txt
##########################################
FILE="Names-found-in-1st-Kings.txt" # output, names to references
echo `date +'%Y_%m_%d-%H%M'` > $FILE # way to reset the file and / or keep track
echo "======================" >> $FILE # eye candy separator
for x in `cat 1Kings-NAS-KEYWORDS.txt` # x is the name to search on
do
echo $x >> $FILE # list the name
echo "----------" >> $FILE # eye candy
grep $x 1Kings.txt | wc -l >> $FILE # number of times name is found in book
echo "----------" >> $FILE # eye candy
grep $x 1Kings.txt | awk '{print $1" "$2}' >> $FILE # print book and reference
echo "======================" >> $FILE # eye candy
done
##########################################
OUTPUT:
##########################################
# 2017_04_13-2209
# ======================
# Abel-meholah,
# ----------
# 2
# ----------
# 1Ki 4:12
# 1Ki 19:16
# ======================
- NEXT CSV - modify script to insert commas for CSV formatting... (not complete, need to wrap lines)
##########################################
#!/bin/bash
# Find-Keywords-SET-AS-CSV-1Kings.sh.txt 13 Apr 2017 jm
##########################################
# 1Kings-NAS-KEYWORDS.txt - grep'd Names, manually edited file
# 1Kings.txt # NAS 1 Kings text
##########################################
FILE="Names-found-in-1st-Kings-CSV-FORMAT.csv.txt"
for x in `cat 1Kings-NAS-KEYWORDS.txt`
do
echo $x >> $FILE # no eye candy, name, followed by a comma, number of times, references
echo ", ">> $FILE
grep $x 1Kings.txt | wc -l >> $FILE
echo ", " >> $FILE
grep $x 1Kings.txt | awk '{print $1" "$2", "}' >> $FILE
echo "====================== \n" >> $FILE
done
##########################################
# OUTPUT:
##########################################
Abel-meholah
,
2
,
1Ki 4:12,
1Ki 19:16,
======================
Abiathar
,
9
,
1Ki 1:7,
1Ki 1:19,
1Ki 1:25,
using grep to get the number of times a word appears in the NAS, and getting the references for study
694 grep -i love NAS.txt | wc -l # 573
695 grep -i hate NAS.txt | wc -l # 312
696 grep -i Lord NAS.txt | wc -l # 6707
697 grep -i sin NAS.txt | wc -l # 1566
698 grep John NAS.txt | wc -l # 136
699 grep Jesus NAS.txt | wc -l # 946
700 grep Joseph NAS.txt | wc -l # 233
701 grep Rebecca NAS.txt | wc -l # 0
702 grep Rebeka NAS.txt | wc -l # 31
703 grep Paul NAS.txt | wc -l # 161
704 grep -i faith NAS.txt | wc -l # 404
705 grep -i faithful NAS.txt | wc -l # 163
--> grep -i faith NAS.txt | awk '{print $1" "$2}'
Gen 32:10
Gen 47:29
Lev 5:15
...
Rev 19:11
Rev 21:5
Rev 22:6
Session #94 - April 7, 2017 - FRIDAY - Linux Commands in use.
|
note on html: color for cell above is "abdddf" (rrggbb)
Notes for this session:
- win10 freebie: ALL COMMANDS from Win explorer: mkdir "c:\ALL_CMDS.{ED7BA470-8E54-465E-825C-99712043E01C}"
http://johnmeister.com/linux/Microsoft/SHOW_ALL_COMMANDS-win7-8-10.html
- got MATE working on SuSE: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
- DISPLAYMANAGER_XSERVER="Xorg"
- DISPLAYMANAGER="sddm"
- DISPLAYMANAGER_REMOTE_ACCESS="yes"
- DISPLAYMANAGER_ROOT_LOGIN_REMOTE="no"
- DISPLAYMANAGER_STARTS_XSERVER="yes"
- DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
- DISPLAYMANAGER_AUTOLOGIN="john"
- Default: "yes"
- DISPLAYMANAGER_PASSWORD_LESS_LOGIN="yes"
- DISPLAYMANAGER_AD_INTEGRATION="no"
- DISPLAYMANAGER_SHUTDOWN="auto"
- this is SuSE still runnig xcfe, will fix...: --> cat /etc/sysconfig/displaymanager | grep -v -E '(^#|^$)'
- DISPLAYMANAGER_XSERVER="mate-desktop"
- DISPLAYMANAGER="lightdm"
- DISPLAYMANAGER_REMOTE_ACCESS="yes"
- DISPLAYMANAGER_ROOT_LOGIN_REMOTE="yes"
- DISPLAYMANAGER_STARTS_XSERVER="yes"
- DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN="yes"
- DISPLAYMANAGER_PASSWORD_LESS_LOGIN="no"
- DISPLAYMANAGER_AD_INTEGRATION="no"
- DISPLAYMANAGER_SHUTDOWN="auto"
- DISPLAYMANAGER_AUTOLOGIN=""
- further refinement of process to build study tools, worked through comparing line by line, editing and checking:
--> more mergefiles.sh.txt
#!/bin/bash
# john 30 Mar 2017 merge-files-project.sh.txt
################################################
cp KJV.txt KJVb.txt
cp NAS.txt NASb.txt
perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt
perl -pi -e 's/(.*)/$1(nas)/' NASb.txt
paste -d '\n' KJVb.txt NASb.txt > KNb.txt
#######################
- To test I'd view the combined file, if I found an error,
I'd edit the NAS version and repeat, put it all on one command line:
--> vi NAS.txt ; sh ./merge-files-project.sh.txt ; more KNb.txt
The first books I configured had more issues, but once my updated scripts had been
used it went quickly; took about 3 evenings to finish.
This also found the one discrepancy between the translations, as noted in 3Jn 1:15
, where I added a line to the kjv to match:
--> grep "3Jn 1:1" KNb.txt
# note: if you forget the quotes you get everything that starts out 3Jn and the name of the file as well...
# note: the indent on the second line for nas - this is found in the NAS.txt file, one space leading
...
3Jn 1:13 I had many things to write, but I will not with ink and pen write unto thee: (kjv)
3Jn 1:13 I had many things to write to you, \
but I am not willing to write them to you with pen and ink; (nas)
3Jn 1:14 But I trust I shall shortly see thee, and we shall speak face to face. \
Peace be to thee. Our friends salute thee. Greet the friends by name.(kjv)
3Jn 1:14 but I hope to see you shortly, and we will speak face to face. (nas)
3Jn 1:15 THIS VERSE DOES NOT APPEAR IN THE KJV BUT DOES IN THE NASB.(kjv)
3Jn 1:15 Peace be to you. The friends greet you. Greet the friends by name. (nas)
- So, now I have several versions to work with for searching key words and phrases.
The kjv with a verse on each line with and without (kjv), and the same for nas.
The combined version has (kjv) and (nas) for clarity.
The NAS.txt and KJV.txt are the base copies, but I have both translations by chapter and by book.
Using grep I have control for searches on the combined file showing any differences in the translations.
Looking for a common phrase:
------------------------------------------------
--> grep -i "Jesus wept" KNb.txt
Jn 11:35 Jesus wept. (kjv)
Jn 11:35 Jesus wept. (nas)
------------------------------------------------
When I check a phrase initially I'll often have the screen flash by with lots of info, so this helps:
--> grep -i "know that I am the Lord." KNb.txt | wc -l
154
------------------------------------------------
So, then I use a pipe and grep another key word to narrow the results...
But sometimes the phrase or keys aren't the same in both and I'll only see one version:
------------------------------------------------
--> grep -i "you will know that I am the Lord." KNb.txt | grep Zion
Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain.
So Jerusalem will be holy, And strangers will pass through it no more. (nas)
------------------------------------------------
By reducing the phrase and dropping "you will" from the string, then both versions are shown.
------------------------------------------------
--> grep -i "know that I am the Lord." KNb.txt | grep Zion
Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain: then
shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv)
Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain.
So Jerusalem will be holy, And strangers will pass through it no more. (nas)
-----------------------------------------------
another variation that may work:
--> grep -i "[shall ye|will] know that I am the Lord." KNb.txt | grep Zion
Joe 3:17 So shall ye know that I am the LORD your God dwelling in Zion, my holy mountain:
then shall Jerusalem be holy, and there shall no strangers pass through her any more. (kjv)
Joe 3:17 Then you will know that I am the Lord your God, Dwelling in Zion, My holy mountain.
So Jerusalem will be holy, And strangers will pass through it no more. (nas)
-----------------------------------------------
When I know there is a difference in the translations, or suspect it,
and I have the reference, I can grep on that:
--> grep "Jn 8:58" KNb.txt
Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv)
Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas)
- now I can use the file KNb.txt, or KJV.txt, NAS.txt to search out phrases or key words
------------------------------------------------
--> grep -i "know that I am the Lord." KNb.txt | more # both translations with (kjv|nas) at the end
--> grep -i "know that I am the Lord." KJV.txt | more # KJV without (kjv) at the end
--> grep -i "know that I am the Lord." KJVb.txt | more # KJV with (kjv) at the end
--> grep -i "know that I am the Lord." NAS.txt | more # NAS without (nas) at the end
--> grep -i "know that I am the Lord." NASb.txt | more # NAS with (nas) at the end
- an example of a passage to search out is one using "I am", a statement that declared diety.
In the Torah when Moses asked who he should say told him, the response was "I AM that I AM".
------------------------------------------------
So, I search: --> grep -i "I am that I am" KNb.txt
Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said,
Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv)
------------------------------------------------
ok... but where's the nas version? No problem, simply grep for the passage:
--> grep "Exo 3:14" KNb.txt
Exo 3:14 And God said unto Moses, I AM THAT I AM: and he said,
Thus shalt thou say unto the children of Israel, I AM hath sent me unto you. (kjv)
Exo 3:14 God said to Moses, "I A M W H O I A M"; and He said,
"Thus you shall say to the sons of Israel, 'I A M has sent me to you.'"(nas)
------------------------------------------------
AND GUESS WHAT? I just found another edit I need to fix in NAS.txt... :) That's why it didn't show up.
Perhaps I should write a script to update all my computers where I have these files now...
------------------------------------------------
But, continuing the search for the other statement of diety...
------------------------------------------------
--> grep -i "I am" KNb.txt | wc -l
1641
------------------------------------------------
well, need to narrow that down a bit
------------------------------------------------
--gt; grep -i "I am" KNb.txt | grep Jesus | wc -l
66
------------------------------------------------
still too many verses... going to guess the passage is in the Gospels
------------------------------------------------
--> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | wc -l
33
------------------------------------------------
Ah, just remembered, Abraham was mentioned in the context... adding a grep:
------------------------------------------------
--> grep -i "I am" KNb.txt | grep Jesus | grep -E '(Mat|Mk|Lk|Jn)' | grep Abraham
Jn 8:58 Jesus said unto them, Verily, verily, I say unto you, Before Abraham was, I am. (kjv)
Jn 8:58 Jesus said to them," Truly, truly, I say to you, before Abraham was born, I am."(nas)
------------------------------------------------
Of course I could just memorize the verse with the passage reference... well, I did, but I forgot it...
------------------------------------------------
So... now I've got a tool that allows me to search on key words, phrases and known references!
------------------------------------------------
I'm a happy student... and I need to fix /etc/sysconfig/displaymanager and test,
and find a way of checking and fixing NAS.txt in Exo 3:14,
and coming up with a way of copying or updating all my systems where I have these notes...
------------------------------------------------
Ok, if we make it this far and have time...
fixing NAS.txt - LET'S use grep and awk to find any other instances
like Exo 3:14 and create a script or command that lists those passages for us to review,
and then possibly create a script using perl or sed to fix those known instances specifically
so we don't create other problems in the file. We will need to review those passages
to make sure they are the same as the source document.
It's tedious and laborius, but important, and using these powerful regex tools in Linux
at the command line makes the creation and editing of this tool possible.
And if we still have time we can generate an rsync script to update other directories and systems...
Session #93 - March 30, 2017 - THURSDAY - Linux Commands and Scripts
|
note on html: color for cell above is "abcdef" (rrggbb)
MET on Thursday this week... normally on Friday...
Notes and the script from the session:
--> more merge-files-project.sh.txt
#!/bin/bash
# john 30 Mar 2017 merge-files-project.sh.txt
# three objectives:
# 1) file 1 first, then file 2
(add new line in process to create combined file) - maybe paste isn't the right tool?
# 2) copy files, then annotate each line with designator - file1X.txt file2Y.txt
# 3) preserve original, annotated and combined in plain text
- file1, file1X, file2, file2X AND: BOTHfiles.txt
# 4) create file with html table to show cells for each line,
two cells in one row... 4.3M files! - BOTHfiles.html
# paste -d '\n' file1.txt file2.txt
# perl -pi -e 's/(.*)/<img src=\"$1\"><br>$1<hr>/'
# KJV.txt NAS.txt
################################################
cp KJV.txt K1.txt
cp NAS.txt N1.txt
paste K1.txt N1.txt > KN-test1.txt # file with one long line for both files
paste -d '\n' K1.txt N1.txt > KN-test2.txt
# file with two separate lines in sequence and in order
################################################
# sed 's/$/ [KJV]/' <KJV.txt >KVJb.txt
# perl -pe 's/$/ (kjv)/' <KJV.txt >KVJb.txt
# need to find global replacement, and also how to insert into a string, if possible
################################################
cp KJV.txt KJVb.txt
cp NAS.txt NASb.txt
perl -pi -e 's/(.*)/$1(kjv)/' KJVb.txt
perl -pi -e 's/(.*)/$1(nas)/' NASb.txt
paste -d '\n' KJVb.txt NASb.txt > KNb.txt # file with two separate lines in sequence and in order
head KNb.txt # put a head command in script to test
read -p 'hit enter'
################################################
sed -i 's$.*$<tr><td>&<p></td>$g' K1.txt
# using sed to add html tags on each line
sed -i 's$.*$<td>&<p></td></tr>$g' N1.txt
# using sed to add html tags on each line
head -n 3 K1.txt
head -n 3 N1.txt
read -p 'hit enter'
#######################
echo "<html><title>TEST</title><center><table border='10' \
width='95%'>" > KN1.txt-testX.html
paste K1.txt N1.txt >> KN1.txt-testX.html # the paste command reads one line from each file
echo "</table></center> <a href='copyright-info.txt'>copyright info</a></html>" \
>> KN1.txt-testX.html
# adding closing table and link
#######################
# perl -pi -e 's$Dan 9:$<b><em>Dan 9:</em></b>$g' KN27-Daniel-Ch9.html
# decided to emphasis the book and chapter
# head KN27-Daniel-Ch9.html # put a head command in script to test
#######################
head KN1.txt-testX.html # put a head command in script to test
#######################
Session #92 - March 24, 2017 - FRIDAY - Linux Commands and Scripts
|
discussion of perl, sed, and editing files using vi and scripts.
COMPLETED the filtering of information from the wget files and fixed special characters and formatting.
1) Will review the entire process and scripts used to extract the information and format it,
2)and, also discuss using rsync between systems.
3) Next step, merge two plain text files one line at a time using a loop to create a table in html.
IF YOU HAVE A SCRIPT that can read one line from two files at a time, please share... KISS... SAW...
KISS - keep it simple sir; ... SAW - simple always works...
the text project:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/the-wget-perl-sed-vi-grep-rsync-plain-text-project.txt
Session #91 - March 17, 2017 - FRIDAY - Linux Commands and Scripts
|
- A quick comparison of commands for MS use:
http://johnmeister.com/linux/Microsoft/PowerShell-vs-BASH-commands-compared.html
- http://johnmeister.com/linux/Scripts/wget-perl-WX-details.html
- http://johnmeister.com/linux/Scritps/perl-fix-spaces-remove-brackets.sh.html
- the art of Linux Sys Admin outline, sample vi video
- SHELL - history, process, commands
- discussion on ASCII vs. UTF-8, etc.
Then, continued discussion on using grep, perl, vi and sed to
reformat my favorite translation of the Bible for searching of key words:
we started out using: http://johnmeister.com/linux/Scripts/remove-brackets-w-perl.sh.html to fix brackets,
#!/bin/bash
echo "remove bracketed footnotes"
perl -pi -e 's/\[.\]//g' *.txt
perl -pi -e 's/\[..\]//g' *.txt
after we ran the script in last week's session, see Session #90 notes. So after several books I've
added more filters: http://johnmeister.com/linux/Scripts/perl-fix-spaces-remove-brackets.sh.html
###############################################################################################
#!/bin/bash
echo "remove bracketed footnotes, fixed commas and semicolon spacing"
perl -pi -e 's/\[.\]//g' *.txt # remove bracketed text such as a footnote
perl -pi -e 's/\[..\]//g' *.txt # remove bracketed text such as a footnote
perl -pi -e 's/,/, /g' *.txt # adds a space after a comma
perl -pi -e 's/, /, /g' *.txt # removes any extra spaces after a comma after adding a space
perl -pi -e 's/;/; /g' *.txt # adds a space after a semicolon
perl -pi -e 's/:/: /g' *.txt # adds a space after a colon
perl -pi -e 's$\.$. $g' *.txt # adds a space after a period
perl -pi -e 's/?/? /g' *.txt # adds a space after a question mark
###############################################################################################
# the following sed and perl string replace all the lines below, used brackets for upper-case
###############################################################################################
sed -i 's/[A-Z]/ /g # add space before uppercase letter
perl -pi -e 's/ / /g # change two spaces to one
###############################################################################################
# perl -pi -e 's/A/A /g' *.txt # adds a space after an uppercase A
# perl -pi -e 's/A /A /g' *.txt # removes any extra spaces before an uppercase A
# perl -pi -e 's/T/T /g' *.txt # adds a space after an uppercase T
# perl -pi -e 's/T /T /g' *.txt # removes any extra spaces before an uppercase T
# perl -pi -e 's/W/W /g' *.txt # adds a space after an uppercase W
# perl -pi -e 's/W /W /g' *.txt # removes any extra spaces before an uppercase W
# perl -pi -e 's/Y/Y /g' *.txt # adds a space after an uppercase Y
# perl -pi -e 's/Y /Y /g' *.txt # removes any extra spaces before an uppercase Y
###############################################################################################
# need to figure out bracket in perl: perl -pi -e 's/(.*)/<img src=\"$1\"><br>$1<hr>/'
###############################################################################################
Would like to discuss getting the parameters for using perl
instead of sed for character replacement, as I did here:
http://johnmeister.com/linux/Scripts/wget-perl-script-WX-details.html
that uses $1 for bringing buffer content over.
I would also like to refine my notes on moving numbers to the beginning of the line
as I did several sessions ago.
The filter for the numbers doesn't always get it right, so I've added the filters above.
#####################################
cat $x | grep "text $x" | sed 's/New American/ New American/g' | \
sed 's/[^<]*<\/span>/ /g' | \
sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt
#####################################
The goal is to further refine the process of filtering the text data to carefully remove
issues that I have been manually editing.
Also: would like to be able to replace the " and ' and -
symbols that are NOT standard, need the info to replace those "special" characters.
Session #90 - March 10, 2017 - FRIDAY - Linux Commands and Scripts
|
Discussion on use of wget, grep, vi and sed parameters: http://johnmeister.com/linux/Notes/MANAGE-TEXT/
(discovered that there are issues with Regular Expressions as part of Posix 2 standards!
complicating the use of special characters! see:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/man.regex-7.html)
- process: download files (script: base-get-book.sh )
#####################################
wget -O INFO-1 "http://www...page 1
wget -O INFO-2 "http://www...page 2
wget -O INFO-3 "http://www...page 3
#####################################
- process: strip out formatting to get specific content
(script: extract-text-book.sh)
#####################################
#!/bin/bash
#####################################
mkdir INFO
for x in `ls INFO-*`
do
#####################################
cat $x | grep "text $x" | sed 's/New American/ New American/g' | \
sed 's/[^<]*<\/span>/ /g' | \
sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt
#####################################
mv $x INFO
done
#####################################
- process: make sure all files have proper names for sorting
ls > fix
#####################################
vi fix
remove lines that have digits that will sort: e.g. filename-10.txt
leave lines that need to have a zero added, then:
:%s/.*/mv & &/g
results: mv filename-1.txt filename-1.txt
then cursor to the number on the right, and insert a 0 before 1, to get: filename-01.txt
repeat (usually 1-9) until done.
:wq
#####################################
sh ./fix ; rm -f fix
#####################################
ls -al (should show all files sorted nicely)
#####################################
- process: remove remaining internal brackets
use script to remove brackets:
sh ./remove-brackets.sh *
#####################################
script: remove-brackets.sh
#!/bin/bash
echo "remove bracketed footnotes"
perl -pi -e 's/\[.\]//g' *.txt
perl -pi -e 's/\[..\]//g' *.txt
#####################################
- process: edit all the files to remove errors, line up on one line
vi INFO-*
after making sure there is a number on the left and all the text on one line, then
:%s/.*/INFO 9:&/g
:wn
:%s/.*/INFO 10:&/g
:wn
:%s/.*/INFO 11:&/g
:wn
# where INFO is the book or file name, and 9 is the chapter.
# The ampersand places the text string with line number to the right
repeat until done editing all files
#####################################
- process: create unique summary file and then copy to other locations
cat INFO-* > INFO_number.txt
copy files to proper directories and other servers:
cp INFO-*.txt ../SORTED-INFO
scp -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO
or
rsync -r ../SORTED-INFO/ 192.168.11.11:/home/luser/FILES/SORTED-INFO
#####################################
- repeat until all 1,189 chapters are cleaned up and sorted...
then test to see that there will be 31,102 lines, and based on the version, 781,621 words
cat INFO-* > total-info.txt ; cat total-info.txt | wc
(note: as of 3/7/2017 I'm on 14/66)
Session #89 - March 3, 2017 - FRIDAY - Linux Commands and Scripts
|
Discussion on use of wget, grep, vi and sed parameters: http://johnmeister.com/linux/Notes/MANAGE-TEXT/
Worked with sed strings to extract specific details from a downloaded file:
cat $x | grep "text $x" | sed 's/New American/ New American/g' |\
sed 's/[^<]*<\/span>/ /g' | \
sed 's/<[^>]*>//g' | sed 's/\ [[^]*\]]//g' | sed 's/ \([1-9]\)/\n\1/g' > $x.txt
As we approached the close of the session we ran out of time sorting out how to remove content between brackets... My subsequent RESEARCH
discovered that there are "some" issues with Regular Expressions as part of Posix 2 standards! Thereby complicating the use of special characters! see:
http://johnmeister.com/linux/Notes/MANAGE-TEXT/man.regex-7.html)
for the notes, which confused me for quite some time... reverted to brute force method as shown for session #90.
Session #88 - February 24, 2017 - FRIDAY - Linux Commands and Scripts
|
http://johnmeister.com/linux/Overview/LinuxOverview.pdf
last weekend: Released Simply Linux: Basics
https://www.smashwords.com/books/view/705084
weekend before: "Let's BASH Win10!" https://www.smashwords.com/books/view/703463
(I skipped watching Doctor Who for TWO weeks now to find time to get these done!!! Glad I have TIVO!)
This week: command line searching for KEY WORDS and organizing results.
Will be using grep, awk, cut, wc, script, nl, and a few others to organize the material. Command line fun!
Parsing through text files for KeyWords and organizing them through REGULAR EXPRESSIONS.
the current list of books and videos - 22 February 2017
click for pdf
Session #87 - February 17, 2017 - FRIDAY - Linux Commands and Scripts
|
http://johnmeister.com/linux/Overview/
http://johnmeister.com/linux/Overview/LinuxOverview.pdf
NEWS ITEM: Finished the BASH on Win10 book:
https://www.smashwords.com/books/view/703463
Windows 10 and BASH security issues:
https://msdn.microsoft.com/en-us/commandline/wsl/release_notes
https://www.youtube.com/watch?v=_p3RtkwstNk&feature=youtu.be
https://github.com/ionescu007/lxss/blob/master/The%20Linux%20kernel%20hidden%20inside%20windows%2010.pdf
In other news, info on Linux, USA based distros:
http://distrowatch.com/table.php?distribution=centos -- https://www.centos.org/ -- https://fedoramagazine.org/
https://en.wikipedia.org/wiki/Fedora_%28operating_system%29
https://distrowatch.com/table.php?distribution=elementary
All USA distributions:
http://distrowatch.com/search.php?ostype=All&category=All&origin=USA&basedon=All¬basedon=None&desktop=\
All&architecture=All&package=All&rolling=All&isosize=All&netinstall=All&language=All&status=Active#simple
Session #86 - February 10, 2017 - FRIDAY - Linux Commands and Scripts
...back to the basics... LPIC 1, BASH Win10, etc...
http://johnmeister.com/linux/Overview/
http://johnmeister.com/linux/Overview/LinuxOverview.pdf
http://johnmeister.com/linux/Microsoft/Win10-BASH/Win10-BASH-install.html
Session #85 - January 27, 2017 - FRIDAY - Let's BASH Windows 10!
Discuss commands in Windows 10 BASH shell, 148 commands in /bin
- how they compare to Mac OS X and Linux (YMMV)
prior week install notes:
discuss steps to configure Ubuntu BASH in Windows 10.
Session #84 - January 20, 2017 - FRI - improve performance of Linux and setup BASH in WIN10
Finish discussion from Session #83 below,
then
discuss steps to configure Ubuntu BASH in Windows 10.
cool items found in today's session:
--> rpm -qf /usr/bin/pdftotext
poppler-tools-0.24.3-15.1.x86_64
--> /etc/icewm "MENU" - edit capability plain text!!!! :)
Session #83 - January 13, 2017 - FRI - improving performance of your Linux (desktop/laptop) experience
# quick note on Friday the 13th... the superstition likely started in Europe
before Gutenberg and Luther made Bibles available widely across Europe...
the idea that "Good Friday" was some how "unlucky" because Christ was crucified and
that there were 13 people in the room...
(assuming 12 disciples and Christ) apparently came from this belief.
Two problems with this.
1) This happened on the 14th of Nisan, or our Wednesday night (Last Supper/Passover), and
2) if Christ hadn't given His life on the cross we'd be completely without hope, or "luck".
If you have another explanation for Friday the 13th, please share.
For more information on this time line and issue, see:
http://johnmeister.com/bible/Crucifixion/THURSDAY-Crucifixion-3days-3nights.jpg
IMPROVING desktop/laptop/server performance of Linux by monitoring, removing or tweaking.
Situation: Running XFCE https://en.wikipedia.org/wiki/Xfce on SuSE Linux (either Leap 42.1, or 13.2)
and noticed CPU consumption by a bunch of applications that are NOT used,
NOT wanted, and shouldn't be there... of course YMMV...
I don't need "orage" (calendar tool), Thunar (file manager -
GUI... if you're going to use the GUI don't remove this),
Clipman or Parcellite (GUI clipboard).
I remove gvfs-fuse. because, in part, it produces this error:
find: ‘/run/user/1000/gvfs’: Permission denied
(I don't like the "convenience" of this "feature", it reminds me of Stuxnet,
and the error as root annoys me greatly...)
--------------------------------------------------------------------------
UNNEEDED APPLICATIONS or SERVICES:
Process to remove an unneeded/unwanted application:
top
# observe any heavy hitters or unneeded tools, e.g. parcellite, ModemManager, fuse-gvfs, etc.
ps -ef | sort | uniq | grep -v ...
those things you know are system related or desired...
ps -e | awk '{print $4}' | sort | uniq -c | grep -Ev '(ssh|bash|firefox)' | sort -n
ps -ef | awk '{print $8}' | sort | uniq | grep -Ev '(ssh|bash|firefox)'
ps -ef | awk '{print $8}' | sort | uniq > processes-Jan-4.txt
# examine the output: cat processes-Jan-4.txt
FOUND: ModemManager (on a desktop without a phone line)
# there are some other items that could be removed, gnome-key-ring...
sudo lsof /usr/sbin/ModemManager # this gives info on the application
sudo kill -9 743 # the pid for ModemManager
or better, use service ModemManager status
service ModemManager stop
sudo zypper remove ModemManager
...(1 package to remove: ModemManager (frees up 2.3MB)
...removed ModemManager-1.4.10-3.2.x86_64
sudo zypper -ps -s # to check services using deleted files
--> sudo zypper ps -s
The following running processes use deleted files:
PID | PPID | UID | User | Command | Service
-----+------+------+---------+----------------------------+----------------
831 | 1 | 0 | root | mcelog | mcelog
861 | 1 | 497 | polkitd | polkitd | polkit
1223 | 1 | 0 | root | cupsd | cups
1241 | 1 | 0 | root | sshd | sshd
1398 | 1396 | 0 | root | Xorg | display-manager
1411 | 1396 | 0 | root | xdm |
1499 | 1435 | 1000 | luser | ssh-agent |
1501 | 1 | 1000 | luser | xfconfd |
1504 | 1435 | 1000 | luser | xfce4-session |
1511 | 1 | 1000 | luser | kdeinit4 |
1513 | 1511 | 1000 | luser | kdeinit4 |
1515 | 1 | 1000 | luser | kdeinit4 |
1517 | 1 | 1000 | luser | gconfd-2 |
1522 | 1 | 1000 | luser | xfwm4 |
1526 | 1 | 1000 | luser | xfce4-panel |
1528 | 1 | 1000 | luser | thunar |
1530 | 1 | 1000 | luser | xfdesktop |
1533 | 1 | 1000 | luser | xfsettingsd |
1534 | 1504 | 1000 | luser | mate-volume-control-applet |
1536 | 1504 | 1000 | luser | python2.7 |
1541 | 1504 | 1000 | luser | python3.4 |
1542 | 1 | 1000 | luser | xfce4-power-manager |
1543 | 1504 | 1000 | luser | nm-applet |
1544 | 1504 | 1000 | luser | kdeconnectd |
1550 | 1504 | 1000 | luser | pk-update-icon |
1556 | 1 | 0 | root | upowerd | upower
1565 | 1 | 1000 | luser | pulseaudio |
1595 | 1 | 1000 | luser | at-spi-bus-launcher |
1610 | 1 | 1000 | luser | at-spi2-registryd |
1615 | 1536 | 1000 | luser | python2.7 |
1616 | 1615 | 1000 | luser | python2.7 |
1624 | 1565 | 1000 | luser | gconf-helper |
1630 | 1 | 1000 | luser | python2.7 |
1649 | 1526 | 1000 | luser | wrapper-1.0 |
1650 | 1526 | 1000 | luser | wrapper-1.0 |
1657 | 1526 | 1000 | luser | xfce4-terminal |
1698 | 1526 | 1000 | luser | VirtualBox |
1718 | 1 | 1000 | luser | VBoxXPCOMIPCD |
1723 | 1 | 1000 | luser | VBoxSVC |
1744 | 1723 | 1000 | luser | VirtualBox |
You may wish to restart these processes.
See 'man zypper' for information about the meaning of values in the above table.
alternate, disable service
sudo systemctl disable ModemManager.service
sudo zypper remove tracker-extract
To find out what packages are installed:
--> zypper wp > zypper-wp-SuSE.txt
# NOTE: wp stands for "what-provides"
--> zypper wp > cat zypper-wp-SuSE.txt
# if there is an "i" in front, it's installed.
# there wasn't an "i" in front of zile, so I tried to remove it:
--> zypper remove zile
Root privileges are required for installing or uninstalling packages.
------------------------------------------------
--> sudo zypper remove zile
Loading repository data...
Reading installed packages...
'zile' not found in package names. Trying capabilities.
No provider of 'zile' found.
Resolving package dependencies...
Nothing to do.
------------------------------------------------
# THIS PERPLEXED ME... so... I checked the man page:
--> man zypper | col -b > man.zypper.txt
# found several directories, and then noticed the log and history file.
--> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}'
--> cat /var/log/zypp/history | grep install | gawk --field-separator=\| '{print $2" "$3}' | grep Modem
install libKF5ModemManagerQt6
install ModemManager
so.... one could: sudo zypper remove ModemManager # already did...
------------------------------------------------
see also: Zypper cheat sheet
see also: assorted tools (apt) cheat sheet
see also: Yum cheat sheet
NOTE: If you're using MUD then it would be: sudo apt-get remove ModemManager
IF you realize that you really needed the app, you could reinstall:
sudo zypper install ModemManager (or apt-get install...)
If you're using Centos or RH, sudo yum remove / install .... If you're using RPM, see man pages for options.
--------------------------------------------------------------------------
KERNEL TWEAKS:
ADJUST SYSTEM TO REDUCE SWAP: sysctl -w vm.swappiness=1
When swappiness is reduced the kernel doesn't attempt to increase cache by paging out. Paging is not good...
To change permanently, write vm.swappiness=1 into your /etc/sysctl.conf file.
(NOTE: do NOT do this is you are memory constrained on your system, typical default setting is around 60)
--------------------------------------------------------------------------
APPLICATION TWEAKS:
As far as "recommended" applications (Firefox, Thunderbird and OpenOffice (LibreOffice)
1) set cache to 0 in firefox.
(go to the hamburger thing on the far right side, select preferences, advanced, Cached Web Content)
More tweaks for the mighty trio to come... (feel free to share...)
Session #82 - January 6, 2017 - FRIDAY - managing file archives at the command line
using Linux and vi at the command line to manage files and directories
PROBLEM: have a large server with duplicate directories and files,
need to organize by date, compare and consolidate
OBJECTIVE: Standardize DIRECTORY names for proper sorting: YYYY-MM-DD-filename.txt
Individual file names can have the date as a suffix before the file type,
e.g. filename-YYYY-MM-DD.txt
Files will sort by number, then letter, starting from the left to the right.
step 1: change into directory, for example, the year 2000, June... cd 06_June-2000
step 2: list the files and create a file to edit (see details below)
step 3: use vi to create a quick script to rename files,
when you exit vi, it executes the script, then removes the script
step 4: then you examine the directory to make sure the files were named correctly
and mv them up to the parent directory
step 5: once you're in the parent directory, check the old directory using a recursive listing:
"ls -AlR 06_June-2000"
step 6: If the directory is empty and didn't contain any hidden directories (e.g. .profile ),
then: "rm -rf 06_June-2000"
NOTE: if you're comfortable with this process, then you can add:
" mv * .. ; cd .. ; rm -rf 06_June-2000 " to the script.
DETAILED STEPS:
--> cd /home/luser/files/2000/06_June-2000
------------------------------------------------
luser@penguin [/home/luser/files/2000/06_June-2000]
------------------------------------------------
--> ls > fix ; vi fix ; sh ./fix ; rm -f fix
# NOTE: be sure NOT to use a keyword or command, fix might not be the best choice...
to check the word used:
--> which fix
which: no fix in (/home/....
# we're ok... if it was a keyword or command, just use some other combination, like mymess2...
# while in vi:
# note: you will be in vi immediately, when you quit out it'll attempt to execute the file,
make sure it's right...
:%s/.*/mv & 2000-06--&/g
# global copy and paste with a prefix of information for another editing step (note: where 06 = Jun)
:%s/2000-06--06-Jun-/2000-06-/g
# another global replace to modify the file, putting the date in the preferred order
# while in vi you'll need to edit out the REDUNDANT -2000
(the original file had 06-June-15-2000 as the file format)
# there might be a better way of creating the FROM TO names using perl, sed, and awk,
but this is a quick and dirty script...
# use /-2000 to find the 2nd instance, then delete TWO words using vi/command mode:
"2dw" - if you goof, <esc>u and try again.
# the desired line in the "fix" script
mv 06-Jun-15-2000-SJ-egr-tube 2000-06-15-SJ-egr-tube
# Once the directories are named correctly,
move them up to the main directory and remove the old directory.
--> mv * .. ; cd .. ; ls -AlR 06_June-2000 # if it's all clean, then nuke it...
--> rm -rf 06_June-2000
updating directories for web presentation
After the directories for each year are organized by YYYY, then MM, then DD
(e.g. 2000 06 15) you can add any particular
web content into each directory via a quick and dirty script.
I add a README.html, HEADER.html and for images I add an ALL.html.
The manual process is something like this:
--> cd 1999-12-Marks-J3000 ; mk-webpage "1999-12-Marks-J3000" ; ls -al ; cd ..
# the "mk-webpage" is a custom script to create localized web content
- we've discussed this script in prior sessions
see:
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.html
# So, the process we need to "script" is:
cd 1999-12-Superdawg ; mk-webpage "1999-12-Superdawg" ; ls -al ; cd ..
So, for every directory we need to cd into it, run the mk-webpage script.
PROCESS:
--> cd 2000
--> ls > addall
--> vi addall
# while in vi the process is simple and quick:
:%s/.*/cd & ; mk-webpage "&" ; cd ../g
# the lines should look like:
cd 2000-01-01-BEST ; mk-webpage "2000-01-01-BEST" ; ls ; cd ..
cd 2000-01-01-Superdawg ; mk-webpage "2000-01-01-Superdawg" ; ls ; cd ..
...
then simply escape, write, quit: "<esc>:wq"
then execute the quick and dirty script to update the entire year's worth of directories for your website:
--> sh ./addall
then copy the updated directory, to your web server (unless you edited in place),
and then check one of the directories and try an ALL.html file:
http://johnmeister.com/fotos/2002/2002-02-05-Sun-Solaris-Install/ALL.html
This is an updated version of a Sun Solaris install from February 2002.
------------------------------------------------
Session #81 - December 16, 2016 - FRIDAY - Linux text management
use sed, vi and perl to combine lines in large text files
http://johnmeister.com/linux/Notes/MANAGE-TEXT/
using sed, vi, and perl will demonstrate how to combine lines in a large block of text
in order to use grep to locate key words and display entire content of line.
Session #80 - December 9, 2016 - FRIDAY - Linux and troubleshooting
troubleshooting, diagnosis, monitoring
Have been setting up some systems and having challenges with BIOS, proprietary RAID and hard drive failures.
some issues during setup of dell towers in bios
random bios screens
"Bob" found some interesting links on SMART:
https://en.wikipedia.org/wiki/Comparison_of_S.M.A.R.T._tools - https://www.smartmontools.org/
https://sourceforge.net/projects/smartmontools/ and https://sourceforge.net/projects/smartmontools/files/smartmontools/6.5/
file system recovery
http://johnmeister.com/linux/FileSystems - recovery of failing drives - tune2fs - grub and UUID
http://johnmeister.com/linux/FileSystems/rescuing-failing-drive-w-dd.html
fresh file system setup
http://johnmeister.com/linux/Notes/KERNEL-BOOTING/grub.cfg-example.html
http://johnmeister.com/linux/FileSystems/UUID-details.html
http://johnmeister.com/linux/FileSystems/newdisk-setup.html
http://johnmeister.com/linux/FileSystems/mount-external-drive-rysnc.txt
http://johnmeister.com/linux/FileSystems/setup-LVM.html
in other use: http://johnmeister.com/linux/Notes/using-find-grep-xargs.html
Session #79- December 2, 2016 - FRIDAY - Simply Linux and mysql
open disucssion: current issue: metadata from RAID on Dell
fdisk -l details
SuSE update screen
updated h/w, dell tower T5600 --> T5810 - Linux not able to boot... metadata based on proprietary consumer grade h/w,
also using LVM... this is why SAW (simple always works... and using "new" features often doesn't)
Continued discussion on command line mysql on Linux.
steps to install, activate, and create simple databases as the command line in Linux.
still working on importing CSV, in the meantime, found the commands to make the table...
will capture working commands and populate... will incorporate commands into BASH scripts and continue developing
tools at the command line to manipulate data.
• sudo zypper install mysql (or sudo apt-get install mysql)
• sudo services mysql start
• mysql -u root -p -h 127.0.0.1
• create DATABASE firsttry;
• show databases;
• show tables;
• create table equipment (hostname varchar(20), location varchar(20), OS varchar(10));
• describe equipment;
• insert into equipment(hostname,location,OS) values ('lab1', 'bench 1', 'Mint');
• select * from equipment;
• status;
• help;
Alternate paths for links: http://LinuxMeister
http://shop.oreilly.com/product/0636920050209.do?cmp=ex-prog-books-videos-product-na_moo_card_for_john_e._meister,_jr.\
_for_study_guide_for_lpic_2_exams_201_and_202
http://johnmeister.com/linux/Intro-to-Linux/One-Hour-Linux-Sessions-2016.html
(not updated as frequently - mirrored on different server)
Session #78- November 18, 2016 - FRIDAY - Simply Linux and mysql
- sudo zypper install mysql
- sudo services mysql start
- mysql -u root -p -h 127.0.0.1
- create DATABASE firsttry;
- show databases;
- status;
- help;
Session #77- November 11, 2016 - FRIDAY - Simply Linux: book outline
- Using Linux
- Linux Summary
- Linux Overview and Setup
- Five (5) Commands you need to master
- Shell configuration, History, and "script"
- File Management
- Files and Processes: Special Characters
- Regular Expressions and Filtering Data
- SSH setup and use
- Network Basics
- File Systems - Create, Monitor, Manage
- User Management (System Administration tasks)
- Package Management - Upgrading and Installing
- Appendix A: The LINUX Philosophy
- Appendix B: Exercise #1 - basic commands
Session #76- November 4, 2016 - FRIDAY - Linux: command line
Regular expressions experimenting with "cal" in a loop:
using a loop to determine the day of the week for the 15th of July over various years...
a discussion on ffmpeg with a cool timelapse, and look at x2go.
copy and paste links below, not live
Video created by Uncle Mike: Facebook post Public time lapse video:
https://www.facebook.com/m.d.clay/videos/vb.703246157/10154101081271158/?type=2&theater
A look at another NX technology tool for remote access of Linux systems:
http://wiki.x2go.org/doku.php/doc:newtox2go and https://en.wikipedia.org/wiki/X2Go
Session #75- October 20, 2016 - THURSDAY - Linux: open discussion and follow up
Discuss follow up on ffmpeg, video formats, and Apache configuration and services.
apache2-service-commands.html
Session #74- October 14, 2016 - Friday - Linux: trace a script to encode video
https://gist.github.com/donmelton/7734177 - found this script to encode video, looks interesting...
still don't have ffmpeg working
Session #73- October 7, 2016 - Friday - Linux: fixing crontab email and examining ffmpeg, vlc and mencoder
http://johnmeister.com/linux/Notes/crontab-email-setup.html
Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series
Session #72- September 28, 2016 - Wednesday - Linux: Using grep, awk, cat, diff, wc, and comm
NOTE: ### WEDNESDAY THIS WEEK ###.... WEDNESDAY - COMMAND LINE POWER
using cat, grep, awk, sort, redirection, comm, more, perl (sed), tr
and wc to solve a cool problem
There were multiple files with common values and lots of data.
We needed to find the common values among several files.
Took real world info and completely buggered it up to show ONLY how to find
certain things in a large file.
The content of the files is irrelevant, so it was obscured to help us find only
what we were seeking.
# this is the process used, but was NOT the strings used to corrupt the data...
cat original.file.1.txt | tr c-z a-x | tr 2-8 3-9 | tr A-Y B-Z > 1.RAW
# after completely messing up the content, look for the one string,
"Messedup", and change it to our primary search value:
perl -pi -e 's/Messedup/KEY-VALUE/g' *.RAW
Then we use grep to find that KEY-VALUE among thousands of lines in the file
and look for the info labeled as "Key-Value":
cat .RAW | grep KEY-VALUE | grep - | awk '{print $4}' | sort > 1.rpt
...
comm -12 1.rpt 2.rpt | comm -12 - 3.rpt > common-values.txt
...
we'll test our results with cat and grep...
time permitting we'll convert this into a script... and enhance it...
Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series
Session #71- September 22, 2016 - Friday - Linux: file systems, shell scripts and configurations
The discussion on file systems:
(we've been having an email discussion on ext4 and journaling...)
script in work: http://johnmeister.com/linux/Scripts/setup-diverse-systems-prototype.sh.html
have been working on shell script to gather key info from a variety of linux versions and distros,
seeking to gather config data and either automate a standard configuration,
or at least have an on-screen checklist with commands spelled out that can be used with tweaks.
There are certain environments where managing Linux installs is a challenge because of networking related issues,
primarily those lacking name services, so one must use the IP address or use /etc/hosts to make things work.
This can be a problem as these environments use either DHCP or DDNS. Automation in an uncertain environment is risky...
Alternate paths for links: http://LinuxMeister
O'Reilly Media Linux Sys Admin video series
Session #70 - Sep 16, 2016 - Friday - Linux command line - using USB devices
http://johnmeister.com/linux/Notes/USB-live-Gparted-setup.txt
http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.html
http://johnmeister.com/linux/Notes/failing-drive-system-rescue.html
Session #69 - Sep 9, 2016 - Friday - Linux command line - Discuss Oracle's Virtual Box and Centos Install
INSTALLING CENTOS 7 on a VM...
(distracted: found the limit when we opened 4 VM's (Win10, WinXP, Solaris and Centos)
on a MacBook Air: 2 browsers, Webex, and Thunderbird)
- rather than wait for it to recover, powered down... it did NOT crash though!
continued discussion on using Windows 10 (v1607) BASH (Ubuntu) REGULAR EXPRESSIONS
http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/
http://www.tenforums.com/tutorials/46769-bash-ubuntu-windows-10-enable-disable.html
http://www.omgubuntu.co.uk/2016/08/windows-10-anniversary-update-delete-partition
Session #68 - Aug 26, 2016 - Friday - Linux command line - Digital photography and web pages
continue discussion on setting up pages using vi, html and regular expression,
and also discuss Windows 10 BASH in v.1607.
Creating new script to add exiftool data to pages on web server.
Exiftool extracts exif info from raw and jpg files.
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
http://johnmeister.com/linux/Scripts/exiftool.sh.txt
Session #67 - Aug 19, 2016 - Friday - Linux command line - Digital photography and web pages
Creating new script to add exiftool data to pages on web server. Exiftool extracts exif info from raw and jpg files.
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
http://johnmeister.com/linux/Scripts/exiftool.sh.txt
Session #66 - Aug 10, 2016 - Wednesday - Linux command line discussion
Demonstrate the use of rsync and scp in scripts and setting up cron to automate.
http://johnmeister.com/linux/Scripts/rsync-directories.sh.html
http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html
Session #65 - Aug 2, 2016 - Wednesday - Linux command line discussion
Discussion showing differences between rsync and scp and the effects of placing in the background:
--> rsync -rp --delete linuxbox:/home/luser/files/METRICS/ linuxbox-files/METRICS
IF THE slash on the source is left off... then the entire directory is copied over to the destination directory.
IF the slash is present on the source, then the files are moved into the destination directory.
(the idea is to copy the entire directory as part of the path, replicated to the remote box... if you don't
get the slashes correct the files will be placed in a directory one level up, creating a mess...)
------------------------------------------------
--> echo "THIS WORKS for SCP, need slash, no dest DIR: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/"
scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/ &
THIS WORKS for SCP, need slash, no dest DIR is required: scp -rpq linuxbox:/home/luser/files/LINUX/ linuxbox-files/
The -q is for quiet, keeps the pipe buffer from filling and stopping the process if scp is placed in bg.
Session #64 - July 29, 2016 - Friday - Linux command line discussion
SYSTEM ADMINISTRATOR APPRECIATION DAY!!! http://sysadminday.com
Continue discussion of command line configurations and scripting the setup of printers and workstations.
Review of SuSE LEAP distribution as based on SLE.
Session #63 - July 22, 2016 - Linux discussion
The Art of Linux System Administration - LPIC-2 Study Guide
Session #62 - July 15, 2016 - Linux Course released by O'Reilly! Free videos - bring popcorn
The Art of Linux System Administration -
LPIC-2 Study Guide Will go over the course outline and maybe show one or two of the videos.
Will also review "shell" options: shopt (which works in any BASH environment, Linux, UNIX or MacOSX
-----------------------------------------------
--> shopt -s histverify
-----------------------------------------------
--> shopt
...
histverify on
...
------------------------------------------------
Session #61 - June 24, 2016 - Linux Overview - continued
see Session #60 below for content links - continued command line demonstrations and explanations.
Session #60 - June 17, 2016 - Linux Overview - continued
Linux Overview on one page
...we left off discussing Special Characters in preparation to show the use of Sed, Grep and Awk in a shell...
...of course we also mentioned col -b and creating copies of text files of man pages, see your homework:
http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html man 'command' | col -b > man.command.txt
Then we'll talk about using sort... and other filters... with whatever chunks of data I have at hand...
Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk
Session #59 - June 9, 2016 - Linux Overview
Linux Overview
Sorting of files, REGULAR EXPRESSIONS for filtering and selecting: grep, sed, awk
Session #58 - June 2, 2016 - Linux Tuning Overview
Linux Tuning
Session #57 - April 8, 2016 - using a script to extract system info and create a CSV file
creating a script to create CSV
Session #56 - April 1, 2016 - Debunking Calendar Myths with Calendar
calendar myths
example... 5 Sundays and 5 Saturdays... an email might claim it only happens once every X years,
with X being some massive number... wrong... but who's going to flip through a century's worth of calendars?
Well, Linux will: (adjust Y=2009 to something like Y=1900 to see much more)
--> for ((Y=2009; Y<=2016; Y++)); do for M in 1 3 5 7 8 10 12;\
do cal $M $Y | grep -q ^31 && printf '%02d/%d\n' $M $Y; done; done
05/2009
01/2010
10/2010
07/2011
03/2013
08/2014
05/2015
01/2016
07/2016
Session #55 - March 25, 2016 - ssh configuration files and tunneling
http://johnmeister.com/linux/Scripts/extract-SSH-config-info.sh.html
http://johnmeister.com/linux/Notes/SSH/sample-etc-ssh-config-files.html
http://johnmeister.com/linux/Notes/SSH/ssh-base.html
http://johnmeister.com/linux/Notes/SSH/typical-etc-ssh-conf-files.html
http://johnmeister.com/linux/Notes/SSH/quick-ssh.html
http://johnmeister.com/linux/Notes/SSH/ssh-full-duplex-process.html
http://johnmeister.com/linux/Notes/SSH/ssh-timeout-tweaks.html
http://johnmeister.com/linux/Notes/SSH/reverse-tunnel.html
http://johnmeister.com/linux/Notes/SSH/ssh-tunnel.jpg
Session #54 - March 18, 2016 - Compile the Kernel and recovery
http://johnmeister.com/linux/Notes/Compile-a-Kernel.html
http://johnmeister.com/linux/Notes/Dracut-recovery-shell.html
Session #53 - March 11, 2016 - Kernel, booting and GRUB
Discussion on Kernels, booting, and GRUB - Grand Unified Boot Loader
http://www.ibm.com/developerworks/library/l-linuxboot/
http://johnmeister.com/linux/Notes/grub.cfg-example.html
Session #52 - March 4, 2016 - NFS basics and RAID
Discussion on Network File Systems and RAID
http://johnmeister.com/linux/FileSystems/NFS-info.html
http://johnmeister.com/linux/FileSystems/RAID-info.html
Session #51 - February 26, 2016 - The use of "script" and col -b
The use of "script", as root user, from /root, type:
script setup-new-user-home-luser.RAW
optional ex: (add user to /etc/passwd, /etc/shadow, /etc/group - edit files manually or use "useradd -m username")
mkdir -p /home/luser/.History ; mkdir -p /home/lusers/.ssh ; mkdir -p /home/lusers/bin
cp /root/users.bashrc /home/luser/.bashrc
man bash | col -b > /home/luser/bin/man.bash.txt
ssh --help | col -b | tee -a /home/luser/bin/man.ssh.txt
chown -R luser:users /home/lusers
exit
cat setup-new-user-home-luser.RAW | col -b > setup-user-home.txt ; rm -f setup-new-user-home-luser.RAW
ls -alR /home/luser | more
vi setup-user-home.txt (replace "luser" with variable for user, and modify as a wrapper script to use with tool)
----------------------------------------------------------------------------------------------------------------------------
Notes, general (SuSE and MINT):
--> which useradd: /usr/sbin/useradd
--> useradd --help: Options: -m, --create-home create the user's home directory
SYNOPSIS
useradd [options] LOGIN
useradd -D
useradd -D [options]
----------------------------------------------------------------------------------------------------------------------------
Notes, MINT (MUD):
SYNOPSIS
adduser [options] [--home DIR] [--shell SHELL] [--no-create-home] [--uid ID] [--firstuid ID] [--lastuid ID]
[--ingroup GROUP | --gid ID] [--disabled-password] [--disabled-login] [--gecos GECOS] [--add_extra_groups]
[--encrypt-home] user
DESCRIPTION
useradd is a low level utility for adding users. On Debian, administrators should usually use adduser(8) instead.
When invoked without the -D option, the useradd command creates a new user account using the values specified on the
command line plus the default values from the system. Depending on command line options, the useradd command will
update system files and may also create the new user's home directory and copy initial files.
By default, a group will also be created for the new user (see -g, -N, -U, and USERGROUPS_ENAB).
----------------------------------------------------------------------------------------------------------------------------
Session #50 - February 19, 2016 - Script Templates
Script template basics:
- #!/bin/bash (shebang: #!) - shell directive
ALTERNATE: #!/usr/bin/env bash
ANOTHER ALTERNATIVE: #!/bin/bash -e
- Author
- Date and version info
- comments about changes, requirements, etc.
- Title or purpose of script
- USAGE variable - or comments on how to use the script
- Variable declaration.
DISCUSSION on "fix-filename": (without appropriate comments, authorship and dates - just the basics)
#!/bin/bash
for f in *
do
mv -v "$f" `echo $f | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | sed 's/_-_/_/g'`
done
http://stackoverflow.com/questions/14008125/shell-script-common-template
https://gist.github.com/KylePDavis/3901321
http://linuxcommand.org/lc3_new_script.php
Session #49 - February 12, 2016 - Syncronizing files between systems
Osync script example: http://johnmeister.com/linux/Scripts/EXAMPLE-OSYNC/
discussion will include a review of this very complex BASH script, and how it can be done
with less code using rsync, scp and ssh commands...
http://www.netpower.fr/projects/osync/Osync_v1.00a.html#toc-Subsection-1.1
Session #48 - February 5, 2016 - The Art of Linux Systems Administration
Continued Topics for discussion:
- Understanding the Linux philosophy and how it affects commands and tools
- Systems Administration overview: what is a sys admin and what do they do?
- Developing the Art of Systems Administrator
- master the vi editor...
- Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
(any artists? would like to draw a snail... purchase and install will be the tail then heading around to deal
with the other parts that are an iterative process, not unlike the SDLC.
Session #47 - January 29, 2016 - The Art of Linux Systems Administration
Topics for discussion:
- Understanding the Linux philosophy and how it affects commands and tools
- Systems Administration overview: what is a sys admin and what do they do?
- Developing the Art of Systems Administrator
- master the vi editor...
- Linux Systems Lifecycle: purchase, install, analyze, configure, troubleshoot, modify, deploy and support
(any artists? would like to draw a snail... purchase and install will be the tail then heading around to deal
with the other parts that are an iterative process, not unlike the SDLC.
Session #46 - January 22, 2016 - FIRST Session 2016 - LPIC - Administering Small to Medium sized Linux Networks
 LPIC-2 - administering small to medium-sized mixed Linux networks.
2015 sessions and links below
Session #45 - December 16, 2015 - LAST Session 2015 - LPIC - Administering Small to Medium sized Linux Networks
LPIC-2 - administering small to medium-sized mixed Linux networks.
BOOK: BOOK on Microsoft Windows
Session #44 - December 9, 2015 (Wednesday) - NESTED FOR loop in a Batch file and BASH script
BATCH file to setup a workstation - load registry hacks
BATCH FILE explicitly coded for each box - "before" the FOR loop
BATCH using FOR LOOPS - significant savings in coding!
BOOK: BOOK (includes background on the TEMP issue)
could be used to automate cleanup LINUX SCRIPTS USING FOR LOOP:
variable definition less complex in BASH
using for loops to gather info quickly - less stringent syntax than MS
this wouldn't be possible in an MS Batch file
Session #43 - December 2, 2015 (Wednesday) - VirtualBox Strategy, Plan and Goals
http://johnmeister.com/linux/Notes/VirtualBox-VM-Plan
Session #42 - November 25, 2015 (Wednesday) - update on Performance Tests; VirtualBox: SuSE on SuSE
Main pages for #42 presentation:
http://johnmeister.com/linux/Microsoft/LoadTest-MacOSx.sh.txt update on performance testing
http://johnmeister.com/linux/Microsoft/LOADTEST-OS-COMPARISON.html
http://johnmeister.com/linux/Notes/VM_setup_SuSE_13.2/ALL.html
Step-by-step screen shots of install of SuSE 13.2 Linux on VirtualBox Virtual Machine on SuSE Linux
exercises: (use VirtualBox with two VMs, or Cygwin, or a Linux system) (continuing Q and A and review)
the goals:
1) work with the command line and SSH between systems – learn basic Linux functions and shell commands
2) later, build VMs for Windows testing and interoperability
- demonstrate the use of Linux as a SECURE and reliable host for Microsoft
- will build Win7 VM's for testing browsers and java by test team
- showing stability of Linux as VM host/cost saver
- will build various NON-networked Windows Server versions for testing (w2k8r2, w2k12 and w2k16)
3) networking between 2 VM's but isolated from local network
- Allow active testing of unapproved Operating Systems (win and linux)
4) remote access from Microsoft to Linux system running a Microsoft system to permit active development and testing
REFERENCE notes and exercises FOR VirtualBox:
- http://johnmeister.com/linux/Notes/VirtualBox-notes.html
- http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
- http://johnmeister.com/linux/Notes/bashrc-simple.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
- http://johnmeister.com/linux/Scripts/chksys.sh.html
- http://johnmeister.com/linux/Scripts/while-loop.sh.html
- http://johnmeister.com/linux/Notes/VM_setup_CENTOS_7/ALL.html - coming soon
- http://johnmeister.com/linux/Notes/Networking-btwn-VBox-VMs.html - coming later
note: there was no session November 18, 2015
Session #41 - November 11, 2015 - SHORT SESSION: VirtualBox - discussion continues
Short session today... starting 11:15, ending before noon... Volunteered for a Veterans' Day activity...
- http://johnmeister.com/linux/Notes/VirtualBox-notes.html
- http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
- http://johnmeister.com/linux/Notes/bashrc-simple.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
- http://johnmeister.com/linux/Scripts/chksys.sh.html
- http://johnmeister.com/linux/Scripts/while-loop.sh.html
Session #40 - November 4 2015 - Q and A - VirtualBox - lab exercise
QandA - VirtualBox - lab exercises - review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
(continuing QandA and review) working with the command line once you have a VirtualBox Linux shell.
Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
http://johnmeister.com/linux/Notes/zypper-update-ps.html
http://johnmeister.com/linux/Notes/apt-get-update-upgrade.html
- http://johnmeister.com/linux/Notes/VirtualBox-notes.html
- http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
- http://johnmeister.com/linux/Notes/bashrc-simple.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
- http://johnmeister.com/linux/Scripts/chksys.sh.html
- http://johnmeister.com/linux/Scripts/while-loop.sh.html
Session #39 - October 28, 2015 - using the VirtualBox - lab exercise
Linux command review and exercises: (using VirtualBox with two VMs, or Cygwin, or a Linux system)
- http://johnmeister.com/linux/Notes/VirtualBox-notes.html
- http://johnmeister.com/linux/Intro-to-Linux/Slides/ALL.html
- http://johnmeister.com/linux/Notes/bashrc-simple.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-with-VM-servers.html
- http://johnmeister.com/linux/Scripts/chksys.sh.html
- http://johnmeister.com/linux/Scripts/while-loop.sh.html
Session #38 - October 21, 2015 - SSH One Command Setup, C programming Overview & CENTOS VirtualBox install
- ONE LINE SSH setup!
http://johnmeister.com/linux/Notes/Setup-SSH-Key-on-ONE-line.html
- brief C programming overview:
http://johnmeister.com/linux/Notes/C.html
- live SETUP CENTOS VM:
http://johnmeister.com/linux/Notes/VirtualBox-notes.html
Session #37 - 14 October 2015 - Troubleshooting Overview
How do you solve system problems? What tools do you use? Is there a simple way of determining the problem?
http://johnmeister.com/linux/Notes/Systems-Adminstration-Overview.html
Session #36 - 7 October 2015 - VirtualBox Overview
VirtualBox basics - setup and uses
http://johnmeister.com/linux/Notes/VirtualBox-notes.html
Session #35 - 30 September 2015 - Systems Administration Overview
so you think you want to be a Sys Admin?
http://johnmeister.com/linux/Notes/Systems-Adminstration-Overview.html
Session #34 - 9 September 2015 - System Configuration Files
What files and directories are important?
http://johnmeister.com/linux/Notes/System-Configuration-Details.html
Session #33 - 2 September 2015 - GRUB and Wrapper Scripts
- http://johnmeister.com/linux/Notes/date-suffix.html
- http://johnmeister.com/linux/Scripts/wrapper-script.html
- http://johnmeister.com/linux/Notes/grub.cfg-example.html
- http://johnmeister.com/linux/Notes/grub2-notes.html
Session #32 - 19 August 2015 - How to Dual Boot Microsoft Windows 10 and SUSE Linux 13.2
- http://johnmeister.com/linux/Microsoft/Dual-Boot-Win10-SUSE13.2/
Session #31 - 12 August 2015 - Security Basics
The vulnerabilities, exploits, attack vectors, actual risks and payoffs
- http://johnmeister.com/linux/Notes/SecurityOverview/
Session #30 August 5, 2015 - CSV, XML, Spreadsheets and the use of AWK to filter output
- https://tools.ietf.org/rfc/rfc4180.txt
- http://www.w3schools.com/xml/
- https://msdn.microsoft.com/en-us/library/bb387090.aspx
- http://johnmeister.com/linux/Commands/awk-example.html
Session #29 July 8, 2015 - performance analysis tools
http://johnmeister.com/linux/Notes/Tuning-Linux.html
Session #28 June 24, 2015 - as requested: REAL WORLD SCRIPTING
practical scripts: creating, dating, renaming, numbering and managing files
-
http://johnmeister.com/linux/Scripts/Lab-exercise-create-text-man-pages-for-vi-practice/
- a basic loop to create man pages for plain text reading
-
http://johnmeister.com/linux/Scripts/man-page-create-textfiles.sh.txt
- the script for the exercise above
-
http://johnmeister.com/linux/Scripts/replace-spaces-plus.txt
- using tr, sed and mv to change case and/or remove spaces in file names
-
http://johnmeister.com/linux/Scripts/removing-spaces-and-special-characters.html
- expansion of prior script, includes dealing with special characters
-
http://johnmeister.com/linux/Scripts/filedate.sh.html
- using grep, sort, awk, sed to create a list of files to be "mv"'d (renamed) with a date PREFIX for sorting
-
http://johnmeister.com/linux/Scripts/number-file-names.sh.html
- create a number prefix for files to aid in sorting, used for mp3 files.
-
http://johnmeister.com/linux/Scripts/mail-list.sh.html
- use for loop to send mail/files to several friends one by one
-
http://johnmeister.com/linux/Scripts/mk-webpage-2015-05-08.txt
- create a web page in a directory of pictures
-
http://johnmeister.com/linux/Scripts/mk-timelapse-avi-on-2-sites-2015-05-08.txt
- one of the scripts to create time lapse videos from pictures
-
http://johnmeister.com/linux/Scripts/chksys.sh.html
- check on the health of your system, including disk usages (drives fill up fast with images)
Session #27 June 17, 2015 - server name link changes, drive fail/system rescue
- http://johnmeister.com/linux/Notes/Update-amazon-ad-links.html
- http://johnmeister.com/linux/Notes/failing-drive-system-rescue.html
Session #26 June 10, 2015 - Networking basics, analysis/tools and differences
- http://johnmeister.com/linux/Notes/basic-networking.html
-
http://johnmeister.com/linux/Notes/networking-analysis-and-tools.html
- http://johnmeister.com/linux/Notes/networking-differences-by-distros.html
differences in network and basic commands between SuSE, MUD and RedCents) (MUD=Mint, Ubunutu, and Debian)
Session #25 June 3, 2015 - /var/log/messages - errors, fixing... attacks, stopping... good networks...
-
http://johnmeister.com/linux/Notes/slapd-unsupported-operation.html
-
http://johnmeister.com/linux/Notes/using-iptables-to-block-spammers-or-attackers.html
-
http://johnmeister.com/linux/Notes/Network-Analysis-and-Observations.html
NOTE: NO CLASS on May 27, 2015
insomnia prevention classes resumes on 3 June 2015 with Session #25!
Session #24 May 20, 2015 - NOTE: (no class on the 27th)
conclusion: Detailed analysis of TimeLapse scripts, page creation and crontab
-
http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html - what failed...
- describe the troubleshooting process used to learn why the cron job failed and
how a "hunch" solved a "newly created" problem.
-
http://johnmeister.com/linux/Notes/ManagingFiles-ssh-rsync.html setup of ssh, using rsync... or scp... and
-
http://johnmeister.com/linux/Notes/wget-n-crontab.html when to use wget...
-
http://johnmeister.com/linux/Notes/ftp-fixes-for-vsftp.conf.html and when to use ftp.
-
http://johnmeister.com/linux/Microsoft/Setup-TaskManager-AutoReboot.html
Comparing a Microsoft captive user interface (GUI) "cron job" to cron, at and nohup.
- in other news: headhunter emailed, looking for:
Intermediate to Expert (command line heavy) knowledge of
one or more of the following Unix operating systems:
70%: AIX v.5.3 and/or 7.1 - 25%: SUSE (v.11 Enterprise) - 5%:
Red Hat Enterprise Linux (v.11 (ed.???) Working with Core Technologies (Overall 10%):
Working familiarity with: Apache Server and/or Tomcat - OpenSSH (Unix) and SSL
(Windows + Unix; configuration + troubleshooting)
- S/FTP (Unix-based configuration + troubleshooting) -
Moderate shell scripting ability (Perl considered a plus)
- references: http://johnmeister.com/linux/Commands/MAN/man.chmod.txt,
http://johnmeister.com/linux/Commands/MAN/man.at.txt, http://johnmeister.com/linux/Commands/MAN/man.nohup.txt,
http://johnmeister.com/linux/Commands/MAN/man.ftp.txt, http://johnmeister.com/linux/Commands/MAN/man.ssh.txt,
http://johnmeister.com/linux/Commands/MAN/man.scp.txt, http://johnmeister.com/linux/Commands/MAN/man.ssh.txt,
http://johnmeister.com/linux/Commands/MAN/man.wget.txt, http://johnmeister.com/linux/Commands/MAN/man.rsync.txt,
http://johnmeister.com/linux/Commands/MAN/man.bash.txt, etc...
Session #23 May 13, 2015 - Detailed analysis of TimeLapse scripts, page creation and crontab
http://johnmeister.com/linux/Notes/Setup-TimeLapse-Creation.html
Session #22 May 6, 2015 - follow up of LVM, discussion of scripts
http://johnmeister.com/linux/FileSystems/setup-LVM.html
Session #21 Apr 29, 2015 - resize2fs and LVM
http://johnmeister.com/linux/FileSystems/setup-LVM.html
Session #20 Apr 22, 2015 - LVM configuration
http://johnmeister.com/linux/FileSystems/setup-LVM.html
Session #19 Apr 15, 2015 - ftp, vsftp, scp, ssh
Session #18 - Apr 8, 2015 - Systemd
http://johnmeister.com/linux/Notes/systemd-info.html systemd and
the general boot up process of Linux systems. (comparing /etc/init.d/)
for additional reading: https://www.novell.com/docrep/2015/01/systemd_in_suse_linux_enterprise_12_white_paper.pdf
Session #17 - Apr 1, 2015 - proxy/vnc
http://johnmeister.com/linux/Notes/vnc-setup.html
http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
http://johnmeister.com/linux/Scripts/run-now-on-all-hosts.sh.html
Session #16 - Mar 25, 2015 - ssh, scp, and rsync
use of ssh to examine, monitor and scp to copy files, create a script and use rsync
to manage several systems from one.
Walk through the process and establish a wrapper script to make archive copies on each system.
http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
http://johnmeister.com/linux/Scripts/run-now-on-all-hosts.sh.html
Session #15 - Mar 18, 2015 - systemd, tree, ssh and scp
discussion on systemd, tree command, use ssh to examine and scp to copy
files, create a script and use rsync to manage several systems from one.
Session #14 - Mar 11, 2015 - general discussion on Linux shells, commands - open discussion with group
Session #13 - Mar 4, 2015 - 1) general Linux info, 2) basics, and 3) advanced topics
Session #12 - Feb 25, 2015 - The power of the command line: grep
Discussion of GREP: example: grep -v ^[[:space:]]$ and grep -v [[:space:]]
Session #11 - Feb 18, 2015 -The basics of Apache2 configuration, sed examples and use.
1) http://johnmeister.com/linux/Notes/?C=M;O=D
(general updates can be seen near the top of this page)
2) http://LinuxMeister/Notes/basic-Apache-setup.
(basic install and config, and major config files)
3) http://LinuxMeister/Notes/apache-tricks.
(virtual server configurations and settings)
4) http://johnmeister.com/linux/Notes/apache-tomcat-topic-list.
(the list of modules used for development purposes)
5) http://johnmeister.com/linux/Notes/setup-LAMP-on-MINT.
(adding apache to Ubuntu, Debian or Mint - not there by default)
Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic
Session #10 - Feb 11, 2015 - Using Linux to Analyze Traffic-
wget, crontab, ls , vi, mencoder, sed, grep, cat, tac, lsof, netstat.
1) http://johnmeister.com/linux/Notes/?C=M;O=D
2) http://johnmeister.com/tech/Traffic/TUES-PM/00_Tues-PM.html
3) http://johnmeister.com/linux/Notes/wget-n-crontab.html
4) http://johnmeister.com/tech/Traffic/
5) http://johnmeister.com/tech/Traffic/files.txt
6) http://johnmeister.com/tech/Traffic/Make-Commute-pages.txt
7) http://johnmeister.com/linux/Notes/setup-in-irfanview.jpg
8) http://johnmeister.com/linux/Notes/mencoder-info.html
9) http://johnmeister.com/linux/Notes/zypper-install-mencoder.html
10) http://johnmeister.com/linux/Notes/cat-tac-and-sed.html
11) http://johnmeister.com/linux/Notes/Network-Analysis-and-Observations.html
Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.
Session #9 - Feb 4, 2015 - SSH, rsync, interoperability, and wrapper scripts to manage data across several computers.
1) http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
2) http://johnmeister.com/linux/Notes/RSYNC-cmd.html
3) http://LinuxMeister/Notes/ManagingFiles-ssh-rsync.html
Will discuss details on how to configure directory structure on Windows and Linux systems
to synchronize a set of Files on several computers.
Will discuss setup of bi-directional ssh passwordless connection between two Linux systems,
and RSYNC configuration.
Will discuss wrapper scripts to make backup copies for files that will be rotated and replaced,
preventing redundant or corrupt files.
http://johnmeister.com/linux/Intro-to-Linux/Notes/FileMgtPlan.txt
If we have time we'll also discuss WinSCP and how it can be used from Microsoft systems. Emphasis will be on the Linux aspect of
ssh and rsync, but also on general file management and config mgt for end users.
If time, will discuss SSH tunneling through firewalls/VPNs/NATs: http://johnmeister.com/linux/Notes/ssh-tunnel.jpg
NOTE: updated Gparted page from last week:
http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.html
Session #8 - 28 January 2015 - discuss the use of cal, grep, tee and the use of a simple loop in a shell
Work with cal, grep, and a do-while loop in a BASH script to determine if it there is truth to the internet claim that:
December 2012 having 5 Saturdays, 5 Sundays and 5 Mondays only happened once every 823 years!
Commands covered: cal, cat, man, more, ls, grep, tee and basic do while loops and tests in BASH
http://LinuxMeister/Intro-to-Linux/28jan2015/grep.help.txt
http://johnmeister.com/linux/Scripts/BUST-MYTHS-WITH-CAL-GREP/ http://johnmeister.com/linux/Commands/grep-awk-uniq.
http://johnmeister.com/linux/Overview/LinuxOverview.pdf
Brief discussion on recovery of /home on a failing or corrupted system:
http://johnmeister.com/linux/Notes/Gparted-for-Recovery/ALL.
http://LinuxMeister/Intro-to-Linux/28jan2015/how-to-preserve-home.txt
tar cvfz home.tar /home(note: will need to be root or use sudo, make sure you have enough space to create the file)
Session #7 - 21 January 2015
http://opensuse-guide.org/installation.php
https://software.opensuse.org/132/en
https://en.opensuse.org/Portal:Installation
https://activedoc.opensuse.org/book/opensuse-start-up
https://activedoc.opensuse.org/book/opensuse-start-up/chapter-1-installation-quick-start
http://johnmeister.com/linux/http://johnmeister.com/linux/Overview/LinuxOverview.pdf
https://www.smashwords.com/profile/view/johnmeister
https://www.smashwords.com/books/view/505731
Session #6 - 14 January 2015
http://LinuxMeister/Intro-to-Linux/ misc notes and screenshots found here… not organized.
Files created during session found here.
In particular http://LinuxMeister/Intro-to-Linux/14jan2015/reg-exp-users.txt
demonstrated the use of script and the creation of the
http://LinuxMeister/Intro-to-Linux/14jan2015/reg-exp-users.RAW file,
which was then filtered using cat {filename}.RAW | col -b > {filename}.txt
Regular Expressions and the power of the command line.Continued demonstrating filtering.
http://LinuxMeister/Intro-to-Linux/
Demonstration of:
grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo, man, --help, ifconfig, dmesg and which
Session #5 - 7 January 2015 http://LinuxMeister/Intro-to-Linux/
Demonstration of:
grep, sed, awk, perl, wc -l, cat, strings, tee, head, tail, col -b, echo,
man, --help, ifconfig, dmesg and which
SESSION #4 - December 17, 2014: pkg mgt, rsync, wine, mount
http://johnmeister.com/linux/PDF-info/PackageManagementCheatSheet.pdf
http://johnmeister.com/linux/FileSystems/newdisk-setup.txt
http://johnmeister.com/linux/Notes/USB-live-Gparted-setup.txt
http://johnmeister.com/linux/Notes/Setup-SuSE-13.2-select-screens/ALL.
http://johnmeister.com/linux/Notes/zypper-installs.
http://johnmeister.com/linux/FileSystems/setup-LVM.txt
http://johnmeister.com/linux/Commands/rsync-example-man. RSYNC basics
http://johnmeister.com/linux/FileSystems/mount-external-drive-rysnc.txt
http://johnmeister.com/linux/FileSystems/rsync-disks-cron.sh.txt
http://johnmeister.com/linux/Notes/Firefox-about-support-Tools.
Firefox about features showing add-ons, in the URL type: about:support
http://johnmeister.com/linux/Notes/wine-examples-manpage. using wine - some examples and the man page
SESSION #3 - December 10, 2014 - basic commands
discuss ssh files, show how to use Gparted,
exercise on real world Linux commands, discuss VNC setup
continue discussion on regularly used commands,
e.g. grep, sed, awk, sort, uniq, file permissions and chmod, chown
exercise #2 - real world examples of commands and file permissions.
----------------------------------------------------------------------------
Links:
- http://johnmeister.com/linux/Overview/
- http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf
- http://johnmeister.com/linux/Notes/bashrc-the-dotfile.html
- http://johnmeister.com/linux/Notes/SSH-setup-and-details.html
- http://johnmeister.com/linux/Notes/ssh-authorized-keys-setup-example.html
- http://johnmeister.com/linux/Notes/Gparted-for-Setup-or-Rescue.html
- http://johnmeister.com/linux/Notes/vnc-setup.html
- http://johnmeister.com/linux/Notes/Real-world-Linux-Commands.html
- http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-2.html
SESSION #2 - December 3, 2014 - SAW:"simple always works"
provided link to exercise #1, discussed environment (.bashrc), path, chmod, discussed "SAW" -
SAW:"simple always works",
History files viewed with sort, uniq, and how to create notes for reference: ls, sort, grep, uniq, wc -l
(after using a command, use history recall, add echo and quotes around it,
and then append to ~/bin/cool-commands.txt e.g.
echo "ls *.jpg > ALL.html ; perl -pi -e 's/(.*)/<img src="$1"><BR>$1<HR>/g' ALL.html ; \
perl -pi -e 's/<img src=""><BR><HR>//g' ALL.html ; cat ALL.html" >> ~/bin/cool-commands.txt
----------------------------------------------------------------------------
Links: http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html
(do this on your system, use "script exercise-1.raw",
when finished type exit, then cat exercise-1.raw | col -b > exercise-1.txt - then edit in vi and save)
http://johnmeister.com/linux/Scripts/chksys.sh.html (use the vi editor to create and run on your system)
http://johnmeister.com/linux/Scripts/man-page-create-textfiles.sh.txt
(mkdir and cd LAB, then create this script in vi and run)
http://johnmeister.com/linux/Commands/grep-awk-uniq.html
(the output of the script above is needed to use the commands, YMMV, counts may be different)
SESSION #1 - November 26, 2014 - OVERVIEW
provided overview of Linux using the pdf: http://johnmeister.com/linux/Overview/LinuxOverview.pdf
5 basic commands: man, ls (ls -al, ls -Al), cd, pwd, more
discussed .bashrc and showed a few script examples, talked about "script"
----------------------------------------------------------------------------
Links: http://johnmeister.com/linux/Overview/LinuxOverview.pdf (print out and use as a guide)
http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and keep as a guide)
http://johnmeister.com/linux/Notes/bashrc-the-dotfile.html (copy and build your own .basrhc for Cygwin or Linux)
http://johnmeister.com/linux/Scripts/chksys.sh.html
http://johnmeister.com/linux/Intro-to-Linux/lab-exercise-1.html (use the commands and the vi editor)
THE CORE MATERIAL:The power of the command line - Simply Linux: Basics
will work our way through the Linux book (Simply Linux: Basics)
under construction based on http://johnmeister.com/linux/Overview/
http://johnmeister.com/linux/Overview/LinuxOverview.pdf (print out and use as a guide)
http://johnmeister.com/linux/Overview/Linux-PowerPoint-2004-overview.pdf
http://johnmeister.com/linux/Notes/Real-world-Linux-Commands.
http://johnmeister.com/linux/Intro-to-Linux/Special-Characters.pdf (print out and use as a guide)
http://johnmeister.com/linux/Notes/bashrc-the-dotfile. (copy and build your own .basrhc for Cygwin or Linux)
ONE HOUR Linux SESSIONS
 
|
