Notes
Slide Show
Outline
1
Get a CLUE* with Linux:
*command line user environment
  • Get a CLUE with Linux;

     Or, how to save 182 manhours
     analyzing test results by using
     LINUX regular expression tools
    at the command line
     rather than in Microsoft Excel.
  • (spreadsheets are amazing tools!)
2
Get a CLUE* with Linux -
*Command Line User Environment
  • Regular Expressions in Linux saved over 182 Manhours during a 19 test cycle with 10 reports for each cycle.
  • Linux saved 13 hours per test cycle of sorting and comparing in Excel by using regular expression commands in a Linux shell, taking roughly 15 minutes.
  • A BASH shell script was written to automate the process;
    it runs in less than FIVE SECONDS.
  • 13 hours to 15 minutes to less than 5 seconds.
  • First an overview of Linux; then the script will be explained.
3
Linux consists of TWO THINGS:
Files and Processes
4
 
5
 
6
To learn Linux, start with these 5 commands:
man, ls, cd, pwd, more
7
"Setting up your SHELL environment..."
  • Setting up your SHELL environment will help productivity, visibility and create custom tools:
    see also #1 of the lesser tenets: allow the user to tailor the environment...
    http://linuxmeister.net/Intro-to-Linux/bashrc-n-history-details.html
    http://linuxmeister.net/Notes/bashrc-basic.html
8
Files, filesystems and terminology
9
You need to be aware of "key words", built-ins and commands,
as well as special characters.
10
Sorting things out:
  naming files and directories (NEVER use spaces or most special characters)
11
the best editor in the world:  vi
to enable at the command line:  set -o vi
12
Regular Expressions
  • awk – Aho, Alfred V.; Weinberger, Peter J.; and Kernighan Brian W. (1977)
    https://en.wikipedia.org/wiki/AWK#Versions_and_implementations
    http://linuxmeister.net/Commands/awk-example.html      http://linuxmeister.net/Notes/tabs-convert-to-spaces.html
  • sed – stream editor (also used with Perl)
    http://linuxmeister.net/Notes/using-sed-to-count-executable-files.html 
    ls `env | grep PATH | grep -v XNLSPATH | sed -e 's/^PATH=//' | sed -e 's/\:/ /g'`  2>/dev/null  | wc -l
  • grep – global regular expression search
    http://linuxmeister.net/Notes/using-find-grep-xargs.html    http://linuxmeister.net/Commands/use-of-grep.html
    find . -type f -name *.html | xargs grep johnmeister.com/linux | perl -pi -e 's$http://johnmeister.com/linux$http://linuxmeister.net$g'


13
SORTING TEST RESULTS USING REGULAR EXPRESSIONS
  • A computer system under test in the early development stages monitored an ARINC bus and reported statuses. Some of the notifications were considered nuisance messages because of intial settings.

    Some might be related to a sensor with too narrow a hysteresis, a parameter incorrectly set in the s/w,
    an out of adjustment gadget, or less likely, an actual defective component.

    note: the content of these reports and the specific systems involved are not included, "dummy data is used“
  • 10 reports were generated for each test cycle of the system.
  • 19 test runs were made under different conditions to ensure full compliance.
  • a USB device is used to copy the text files from the system.
  • The 10 reports were copied into Excel
  • each report would be sorted in Excel by ATA chapter
  • copying and sorting the reports would take an hour.
  • after all 10 reports were sorted, the engineer compared them, looking for common errors.
  • It took another 3 hours to compare those 10 sorted reports
  • The engineer had 13 hours invested in this analysis for just ONE test cycle, there were 19.
  • The engineer then examined each of the common errors in the original reports.
  • Once he determined the issue, he'd resolve that one and move to the next.

  • Analyzing his process and test results, we determined the plain text files had a consistent format
    with an easily spotted "key field", i.e. the ATA Maintenance Chapter.

    The reports had names unique to the product under test. This info was also in each report.

    To protect the original reports and to make handling the sorts easier,
    We copied the originals with new names from 01.txt to 10.txt, with a "txt" suffix for interoperability.
14
 
15
basic info found in the reports and scoping the file sizes
16
the manual process (15 minutes)
17
Use grep to globally find the regular expression “21-18891” and then display 23 lines After that in file 01.txt
18
Test Result Analysis script – find common messages
19
...and executing the script...
20
...performance of script...  compare OS & HW
21