Introduction To Bash Shell Scripting
References
Learning the Bash Shell, Cameron Newham & Bill Rosenblatt (O'Reilly & Associates, Inc)
Linux in a Nutshell, Jessica Perry Hekman (O'Reilly & Associates, Inc)
The UNIX Programming Environment, Brian W Kernigham & Rob Pike (Prenice Hall)
Unix Power Tools, Jerry Peek, Tim O'Relly, & Mike Lookides (O'Reilly & Associates,Inc)
Advance Bash Scripting Guide : http://www.tldp.org/LDP/abs/html
Appendix B has some good Reference Cards
Review
Getting Help
man Get off information about commands
man -k Get off information via keyword
info Read info documents
help Show help on shell built-in commands
command --help Help on command .i.e. cp --help Will give help on cp
bash -c "help" Short help on bash
bash -c "help set" Short help on bash options
Special Characters
Space Argument separator
\ Quote a single character
A \ followed by a carriage return extend the current line
' Quote rext with spaces in it i.e. 'Hello world'
$' Quote allows string expansion, backslash-escaped characters
" Quote rext, expands variables and command substitution
` Command substitution i.e. echo "The date is `date`"
$ Denote a shell variable
# Comments the rest of the line
; Commands separator
(commands) Run multiple commands in a subshell
Control-C Interrupt a command
Control-D Sends End of File from terminal
Control-U Erases the entire command line
Control-\ Is a stronger terminate than Control-c
Control-Z Suspend process
Redirection, Pipes and Filters
- Standard In for some commands
0 Standard In
1 Standard Out
2 Standard Error
>2&1 Redirect standard error to standard out
&> Redirect standard error & standard out
< Redirect input
> Redirect output
>> Concatenate output to file
<< Here Is File for scripts
| Pipe
tee Command write to file and standard out
tee -a FILE #Allow appending to FILE
xargs Build command from standardin
Wildcards
* Wildcard for any character(s)
? Wildcard for single character
[set] Wildcard for character in set
[^set ] Wildcard for not the character in the set
[!set ] Wildcard for not the character in the set
{ab,dc} Wildcard for alternate between commas
All wildcard work with existing files
Only {} alternate work to create files
Process Control
(command1; command) Run command1, then command2 in subshell
command1&&command Run command1, then command2 if command1 successes
command1||command Run command1, then command2 if command1 fails
Regular Expressions
Definition: Text pattern of text character and meta characters
Some Meta characters
Escape character \
Single Character Meta characters
. Matches any one character
[...] Matches any one character in a set
[^...] Matches any one character not in the set
Quantifiers
>
* Matches the previous character zero or more times
\{n\} Matches the previous character n times
\{n,m\} Matches the previous character at least n & at most m
\{n,\} Matches the previous character n or more times
Anchors
^ Matching at the start the line
$ Matching at the end of line
Grouping \( \)
Commands that use regular expressions
awk Pattern scanning and text processing language
ed Line-oriented text editor
egrep extended grep
emacs Emacs full screen text editor
ex Line-oriented text editor
expr Command evaluates an expression
fgrep Grep from patterns in a file
gawk GNU pattern scanning and processing language
grep Searches file for pattern (also see fgrep & egrep)
grep [OPTIONS] PATTERN [INPUT-FILE...]
-E same as egrep
-c Count
-e pattern (for multiple pattern on line)
-f same as fgrep
-i Ignore case
-l Only list files containing pattern
-q Quit (No output, only Return Code)
-v Invert sense mode
perl Perl scripting
python Python scripting
sed Applies a set of user-specified editing command to a file
sed [OPTIONS] 'sed_command' [INPUT_FILE...]
-n Suppress automatic printing
-e expression - sed_command
substitute other_text for some_text
sed 's/some_text/other_text/g' FILE > NEWFILE
multiple changes
sed -e 's@abc@def@g' -e 's@xyz@mno@g' FILE
print out line with faq in them
sed -n '/faq/p' FILE
change Page ### to (Page ###) at end of line
sed 's/Page [0-9]+$/(&)/' file # & replace the match
delete blank lines
sed '/^[ \t]*$/d
tcl Tool command language
vi Full screen text editor
Shell Variables
Built-in Shell Variables
CDPATH Path of shortcuts for cd (like PATH)
COLUMNS Numbers of columns on the display
EDITOR Path for editor
HISTSIZE Number of commands in command history (default 500)
IFS Input Field Separator
LINES Numbers of lines on the display
OFS Output Field Separator
SECONDS Seconds that this shell is running
SHELLOPT Colon separate list of shell options
Environment Variables
export var Will make a variable an environment variable
HOME User's home directory
LOGNAME User's name
MAIL Name of user's mailbox
PATH List of directories to be search by the shell
to find programs whose names are type as commands
PS1 String that is used by the shell prompt
PWD Name of current directory
SHELL Name of current shell
TERM The kind of terminal being used
Environment variable are global to shell and subshells
User Variables
Can be either upper or lower case
var=value Define a variable
var="" Define a variable as null
local var Define a variable local to its scope
Positional Variables
$0 Name of function or script being called
$1 ... $9 Replace by arguments to shell or function
${n} Replace by n-th arguments to shell or function
required if number of argument is over 9
Special Variables
$? Exit status or return code of last command
$# Number of arguments
$@ Argument 1 thru n with Input Field Separator
$* "$1" $2" ... $n
$! Process id of last background process
$$ Process id of shell running this script
$- The current shell flags
Some Command Useful With Scripts
basename Strip directory and option suffix
declare Built-in command declares variable
dirname Strip non-directory part
echo Built-in command display message to standard out
echo -n Built-in command display message to standard out without newline
echo -e Builtin Command display message to standout with escape sequences
enable Built-in command to enable/disable (-n) built-in commands
env Built-in command displays environment variables
eval Built-in command evaluate arguments before executing results
exec Built-in command runs command as the shell process
exit Built-in command exits shell
expr Evaluates an expression and output its value
false Built-in command always return false condition
local Built-in command make variable local
read Built-in command reads data into variable from standard in
seq Print a sequence of numbers
seq [OPTION]... LAST
seq [OPTION]... FIRST LAST
seq [OPTION]... FIRST INCREMENT LAST
-w equalize width by padding with leading zeroes
sleep Command causes execution to stop for a
specified number of seconds
test Built-in command tests for various conditions, such as
existence of a file, useful for controlling
conditional script execution
time Built-in command times commands
times Print accumulated user and system times.
true Built-in command always return true condition
type Built-in command show what word is
wait wait for the completion of background
processing; is used to ensure that critical
processing is complete before proceeding in a script
Making Scripts Colorful
tput sgr0 #Reset text attributes to normal without clearing screen
Escape sequence to change colors
#\e[${forground};${background}m
example white on black sequence is \e[37;40m
#Forground colors Backgrounfd Color
black=30; bgblace=40
red=31; bgred=41
green=32; bggreen=42
yellow=33; bgyellow=43
blue=34; bgblue=44
magenta=35; bgmagenta=45
cyan=36; bgcyan=46
white=37; bgwhite=47
Turn text attribute off \e[0m
Change text attribute bold \e[1m
Change text attribute underline \e[4m
Change text attributer everse \e[7m
Shell Functions and Scripts
Functions
Function must be sourced just like .bashrc
type function will list the function
function functionname {
shell commands
}
functionname () {
shell commands
}
Example - a helpful function
function givehelp { exec $1 --help | more; }
Scripts
#!/bin/bash Script name run script is run if execute bit is set
#!/bin/bash -x As above, but script lines are displayed
bash -x script As above
/usr/bin/env bash Use the environment bash for the script
Script Debugging options
set -o OPTION Command line Action
set -o noexec sh -n Don't run command, just check for syntax
set -o verbose sh -v Echo commands before running them
set -o xtrace sh -x Echo commands after running them
set +o OPTION Turns OPTION off
Operation With Variables
Variable Usage
$var Value of variable var
"$var" Is null if variable is undefined
avoids some shell syntax errors if variable is undefined
${var} Value of variable var
avoids confusion when concatenating with text
${#var} Gives the length of the string contained in var
${var:FIRST:N} Extract string from var starting at FIRST position
and continuing for N-1 characters. Note FIRST starts at 0.
Passing an variable to a program or script
echo $var | command
Arrays
declare -a name Declare an array
name[index]=value Just assigning a value defines it
Index starts at zero
No maximum limit
Need not be contiguous
Setting Variable From Execution of Command
var=`command` var is set to output of command
var=$(command) same as above
Arithmetic Operation
$(( expression )) Almost the same as 'expr expression'
$[ expression ] Same as above
( expression ) Groups expression within $(( ... ))
expr expression Note must quote *
Arithmetic operator
+ add
- substract
* multiply
/ divide
% remainder
# number conversion (Only in bash not expr)
Test Operation
[ -option arg ] Same as 'test -option arg
[[ -option arg ]] Same as 'test -option arg
[ arg1 -option arg2 ] Same as 'test arg1 -option arg2'
[[ arg1 -option arg2 ]] Same as 'test arg1 -option arg2'
See test command for test complete list of options
string1 = string2 String 1 equals string 2
string1 !=string2 String 1 not equals string 2
-n string String is not zero length
-z string String is zero length
-d FILE File is a directory
-e FILE File exists
-f FILE File exists and is a regular file
-r FILE File exists and is readable
-s FILE File exists and has length greater than zero
-w FILE File exists and is writable
-x FILE File exists and is executable
num1 -eq num2 Number 1 equals number 2
num1 -ne num2 Number 1 not equals number 2
num1 -lt num2 Number 1 less than number 2
num1 -le num2 Number 1 less than or equals number 2
num1 -gt num2 Number 1 greater than number 2
num1 -ge num2 Number 1 greater than or equals number 2
String Operation
${varname:-word} Return var if exists and is not null, else word
${varname:+word} Return word if var exists and is not null, else null
${varname:?mess} Return var if exists and is not null, else
display mess and return from script with error
Pattern Operations
${variable#pattern} If the pattern matches the beginning of the
variable value, delete the shortest part that
matches and return the rest
${variable##pattern} If the pattern matches the beginning the of
variable value, delete the longest part that
matches and return the rest
${variable%pattern} If the pattern matches the end of the variable
value, delete the shortest part that matches and
return the rest
${variable%%pattern} If the pattern matches the end of the variable
value, delete the longest part that matches and
return the rest
The handy 'expr'
expr string : regrep Anchored pattern match
expr match string pattern Same as above
expr substr string pos {len] Substring beginning at pos of length len
expr length string Length of string
Flow Control
if - general information
if command Test return code of command
if command1 && command2 Test return code of command1 and command2
if command1 || command2 Test return code of command1 or command2
command can be condition i.e. [ $# -eq 0 ]
if/then/else or if/then/elif..
if condition
then statements...
[else statements...]
fi
if condition
then statements...
[elif condition
then statements...]
[else] statements...]
fi
return from Function
return [ numeric expression or variable ] This is the return code
for
for name [ in list ] do
statements
done
for variable = start to end do
statements
done
while/until
while condition do
statements
done
until condition do
statements
done
break/continue
break [n] Break out of loop & select
continue [n] Continue next iteration of loop
case
case expression in
pattern1[|pattern11] } statements ;;
pattern2[|pattern21] } statements ;;
...
esac
User Interfaces
select
select name [in list] do
statements that can use $name
done
Generate a menu of each item in the list formatted with
a number for each choice
Prompt the user for the number
Store the selected choice in name. The number is stored in REPLY
execute the statements in the do
Repeat the process again
Command Line Options
shift Shift 1st argument form the argument list
getopts Used to process command line options
OPTIND Variable contains number of options
OPTARG Default variable for option
Example
while getopts ":ab:c" opt; do
case $opt in
a) process_option_a;;
b) process_option_b
$OPTARG is the option's argument;;
c) process_option_c;;
esac
done
shift $(($OPTIND - 1))
I/O in Scripts
printf Built-in command for formatted output
read Built-in command to read one line into variable(s)
read #Everything entered goes to REPLY
read var #Everything is read into var
read a b #Read 1st word in to a and rest into b
read -t 300 var #Read with 300 second timeout
Example how to use read from $file
while read line; do
do_something_with_line
done <$file
Process Handling
Signals
Name Number Control Character
EXIT 0 Used to trap exiting script
HUP 1 Logout
INT 2 Control-C
QUIT 3 Control-\
KILL 9 can not be ignored or trapped
TERM 15 Default kill
TSTP 24 Control-Z
Traps
trap "" signal-list Ignore signal
trap "cmds" signal-list Execute commands if signal is caught
trap signal-list Reset signal to original condition
trap : signal-list (undocumented) ignore signal, pass to child
Signal are normally not passed to subprocesses
Examples
trap 'rm tmpfile; exit' 0 1 2 #remove tmpfile on exit, logout, interrupt
trap "echo 'You hit Control-C'" INT
while true ; do
sleep 60
done
Example parent child process
#!/bin/bash #parent
echo parent running
trap 'echo parent exiting; exit' 0
trap :2
child
sleep 1000
#!/bin/bash #child
echo child started. pid is $$
trap 'echo child got signal 2; exit' INT
sleep 1000
Examples Useful Scripts or Function
# Function top5 Example how to set defaults
# Usage top5 {n} #list n processes
function top5 {
ps -ef | head -${1:-5}
}
# Function hereis Example of HERE IS FILE and handling arguments
# Usage hereis word1 word2 ...
function hereis {
for name in "$@"
do
cat </dev/tty #ask user to select
read ans #read answer from standard in
case $ans in #Check choices
y*) echo $name;; #selected
q*) break;; #skip rest of arguments
*) continue;; #skip item
esac
done
}
# Function acal Display a nicer calendar
# but will accept Alphabetic month
function acal {
m=""
case $# in
0) cal; return;; #no arguments
1) m=$1; y=`date +%Y`;; #1 argument
2) m=$1; y=$2;; #2 arguments
esac
case $m in
Jan*|jan* ) m=1;;
Feb*|feb* ) m=2;;
Mar*|mar* ) m=3;;
Apr*|apr* ) m=4;;
May|may } m=5;;
Jun*|jun* ) m=6;;
Jul*|jul* ) m=7;;
Aug*|aug* ) m=8;;
Sep*|sep* } m=9;;
Oct*|oct* ) m=10;;
Nov*|nov* ) m=11;;
Dec*|dec* ) m=12;;
[1-9]|1[0-2] ) ;; #numeric month
*) ) y=$m; m="";;
esac
cal $m $y
}
## Function selectex - Example select
#
function selectex () {
choices="/bin /usr /home"
select selection in $choices; do
if [ $selection ]; then
ls $selection
break
else
echo 'Invalid selection'
fi
done
}
# Function fwhich Which command in $PATH is executed
#
function fwhich {
if [[ $# -eq 0 ]] ;
then cat << EndOfHelp 1>&2; return 2
Usage fwhich command #Example of parsing the $PATH
Return 0 - found
Return 1 - not found
Return 2 - No arguments
EndOfHelp
fi
for path in `echo $PATH | sed 's/^:/.:/
s/::/.:/g
s/:$/:./
s/:/ /g'`
do
if [[ -x $path/$1 ]] ; # does executable file exists here?
then echo $path/$1 # found it
return 0
fi
done
return 1 # not found
}
# Name: overwrite Copy standard input to output after EOF
function overwrite {
if [[ $# -lt 2 ]] ;
then echo "Usage: overwrite file command [args]" 1>&2; return 2
fi
file=$1; shift
new=/tmp/overwrite1$$; old=/tmp/overwrite2$$
trap 'rm -f $new $old; return 1' 1 2 15 # clean up files
"$@" > $new
if [[ $? -eq 0 ]] ; # collect output
then # command completed successfully
cp $file $old # save original file
trap '' 1 2 15 # we are committed; ignore signals
cp $new $file # copy new file into file
rm -f $new $old # remove temp files
else
echo "overwrite: $1 failed, $file unchanged" 1>$2
return 1
fi
}
# Name: zgrep
# Purpose: caseless grep of gzip files
# Usage: zgrep text files.gz
#
function zgrep {
if [ $# -eq 0 ] ; then
echo "Usage: zgrep grep_text files.gz"
return 2
fi
text=$1
shift
while [ $# -gt 0 ]
do
echo $1
gzip -cd $1 | grep -i $text
shift
done
}
# Name: hgrep
# Purpose: highlighting grep
# Usage: hgrep pattern files
#
function hgrep {
if [ $# -lt 2 ] ; then
echo "Usage: hgrep pattern files"
return 2
fi
pattern=$1;shift
sep=$'\001' #note use of $' ' to create control characters
bold=$'\e[1m'; off=$'\e[0m'
underline=$'\e[4m'; reverse=$'\e[7m' #other choices of highlighting
sed -n "s${sep}${pattern}${sep}${reverse}&${off}${sep}gp" $*
}