Department of
Computer & Electrical Engineering & Computer Science
California State University, Bakersfield

Shell Programming Basics

Written by Donna Meyers in February 2002, editted by Alberto Cruz March 2026

CREATING A SCRIPT

Each UNIX shell (bash, csh, ksh, sh) has a specific programming (scripting) language. Scripts in this article use bash (a derivative of the bourne shell). Scripts must be ASCII text and written with a text editor such as emacs, vi, or pico. A shell script contains instructions like source code for programming languages such as C/C++, but scripts are not compiled into an executable. Scripts are interpreted and executed line by line by the shell. Scripts must have execution permission. The first line of a script controls which program will execute the script (in the example below it is bash). The syntax is #! followed by the full path of the program.

#!/usr/bin/env bash

Assume you have written and saved a script called myscript. Since the default permission set for text files is 644, you must explicitly make your script executable. This command will make myscript executable for you alone and no permission otherwise:

chmod 700 myscript

Start your script by typing:

myscript

or

./myscript (if you do not have a . in your PATH)

DEBUGGING

Simple debugging can be done with echo. You can use echo to print specific variables around the place where you suspect the mistake. Scripts can be executed in debug mode by adding the following line at the beginning of the script:

set -x

COMMENTS

Comments in shell programming start with # and go until the end of the line.

VARIABLES

In shell programming all variables have the datatype string. You do not need to declare variables before use. The following statements will assign values to variables. The use of quotes mirrors the use of quotes at the command line:

myfirstname=samantha
myname="samantha spade"
myid=54321

To get the value back you just put a dollar sign in front of the variable:

#!/usr/bin/env bash
# assign a value:
a="hello world"
# now print the content of "a":
echo "A is:"
echo $a

This script will print:

A is:
hello world

Use {} to separate a variable embedded in a literal:

num=2
echo "this is the $numnd"
echo "this is the ${num}nd"

The first statement will print "this is the " (not "this is the 2nd") because the shell searches for a variable called numnd which has no value.

From within a script you also have access to user-defined environment variables (variables set by the user in a script and exported for use) and shell variables (read-only variables set by the shell). These variables are always completely uppercase. Typical user-defined environment variables are TERM, PATH, EDITOR, USER. In a script type $varname, e.g. $PATH to access the variable. Shell variables are set by the shell at login. Examples are BASH_VERSION, HOME.Type 'man bash' for a complete listing of all shell variables (search for 'Shell Variables'. Type 'set' for a listing of all your current environment variables.

SHELL COMMANDS AND CONTROL STRUCTURES

A shell script can use any UNIX command that can be typed at the command line. Shell scripts also have access to shell control structures that will work only in a script. UNIX commands used in this document include:

Just as at the command line, the output of a UNIX command can also be used as a command line argument for another command or assigned to a variable. This is called command substitution. In the past it wass denoted by a set of backquotes or backticks (`). With modern scripting, you enclose the command with $()$.

tar -zcvf lastmod.tar.gz $(find . -mtime -1 -type f -print)

or

today=$(date | cut -f 1 -d " ")
echo $today

# evaluate the expression and store in result
result=$(expr 2 + 3)

The shell script language offers the same control structures that programming languages offer: conditions, loops, and breaks. Control structures only make sense in a script, not at the command line. The if statement tests if the condition is true (exit status is 0 for success). If the condition is true, the "then" part gets executed:

if ....; then
    ....
elif ....; then
    ....
else
    ....
fi

The test operator can be used to compare strings or test if a file exists, is readable etc... (type 'man test' for details). The "test" command is written as square brackets " [ ] ". Note that space is significant here: Make sure that you always have spaces around the brackets. Examples:

Example:

#!/usr/bin/env bash
if [ "$SHELL" = "/usr/bin/bash" ]; then
    echo "your login shell is the bash (bourne again shell)"
else
    echo "your login shell is not bash but $SHELL"
fi

if [ -f $HOME/.profile ]; then
    echo "your .profile exists"
else
    echo "your .profile does not exist"
fi

The AND (&&) and OR (||) operators can be used as shorthand if statements. You can do everything without the ANDs and ORs using just if-statements; sometimes the shortcuts AND and OR are more convenient. In the AND statement the right side is executed if the left is true; in the OR statement the right side is executed if the left is false. In the following, the echo statement will be executed if the file /etc/shadow exists.

[ -f "/etc/shadow" ] && echo "This computer uses shadow passwords"

In this script an error message is displayed and the script is exited if the mailfolder is NOT readable:

#!/usr/bin/env bash
mailfolder=/var/spool/mail/james
[ -r "$mailfolder" ] || { echo "Can not read $mailfolder" ; exit 1; }
echo "$mailfolder has mail from:"
grep "^From " $mailfolder


A few notes about this code:

• the syntax of the shorthand OR allows for only one command on the right. To treat two statements as one command, group them together in an anonymous function using curly braces.
• the variable $mailfolder is enclosed in quotes to allow for the possibility of a null string (a null string might produce an error message). you should do this whenever you cannot quarantee the value of a variable

The case statement is a conditional control structure that can be used to match (using shell wildcards such as * and ?) a given string against a number of possibilities. The ';;' is required and marks the end of a case option.

case ... in
    pattern)
        do something here
        ;;
    another pattern)
        do something else here
        ;;
esac

Assume you have a gzipped file named lf.gz. The command:

file lf.gz

Returns:

lf.gz: gzip compressed data, deflated, original filename, last modified: Mon Aug 27 23:09:18 2001, os: Unix

This script called smartzip will uncompress bzip2, gzip and zip compressed files automatically :

#!/usr/bin/env bash
ftype=$(file "$1")
case "$ftype" in
    "$1: Zip archive"*)
        unzip "$1"
        ;;
    "$1: gzip compressed"*)
        gunzip "$1"
        ;;
    "$1: bzip2 compressed"*)
        bunzip2 "$1"
        ;;
    *)
        error "File $1 can not be uncompressed with smartzip"
        ;;
esac

Note the special variable called $1. This variable contains the first of any number of command line arguments. See the section below for details. Here is an example:

#!/usr/bin/env bash
echo "What is your favourite OS?"
select var in "dUNIX" "Solaris" "Free BSD" "Other"; do
    break done
echo "You have selected $var"


Here is what the script does:

What is your favourite OS?"
1) dUNIX "
2) Solaris "
3) Free BSD"
4) Other"
#? 1"
You have selected dUNIX

In the shell you have the while-loop, for-loop, and until-loop control structures:

while ...; do     .... done

The while-loop will run while the expression that we test for is true. The keyword "break" can be used to leave the loop at any point in time. The keyword "continue" forces the loop to continue with the next iteration and skips the rest of the loop body.

The for-loop takes a list of strings (strings separated by space) and assigns them to a variable:

for var in ....; do
    ....
done

E.g. the following will print the letters A to C on the screen:

#!/usr/bin/env bash
for var in A B C ; do
    echo "var is $var"
done

ARITMETIC AND RELATIONAL OPERATORS

COMMAND LINE ARGUMENTS

The script variables $* and $1, $2 ... $9 contain the arguments that the user specifies on the command line. $* is the complete command line, $1 is the first argument, $2 the second, etc. The special variable $# contains the number of arguments in $*; i.e., if there are 4 arguments $# is 4. The shift operator moves the arguments in $* one place to the left; i.e. if there are 3 arguments ($# = 3) the following occurs: $1 = $2, $2 = $3, the original value of $1 is lost, and $# = 2.

When the command line syntax of your script is complicated; i.e. you cannot guarantee the number or order of the arguments you will need a sophisticated way to parse the command line. One such method is to use a while loop combined with a case and shift statement.

#!/usr/bin/env bash
# Funtion: command line parser.
#
# USAGE EXAMPLE: parser [-m hello] filename [-h]
if [ $# -eq 0 ];
then
echo "USAGE: parser [-m hello] filename [-h]"
fi
# while [ $# -ne 0 ]; do
while [ -n "$1" ]; do
case $1 in
    -h) echo "USAGE: parser [-m hello] filename [-h]";
        shift
        ;;
    -m) message=$2;
        # grab the next argument past -d
        echo "message: $message"; # display it
        shift;
        shift
        ;; # move past 2 command line arguments
    -*) echo "error: no such option $1. -h for help";
        exit 1
        ;;
    *) filename=$1;
        echo "filename: $filename";
        shift
        ;; esac
done


If you run the script with:

parser -m hello somefile -h

It produces

message: hello
filename: somefile
USAGE: parser [-m hello] [filename] [-h]

The script loops through all arguments and matches them against the case statement. If it finds a matching one it sets a variable and shifts the command line by one or two and continues on. The script will work regardless of the order and number of arguments.

QUOTING

Before passing any arguments to a program the shell tries to expand wildcards and variables. To expand means that the wildcard (e.g. *) is replaced by the appropriate file names or that a variable is replaced by its value. To change this behaviour you can use quotes: Let's say we have a number of files in the current directory. Two of them are jpg-files, mail.jpg and tux.jpg.

#!/usr/bin/env bash
echo *.jpg

This will print "mail.jpg tux.jpg".

Quotes (either single or double) will prevent this wildcard expansion:

#!/usr/bin/env bash
echo "*.jpg"
echo '*.jpg'

This will print "*.jpg" twice. Single quotes are most strict. They prevent even variable expansion. Double quotes prevent wildcard expansion but allow variable expansion:

#!/usr/bin/env bash
echo $SHELL
echo "$SHELL"
echo '$SHELL'

This will print:

/usr/bin/bash
/usr/bin/bash
$SHELL


Finally there is the possibility to take the special meaning of any single character away by preceeding it with a backslash:

echo \*.jpg
echo \$SHELL


This will print:

*.jpg
$SHELL

HERE DOCUMENTS

Here documents are a nice way to send several lines of text to a command. It is quite useful to write a help text in a script without having to put echo in front of each line. A "Here document" starts with << followed by some string that must also appear at the end of the here document. Here is an example script, called ren, that renames multiple files and uses a here document for its help text:

#!/usr/bin/env bash
# we have less than 3 arguments. Print the help text:
if [ $# -lt 3 ] ; then
cat << HELP
ren -- renames a number of files using sed regular expressions

USAGE: ren 'regexp' 'replacement' files...

EXAMPLE: rename all *.HTM files in *.html:
ren 'HTM$' 'html' *.HTM
HELP
    exit 0
fi
OLD="$1"
NEW="$2"
# The shift command removes one argument from the list of
# command line arguments.
shift
shift
# $* contains now all the files:
for file in $*; do
    if [ -f "$file" ] ; then
        newfile=$(echo "$file" | sed "s/${OLD}/${NEW}/g")
    if [ -f "$newfile" ]; then
        echo "ERROR: $newfile exists already"
    else
        echo "renaming $file to $newfile ..."
        mv "$file" "$newfile"
    fi
fi
done


The first if-statement tests if we have provided at least 3 command line parameters. (The special variable $# contains the number of arguments.) If not, the help text is sent to the command cat which in turn sends it to the screen. After printing the help text we exit the program. If there are 3 or more arguments we assign the first argument to the variable OLD and the second to the variable NEW. Next we shift the command line parameters twice to get the third argument into the first position of $*. With $* we enter the for loop. Each of the arguments in $* is now assigned one by one to the variable $file. Here we first test that the file really exists and then we construct the new file name by using find and replace with sed.

The backticks are used to assign the result to the variable newfile. Now we have all we need: The old file name and the new one. This is then used with the command mv to rename the files.

FUNCTIONS

As soon as you have a more complex program you will find that you use the same code in several places and also find it helpful to give it some structure. A function looks like this:

functionname()
{
    # inside the body $1 is the first argument given to the function
    # $2 the second ...
    body
}

You need to "declare" functions at the beginning of the script before you use them. Here is a script called xtitlebar which you can use to change the name of a terminal window. If you have several of them open it is easier to find them. The script sends an escape sequence which is interpreted by the terminal and causes it to change the name in the titlebar. The script uses a function called help. As you can see the function is defined once and then used twice:

#!/usr/bin/env bash
# vim: set sw=4 ts=4 et:

help()
{
    cat << HELP
xtitlebar -- change the name of an xterm, gnome-terminal or kde konsole

USAGE: xtitlebar [-h] "string_for_titlebar"

OPTIONS: -h help text

EXAMPLE: xtitlebar "cvs"

HELP
    exit 0
}

# in case of error or if -h is given we call the function help:
[ -z "$1" ] && help
[ "$1" = "-h" ] && help

# send the escape sequence to change the xterm titlebar:
echo -e "\033]0;$1\007"
#


It's a good habit to have extensive comments inside your scripts.

EXAMPLES

The following script b2d converts a binary number (e.g 1101) into its decimal equivalent. It is an example that shows that you can do simple mathematics with expr:

#!/usr/bin/env bash
# vim: set sw=4 ts=4 et:
help()
{
    cat << HELP
b2h -- convert binary to decimal

USAGE: b2h [-h] binarynum

OPTIONS: -h help text

EXAMPLE: b2h 111010
will return 58
HELP
    exit 0
}

error()
{
    # print an error and exit
    echo "$1"
    exit 1
}

lastchar()
{
    # return the last character of a string in $rval
    if [ -z "$1" ]; then
        # empty string
        rval=""
        return
    fi
    # wc puts some space behind the output this is why we need sed:
    numofchar=$(echo -n "$1" | wc -c | sed 's/ //g' )
    # now cut out the last char
    rval=$(echo -n "$1" | cut -b $numofchar)
}

chop()
{
    # remove the last character in string and return it in $rval
    if [ -z "$1" ]; then
        # empty string
        rval=""
        return
    fi
    # wc puts some space behind the output this is why we need sed:
    numofchar=$(echo -n "$1" | wc -c | sed 's/ //g' )
    if [ "$numofchar" = "1" ]; then
        # only one char in string
        rval=""
        return
    fi
    numofcharminus1=$(expr $numofchar "-" 1)
    # now cut all but the last char:
    rval=$(echo -n "$1" | cut -b 0-${numofcharminus1})
}


while [ -n "$1" ]; do
case $1 in
    -h) help;shift 1;; # function help is called
    --) shift;break;; # end of options
    -*) error "error: no such option $1. -h for help";;
    *) break;;
esac
done

# The main program
sum=0
weight=1
# one arg must be given:
[ -z "$1" ] && help
binnum="$1"
binnumorig="$1"

while [ -n "$binnum" ]; do
    lastchar "$binnum"
    if [ "$rval" = "1" ]; then
        sum=$(expr "$weight" "+" "$sum")
    fi
    # remove the last position in $binnum
    chop "$binnum"
    binnum="$rval"
    weight=$(expr "$weight" "*" 2)
done

echo "binary $binnumorig is decimal $sum"
#


The algorithm used in this script takes the decimal weight (1,2,4,8,16,..) of each digit starting from the right most digit and adds it to the sum if the digit is a 1. Thus "10" is:

0 * 1 + 1 * 2 = 2

To get the digits from the string we use the function lastchar. This uses wc -c to count the number of characters in the string and then cut to cut out the last character. The chop function has the same logic but removes the last character, that is it cuts out everything from the beginning to the character before the last one.

Perhaps you are one of those who save all outgoing mail to a file. After a couple of months this file becomes rather big and it makes the access slow if you load it into your mail program. The following script rotatefile can help you. It renames the mailfolder, let's call it outmail, to outmail.1 if there was already an outmail.1 then it becomes outmail.2 etc...

#!/usr/bin/env bash
# vim: set sw=4 ts=4 et:
ver="0.1"
help()
{
    cat << HELP
rotatefile -- rotate the file name

USAGE: rotatefile [-h] filename

OPTIONS: -h help text

EXAMPLE: rotatefile out
This will e.g rename out.2 to out.3, out.1 to out.2, out to out.1
and create an empty out-file

The max number is 10

version $ver
HELP
    exit 0
}

error()
{
    echo "$1"
    exit 1
}
while [ -n "$1" ]; do
case $1 in
    -h) help;shift 1;;
    --) break;;
    -*) echo "error: no such option $1. -h for help";exit 1;;
    *) break;;
esac
done

# input check:
if [ -z "$1" ] ; then
error "ERROR: you must specify a file, use -h for help"
fi
filen="$1"
# rename any .1 , .2 etc file:
for n in 9 8 7 6 5 4 3 2 1; do
    if [ -f "$filen.$n" ]; then
        p=$(expr $n + 1)
        echo "mv $filen.$n $filen.$p"
        mv $filen.$n $filen.$p
    fi
done
# rename the original file:
if [ -f "$filen" ]; then
    echo "mv $filen $filen.1"
    mv $filen $filen.1
fi
echo touch $filen
touch $filen


How does the program work? After checking that the user provided a filename we go into a for loop counting from 9 to 1. File 9 is now renamed to 10, file 8 to 9 and so on. After the loop we rename the original file to 1 and create an empty file with the name of the original file.