Lab 2: Bash Scripting

Shell comments and the curious #!

In the shell, # starts a comment. So in the bash shell we can do this

bear[1][~/]> echo $USER # This will print my user name.
wcbrown

See how the # and everything after is ignored? Now, on the command-line this isn't very interesting. But of course in a file of commands that we are going to evaluate in the shell using source, it can be handy.

# This adds my personal bin directory to my path
PATH=$PATH:${HOME}/bin

There is one very important comment. If the first line of a file full of bash commands starts with #!/bin/bash and the file has executable permissions set, the file can be executed as a program, and and the result is (more or less) what you would've gotten if you'd use source.

file wwww example runs

#!/bin/bash echo ${USER}@${HOST} date pwd

bash-3.00$ source wwww wcbrown@michcsdbrownu Sun Jan 25 23:37:28 EST 2009 /home/wcbrown/courses/IC221/classes/L06 bash-3.00$ ./wwww wcbrown@michcsdbrownu Sun Jan 25 23:37:40 EST 2009 /home/wcbrown/courses/IC221/classes/L06

Remember to make the file executable! This kind of "program" is called a shell script, and these can be very powerful. In fact, if there is a #!whatever at the beginning of an executable file, the shell tries to run whatever as a program with that file as an argument. It doesn't need to be a shell. At any rate, there is one difference between source www and wwww: in the former case the commands in the file wwww are executed by the current shell, in the later case, a new shell is spawned and the commands executed in the new shell. The most important consequence of this is that any variables that have not been exported won't be there in the new shell that's executing the commands.

Lab Part 1: Create a directory lab02, copy /home/wcbrown/courses/IC221/labs/L02/labdata.tar into the lab02 directory, and untar it. Create a shell script lab2p1 that does nothing more than write out the size (in bytes) of the file foo.txt (which was created when you unpacked labdata.tar). You should be able to write an ls/cut or a wc shell command line that accomplishes this and stick it in your lab2p1 script.
You should be able to run your lab2p1 script like this:
bash$ lab2p1
42 
	  
Make sure that your name appears in a comment within the file!!!

Command-line arguments in scripts

Since you're calling shell scripts as programs, they should have command-line arguments, right? Inside your script, the special variables $1, $2, $3, ... reference the first, second, third, .... arguments. The variable $# evaluates to the number of command-line arguments. Here's an example of a simple script that uses a command-line argument. You give it an alpha code of a mid with a Unix account with the department, and it'll give you the mid's name.

alpha2name example runs

#!/bin/bash finger -m m$1 | head -n 1 | cut -d':' -f3

bash-3.00$ ./alpha2name 129999 Midn Albert Michelson {SCS}

The finger command takes a username and gives some info about the name, info which we extract with head and cut. Since $1 is expected to be the alpha number, we prefix it with an "m" to get the user name.

Lab Part 2: Copy your lab2p1 script to a new file named lab2p2. Extend/modify your previous script, now copied in lab2p2, so that the file whose size is getting checked is not hardcoded as foo.txt, but instead, the filename comes from the user as a command-line argument. You should be able to run your lab2p1 script like this:
bash$ lab2p2 foo.txt
42 
bash$ lab2p2 bar.txt
100
	  

Conditionals: ifs, elses, ifelses

If there are no command line arguments, alpha2name really ought to print out a usage message. To do this we need conditionals. The syntax for if's in bash is:

if [ test-command ]
  then
    commands
  else
    commands
fi

NOTE: the spaces around the [ ]'s on both sides is required, if the then appears on the same line as the [ ]'s there must be a semi-colon before the "then", and the else part can just be dropped if you want. [In fact, then, else and fi are all commands (just like if, so they all need to be separated by newlines or ;'s, just like cd and ls in cd ~ ; ls -l. ] Also note that the command exit takes you out of the script immediately. Thus, in our script we get

alpha2name

#!/bin/bash if [ $# -lt 1 ] then echo "alpha2name <alpha-code>" else finger -m m$1 | head -n 1 | cut -d':' -f3 fi

You'll notice we use -lt as a less-than operator. For numerical values the comparison operators are -eq, -ne, -lt, -gt, -le, -ge. For strings, = and !=. The and operator is -a, or is -o, and not is !. You can group boolean expressions with ( )'s. Additionally, you can test file names in several interesting ways:

fnt

#!/bin/bash if [ -d $1 ] ; then echo "$1 exists and is a directory!" ; fi if [ -e $1 ] ; then echo "$1 exists!" ; fi if [ -f $1 ] ; then echo "$1 exists and is not a directory!" ; fi if [ -r $1 ] ; then echo "$1 exists and is readable!" ; fi if [ -s $1 ] ; then echo "$1 exists and has size greater than zero!" ; fi if [ -w $1 ] ; then echo "$1 exists and is writable!" ; fi if [ -x $1 ] ; then echo "$1 exists and is executable!" ; fi # NOTE: Since everything's on the same line, I need ;s between # the if and the then and between the then and the fi.

bear[2] [~/]> ./fnt alpha2name alpha2name exists! alpha2name exists and is not a directory! alpha2name exists and is readable! alpha2name exists and has size greater than zero! alpha2name exists and is writable! alpha2name exists and is executable! bear[3] [~/]> ./fnt /usr/bin /usr/bin exists and is a directory! /usr/bin exists! /usr/bin exists and is readable! /usr/bin exists and has size greater than zero! /usr/bin exists and is executable!

Lab Part 3: Copy your lab2p2 script to a new file named lab2p3. Extend/modify your previous script, now copied in lab2p3, so that
when called with no arguments, lab2p3 prints out a usage statement
lab2p3 [FILE] ...
	      
When called with an argument that is not a valid pathname prints the following error message to stderr:
lab2p3: cannot access <file>: No such file or directory
	      
... where <file> is the argument. Now, most messages in shell scripts are written with echo, and echo sends its output to stdout. There's a clever little way to get the job done: we can redirect stdout of echo to stderr! How? Well, there are special "files" in Unix, /dev/fd/0, /dev/fd/1, and /dev/fd/2 that actually refer to a process's stdin, stdout and stderr, respectively. so here's an example of redirecting echo's stdout to stderr:
echo "This message goes to stderr" > /dev/fd/2
	      
The > redirects stdout, like always, but where does stdout get redirected? To /dev/fd/2, which is just stderr.
When called with an argument that is a directory, rather than a file writes out a zero as its result, but prints no error messages.
You should be able to run your lab2p3 script like this:
bash$ lab2p3 
lab2p3 [FILE] ...
bash$ lab2p3 foo.txt
42
bash$ lab2p3 ~                 # since this is a directory we should get zero
0
bash$ lab2p3 lkj               # presumably there is no such file
lab2p3: cannot access lkj: No such file or directory
bash$ lab2p3 lkj > /dev/null   # this test makes sure you are really writing to stderr
lab2p3: cannot access lkj: No such file or directory
	  

For loops

Suppose we wanted to allow for many alphas on the command-line and get the names for all of them? The $@ variable is all the command-line arguments, so we want to loop over each element of $@. We'll use a for loop to do this. For loops in bash are very different from C/C++ ... different from basic Java stuff too, though similar to looping over a collection. By default, for iterates through each of the command-line arguments. The syntax is

for var-name
do
  commands using var-name
done

Whatever name you choose for var-name, it takes on each of the values on the command line in consecutive iterations. Now, the following version of alpha2name finds the names of mids from any number of alpha codes.

#!/bin/bash

if [ $# -lt 1 ]
   then
     echo "alpha2name <alpha-code-1> ... <alpha-code-k>"
   else
     for alpha
     do
         finger -m m$alpha | head -n 1 | cut -d':' -f3
     done
fi

In general, you can iterate over the white-space-separated words in any string.

`junk`	sample run
#!/bin/bash foo="the rain in spain falls mainly on the plain" for x in $foo do echo $x done	bear[4] [~/]> ./junk the rain in spain falls mainly on the plain

Lab Part 4: Copy your lab2p3 script to a new file named lab2p4. Extend/modify your previous script, now copied in lab2p4, so that more than one argument can be given, and the result is simply to print the file size (or zero for directory, or error message for non-existent path) for each argument, one-per-line. You should be able to run your lab2p4 script like this:
bash$ lab2p4
lab2p4 [FILE] ...
bash$ lab2p4 foo.txt ~ lkj bar.txt 
42
0
lab2p4: cannot access lkj: No such file or directory
100
bash$ lab2p4 foo.txt ~ lkj bar.txt > /dev/null
lab2p4: cannot access lkj: No such file or directory
	  

$( ) for command evaluation, $(( )) for numeric evaluation

Sometimes you wnt to store the output of a command in a variable. If you wrap a command in $( ), the output (stdout) of the command is returned as a string. So, for example, the following script prints the names of all the Class of 2009 with CS Department accounts.

#!/bin/bash

alphas09=$(grep m09 /etc/passwd | cut -d":" -f1 | cut -d"m" -f2)
for alpha in $alphas09
do
  alpha2name $alpha
done

Of course, since we have our new and improved alpha2name that processes an arbitrary number of alphas from the command-line, we can also do it all like this:

alpha2name $(grep m09 /etc/passwd | cut -d":" -f1 | cut -d"m" -f2)

... because the $( ) expression will expand to the list of alphas, which then become the arguments to alpha2name. In other words, the bash shell evaluates the $( ) expression before it spawns off a process for alpha2name, so the argument list for alpha2name is the result of the $( ) expression.

To evaluate arithmetic expressions, you need to wrap the expression up in $(( )).

bash-3.00$ foo=5
bash-3.00$ echo foo
foo
bash-3.00$ echo $foo
5
bash-3.00$ echo $foo + 1
5 + 1
bash-3.00$ echo $(($foo + 1))
6

Here's a cute little program that does some basic arithmetic.

#!/bin/bash

# This program prints the number of 
# files in this directory

tot=0
sizes=$(ls -l)
for x in $sizes
do
  tot=$(($tot + 1))
done
echo $tot

Lab Part 5: Copy your lab2p4 script to a new file named lab2p5. Extend/modify your previous script, now copied in lab2p5, so that You should be able to run your lab2p4 script like this:
bash$ lab2p5 foo.txt ~ lkj bar.txt 
lab2p5: cannot access lkj: No such file or directory
142
bash$ lab2p5 foo.txt ~ lkj bar.txt > /dev/null
lab2p5: cannot access lkj: No such file or directory
	  
Finally, there's another test to run that gives us a nice opportunity to learn a good shell trick. The shell allows the wildcards * and ? in describing filenames. The * matches zero or more characters, the ? matches one character. So,
bash$ lab2p5 fed*.txt   # try   echo fed*.txt    to see what files you got
49267
bash$ lab2p5 fed?.txt   # try   echo fed?.txt    to see what files you got
19585
bash$ lab2p5 fed??.txt  # try   echo fed??.txt   to see what files you got
29682
bash$ lab2p5 *          # try   echo *           to see what files you got
???

while and case

Bash also has while loops, the syntax is

while test-command
do
  commands
done

Bash also has case statements. See pages 460-462 A Practical Guide to Linux, or look it up online or in the man page.

Input and output

Usually shell scripts use echo to provide output. Input usually comes from the command-line, but you can also read from stdin with the read command.

$ read foo
ImTypingThis
$ echo foo is $foo
foo is ImTypingThis

Often this is used to get user feedback like

#!/bin/bash

echo "I'm going to delete all your files, is this OK? (y/n)"
read response
if [ $response = "y" ]
then
  echo "Really, I was only kidding!"
else
  echo "Good call!"
fi

Submission

This lab is due by the start of your lab period on 1 February. Submit your lab02 directory using the same submit script you used for Lab 1. Your submission must include:

A README file that is a simple text file with your name and alpha and a clear indication of which part of the lab you are submitting for a grade. You will submit the last part that you actually got working completely correctly. If it doesn't work, don't list it as the part you are submitting.
The script file lab2pX, where your README file indicated that you were submitting a Part X solution to be graded. (Make sure your name appears in this file!)

A file ans.txt containing answers to the following questions (make sure your name appears in the file!)

1. Remember, if the first two characters in a file are #!
   then the file is treated as a script.  What immediately follows the
   "#!" is the name of the program to be executed, and that programs 
   stdin is set to come from the scriptfile.  Create a file tmp with 
   the initial line 

   #!/foobar 

   set its permissions to executable, and execute it.  
   
   a) What message do you get?

   b) Who's giving the error message?

   c) Why are you getting an error message?

2. The lab instructs you to begin your script's error message with
   the name of the script itself.  In fact most utilities work like
   this --- rm for instance.  But isn't this stupid?  I (the user)
   called rm so I don't need it to tell me that it's the one writing
   out the error message.  What's wrong with this logic, i.e. why is
   it really important to begin error messages with the name of the
   program that's writing the message?
   
3. A lot of big applications are launched by a script.  For example,
   if you type "firefox" at the shell prompt, what the shell starts
   executing is not the firefox binary, but rather a script that does
   a bit of work and then launches the actual compiled application.
   Another program like that is netbeans: /usr/local/bin/netbeans 
   is actually a shell script, but not with bash shell.  What shell
   is it scripted in?  How did you figure it out?

4. Explain what  /usr/*/x??  means on the command-line.  What would it
   take for a file to match this?

Dave Stahl

`alpha2name`	example runs
#!/bin/bash finger -m m$1 \| head -n 1 \| cut -d':' -f3	bash-3.00$ ./alpha2name 129999 Midn Albert Michelson {SCS}