COMP2100/2500
Lab 5: Shell programming

Summary

You will get some experience interacting with the shell and writing some simple bash scripts.

Aims

Preparation

Exercise 1

Open a terminal window and enter the following command:

[comp2100@partch]$ echo Hello world

You should see the response Hello world appear as a result.

That command was interpreted by tcsh, your default shell. For these exercises we want to use the bash shell. To start bash just enter the command bash. When you do this, the prompt will change to something ending with a ‘$’, the default prompt for all shells in the Bourne shell family.

Enter the command to print ‘Hello world’ again, this time to bash. You should get the same result. One shell is pretty much like another when processing simple commands.

You can exit bash either by typing the command exit, or by typing Ctrl-D (hold down the Control key and press d). In UNIX, Ctrl-D signals the end of input.

Exercise 2

Create a directory called bin in your home directory (if you don’t already have one).
Don't use the File Manager GUI to do this, use the ‘mkdir’ command. If you don't know how it works, look up the manual page by typing ‘man mkdir’.
This directory should be one place your shell will go looking for commands that are not built in to the shell. If you write a program of your own that you want to use on a regular basis, just copy it to this directory. Then, whenever you type the name of your program as a command, the shell will find it here and run it.
Check your shell PATH.
For a complete list of all the places the shell will look, known as a ‘search path’, enter the command echo ${PATH}. Do this now to check that your bin directory is on your path.)

If the value of the PATH variable does not include your home bin (it should appear as ~/bin or similar) then change the value of your PATH variable to add a colon character and the name of your bin directory: 

PATH=${PATH}:~/bin
Create your hello command file

Create a file called hello in your bin directory with the following contents:

#!/bin/bash
echo Hello world

Next set the executable bit on this file with the command ‘chmod +x hello’.

Execute your own command

You should now be able to run your new command by typing its name hello. You don't have to be in the same directory as your command for this to work. Try moving to another directory and typing hello again.

Search paths: the shell is one of the Unix tools that knows how to search for named files along a pathway. Other tools have pathways that are defined separately: PATH for the shell(s), MANPATH for the man command, classpath and sourcepath (for the Java compiler), for example. In many cases the path is defined as the value of a shell environment variable.

Exercise 3

Make a working directory

Make a new directory (say comp2100/labs/lab4 or something like that) and download the following Java program StringCount.java into it. Compile the program by typing

javac StringCount.java

This program takes two string arguments from the command line and returns a count of how many times the first string can be found in the second. Occurrences must be contiguous; for example, ‘cat’ does not occur in ‘cxaxt’. Occurrences may overlap, for example ‘wowow’ occurs twice in ‘wowowow’.

Well, that's what it's supposed to do. In reality, it doesn't come close to working. Try it out by typing

java StringCount abc abcd

You should get the answer ‘1’, but you won't.

It's going to take a long time to test this program if we have to enter each test case by hand, and re-enter it by hand after every modification. Chances are we'll end up making as many mistakes as the program. You could use JUnit and write a test class and methods. But this time let's write a shell script to do it instead.

Create and save the test case inputs
Start by creating a file called cases to contain your test cases. Each line should contain two strings. Here are a few lines to get you started. Add some more of your own.

x x
x xx
wowow wowowowow
Create a prototype shell script file

The following simple shell script will read two words from each line of its input and print those words out again. Type (or cut and paste) the following text into a file named tester.

#!/bin/bash
while read x y
do
  echo ${x} ${y}
done

The key feature of the script is the read command, which we didn't cover in lectures. It reads a line of input and assigns the first word to the variable x and the rest of the line to the variable y. The read command gives an exit code of 0 (= success = True) if was was able to read the words, or 1 (= failure = False) if it encounters the end of its input. That makes it perfect for using in a while loop, as in this example. When the input runs out, it returns 1, and the loop terminates.

Save, make executable, and run

Save this script in a file called tester, make it executable, and run it over your test input by typing 

./tester < cases

Hint for emacs users: When editing shell scripts you may find it helpful to put Emacs into ‘shell-script-mode’. If you edit an existing script, this should happen all by itself, but when you start a new script you need to tell Emacs to change mode by typing M-x shell-script-mode. (The M stands for ‘Meta’, and it works just like Shift or Control: hold down the Alt key while you type an ‘x’, or the <‘code>Esc’ key followed by ‘x’.)

Exercise 4

This is where the thinking work starts.

Modify tester so that for each input line of the form ‘string1 string2’, it outputs the number of occurrences of string1 that StringCount claims are in string2.

Note: This does not mean rewrite the Java program as a bash script. It means write a bash script that runs the Java program on each line of its input and formats the results as a readable report.

Exercise 5

Save this version and continue development.

Save the current version of your script and your cases file for later. (You will need these again in Exercises 8 and 9.) Do this by copying tester to tmp_tester and cases to tmp_cases using the cp command.

Define the test cases.

Modify your cases file so that each line now contains the expected test case output as well as the inputs. Each line now has 3 things, namely: the two words as before, plus the correct number of occurrences of the first word in the second. For example, your file will start:

x x 1
x xx 2
wowow wowowowow 3
Make the script check the expected output.

Modify your tester script so that for each test it prints a line of the following form:

Test i: string1 string2: Expected n, got m

where i is the line number of this test, n is the correct number of occurrences of string1 in string2, and m is the number found by StringCount.

Exercise 6

Report success!

Modify your tester so that when StringCount actually gets the right answer (a rare occurrence, but you should be able to find one), then it instead prints a line of form:

Test i: Passed

The key to this (unsurprisingly) is to use an if statement in your shell script. Remember that the form of an if statement is as follows:

if command-list
then
    command-list
else
    command-list
fi

Note: The line breaks (or semicolons in their place) are required. You can leave out the else part if you don't need it.

Look at the manual page for the test command (by typing man test or info test). That should tell you everything you need to know now to complete this exercise. The test command evaluates a boolean expression (as well as being able to test for all sorts of other conditions). It basically lets you program in Bash in a way very like what you would do in Java.

Exercise 7

The test command is so commonly used in shell scripts that it has a short-hand notation in which  [ stuff ]  means the same as  test stuff  (those spaces inside the square brackets are required). Use this notation to simplify your solution to Exercise 6.

Exercise 8

For this exercise you should return to using a version of your test cases file where each line just contains the two strings to be tested. Make a second file called expected that contains just the correct answers for your test cases, one per line. Now, create a new tester script that writes the output from each of your tests to a file called results.

Hint 1: The line command > file redirects the output of command into file. Similarly, command >> file appends the output of command to the existing content of file (creating it if it did not previously exist).

Hint 2:: You may wish to solve this problem by first removing the file results and then adding output to it one line at a time. Check out the manual page for the rm command to find options that will cause it to work cleanly even when the file to be removed does not exist.

Exercise 9

Create another version of the tester script that works like the one from Exercise 8, but then uses diff to check if the results produced are the same as the expected answers. If yes, have your script print ‘All tests passed’, otherwise have it print ‘Some tests failed’.

Hint: diff is noisy and produces output that you may not want to be seen. Output redirected to the special file /dev/null will never be seen again. Alternatively, find how to make diff run silently.

Exercise 10

Develop bash shell commands (and a shell script file) to answer the question: How many Java source files do you have in your file space?

Hint 1: Commands can tell if their output has been redirected to a file or a pipe, and can change their behaviour as result. Check out the difference between the output of ls and ls | cat. The cat command just copies its input to its output, so any differences have come about because ls sensed it was connected to a pipe and modified its behaviour. Why would it do that?

Hint 2: You should be able to complete this exercise by using only the commands ls, wc, and grep.

Hint 3: Check out the -R option on ls and all the options on wc. (Use the man command.)

Hint 4: The grep pattern '\.java$' (those quote marks are important) will match lines ending in .java.

Exercise 11*

Starred exercises are extension work, and should not be attempted until you have solved all the regular exercises. They are optional, but remember that the more you do, the more you will learn.

So, where are all these Java files then? Just using ls and grep can get you the names of all your Java files, but it won't tell you the full pathname i.e where to find them. Read the manual page for the find command, and then use it to print the locations of all your Java files.

Hint: don't despair, find is not an easy command to use, and its switches and argumentrs are quite unlike almost any other command.

Exercise 12*

Fix the StringCount Java program to be correct according to the specification, and check your fixes by using the testing script and test data you developed earlier.

Copyright © 2006, 2009, Jim Grundy & Ian Barnes, Chris Johnson, The Australian National University
$Revision: 1.8 $ $Date: 2009/05/28 06:30:46 $ $Author: cwj $
Feedback & Queries to comp2100@cs.anu.edu.au