Lab 5 - COMP2100/2500 - DCS

ANU

____________________________________________________

[ANU] [DCS] [COMP2100/2500] [Description] [Schedule] [Lectures] [Labs] [Homework] [Assignments] [COMP2500] [Assessment] [PSP] [Java] [Reading] [Help]

____________________________________________________

COMP2100/2500
Lab 5: Make & RCS

Summary

Some exercises to introduce the build tool ‘make’ and the version control system ‘RCS’.

Aims

Give you some experience using make and writing simple Makefiles.

Give an introduction to using the RCS version control system.

Hints

The UNIX command touch is very useful when experimenting with make. Running the command ‘touch f’ has the following effect:

If the file f does not exist, then an empty file with that name is created.

If the file f exists and is writable, then the time at which it was last updated is set to the time when the touch command was run.

Another useful thing to remember when experimenting with make is the -l option to ls. Typing ls -l shows the times at which files were last updated.

Note that the commands in a Makefile must be indented by a single Tab character, not 8 spaces. The file must also end with a new-line character, something Emacs does not always guarantee. These two invisible restrictions on Makefiles are the most annoying features of Make, and the most common source of errors by new users.

Exercise 1 - Doing it by hand

Download the following files:

sort.c: A simple insertion sort routine.

sort.h: A header describing the sort.

harness.c: A simple, and quite ugly, test harness for the sort routine.

tests.txt: A list of test cases for the harness.

Note: the selection of these test cases leaves a lot to be desired.

expected.txt: The corresponding list of expected answers.

Now, perform the following steps:
Compile sort.c to get sort.o with the following command:
gcc -c -Wall sort.c
The ‘-c’ means compile only; do not attempt to link with other modules to form an executable.
Compile harness.c to get harness.o similarly.
Link sort.o and harness.o to make an executable called harness with the following command:
gcc -o harness sort.o harness.o
Run harness on tests.txt and place the output in output.txt.

Hint: You can do this using the UNIX shell I/O redirection symbols ‘<’ and ‘>’.
If you compare the files output.txt and expected.txt (using the diff command), you will see there is a difference for Test 1. This is, in fact, caused by an error in the expected output for Test 1, but we'll come back to that later.

Exercise 2 - Dependency diagram

Draw a diagram showing the dependencies between the various files in Exercise 1. Use the diagrams from Lecture 24 as your model.

Exercise 3 - Simple Makefile

Construct a Makefile, which should be called ‘Makefile’, that can be used to rebuild output.txt with the minimum amount of work whenever it is out of date with respect to the other files.

Test your Makefile by using the touch command to alter the time stamp on some of the other files, and then examining the actions that make subsequently performs. See the hint at the beginning of these exercises or the manual page for touch for more information on this command.

Exercise 4 - How make deals with return codes

Extend your Makefile so that it can automatically produce a file called ‘errors.txt’ that contains the differences between output.txt and expected.txt.

Hint: Actually, this exercise is a little harder than it looks, and you do not yet have all the information you need to complete it. Nevertheless, the best way to proceed is to try to complete the exercise anyway. You should be able to get something that looks right and almost works. The following paragraph will help you finish it off.

By now you should have extended your Makefile with a rule to build the file errors.txt using the diff command. You should also have found than when you try to make errors.txt, the make program complains about an error. Here's why: Make checks the return code of each command it executes. If any command returns a nonzero code, which usually indicates an error, then it reports the error and aborts the run. The problem is that the diff program does not obey this convention: it returns 0 if the files compared were the same, 1 if the files were different, and 2 if an error occurred. The result is that when diff is used in a Makefile and the files compared differ, then Make will report an error. You can tell Make not to check the return code of a command by inserting a ‘-’ in front of it. For example, in your Makefile you should have a line of the form:
diff expected.txt output.txt > errors.txt
If you replace that with the following line, your Makefile should be correct.
-diff expected.txt output.txt > errors.txt
Exercise 5 - Registering the source files with RCS

We would now like to use RCS to manage the maintenance of the source files of this system, i.e., those files that are not created by your Makefile.

First, create a subdirectory called ‘RCS’. (Just create it using the ‘mkdir’ command, don't ‘cd’ into it.) Place the file sort.c under the control of RCS with the command ‘ci sort.c’, to ‘check in’ the file. Since this file is new to RCS you will be asked to provide a short description of it. One line of text should do, perhaps the following:
A function to sort an array of integers.
You can ‘check out’ a read-only copy of sort.c by using the command co sort.c. Check in (and out) all the other source files.

Exercise 6 - Headers, locking, modifying and checking in

Use the command co -l sort.c to get a writable, ‘locked’, copy of sort.c. Add the following line near the start of the file (outside any function).
char sort_header[] = "$Header:$";
Check the file back in to RCS using the command ci -u sort.c to leave yourself with a readable, ‘unlocked’, copy.

Have a look at sort.c now. What happened to the line you added?

Add a similar line defining a string called ‘harness_header’ to harness.c, and check that in to RCS as well.

Use the command make harness to recompile the harness, then type the command ident harness. What does its output tell us about the harness program?

Exercise 7 - Making the program tell you what version it is

The string ‘ $Revision:$ ’ also has a special significance to RCS, but it is replaced with just the revision number of a file. Modify the system so that the harness prints out which revision of the sort module it is using before testing begins.

You don't have to do any fancy formatting; a message like the following is acceptable:
Using sort $Revision:$.
Hint: Since this year you are not being asked to write C code, this question is a bit too hard. So here's the answer. Follow these steps and then look at the results. Think about what you're doing; even though you're not expected to write your own C code, you are expected to read and understand it.
Add the line:
extern char sort_version[];
to sort.h. This tells all users (clients) of the sort routine that there is a string called sort_version somewhere.
Add the line:
char sort_version[] = "$Revision:$";
to sort.c. This will be replaced by $Revision: a.b$ by RCS (where a.b is the number of the current revision, for example 1.2).
Add the line:
printf("Using sort %s\n", sort_version);
at the beginning of harness.c between the declaration of local variables in main and the beginning of the first for loop. This causes the test harness to print the words ‘Using sort ’ followed by the contents of the string sort_version and a newline at the start of its output.
Exercise 8 - Using RCS from within Emacs

You can also access the RCS commands to check files and in and out from the Emacs Tools menu. Use Emacs to edit the file expected.txt. The Emacs status line should start with something like this:
--:%%  expected.txt
The ‘%%’ part of the status line indicates that the file is read-only. To check out and lock the file for changing you should select the Check In/Out option from the Version Control submenu of the Tools menu. The percent symbols should disappear, indicating that the file may now be changed. Fix the error in this file, then save it and check it back in using the Check In/Out option again.

When you look at the menu, you'll see that there is a command key equivalent for this action; it's C-x v v.

When you check the file back in using Emacs, it will prompt you for a description of the changes by splitting the window in half and putting the cursor in the bottom part. Type a brief (usually one line is enough) description, and then finish the process with the command keys C-c C-c. It tells you this only briefly, and the message disappears before most first-time users have read it...

Exercise 9 - Looking at the revision history

The ‘rlog’ command lets you inspect the revision history of any file under RCS control. Use it to inspect the history of your files.

Exercise 10 - Going back to an old version

The command ‘co -rn foo’ retrieves revision n of the file foo. Use this command to retrieve and rebuild the original version of the system.

Exercise 11 - Inspecting the internal format

The versions of each file foo are recorded by RCS in the versions file RCS/foo,v. Have a look at the versions files for each of the source files. Can you see how RCS is able to store many versions of the same file efficiently?

Exercise 12* - Build management for Java (New in 2005)

The build script I gave you for the project is overkill. It recompiles every class every time, whether or not it has changed. In principle, you should now be able to write a Makefile that recompiles the project without waste.

Don't try it now though, because it's not that simple. Read on instead.

To start with, you would need rules expressing the dependency between X.java and X.class (for each class X) so that if you modify X.java it will recompile it. You would need lots of these, one for each class in the system. In fact there is a better way: it's called “Implicit rules”. You can have one implicit rule that expresses that relationship for all Java classes, like this:
%.class: %.java
        javac $<
(The special variable $< gets replaced by the first name in the list of dependencies for the rule: here the .java file.)

Whether you use implicit rules or just have a rule for every class, you then need a default target (just make it the first one at the top of the file) that does nothing but depends on all the class files. Again, there is an easy way that requires lots of typing, and a harder way that requires you to look up the wildcard function in the Make documentation.

OK, that looks good. But unfortunately it's not enough. This is harder than it looks. Java classes are not all independent. Some of them depend on others. This can be through use (one class is named in a declaration somewhere in the other), or through inheritance (one class extends another class or implements an interface). In order to compile a class correctly, the compiler needs an up-to-date .class file for all the classes it depends on. Now the Java compiler knows a bit about dependencies: suppose class X depends on class Y, and class Y has been changed since the last compilation. If I tell the Java compiler to compile X, it will check whether Y.class is up to date, and will recompile it if it isn't. That's good. But if I ask it to compile class Y, since it is the only one that changed, the Java compiler has no way of knowing that class X depends on it, and should now be recompiled too. This is what our Makefile would do, and it's wrong, and can lead to really weird problems at runtime. This is all a bit of a nightmare, and generally leads to people adopting a conservative build management strategy: recompile everything every time, just in case.

The solution to this seems obvious, and Make was made for it. If class X depends on class Y, we put a line in our Makefile like this:
X.class: Y.class
We don't need the action part (the second line with the instruction indented by a tab), since the implicit rule tells Make how to build X.class if it needs to. All this does is express the information that if Y has changed, we need to recompile X.

All done, right? Wrong. And this is a killer. The problem is that many Java programs have cyclic dependencies between the classes. Look at the Visitor pattern in the project for example. The interface depends on class XmlContainerElement and XmlContainerElement depends on Visitor. If either one of these changes, the other one should be recompiled. You need both dependencies in your Makefile.
Visitor.class: XmlContainerElement.class
XmlContainerElement.class: Visitor.class
The problem is that when Make see that, it gives up. Make works on the model where dependencies form a tree (like the diagrams in the lecture). This is valid for C programs, but not for Java. When Make sees a cyclic dependency, it prints an error message and stops. The result is that we can't write a Makefile for our project.

What now? All is not lost. There is more to build tools than Make. Look up Apache Ant. Ant is a Java-based build tool for Java. Instead of a Makefile, it uses an XML file called build.xml. It takes a little learning, but it can do everything we want. Ant is installed on the student system. You may need to reboot your machine before you can see it. You will need to define the following environment variables before it will work correctly.
setenv ANT_HOME /usr/share/ant
setenv JAVA_HOME /usr/local/java
Put those lines in your .cshrc file if you are going to use Ant regularly.

I don't have time to write an Ant tutorial here, but there are several on the web. I found this one useful: Apache Ant Demystified, but you may find something else on the web that works better for you. You'll also want to look at the Apache Ant User Manual.

The way most people use Ant, it doesn't manage the dependencies between classes at all. (Amazing!) They just have it delete all the class files and recompile everything all over again. (See the rebuild target in the example in that tutorial.) But Ant has a builtin “task” called depend that will go through your program and delete any out-of-date .class files, and the .class files of any classes that depend on them. It does this without you telling it what the dependencies are. (It can read the dependencies from the .class files all by itself.)

If you want Ant to be super-conservative, you can turn on the closure attribute of the depend task, and it will delete any .class file that depends on anything that depends on anything that is out-of-date, and so on until it has followed all chains of dependencies to the end. The documentation says this is usually unnecessary, and sure enough, when I tried it on the expression tree example (from Lab 2) it deleted all the .class files if I touched any of the source files. But if you want to be safe, this is the right thing to do.

So if your compile target has the depend task before the javac task, it will delete all potentially out-of-date .class files, then recompile them, which is exactly what we wanted.

By the way, both Make and Ant can be used to do much more than just compilation. They can run unit tests, bundle up software for distribution, make backups, upload material to a web site. Used creatively, these are incredibly powerful tools.