COMP2100
Assignment 1 Hints and FAQ

More Hints

If you've forgotten everything you ever knew about unit testing, you can refresh your memory by looking at the relevant lecture notes and lab instructions from COMP1110 Foundations of Software Engineering. Note that the COMP1110 website is the one for the course as it was run in the summer session. You may find that what you are looking for is easier to find if you follow the link to Semester 2, 2003 (or an even earlier version of the course).

This assignment is about black-box testing, so you'll find that section of the notes more relevant than the notes on white-box testing.

If you're having trouble understanding the sample Bash script output with all those * and - signs and ! characters and stuff, then read the info documentation for the diff command. At the command line type `info diff'. To navigate around the info pages, move the mouse to a star (indicating a link to another page) and hit `Enter', or type `u' to go up, `n' to go to the next page or `p' to go to the previous page. Type `q' to quit. All the information you need is in the info pages.

---

Frequently Asked Questions

1. Any chance you could release a Solaris, Windows, MacOS, OS2, Amiga, BeOS, HP-UX, IRIX, VMS, Apple II, Commodore 64... version of the test_formatter program?

   Sorry, but no. I don't have access to a suitably set up Windows, MacOS, OS2, Amiga, BeOS, HP-UX, IRIX, VMS, Apple II, Commodore 64... machine.

   Seriously though, you should be aware that most of the work for this assignment can be done with pen and paper. The `sitting in front of the computer' component is relatively light. So I don't think having to come in and do it in the labs is too much of a hardship.

2. Will we also be getting a copy of the source code for class OUTPUT_FORMATTER or do we just have to work on the executable test_formatter file?

   No source code. This assignment is about black box testing, so I'm deliberately not showing you the source code.

3. Should an empty input stream really result in no output at all?

   Yes. The test program never enters the body of its main loop.

   Added 17/3/2004: Oops! No! That was wrong. The way the test program is written, even an empty input stream will still result in one call to blank_line, which will put at least one new-line character on the standard output.

   This may or may not be a worthwhile test to include in your assignment -- that's up to you to decide -- but it raises an important issue. You should always think about boundary cases (look at the COMP1110 lecture notes on black box testing again). An empty input file is often an important boundary case.

4. Around how many test cases would you expect us to have?

   I expect most students to have around 20 tests, but for a really thorough job I wouldn't be surprised if you had more. But think hard about how to avoid repetition. Perhaps you can test several variations of the same test in the one short file?

5. The method I have for calculating percentages in my shell script leads to rounding errors, for example 10/11 is calculated as 90%, whereas it really should be 91%. It doesn't seem to be possible to use floating point numbers in bash. Can you give any hints?

   OK, so what you've discovered is that bash truncates rather than rounding. The standard way to round is add 0.5 then truncate, right? You just have to figure out a way to do this using integer arithmetic.

6. For my shell script to work correctly, there must be a certain number of test input files, numbered in order test01.in to testnn.in. If this order is not adhered to, the script will not work correctly. Is this acceptable?

   Do you mean that the number of test files is hard coded into your script, so that if you add a new test you have to modify the script? If so, then I think you can do better.

   I don't have a problem with you assuming that there are less than 100 tests, that the first one is test01, and that they are numbered consecutively without any gaps. But I don't think you should have to change the script when you add a new one.

7. Do we have to test whether the software performs correctly on large files? Binary files? Other really weird stuff?

   No. Do not attempt to do this. Make every effort to keep your test files as short as possible.

   To be more explicit, I would regard submitting huge (or binary or weird) test files as vandalism. Remember that in order for your tutor to mark your assignment, it will have to be printed out, along with the output of running your test suite on at least two different versions of the test_formatter program. So keep it reasonable. No large files. No binary files.

8. I've been working on this assignment for hours and hours and I don't seem to be even nearly finished. What should I do?

   Don't worry. Work on the assignment for as long as you think is reasonable, then hand in what you have done. Make sure that you have a working test script and that the tests you have completed are documented properly in your test plan and test report. Write a note in your test plan telling your tutor that you think there are more issues to consider, and briefly what they are. Then hand it in and move on. This is one assignment in a busy semester. It's possible that I have misjudged the size of it. There is no point in making unreasonable efforts.

9. I can't get any output out of test_formatter. I tried putting the input on the command line like this: "test_formatter abc" but nothing happens. Then I tried putting it in a file and doing this: "test_formatter test01.in" but still nothing. What am I doing wrong?

   If you read the file test_formatter.e carefully, you'll see that it reads from standard input and writes to standard output. In the language of Lecture 4, the test_formatter program is a filter. That means that if you just say "test_formatter", then it will wait for input from the keyboard. To read input from a file as you will need to in your script, you need to use redirection. It's all explained in Lecture 4.

10. The executable test_formatter program isn't on my path like you said it would be. What can I do?

    If you type the line

    setenv PATH /dept/dcs/comp2100/bin:${PATH}

    it will add that directory to your path and everything will be fine until you log out. To make this permanent, add that line to a file called ".cshrc" (the dot is important) in your home directory. This file gets executed every time you start a new shell window.

    (What should be happening is that every time you start a new shell, it runs the file /dept/dcs/comp2100/login/Cshrc. This file contains settings for all COMP2100 students. It's not working at the moment, mostly because we're having some problems getting the new student systems stable, but it should be working soon.)

11. I found the test_formatter executable, but when I try to run it, I get this nasty error message:

    /dept/dcs/comp2100/bin/test_formatter: Exec format
          error. Binary file not executable.

    What's wrong?

    The executable is a Linux executable that works on the new machines in the labs. The servers iwaki, challender, karajan and rattle are all running Solaris (Sun Unix). The two operating systems are not binary compatible. You will get that error message if you try to run that executable on one of the Solaris servers.

    If you're working remotely from home, logging in via ssh, you need to ssh to partch.anu.edu.au rather than one of the Solaris servers. Then everything should work OK.

    If you're working at home and you have a Linux machine, you can copy the executable across. Make sure you put it somewhere that's on your path, so that you can run it just by typing its name (without having to specify the path to it).

12. I can't figure out what these lines in the sample output are:

    *** 1,3 ****
    --- 1,3 ----

    What do they mean? How do I get them?

    That is all explained in the info documentation for the diff command. Read it carefully and you'll find what you're looking for.

    You do not have to modify the diff output in any way. It's just a matter of calling diff with the correct options.

    One of the objectives of this assignment is to get you reading online documentation. It's a very important skill. If you let your friends tell you the answer, you're missing out on an important exercise. Be aware that this skill will also be tested in the lab exam, and you won't be able to rely on your friends then.