COMP4300: Formal Unit Description

Also see COMP4300 activity timetable

COMP4300: Parallel Systems

(6 credit points) Group C
Second Semester
Twenty-six one-hour lectures, four two-hour laboratory/tutorials.

Lecturer: Dr Alistair Rendell

Prerequisites

COMP2310; and 24 credit points of 3000-level COMP units including either COMP3320 or COMP3600.

Syllabus

A practically oriented introduction to programming paradigms for parallel computers. Considers definitions of program efficiency on parallel computers, addresses the modelling, analysis and measurement of program performance. Description, implementation and use of parallel programming languages, parallel features of operating systems, library routines and applications.

Description

A mainly practical introduction to the arts of programming high-performance parallel computers for representative problems, with the emphasis on performance and programming paradigms.

Rationale

The leading edge of high performance computing is in computers with highly parallel architecture machines like the ASCI machines and BEOWULF-style architectures. The high-end examples of these computers cost millions of dollars each, the low end hundreds of thousands, but they have important uses in high speed computations: weather forecasting, financial modelling and information databases, so-called ``electronic wind tunnels'', realistic high speed graphics such as film and video animation sequences, scientific visualisation, machine learning, and virtual reality research. Similar computers using parallel processing will become much more widely used in many areas as they are better understood and the cost of building them continues to fall. The ANU has a stable of high performance parallel computers in the Department of Computer Science, the Supercomputing Facility, and the Research School of Information Science and Engineering, that provides later year students with an unparalleled opportunity to work with state-of-the-art computing systems.

Parallel processing is the key to harnessing the power of modern cheap high-powered processing and memory chips. There are many computer designs which attempt answers to the computer architecture question, that is how to combine processors and memory in a parallel computer that can make effective use of their potential power at an acceptable cost. The techniques for programming the resulting machines include many new models of constructing, debugging and measuring performance of programs that are quite different to conventional computing structures.

Objectives

At the completion of this unit the student will:

1. be able to program more than one parallel machine in more than one specialised programming language or programming system; generic graduate attributes: 1,4
2. be able to descriptively compare the performance of different programs and methods on one machine; generic graduate attributes: 3,5
3. be aware of the elements of parallel programming language and system implementation; generic graduate attributes: 3
4. be aware of the history and developments in the field. generic graduate attributes: 3

Assessment

The following assessment modes are used.

final examination

This tests objectives 1, 2, 3, and 4.

assignments

Each assignment will require an in-depth task which interacts with the larger system.

This tests objectives 1 and 2.

Technical Skills

• familiarity in data-parallel languages.
• competence in parallel programming using message passing libraries.

Ideas

This unit presents the ideas of parallel efficiency, speedup, and load balancing, and the associated difficulties of performance evaluation; practical parallel programming with existing languages; data parallel programming, process parallel programming; common parallel programming paradigms and problem decomposition.

Topics

A selection will be made from the following topics:

Foundations:

machine architectures and programming applications.

Parallel programming issues:

driving forces and enabling factors, sample applications (scientific, engineering, AI and database). Efficiency, speedup, load balancing, performance measurement and comparisons. Introduction to parallel algorithms (reduction, sorting). Matrix manipulation.

Programming paradigms:

geometric and process decomposition, worker-farming, spatial decomposition, particle decomposition.

Parallel software:

languages and coordination constructs, parallelising compilers, programming environments.

Practical parallelism:

creating and evaluating programs for Fujitsu AP3000 or Beowulf and other machines as available.

Technology trends:

the future of high performance and parallel computing.

Recommended Reading

• Selim G. Akl. The Design and Analysis of Parallel Algorithms. Prentice Hall, Englewood Cliffs, New Jersey, 1989.
• Rajkumar Buyya. High Performance Cluster Computing: Architectures and Systems. Prentice Hall, Upper Saddle River, New Jersey, 1999.
• Rajkumar Buyya. High Performance Cluster Computing: Programming and Applications. Prentice Hall, Upper Saddle River, New Jersey, 1999.
• Geoffrey C. Fox, Mark A. Johnson, Gregory A. Lyzenga, Steve W. Otto, John K. Salmon, and David W. Walker. Solving Problems on Concurrent Processors. Prentice Hall, 1988.
• Robert G. Babb II. Programming Parallel Processors. Addison-Wesley, Reading, Massachusetts, 1988.
• Ted G. Lewis and Hesham El-Rewini. Introducton to Parallel Computing. Prentice Hall, Englewood Cliffs, New Jersey, 1992.
• Michael J. Quinn. Designing Efficient Algorithms for Parallel Computers. McGraw-Hill, 1987.
• Thomas L. Sterling, John Salmon, Donald J. Becker, and Daniel F. Savarese. How to build a Beowulf. MIT Press, 1999.
• Greg Wilson. Practical Parallel Programming. The MIT Press, 1995.