Skip Navigation CECS Home | ANU Home | Search ANU

COMP3320: High Performance Scientific Computation

(6 units) Group C

Thirty one-hour lectures and six two-hour tutorial/laboratory sessions

Convener: Dr Alistair Rendell

Prerequisites

12 units of 2000-level COMP courses including COMP2100 or COMP2300 or COMP2031 or ENGN2003; and 6 units of 2000-level MATH/STAT/EMET courses or COMP2600.

Incompatible with COMP3061,COMP3067.

Syllabus

This unit provides an introduction to High Performance Computing with an orientation towards applications in science and engineering. Aspects of numerical computing and the design and construction of sophisticated scientific software will be considered. The focus will be on the C and C++ programming languages, although reflecting the reality of modern scientific computation this unit will also touch on other languages such as Python, Java and FORTRAN95. The unit will study high performance computer architectures, including modern parallel processors, and will describe how an algorithm interacts with these architectures. It will also look at practical methods of estimating and measuring algorithm/architecture performance.

The following topics will be addressed: the C++ programming language; basic numerical computing from aspects of floating point error analysis to algorithms for solving differential equations; the engineering of scientific software; general high performance computing concepts and architectural principles; modern scalar architectures and their memory structure; performance and programmability issues, and program analysis techniques for high performance computing; parallel computing paradigms and programming using the OpenMP standard; trends in HPC systems.

Description

This unit provides an introduction to the main principles, issues and applications of High Performance Scientific Computing, including parallel processing. It provides some overlap between the Computer Science and both the Computational Mathematics and Science / Engineering curriculum areas, and is suited for students with an interest in a combination of areas. It is one of the two capstone units in the Bachelor of Computational Science (BComptlSci) degree and the Scientific Computation stream within the BSc.

Rationale

In modern scientific and engineering practice computer modelling and simulation is significantly augmenting, and even partly replacing, experiments. It is likely that tomorrow's professional engineers and scientists will spend an increasing amount of time writing, running and interpreting the results of numerical software.

Although science and engineering have been closely coupled to the historical development of computing the development of scientific software has been relatively independent of trends in computer science over the past two decades. Recently, however, common interest in, e.g. the internet, visualization, high-performance computers and modern programming languages have been bringing the two communities closer again. On the one hand there is a growing enthusiasm in parts of the scientific community to take on modern ideas of software engineering if the productivity of scientific software is to be improved. On the other, some computer scientists are recognising the unique challenge of engineering scientific software - particularly in the context of high performance computing. As a further boost to the educational enterprise, modern software packages which combine interpreted interfaces for numerical work with excellent graphics have rekindled interest in the teaching of numerical computing.

This unit aims to bridge the divide between scientific computing and computer science. It also aims to provide students with experience in experimental computing using simple numerical models - experience which will be able to be carried across to working with much larger engineering packages in the workplace.

Ideas

This unit will be the primary carrier in the computer science curriculum for

• numerical computing
• experimental computing in a numerical context (convergence and extrapolation of numerical results to a limit, error estimation, model building and parameter studies, logical presentation and analysis of numerical results)
• the C++ programming language
• software engineering of scientific software
• the design of modern computer architectures,

• principles of parallel computing,

• practical methods of analysis and measurement of algorithm performance.

It will share responsibility for

• software engineering
• parallel algorithms,

• numerical computing methods,

• advanced principles in the compilation process.

Topics

These will be sourced from the following:

1.

Floating point arithmetic. Error analysis.

2.

Basic numerical methods: linear algebra, interpolation, integration, differentiation, and the solution of ordinary differential equations.

3.

The C++ programming language.

4.

The engineering of scientific software including structured and object-oriented approaches.

5.

The context of high-performance computing for grand challenge problems.

6.

Introduction to general high performance and parallel computing concepts and methods; benchmarking; speedup; scalability.

7.

Modern scalar architectures and their memory structure.

8.

Performance issues including data re-use, operation balancing, and memory access.

9.

Compiler support for optimization, and program analysis for performance.

10.

Overview of parallel computing paradigms: (architectural) shared and distributed memory processors; (conceptual) data-parallel, threaded, message passing.

11.

Parallel computing issues.

12.

Trends in high performance computing systems and applications.

13.

Case studies are to be taken from linear algebra, heat flow PDEs, and graphics.

Knowledge Units

ANU Knowledge Areas

SEn1

Fundamental Problem Solving

SEn3

Software Design

SEn6

Software Construction

CSc1

Numerical Analysis

CSc2

Scientific Computing

CSc5

Modelling and simulation

Fnd1

Complexity Analysis

Fnd8

Program Language Semantics

PTP9

Study of a Programming Language

ACM: (out of date for this field ...)

AR7

Alternative Architectures (3 of 5) (and more)

NU1

Number representation, errors and portability (3 of 3)

NU2

Iterative Approximation Methods (2 of 4)

OS3

Process Coordination and Synchronisation (1 of 4) (from SMP point of view)

PL11

Programming Paradigms (2 of 10)

PL12

Distributed and Parallel Programming Constructs (2 of 3) (parallel loops, SMP threads and message passing programming)

Stone Man:

AL

Algorithms and Data Structures

• Parallel and Distributed Algorithms

CA

Computer Architecture

• Memory System Organization and Architecture (advanced)
• Alternative Architectures
• Performance

CS

Computational Science

• Numerical Analysis
• Scientific Computing
• Parallel Algorithms
• Supercomputing
• Modelling and Simulation

Objectives

Upon completion of the course, students should:

1. appreciate the building blocks of scientific and engineering software. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 1)
2. be able to apply a basic knowledge of numerical computing using an appropriate programming language. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 1,3)

3. be competent in experimental computing in a numerical context and of the optimisation of algorithms on high performance architectures. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 1)

4. be able to reason about the accuracy of mathematical and numerical models of real physical phenomena. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 2,4)

5. have an awareness of the modern field of computational science and engineering and of the impact of high performance computing on science and industry. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 3,7)

6. have an understanding of the various paradigms of high performance computing and their potential for performance and programmability. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 3)

7. be able to write algorithms yielding good performance on high-performance architectures, and to be able to estimate and evaluate their performance. (level in Biggs SOLO taxonomy: ; generic graduate attributes: 1,2,3,5)

Assessment

The following assessment modes are used.

final examination

This tests objectives 1, 2, 4, 5, 6 and 7.

programming exercises

Graded programming tasks will be assigned (generally involving the modification and extension of given codes). Each student will submit the completed codes, for marking.

This tests objectives 2, 3 and 7.

reports

Each assignment will require the submission of a report. This may including experiments, calculation of derived quantities, and an analysis of the results. Suitable conclusions will be drawn.

This tests objectives 4, 6 and 7.

Technical Skills

This unit provides exposure to program optimization techniques and tuning programs via compilers. It also provides skills in data modelling and in the application of software engineering to scientific and engineering applications. It also provides exposure to computer languages used in High Performance Computing, including C++.

It presents the mathematical models for the performance of computations, whose properties can be hypothesised and tested by the scientific method. It introduces scientific experimental skills such as obtaining optimal accuracy for measured and derived quantities, and performing appropriate calculations with limited significant figures.

Relevant Reading

• Kevin Dowd and Charles Severance. High Performance Computing. O'Reilly, 2nd edition, 1998.
• Lloyd D. Fosdick, Elizabeth R. Jessup, Carolyn J. C. Schauble, and Gitta Domik. An Introduction to High-Performance Scientific Computing. The MIT Press, 1996.
• Michael T. Heath. Scientific Computation - An Introductory Survey. McGraw-Hill, 1997.
• Rajkumar Buyya. High Performance Cluster Computing: Programming and Applications. Prentice Hall, Upper Saddle River, New Jersey, 1999.