Lecture Notes

Lecture	Notes
29: Tue May 24	GPU Hardware and Software (cont) We will look at some chapters of the book Programming Massively Parallel Processors by Kirk and Hwu, available here.
27-28: Mon May 23	GPU Hardware and Software We will look at Lectures 1-5 of the course ECE498 at Illinois, available here and (slightly different version) here.
26: Tue May 17	Climate Modelling ( 1up)
24-25: Mon May 16	Big Iron and NCI Tour. Big Iron lecture ( 1up)
23: Tue May 10	Hadoop and MapReduce (cont)
21-22: Mon May 9	Hadoop and MapReduce (ppt)
20: Tue May 3	Assignment 2 and Mid-Semester feedback
18-19: Mon May 2	The Cilk Programming Language. Charles Leiserson's lectures are available from the MIT Cilk project web site (we'll cover at least his first lecture).
17: Tue Apr 5	The OpenMP programming model ( 1up) The OpenMP web site LLNL OpenMP Tutorial
XX: Mon Apr 4	Mid-Semester Exam
16: Tue Mar 29	Mid-Semester Exam Preparation Lecture
14-15: Mon Mar 28	Shared-Memory Programming (cont) Memory Consistency Models ( 1up) Memory consistency will take some time for you to fully appreciate, but it is fundamental to concurrent and distributed computing. It should also make you reconsider some of the initial material presented in COMP2310! Shared Memory Consistency Models: A Tutorial Sarita V. Adve. Kourosh Gharachorloo Micro-benchmarks for Cluster OpenMP Implementations: Memory Consistency Costs: paper by computer systems group PhD student Jin Wong about memory consistency issues primarily (but not totally) in the context of software distributed shared memory systems.
13: Tue Mar 22	Shared Memory; Programming (1up) As noted under Mar 21 lecture the lab work related to this material will not begin until immediately after the mid-semester break. Chapter 6 of Lin and Synder Chapter 8 of Wilkinson and Allen Chapter 7 of Grama, Gupta, Karypis and Kumar POSIX Threads Tutorial from LLNL
11-12: Mon Mar 21	Graph Algorithms (continued from Mar 15) Shared Memory; Hardware (1up) We now begin our discussion about shared memory systems. If you are still trying to come to grips with distributed memory message passing don't panic. The labs will continue to focus on MPI until after the mid-semester break! Material in this lecture is drawn from: Chapter 2 of Lin and Synder Chapter 2 of Grama, Gupta, Karypis and Kumar SGI Origin architecture AMD Northbridge Architecture
10: Tues Mar 15	Graph Algorithms (1up) See Chapter 10 of Grama, Gupta, Karypis and Kumar; also (background reading for designing and debugging parallel programs) Chapter 11 of Lin and Snyder.
9: Tues Mar 8	Parallel Algorithm Design (1up) (continued) See Chapter 3 of Grama, Gupta, Karypis and Kumar; also (background reading) Chapter 4 of Lin and Snyder. Dense Matrix Algorithms (1up) See Chapter 8 of Grama, Gupta, Karypis and Kumar. Getting Good Performance on the Linpack Benchmark (background reading). (A more detailed paper is here.)
7-8: Mon Mar 7	Performance Models (1up) Measuring and modeling performance. Mr Amdahl and more! See Chapter 3 of Lin and Synder and Chapter 5 of Grama, Gupta, Karypis and Kumar. Parallel Algorithm Design (1up) See Chapter 3 of Grama, Gupta, Karypis and Kumar; also (background reading) Chapter 4 of Lin and Snyder. MPI visualization tools: Jumpshot Vampir
6: Tue Mar 1	Basic Message Passing (1up) A quick introduction to MPI, there will be more in lab 1. See also Chapter 7 from the text "Principles of Parallel Programming" (copy of figures). Some references: MPI forum Online MPI-1 Book Chapter 2 in Wilkinson and Allen
4-5: Mon Feb 28	First we will complete overview of parallel hardware from last Tue Basic Communications (1up) This material lays the ground work for message passing and should make you appreciate the complexities of writing a good message passing library for a given platform. See parts of Chapter 2 and 4 in the book by Grama et al for more info.
3: Tue Feb 22	Overview of Parallel Hardware (1up) A quick review of single CPU hardware followed by an overview of parallel hardware issues. Material presented here will be picked up on in later lectures. ...we finished single processors in this lecture. For a very detailed discussion about single processor performance see the reference book by Bryant and O'Hallaron. Chapters 2 and 3 from Lin and Snyder contains an overview of single and multiple processor architectures similar to that outlined here. See also the lecture notes from Calvin Lin: "A success Story: ISA": more details of how parallelism is a key part of the instruction set architecture (ISA) of all modern chips Parallel Architectures: Slide 12 and beyond is relevant at this point
1-2: Mon Feb 21	Unit Introduction (ppt) NCI NF Vayu System NF Xe System ANU Supercomputer Facility The Bunyip Project Worlds Fastest Computer