Information Theory COMP2610
Workload
Twenty-six one-hour lectures and five two-hour tutorial sessions.
Study schedule
More information may be available for enrolled students on the course website on Wattle
More information may be available for enrolled students on the course website at http://cs.anu.edu.au/courses/info/COMP2610
This course consists of two major sections and a series of guest lectures at the end.
These cover the following broad topics:
- Probability, information, and inference (Weeks 1-5)
- Coding for compression and error correction (Weeks 6-7,10-13)
- Algorithmic Information Theory and Kolomgorov complexity (Weeks 14-15)
- There is a two week teaching break between weeks 7 & 8 below.
- Schedule is tentative - slight changes may be made to the order the material is presented.
- See the Wattle site for the definitive schedule
Theme / Topic / Module
Overview, Introduction, and Administration
Face to face activities
Lecture 1 - Admin & Overview
Lecture 2 - Introduction & Motivation
Theme / Topic / Module
Introduction to Bayesian Inference
Face to face activities
Lecture 3 - Probability Theory and Bayes' Rule
Lecture 4 - Bayesian Inference
Theme / Topic / Module
Bayesian Inference and Coding
Face to face activities
Lecture 5 - Useful Discrete Probability Distributions
Lecture 6 - Bayesian Inference and Coding
Tutorial 1: Probability and Bayesian Inference
Theme / Topic / Module
Simple Linear Block Codes and Entropy
Face to face activities
Lecture 7 - Introduction to Block Codes
Lecture 8 - Entropy
Theme / Topic / Module
Entropy & Information
Face to face activities
Lecture 9 - Entropy & Information
Lecture 10 - Some fundamental inequalities
Tutorial 2: Coding, Entropy and Information
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 11 - Entropy & Coding
Lecture 12 - Lossy Compression
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 13 - Symbol Codes for Lossless Compression
Lecture 14 - Source Coding Theorem for Symbol Codes
Tutorial 3: Coding and Compression
Theme / Topic / Module
Stream Codes
Face to face activities
Lecture 15 - Arithmetic Coding
Lecture 16 - Arithmetic Coding (cont.)
Theme / Topic / Module
Stream Codes / Noisy Channels
Face to face activities
Lecture 17 - Lempel-Ziv Coding
Lecture 18 - Noisy Channels
Tutorial 4: Arithmetic Coding
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 19 - Noisy-Channel Coding
Lecture 20 - The Noisy-Channel Source Coding Theorem
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 21 - Channel Capacity
Lecture 22 - Noisy-Channel Codes in Practice
Tutorial 5: Noisy Channel Coding
Theme / Topic / Module
Algorithmic Information Theory
Face to face activities
Lecture 23 - Kolomgorov Complexity
Lecture 24 - Algorithmic Information Theory
Theme / Topic / Module
Algorithmic Information Theory
Face to face activities
Lecture 25 - Algorithmic Information Theory & Sequential Prediction
Lecture 26 - Summary, Final remarks, Q&A


