Information Theory COMP2610
Workload
Twenty-six one-hour lectures and five two-hour tutorial sessions.
Study schedule
More information may be available for enrolled students on the course website on Wattle
More information may be available for enrolled students on the course website at http://cs.anu.edu.au/courses/info/COMP2610
This course consists of two major sections and a series of guest lectures at the end.
These cover the following broad topics:
- Probability, information, and inference (Weeks 1-5)
- Coding for compression and error correction (Weeks 6-11)
- Algorithmic Information Theory and Kolomgorov complexity (Weeks 12-13)
Theme / Topic / Module
Overview, Introduction, and Administration
Face to face activities
Lecture 1 - Admin & Overview
Lecture 2 - Introduction & Motivation
Other activities
Course Outline & Chapter 1 (MacKay)
Theme / Topic / Module
Introduction to Bayesian Inference
Face to face activities
Lecture 3 - Probability Theory and Bayes' Rule
Lecture 4 - Bayesian Inference
Other activities
Chapter 2 (MacKay)
Theme / Topic / Module
Bayesian Inference and Coding
Face to face activities
Lecture 5 - Useful Discrete Probability Distributions
Lecture 6 - Bayesian Inference and Coding
Other activities
Chapters 1 & 23 (MacKay)
Theme / Topic / Module
Simple Linear Block Codes and Entropy
Face to face activities
Lecture 7 - Introduction to Block Codes
Lecture 8 - Entropy
Tutorial 1: Probability and Bayesian Inference
Other activities
Chapter 1 (MacKay); Chapter 2 (Cover and Thomas)
Theme / Topic / Module
Entropy & Information
Face to face activities
Lecture 9 - Entropy & Information
Lecture 10 - Some fundamental inequalities
Other activities
Chapters 2 & 8 (MacKay)
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 11 - Entropy & Coding
Lecture 12 - Lossy Compression
Tutorial 2: Coding, Entropy and Information
Other activities
Chapter 4 (MacKay)
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 13 - Symbol Codes for Lossless Compression
Lecture 14 - Source Coding Theorem for Symbol Codes
Other activities
Chapter 5 (MacKay)
Theme / Topic / Module
Stream Codes
Face to face activities
Lecture 15 - Arithmetic Coding
Lecture 16 - Arithmetic Coding (cont.)
Tutorial 3: Coding and Compression
Other activities
Chapter 6 (MacKay)
Theme / Topic / Module
Stream Codes / Noisy Channels
Face to face activities
Lecture 17 - Lempel-Ziv Coding
Lecture 18 - Noisy Channels
Other activities
Chapters 6 & 9 (MacKay)
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 19 - Noisy-Channel Coding
Lecture 20 - The Noisy-Channel Source Coding Theorem
Other activities
Chapters 9 & 10 (MacKay)
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 21 - Channel Capacity
Lecture 22 - Noisy-Channel Codes in Practice
Tutorial 4: Noisy Channels
Other activities
Chapter 10 & parts of Chapters 11, 13 & 14 (MacKay)
Theme / Topic / Module
Algorithmic Information Theory
Face to face activities
Lecture 23 - Kolomgorov Complexity
Lecture 24 - Algorithmic Information Theory
Theme / Topic / Module
Algorithmic Information Theory
Face to face activities
Lecture 25 - Algorithmic Information Theory & Sequential Prediction
Lecture 26 - Summary, Final remarks, Q&A
