Information Theory COMP2610
Workload
Twenty-six one-hour lectures and five two-hour tutorial sessions.
Study schedule
More information may be available for enrolled students on the course website on Wattle
More information may be available for enrolled students on the course website at http://cs.anu.edu.au/courses/info/COMP2610
Theme / Topic / Module
Overview, Introduction, and Administration
Face to face activities
Lecture 1 - Admin & Overview
Lecture 2 - Introduction
Other activities
Course Outline & Chapter 1 (MacKay)
Theme / Topic / Module
Introduction to Bayesian Inference
Face to face activities
Lecture 3 - Probability Theory and Bayes' Rule
Lecture 4 - Bayesian Inference
Other activities
Chapter 2 (MacKay)
Theme / Topic / Module
Bayesian Inference and Coding
Face to face activities
Lecture 5 - Useful Discrete Probability Distributions
Lecture 6 - Bayesian Inference and Coding
Other activities
Chapters 1 & 23 (MacKay)
Theme / Topic / Module
Simple Linear Block Codes and Entropy
Face to face activities
Lecture 7 - Introduction to Block Codes
Lecture 8 - Entropy
Tutorial 1: Probability and Bayesian Inference
Other activities
Chapter 1 (MacKay); Chapter 2 (Cover and Thomas)
Theme / Topic / Module
Entropy & Information
Face to face activities
Lecture 9 - Entropy & Information
Lecture 10 - Some fundamental inequalities
Other activities
Chapters 2 & 8 (MacKay)
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 11 - Entropy & Coding
Lecture 12 - Lossy Compression
Tutorial 2: Coding, Entropy and Information
Other activities
Chapter 4 (MacKay)
Theme / Topic / Module
Symbol Codes
Face to face activities
Lecture 13 - Symbol Codes for Lossless Compression
Lecture 14 - Source Coding Theorem for Symbol Codes
Other activities
Chapter 5 (MacKay)
Assessment milestones
Assignment 1 released (9 Sept.)
Theme / Topic / Module
Stream Codes
Face to face activities
Lecture 15 - Arithmetic Coding
Lecture 16 - Arithmetic Coding (cont.)
Tutorial 3: Coding and Compression
Other activities
Chapter 6 (MacKay)
Assessment milestones
Assignment 1 due (30th Sept)
Theme / Topic / Module
Stream Codes / Noisy Channels
Face to face activities
Lecture 17 - Lempel-Ziv Coding
Lecture 18 - Noisy Channels
Other activities
Chapters 6 & 9 (MacKay)
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 19 - Noisy-Channel Coding
Lecture 20 - The Noisy-Channel Source Coding Theorem
Other activities
Chapters 9 & 10 (MacKay)
Assessment milestones
Assignment 2 released (17th Oct.)
Theme / Topic / Module
Noisy-Channel Coding
Face to face activities
Lecture 21 - Channel Capacity
Lecture 22 - Noisy-Channel Codes in Practice
Tutorial 4: Noisy Channels
Other activities
Chapter 10 & parts of Chapters 11, 13 & 14 (MacKay)
Theme / Topic / Module
Advanced Inference
Face to face activities
Lecture 23 - Exact Marginalization over discrete variables
Lecture 24 - Exact Marginalization over continuous variables
Other activities
Chapter 23, 24, 25 (MacKay)
Assessment milestones
Assignment 2 due (31st Oct.)
Theme / Topic / Module
Advanced Inference
Face to face activities
Lecture 25 - Exact Marginalization over continuous variables (cont.)
Lecture 26 - Summary, Final remarks, Q&A
Tutorial 5: Advanced Inference
Other activities
Chapter 24 (MacKay)
