Information Theory COMP2610

Workload

Twenty-six one-hour lectures and five two-hour tutorial sessions.

Study schedule

More information may be available for enrolled students on the course website on Wattle

More information may be available for enrolled students on the course website at http://cs.anu.edu.au/courses/info/COMP2610

This course consists of two major sections and a series of guest lectures at the end.

These cover the following broad topics:

1. Probability, information, and inference (Weeks 1-5)
2. Coding for compression and error correction (Weeks 6-7,10-13)
3. Algorithmic Information Theory and Kolomgorov complexity (Weeks 14-15)

• There is a two week teaching break between weeks 7 & 8 below.
• Schedule is tentative - slight changes may be made to the order the material is presented.
• See the Wattle site for the definitive schedule

Week

1

Theme / Topic / Module

Overview, Introduction, and Administration

Face to face activities

Lecture 1 - Admin & Overview
Lecture 2 - Introduction & Motivation

Week

2

Theme / Topic / Module

Introduction to Bayesian Inference

Face to face activities

Lecture 3 - Probability Theory and Bayes' Rule
Lecture 4 - Bayesian Inference

Week

3

Theme / Topic / Module

Bayesian Inference and Coding

Face to face activities

Lecture 5 - Useful Discrete Probability Distributions
Lecture 6 - Bayesian Inference and Coding
Tutorial 1: Probability and Bayesian Inference

Week

4

Theme / Topic / Module

Simple Linear Block Codes and Entropy

Face to face activities

Lecture 7 - Introduction to Block Codes
Lecture 8 - Entropy

Week

5

Theme / Topic / Module

Entropy & Information

Face to face activities

Lecture 9 - Entropy & Information
Lecture 10 - Some fundamental inequalities
Tutorial 2: Coding, Entropy and Information

Week

6

Theme / Topic / Module

Symbol Codes

Face to face activities

Lecture 11 - Entropy & Coding
Lecture 12 - Lossy Compression

Week

7

Theme / Topic / Module

Symbol Codes

Face to face activities

Lecture 13 - Symbol Codes for Lossless Compression
Lecture 14 - Source Coding Theorem for Symbol Codes
Tutorial 3: Coding and Compression

Week

8

Theme / Topic / Module

Stream Codes

Face to face activities

Lecture 15 - Arithmetic Coding
Lecture 16 - Arithmetic Coding (cont.)

Week

9

Theme / Topic / Module

Stream Codes / Noisy Channels

Face to face activities

Lecture 17 - Lempel-Ziv Coding
Lecture 18 - Noisy Channels
Tutorial 4: Arithmetic Coding

Week

10

Theme / Topic / Module

Noisy-Channel Coding

Face to face activities

Lecture 19 - Noisy-Channel Coding
Lecture 20 - The Noisy-Channel Source Coding Theorem

Week

11

Theme / Topic / Module

Noisy-Channel Coding

Face to face activities

Lecture 21 - Channel Capacity
Lecture 22 - Noisy-Channel Codes in Practice
Tutorial 5: Noisy Channel Coding

Week

12

Theme / Topic / Module

Algorithmic Information Theory

Face to face activities

Lecture 23 - Kolomgorov Complexity
Lecture 24 - Algorithmic Information Theory

Week

13

Theme / Topic / Module

Algorithmic Information Theory

Face to face activities

Lecture 25 - Algorithmic Information Theory & Sequential Prediction
Lecture 26 - Summary, Final remarks, Q&A