Information Theory COMP2610

Course overview

Assumed knowledge & required skills

Some background in elementary statistics and probability. Ability to differentiate and manipulate expressions involving logarithms and exponentials.

Course description

Information theory studies the fundamental limits of the representation and transmission of information. This course provides an introduction to information theory, studying fundamental concepts such as probability, information, and entropy and examining their applications in the areas of data compression, coding, communications, pattern recognition and probabilistic inference.

Textbooks

Information Theory, Inference, and Learning Algorithms by David MacKay, Cambridge University Press, 2003.
Additional reading: Elements of Information Theory by Cover and Thomas, 2nd Edition, New York, Wiley, 2006.

Workload

Twenty-six one-hour lectures and five two-hour tutorial sessions.