Advanced Topics in Artificial Intelligence COMP4620/COMP8620 
Welcome to the Advanced AI course at the ANU !
The course focuses on the Foundations of AI, including inductive inference, sequence prediction, decisionmaking under uncertainty, reinforcement learning, intelligent agents, information theory, philosophical foundations, and others.News
10May19: website contents createdFormalities/Miscellaneous/Summary
Offered By: Intelligent Agents Team @ The AI Group @ Research School of Computer Science @ Australian National UniversityOffered In: Second Semester, 2019 (22 July to 17 November). See Schedule below
Course Coordinator: Marcus Hutter
Lecturers: Elliot Catt and Sultan J. Majeed
Tutors/Labs/Assistance: Samuel YangZhao Tianyu Wang
Target: Undergraduate (COMP4620) and Graduate (COMP8620) students. Others welcome.
Enrollment: Undergraduate & Masters: The usual way via ISIS. Honors&Others: Contact admin/lecturer.
Admin: Sandra Harrison student.services@cecs.anu.edu.au or studentadmin.cecs@anu.edu.au, 02 6125 4450, Office N202, Level 2 CSIT Bld.108
Class representative: See here for further information.
Course Subjects: Computer Science & Mathematics & Statistics
Unit Value: 6 units
Time Table: See Schedule below for details
Indicative Assessment: Initial Hurdle Assessment (pass/fail); Assignments (45%); Seminar (10%); Examination (45%) [details]
Indicative Workload: ~25h lectures, ~6h tutorial, ~10h lab, ~50h assignments, lots of selfstudy
Prescribed texts: Excerpts from (see resources for details)
 Shane Legg (2008) Machine Super Intelligence
 Marcus Hutter (2005) Universal Artificial Intelligence
 Joel Veness et al. (2011) A Monte Carlo AIXI Approximation
ANU page: http://programsandcourses.anu.edu.au/2019/course/comp4620
Wattle page: https://wattlecourses.anu.edu.au/course/view.php?id=27258
This page: http://cs.anu.edu.au/courses/COMP4620/2019.html
Prerequisites
Machine Learning (COMP4670/COMP8600) or Artificial Intelligence (COMP3620/COMP6320) or Information Theory (COMP2610/COMP6261). Students are expected to be familiar with the material in the AI book Russell&Norvig (2010) Chp. 2, 3, 5.2, 5.5, 13, 15.12, 17.13, 21; and Chapter 1 of Li&Vitanyi (2008), which is a great refresher of basic computability, information, and probability theory.The course requires good math and probability skills in general. It is often regarded as (one of) the hardest CS course(s) at ANU, and definitely is more challenging than the three courses above.
The following selfevaluation may help you determine whether to take this course or not. If you can answer most questions essentially correctly within the suggested time, you are probably prepared for the course. If you have difficulties answering say 3 or more questions even given extra time, the course is probably too difficult for you. Study the material above before you consult the solution.
Note: If you are planning to do a project with me, excelling in this course is a defacto prerequisite, since most of my projects require the knowledge taught in this course.
Course Description
This is an advanced undergraduate and graduate course that covers advanced topics in Artificial Intelligence. The course focuses on the foundations of AI, including inductive inference, decisionmaking, reinforcement learning, information theory, and some game and agent theory. The dream of creating artificial devices that reach or outperform human intelligence is many centuries old. This course presents an elegant parameterfree theory of an optimal reinforcement learning agent embedded in an arbitrary unknown environment that possesses essentially all aspects of rational intelligence. The theory reduces all conceptual AI problems to pure computational questions, and is key to addressing many theoretical, philosophical, and practical AI questions. How to perform inductive inference is closely related to the AI problem. The course covers Solomonoff's theory, which solves the induction problem, at least from a philosophical and statistical perspective. Both theories are based on Occam's razor quantified by Kolmogorov complexity, Bayesian probability theory, and sequential decision theory. The course is for computer science students interested in knowing about or building general AI systems from first principles, and others interested in the formal foundations of intelligence.Learning Outcomes
While the Introduction to AI course taught a diversity of methods for solving a variety of AI problems, this Advanced AI course emphasizes the foundational, unifying, and general aspects of (artificial) intelligence. Course highlights are: Formal definitions of (general rational) Intelligence;
 Optimal rational agents for arbitrary problems;
 Philosophical, mathematical, and computational background;
 Some approximations, implementations, and applications;
 Stateoftheart artificial general intelligence.
 measures, tests, and definitions of intelligence;
 Occam's razor;
 universal Turing machines;
 algorithmic information theory;
 probability theory;
 universal induction;
 Bayesian sequence prediction;
 minimum description length principle;
 intelligent agents;
 sequential decision theory;
 reinforcement learning;
 planning under uncertainty;
 universal search;
 MonteCarlo tree search;
 philosophical foundations.
Schedule
Lectures: Mon (HA T) 10ºº11ºº & Wed (DA Brown 110/108) 9ºº10ºº & Fri (LAW T) 9ºº10ºº;Tutorials: Mon.1517ºº PSYC G5Tue.1315ºº SRES TTue.1618ºº PSYC G8;
Labs: Mon.1517ºº HN Lab 1Tue.1315ºº HN Lab 1Tue.1618ºº HN Lab 2
Lectures, Tutorials, and Labs will not run every day/week!
See schedule below for details.
#  Week  Lecture / Tutorial / Lab 

to be updated throughout the course  
1  22Jul  26Jul do selfevaluation (solution) 
Overview & Introduction [Advertisement] [Slides] Reading:[Legg08.Chp.1] 
2  29Jul 2Aug get assignment 1 
Information Theory & Kolmogorov Complexity [Slides] Reading:[UAIBook.Sec.2.2] Tutorials 
3  5Aug  9Aug  Bayesian Probability Theory [Slides] Reading:[UAIBook.Sec.2.3] Algorithmic Probability & Universal Induction [Slides] Reading:[UAIBook.Sec.2.4] No tutorials. 
4  12Aug  16Aug  Tutorials (only). No lectures 
5  19Aug  23Aug  Minimum Description Length [Slides] Optional Reading:[MDL.Chp.1] Universal Similarity [Slides] Optional Reading:[USM] No tutorials. 
6  26Aug  30Aug  Bayesian Sequence Prediction [Slides] Reading: Parts of [UAIBook.Chp.3] Context Tree Weighting [Slides] Reading:[CTW] Tutorials 
2Sep  13Sep  break  
7  16Sep  20Sep hand in assignment 1 get assignment 2 
Universal Rational Agents
[Slides]
Reading:[UAIBook.Chp.4.1&4.2&5.1.1] MCAIXICTW [Slides] Reading:[MCAIXICTW] Orientation Lab for Assignment 2 
8  22Sep  27Sep  Theory of Rational Agents [Slides] Reading:try [UAIBook.Chp.5] Q&A Lab for Assignment 2 
9  30Sep  4oct  Approximations and Applications
[Slides] Q&A Lab for Assignment 2 
10  7oct  11oct  Solutions to Assignment 1 (in lecture slots for all students) Q&A Lab for Assignment 2 No Monday Lecture or Lab 
11  14oct  18oct hand in assignment 2 
Discussion
[Slides]
Reading:[UAIBook.Chp.8] Q&A Lab for Assgnment 2. 
12  21oct  25oct 
Student Presentation of Individual Contribution to Practical Assignment. Send slides in advance to Samuel YangZhao 
Assignments
Theory Assignment 1: The theory assignment is to be done individually, and will involve various mathematical exercises that will deepen the understanding of the lectured material. Samuel YangZhao will be tutor and primary contact for the theory assignment.Practical Group Assignment 2: The practical assignment will be a group project. Goal is to implement the MCAIXICTW model, which is a recent practical scaleddown version of the theoretical universal AI agent AIXI. Students will acquire firsthand experience how a single algorithm can autonomously learn to solve various toy problems like playing TicTacToe or PacMan or Poker just based on experience and reward feedback without ever being told the rules of the game. A code skeleton in very light python will be provided, but a group can request to use a different programming language at their own risk. Particular emphasis is on ease of use (installation, compilation, running, modification) and good documentation. The project involves programming of various sophisticated functions, and requires and furthers the understanding of the theoretical material taught in the main class.
Each group will consist of 58 students. A group can selforganize and distribute work internally. The various modules/tasks/domains can be implemented by different students, each responsible for delivering a welltested module including source and documentation. The group is responsible to deliver a final product consisting of documented source code, experimental results, and a final joint report.
A Lab director will supervise the practical group project during lab sessions.
Tutorials/Labs
Rehearsal of lecture material and help with assignments: See WattleAssessment
Hurdle:Initial Hurdle Assessment (pass/fail) in Week 1 or 2 to ensure enrolled students have the required background.Theory: Individual Theory Assignments (20%). Late submissions will not be accepted, and result in 0 points, hence failure of the course.
Practice: Practical Group Assignment (25%). Late submissions will not be accepted, and result in 0 points, hence failure of the course.
Seminar: Seminar = 5 minute presentation of individual contribution to group assignment (10%).
Exam: Final written examination (45%) Exam (120min, written, closedbook,informal&math questions similar to Ass.1).
Know: What to know for the exam: Material in the course slides.
The other provided reading material should help you to better understand the slides, but will itself not be examined.
Math questions will be similar to Assignment 1 but no long or hard proofs. Informal questions test knowledge&comprehension.
Pass: To pass the course, students must pass each assignment and the final exam.
Grading: Final course marks will be subject to moderation, so may differ from the raw sum of assessment marks.
Plagiarism: Misconduct will result in failure of the course and disciplinary consequences (no exceptions)
and possibly expulsion from the ANU. See: [AcademicHonesty@ANU] [Student Handbook@RSCS]
Resources
 Slides and assignments: See links in schedule.

Marcus Hutter (2005) Universal Artificial Intelligence
The lectures will draw heavily from this (tough) book, but only the easier parts will be covered.
It is not necessary that students have a copy of this book, but it can help understanding the slides.
It is available at the ANU library, the ANU bookshop or cheaper at the BookDepository. 
Shane Legg (2008) Machine Super Intelligence
This is a gentle more philosophical, less mathematical introduction to the subject. It is highly recommended. It costs less than $20 and the pdf is even free. 
Samuel Rathmanner and Marcus Hutter (2011) A Philosophical Treatise of Universal Induction
Another more philosophical, less mathematical introduction, specificially on Universal Induction. 
Joel Veness et al. (2011) A Monte Carlo AIXI Approximation
This is a (tough) research paper, which builds the basis for the group implementation project. 
The lectures will also draw from the following paper(s)
F. Willems and Y. Shtarkov and T. Tjalkens
The contexttree weighting method: Basic properties
IEEE Transactions on Information Theory (41), 653  664, 1995
A more readable version of the same paper is here  If you're curious what's out there else (but this is clearly beyond the course), see further recommended AI books and the papers read in the RL reading group (which everyone is welcome to attend).
 If you do not possess the required prerequisites to take this course now, but want to prepare for it, see the undergraduate studies guide.