INTELLIGENT AGENTS & DECISION (CS_533_001_S2018)

Lecture Schedule and Slides

Supplementary Online Textbook

Reinforcement Learning: An Introduction - Draft
Richard Sutton and Andrew G. Barto
Second Edition, in progress
MIT Press

Instructor Note about Textbook: This book draft just came out and I will be experimenting with using it as supplementary reading material throughout the course. The course lecture slides from the instructor are relatively self-contained, but the supplementary textbook offers many valuable perspectives and examples. Note that the mathematical notation in the book and the course slides will not always be consistent. 

Description

In this course we will study models and algorithms for automated planning and decision making. The course will be divided into four main sections.

1) We will study planning in the context of Markov decision processes (MDPs) where the environment is allowed to be stochastic. We will cover the basic theory and algorithms for explicit state-space MDPs for exactly solving small to moderately sized problems.

2) We will study the area of Monte-Carlo planning, which is a middle ground between reinforcement learning and MDP planning, where a simulator of the system to be controlled is available and can be used to make intelligent action choices.

3) We will study the basic theory and algorithms for reinforcement learning, where the agent is not given a model of the environment, but instead must learn to act in the world by directly interacting with the environment. We will learn about a two of the primary RL paradigms, temporal-difference learning and policy gradient methods, and learn how they can be applied for both linear and non-linear agent architectures.

Assignments

There will be a number of assignments. Each will generally involve implementing and evaluating one or more algorithms and reporting their results. You are free to use any programming language that you would like to complete the assignments.

There will also be occasional written problems posted. Those problems will not be collected or graded. However, they will be critical to performing well on the mid-term.

Mid-Term

There will be an in class mid-term exam, which will cover material from the lectures and the written problems.

Final Project

Students will work on a final project during the last month of the course. Small teams are allowed. The topic of the final project is up to the students, but must be relevant to the course content and approved by the instructor. Each team will present their final project to the class during the week of final exams. 

Grades

The final grade will be calculated as follows: Assignments 60%, Mid-Term 15%, Final Project 25%

Honor Code

Collaboration on assignments is not permitted. The instructor will actively check for copying of code and solutions. The work you hand in should be your own. Any violation of these rules will result in failing the course.