Time/Location

Instructors

Stefano Ermon
Aditya Grover

Course Assistants

Yang Song
Jiaming Song
Rui Shu
Casey Chu
Nishith Khandwala
Kristy Choi

Course Description

Generative models are widely used in many subfields of AI and Machine Learning. Recent advances in parameterizing these models using deep neural networks, combined with progress in stochastic optimization methods, have enabled scalable modeling of complex, high-dimensional data including images, text, and speech. In this course, we will study the probabilistic foundations and learning algorithms for deep generative models, including Variational Autoencoders (VAE), Generative Adversarial Networks (GAN), autoregressive models, and normalizing flow models. The course will also discuss application areas that have benefitted from deep generative models, including computer vision, speech and natural language processing, graph mining, and reinforcement learning.

Prerequisites

Basic knowledge about machine learning from at least one of CS 221, 228, 229 or 230. Students will work with computational and mathematical models and should have a basic knowledge of probabilities and calculus. Proficiency in some programming language, preferably Python, required.

Piazza
Course notes

Suggested Reading

There is no required textbook. Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville. Online version available free here.

Grading Policy

Project Guidelines

The course project will give the students a chance to explore deep generative modeling in greater detail. Course projects will be done in groups of up to 3 students and can fall into one or more of the following categories:

Collaboration Policy and Honor Code

You are free to form study groups and discuss homeworks and projects. However, you must write up homeworks and code from scratch independently without referring to any notes from the joint session. You should not copy, refer to, or look at the solutions in preparing their answers from previous years’ homeworks. It is an honor code violation to intentionally refer to a previous year’s solutions, either official or written up by another student. Anybody violating the honor code will be referred to the Office of Judicial Affairs.

Submission Instructions

We will be using the GradeScope online submission system. All students (non-SCPD and SCPD) should submit their assignments electronically via GradeScope. Students can typeset or scan their homeworks.

To register for GradeScope,

Written Assignments: Homeworks should be written up clearly and succinctly; you may lose points if your answers are unclear or unnecessarily complicated. You are encouraged to use LaTeX to writeup your homeworks (here is a template), but this is not a requirement.

Late Homework: You have 6 late days which you can use at any time during the term without penalty. For a particular homework, you can use only two late days. Once you run out of late days, you will incur in a 25% penalty for each extra late day you use. Each late homework should be clearly marked as “Late” on the first page.

Regrade Policy: You may submit a regrade request if you believe that the course staff made an error in grading. Any regrade requests should be submitted through Gradescope within one week of receiving your grade. Please try to be as specific as possible with your regrade request.

Syllabus (tentative, periodically updated throughout the quarter)

Week Starting date Lecture topics Coursework Sections
1 Sep 24 Introduction and Background (slides 1, slides 2)    
2 Oct 1 Autoregressive Models (slides 3, slides 4) HW 1 released Probability and linear algebra, PyTorch
3 Oct 8 Variational Autoencoders (slides 5, slides 6)    
4 Oct 15 Normalizing Flow Models (slides 7, slides 8) HW 1 due, HW 2 released CNNs and RNNs
5 Oct 22 Generative Adversarial Networks Project Proposal due  
6 Oct 29 Guest lecture 1: Tengyu Ma, Energy-based Models HW 2 due  
7 Nov 5 Discreteness in Latent Variable Modeling Mid term  
8 Nov 12 Guest lecture 2: Diederik P. Kingma, Guest lecture 3: TBD Project Progress Report due, HW 3 released  
9 Nov 19 - Thanksgiving break -    
10 Nov 26 Applications: Vision, Speech, Language, Graphs, Reinforcement learning HW 3 due on Monday, Poster presentation on Friday  
11 Dec 3 Guest lectures: TBD    
12 Dec 10 - Finals week - Final project reports due  

Additional Reading: Surveys and tutorials

  1. Tutorial on Deep Generative Models. Aditya Grover and Stefano Ermon. International Joint Conference on Artificial Intelligence, July 2018.
  2. Tutorial on Generative Adversarial Networks. Computer Vision and Pattern Recognition, June 2018.
  3. Tutorial on Deep Generative Models. Shakir Mohamed and Danilo Rezende. Uncertainty in Artificial Intelligence, July 2017.
  4. Tutorial on Generative Adversarial Networks. Ian Goodfellow. Neural Information Processing Systems, December 2016.
  5. Learning deep generative models. Ruslan Salakhutdinov. Annual Review of Statistics and Its Application, April 2015.