- Lectures: Mon/Wed 4:30-5:50 pm, Location: Room 320-105
- Office Hours and Sections: See course calendar
- Mid-term: November 9 (Friday), 6-9 pm, Location: Hewlett 200
Generative models are widely used in many subfields of AI and Machine Learning. Recent advances in parameterizing these models using deep neural networks, combined with progress in stochastic optimization methods, have enabled scalable modeling of complex, high-dimensional data including images, text, and speech. In this course, we will study the probabilistic foundations and learning algorithms for deep generative models, including Variational Autoencoders (VAE), Generative Adversarial Networks (GAN), autoregressive models, and normalizing flow models. The course will also discuss application areas that have benefitted from deep generative models, including computer vision, speech and natural language processing, graph mining, and reinforcement learning.
Basic knowledge about machine learning from at least one of CS 221, 228, 229 or 230. Students will work with computational and mathematical models and should have a basic knowledge of probabilities and calculus. Proficiency in some programming language, preferably Python, required.
There is no required textbook. Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville. Online version available free here.
- Three homeworks (15% each): mix of conceptual and programming based questions
- Midterm: 15%
- Course Project: 40%
- Proposal: 5%
- Progress Report: 10%
- Poster Presentation: 10%
- Final Report: 15%
The course project will give the students a chance to explore deep generative modeling in greater detail. Course projects will be done in groups of up to 3 students and can fall into one or more of the following categories:
- Application of deep generative models on a novel task/dataset
- Algorithmic improvements into the evaluation, learning and/or inference of deep generative models
- Theoretical analysis of any aspect of existing deep generative models
Collaboration Policy and Honor Code
You are free to form study groups and discuss homeworks and projects. However, you must write up homeworks and code from scratch independently without referring to any notes from the joint session. You should not copy, refer to, or look at the solutions in preparing their answers from previous years’ homeworks. It is an honor code violation to intentionally refer to a previous year’s solutions, either official or written up by another student. Anybody violating the honor code will be referred to the Office of Judicial Affairs.
We will be using the GradeScope online submission system. All students (non-SCPD and SCPD) should submit their assignments electronically via GradeScope. Students can typeset or scan their homeworks.
To register for GradeScope,
- Create an account on GradeScope if you don’t have one already.
- Here are some tips for submitting through Gradescope.
Written Assignments: Homeworks should be written up clearly and succinctly; you may lose points if your answers are unclear or unnecessarily complicated. You are encouraged to use LaTeX to writeup your homeworks (here is a template), but this is not a requirement.
Late Homework: You have 6 late days which you can use at any time during the term without penalty. For a particular homework, you can use only two late days. Once you run out of late days, you will incur in a 25% penalty for each extra late day you use. Each late homework should be clearly marked as “Late” on the first page. No late days is allowed for deadlines related to the course project.
Regrade Policy: You may submit a regrade request if you believe that the course staff made an error in grading. Any regrade requests should be submitted through Gradescope within one week of receiving your grade. Please try to be as specific as possible with your regrade request.
Syllabus (tentative, periodically updated throughout the quarter)
|Week||Starting date||Lecture topics||Coursework||Sections|
|1||Sep 24||Introduction and Background (slides 1, slides 2)|
|2||Oct 1||Autoregressive Models (slides 3, slides 4)||HW 1 released||Probability and linear algebra, PyTorch|
|3||Oct 8||Variational Autoencoders (slides 5, slides 6)|
|4||Oct 15||Normalizing Flow Models (slides 7, slides 8)||HW 1 due, HW 2 released||CNNs and RNNs (slides)|
|5||Oct 22||Generative Adversarial Networks (slides 9, slides 10)||Project Proposal due|
|6||Oct 29||Guest lecture 1: Tengyu Ma on Monday,
Evaluation of Generative Models on Wednesday (slides 11)
|HW 2 due|
|7||Nov 5||Combining generative model variants (slides 12), Energy-based models (slides 13)||Mid term|
|8||Nov 12||Guest lecture 2: Diederik P. Kingma on Monday
Discreteness in Latent Variable Modeling on Wednesday
|Project Progress Report due, HW 3 released|
|9||Nov 19||- Thanksgiving break -|
|10||Nov 26||Applications: Vision, Speech, Language, Graphs, Reinforcement learning||Poster presentation on Friday|
|11||Dec 3||Guest lectures: TBD||HW 3 due|
|12||Dec 10||- Finals week -||Final project reports due|
Additional Reading: Surveys and tutorials
- Tutorial on Deep Generative Models. Aditya Grover and Stefano Ermon. International Joint Conference on Artificial Intelligence, July 2018.
- Tutorial on Generative Adversarial Networks. Computer Vision and Pattern Recognition, June 2018.
- Tutorial on Deep Generative Models. Shakir Mohamed and Danilo Rezende. Uncertainty in Artificial Intelligence, July 2017.
- Tutorial on Generative Adversarial Networks. Ian Goodfellow. Neural Information Processing Systems, December 2016.
- Learning deep generative models. Ruslan Salakhutdinov. Annual Review of Statistics and Its Application, April 2015.