Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• 

  Introduction and Background
  tl;dr: Introduction to Deep Generative Models, History, and Applications
  [syllabus] [slides] [recording]
  

  Suggested Readings:

  • History of AI
  • History of Generative Models
  • Advent of Deep Generative Models
  • Applications of Deep Generative Models
• 

  Maximum Likelihood Estimation
  tl;dr: Basics of Probability and Statistics, Information Theory, Maximum Likelihood Estimation
  [slides] [recording]
  

  Suggested Readings:

  • Basics of Probability and Statistics
  • Basics of Information Theory
  • Discriminative vs. Generative Models
  • Basics of Maximum Likelihood Estimation
  • Basics of Optimization
• 

  Classes of Generative Models
  tl;dr: Gaussian Parameter Estimation, Latent Variable Models
  [slides] [recording]
  

  Suggested Readings:

  • Gaussian Parameter Estimation via MLE
  • Introduction to Latent Variable Models
  • Basics of Probabilstic Principal Component Analysis
• 

  Probabilistic Principal Component Analysis
  tl;dr: Probabilistic Principal Component Analysis (PPCA)
  [slides] [recording]
  

  Suggested Readings:

  • Probabilistic Principal Component Analysis (PPCA)
  • Application of PPCA to Generating Images of Faces
• 

  Latent Variable Models
  tl;dr: Latent Variable Models, Variational Inference, Expectation Maximization
  [slides] [recording]
  

  Suggested Readings:

  • Latent Variable Models
  • Introduction to Variational Inference
  • Introduction to Expectation Maximization
  • Expectation Maximization for Gaussian Mixture Model
• 

  Latent Variable Models
  tl;dr: VAE (definition, reparameterization trick)
  [slides] [recording]
  

  Suggested Readings:

  • Variational Auto-Encoders (VAEs)
• 

  Latent Variable Models
  tl;dr: VAE - application + hands on session
  [slides] [recording]
  

  Suggested Readings:

  • Application of VAEs to Generating Images of Handwritten Digits and Faces
• 

  Markov Models
  tl;dr: Markov Models, MLE for Markov Models
  [slides] [recording]
  

  Suggested Readings:

  • Introduction to Markov Models.
  • Maximum Likelihood Estimation for Markov Models.
• 

  Hidden Markov Models
  tl;dr:
  [slides] [recording 1] [recording 2]
  

  Suggested Readings:

  • Introduction to Hidden Markov Models.
  • Inference, Deccoding, and Learning for Hidden Markov Models.
    • The Viterbi Algorithm.
    • The Baum-Welch Algorithm.
• 

  Linear Dynamical Systems
  tl;dr:
  [slides] [recording 1] [recording 2]
  

  Suggested Readings:

  • Introduction to Linear Dynamical Systems.
  • Filtering and Smoothing for Linear Dynamical Systems.
  • State Estimation for Linear Dynamical Systems.
  • EM for Linear Dynamical Systems.