You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

  • Introduction and Background
    tl;dr: Introduction to Deep Generative Models, History, and Applications
    [syllabus] [slides] [recording]

    Suggested Readings:

    • History of AI
    • History of Generative Models
    • Advent of Deep Generative Models
    • Applications of Deep Generative Models
  • Maximum Likelihood Estimation
    tl;dr: Basics of Probability and Statistics, Information Theory, Maximum Likelihood Estimation
    [slides] [recording]

    Suggested Readings:

    • Basics of Probability and Statistics
    • Basics of Information Theory
    • Discriminative vs. Generative Models
    • Basics of Maximum Likelihood Estimation
    • Basics of Optimization
  • Classes of Generative Models
    tl;dr: Gaussian Parameter Estimation, Latent Variable Models
    [slides] [recording]

    Suggested Readings:

    • Gaussian Parameter Estimation via MLE
    • Introduction to Latent Variable Models
    • Basics of Probabilstic Principal Component Analysis
  • Probabilistic Principal Component Analysis
    tl;dr: Probabilistic Principal Component Analysis (PPCA)
    [slides] [recording]

    Suggested Readings:

    • Probabilistic Principal Component Analysis (PPCA)
    • Application of PPCA to Generating Images of Faces
  • Latent Variable Models
    tl;dr: Latent Variable Models, Variational Inference, Expectation Maximization
    [slides] [recording]

    Suggested Readings:

    • Latent Variable Models
    • Introduction to Variational Inference
    • Introduction to Expectation Maximization
    • Expectation Maximization for Gaussian Mixture Model
  • Latent Variable Models
    tl;dr: VAE (definition, reparameterization trick)
    [slides] [recording]

    Suggested Readings:

    • Variational Auto-Encoders (VAEs)
  • Latent Variable Models
    tl;dr: VAE - application + hands on session
    [slides] [recording]

    Suggested Readings:

    • Application of VAEs to Generating Images of Handwritten Digits and Faces
  • Markov Models
    tl;dr: Markov Models, MLE for Markov Models
    [slides] [recording]

    Suggested Readings:

    • Introduction to Markov Models.
    • Maximum Likelihood Estimation for Markov Models.
  • Hidden Markov Models
    tl;dr:
    [slides] [recording 1] [recording 2]

    Suggested Readings:

    • Introduction to Hidden Markov Models.
    • Inference, Deccoding, and Learning for Hidden Markov Models.
      • The Viterbi Algorithm.
      • The Baum-Welch Algorithm.
  • Linear Dynamical Systems
    tl;dr:
    [slides] [recording 1] [recording 2] [recording 3]

    Suggested Readings:

    • Introduction to Linear Dynamical Systems.
    • Filtering and Smoothing for Linear Dynamical Systems.
    • State Estimation for Linear Dynamical Systems.
    • EM for Linear Dynamical Systems.
  • Recurrent Neural Networks
    tl;dr:
    [slides] [recording 1] [recording 2] [recording 3]

    Suggested Readings:

    • Introduction the RNN architecture
    • Unfolding and Backpropagation through Time.
    • Variants of RNNs: LSTMs, GRUs
  • Recurrent Neural Networks and Attention Mechanisms
    tl;dr:
    [slides] [recording]

    Suggested Readings:

    • Introduction the RNN architecture
    • Unfolding and Backpropagation through Time.
    • Variants of RNNs: LSTMs, GRUs
  • Transformers
    tl;dr:
    [slides] [recording]

    Suggested Readings:

    • Introduction the Self-Attention mechanism
    • Multi-Head Attention
    • Attention is All You Need (https://arxiv.org/pdf/1706.03762)
  • BERT and Vision Transformers
    tl;dr:
    [slides] [recording 1] [recording 2]

    Suggested Readings:

    • BERT (https://arxiv.org/abs/1810.04805)
    • ImageGPT (https://cdn.openai.com/papers/Generative_Pretraining_from_Pixels_V2.pdf)
    • ViT (https://arxiv.org/abs/2010.11929)
  • Transformers for Vision and Language
    tl;dr:
    [slides] [recording 1] [recording 2]

    Suggested Readings:

    • ViLBERT (https://arxiv.org/abs/1908.02265)
    • CLIP (https://arxiv.org/abs/2103.00020)
    • DALL-E (https://arxiv.org/abs/2102.12092)
    • LLaVA (https://arxiv.org/abs/2304.08485)
  • Denoising Diffusion Probabilistic Models
    tl;dr:
    [slides] [recording 1] [recording 2]

    Suggested Readings:

    • Introduction to Denoising Diffusion Probabilistic Models
  • Latent Diffusion Models
    tl;dr:
    [slides] [recording]

    Suggested Readings:

    • Stable Diffusion
  • Image Editing with Diffusion Models
    tl;dr:
    [slides] [recording]

    Suggested Readings:

    • Introduction to Denoising Diffusion Probabilistic Models
    • Derivation of Diffusion Models + Stable Diffusion/ControlNet
    • Denoising Diffusion Implicit Models
    • Prompt-to-Prompt (P2P)
  • Score Matching with Diffusion Models
    tl;dr:
    [slides] [recording]

    Suggested Readings:

    • Denoising Score Matching