Schedule

Suggested Readings:

• History of AI
• History of Generative Models
• Advent of Deep Generative Models
• Applications of Deep Generative Models

Suggested Readings:

• Basics of Probability and Statistics
• Basics of Information Theory
• Discriminative vs. Generative Models
• Basics of Maximum Likelihood Estimation
• Basics of Optimization

Suggested Readings:

• Gaussian Parameter Estimation via MLE
• Introduction to Latent Variable Models
• Basics of Probabilstic Principal Component Analysis

Suggested Readings:

• Probabilistic Principal Component Analysis (PPCA)
• Application of PPCA to Generating Images of Faces

Suggested Readings:

• Latent Variable Models
• Introduction to Variational Inference
• Introduction to Expectation Maximization
• Expectation Maximization for Gaussian Mixture Model

Suggested Readings:

• Variational Auto-Encoders (VAEs)

Suggested Readings:

• Application of VAEs to Generating Images of Handwritten Digits and Faces

Suggested Readings:

• Introduction to Markov Models.
• Maximum Likelihood Estimation for Markov Models.

Suggested Readings:

• Introduction to Hidden Markov Models.
• Inference, Deccoding, and Learning for Hidden Markov Models.
  • The Viterbi Algorithm.
  • The Baum-Welch Algorithm.

Suggested Readings:

• Introduction to Linear Dynamical Systems.
• Filtering and Smoothing for Linear Dynamical Systems.
• State Estimation for Linear Dynamical Systems.
• EM for Linear Dynamical Systems.

Suggested Readings:

• Introduction the RNN architecture
• Unfolding and Backpropagation through Time.
• Variants of RNNs: LSTMs, GRUs

Suggested Readings:

• Introduction the RNN architecture
• Unfolding and Backpropagation through Time.
• Variants of RNNs: LSTMs, GRUs

Suggested Readings:

• Introduction the Self-Attention mechanism
• Multi-Head Attention
• Attention is All You Need (https://arxiv.org/pdf/1706.03762)

Suggested Readings:

• BERT (https://arxiv.org/abs/1810.04805)
• ImageGPT (https://cdn.openai.com/papers/Generative_Pretraining_from_Pixels_V2.pdf)
• ViT (https://arxiv.org/abs/2010.11929)

Suggested Readings:

• ViLBERT (https://arxiv.org/abs/1908.02265)
• CLIP (https://arxiv.org/abs/2103.00020)
• DALL-E (https://arxiv.org/abs/2102.12092)
• LLaVA (https://arxiv.org/abs/2304.08485)

Suggested Readings:

• Introduction to Denoising Diffusion Probabilistic Models