Review — Probability, Linear Algebra, and Deep Learning for Generative Models
Stub. Bayes' rule, maximum likelihood estimation, and multivariate distributions; conditional independence and Markov structure; the Expectation-Maximization (EM) algorithm; PCA/SVD; and the basic deep-learning building blocks used throughout the course.
The prerequisite toolkit, compressed into two review lectures. Probability: Bayes’ rule and MLE as the two lenses the whole course alternates between, multivariate (especially Gaussian) distributions, and conditional independence/Markov structure as the assumption that makes high-dimensional models tractable. The centerpiece is Expectation-Maximization — the general recipe for maximum-likelihood estimation with latent variables, which is the direct ancestor of the VAE’s training objective (the ELBO is literally what EM optimizes when the E-step can’t be done exactly). Linear algebra: PCA/SVD as the linear latent-variable model VAEs generalize nonlinearly. Deep learning: the minimal set of building blocks (networks, backprop, basic architectures) assumed from here on.
Outline (TODO — flesh out each)
- Bayes’ rule; maximum likelihood estimation
- Multivariate distributions
- Conditional independence; Markov structure
- Expectation-Maximization (EM)
- PCA and SVD as a linear latent-variable model
- Basic deep learning building blocks