Diffusion Models as Differential Equations — SDEs, ODEs, and an Introduction to Flows
Stub. Formalizing diffusion as a differential equation: the forward process as an SDE driven by Brownian/Wiener noise (or, in the noiseless limit, an ODE — gradient descent read as a continuous-time process), the reverse-time SDE, the Fokker-Planck/Kolmogorov equation linking the SDE to its density's evolution, and numerical solvers (Euler, Runge-Kutta).
Takes the “time-varying force field” hand-wave from the diffusion-models note and makes it precise: the forward noising process is a stochastic differential equation driven by a Wiener/Brownian-motion noise term, and reversing it (running the SDE backward in time) is what generation actually is — with the deterministic (noiseless) special case being an ordinary differential equation, which is exactly gradient descent viewed as a continuous-time limit. The Fokker-Planck (equivalently, Kolmogorov forward) equation is the PDE governing how the density evolves under the SDE, which is what justifies the score-function connection. Closes with the numerical reality of simulating these SDEs/ODEs — Euler and Runge-Kutta solvers — and previews the probability-flow ODE, which is where the flow-matching framework picks up.
Outline (TODO — flesh out each)
- The differential-equation (DE) problem statement
- Gradient descent as an ODE; the reverse ODE
- Stochastic differential equations (SDEs); the Brownian/Wiener process
- The reverse-time SDE
- Fokker-Planck and Kolmogorov equations
- Numerical solvers: Euler’s method, Runge-Kutta