Controlled Diffusion and Composition
Stub. Recent methods for controlling and composing diffusion models — DreamFusion (text-to-3D via score distillation), Progressive Guidance, MultiDiffusion, RB-Modulation, TRAP — plus composition of diffusion models more generally, the Seed tutorial, LLM-based material generation, parallel diffusion, and test-time adaptation (SoTTA).
The course’s capstone survey of where controllable generation is heading: composing multiple diffusion models or multiple spatial/semantic constraints at generation time (MultiDiffusion, RB-Modulation, TRAP), lifting 2D diffusion priors into other domains via score distillation (DreamFusion’s text-to-3D), and progressively refining guidance strength during sampling. Rounds out with a grab-bag of applied/systems concerns: using an LLM to generate structured material/asset descriptions, running diffusion sampling in parallel for speed, and source-free test-time adaptation (SoTTA) for deploying a model under distribution shift.
Outline (TODO — flesh out each)
- Composition of diffusion models
- DreamFusion — text-to-3D via score distillation
- Progressive guidance
- MultiDiffusion
- RB-Modulation
- TRAP
- LLM-based material/asset generation
- Parallel diffusion sampling
- Source-free test-time adaptation (SoTTA)