Random Recurrent Networks and Dynamic Mean-Field Theory
Stub. Analyzing a recurrent neural network with random connectivity as a high-dimensional dynamical system: the geometry and effective dimensionality of its activity, and dynamic mean-field theory — the technique that replaces thousands of coupled, randomly-connected units with a single effective stochastic process, making the network's statistical behavior analytically tractable.
Corresponds to the “High-dimensional and random neural networks” unit of the syllabus (weeks 7–10) — no lecture dates posted yet for this unit, so this stub is organized from the syllabus’s topic list rather than specific lectures; update once lectures are posted. Covers random recurrent networks (units connected by random, rather than trained, weights — the baseline object the rest of the unit analyzes) and the question of how to even describe the resulting high-dimensional activity: its geometry (what shape does the trajectory trace out in activity space) and its effective dimensionality (how many of the network’s many neurons’ worth of degrees of freedom are actually independent). Dynamic mean-field theory is the main analytical tool, replacing the intractable many-body problem of thousands of coupled random units with a single self-consistent effective single-unit process — the same mean-field idea behind the random-matrix spectral results used elsewhere (operators-on-inner-product-spaces) to understand large random systems collectively rather than unit by unit.
Outline (TODO — flesh out each)
- Random recurrent networks
- Activity geometry
- Effective dimensionality of network activity
- Dynamic mean-field theory