Information Theory
Entropy, coding, and the limits of compression and communication.
Entropy, mutual information, source/channel coding, and the theoretical limits on compression and reliable communication.
Documents
Notes
- BCH and Reed-Solomon Codes
- Deletion-Correcting Codes and Trace Reconstruction
- Distributed Storage Codes
- Expander Codes, LDPC Codes, and Fountain Codes
- Finite Fields and Algebraic Preliminaries
- Group Testing
- Linear Codes, Bounds on Code Size, and the MacWilliams Identities
- Molecular and DNA Storage Coding
- Reed-Muller Codes and Polar Codes
- Sphere Packings, Kissing Numbers, and Lattices
- Information Theory and Portfolio Theory
- Kolmogorov Complexity
- Network Information Theory
- Gambling and Data Compression
- Universal Source Coding
- Maximum Entropy
- Information Theory and Statistics
- Rate Distortion Theory
- The Gaussian Channel
- Differential Entropy
- Channel Capacity
- Entropy Rates of a Stochastic Process
- The Asymptotic Equipartition Property
- Information Theory Inequalities
- Data Compression and Source Coding
- Information, Entropy, Relative Entropy, and Mutual Information