Correlation, Covariance, MMSE Estimation, and the Joint Gaussian Distribution
Stub. Correlation and covariance as measures of joint variability; MMSE estimation of one random variable from another (constant, unconstrained, and linear estimators); and the joint Gaussian distribution — built from the standard 2-d normal, its key properties, and the higher-dimensional case.
Covariance quantifies how two random variables move together; MMSE estimation asks how best to predict one from the other under squared error. The chapter’s three estimator classes nest: the best constant estimator is the mean, the best unconstrained estimator is the conditional expectation, and the best linear estimator depends only on means/variances/covariance — which is exactly what makes it tractable, and exactly what ties this note to the joint Gaussian distribution: start from the standard 2-d normal (independent standard components), apply a linear transformation to get the general bivariate normal, and note that for jointly Gaussian variables the linear MMSE estimator is the true (unconstrained) MMSE estimator — the earlier hierarchy of estimators collapses. Covers the key properties that make Gaussians special for estimation (elliptical contours, uncorrelated implies independent, conditionals are Gaussian) and the extension to dimensions via the mean vector and covariance matrix.
Outline (TODO — flesh out each)
NoteTODO — all sections below
This whole note is still a stub. Needs the Hajek MMSE-estimation chapter
and the joint/bivariate Gaussian material (Ross Ch. 6 bivariate normal and
§7.8 multivariate normal — confirm section numbers against the edition in
lit/ross2019-first-course-probability). Source
text not yet extracted into sources/ — do not write from memory.
- Correlation and covariance
- Constant estimators
- Unconstrained estimators — the conditional expectation as MMSE estimator
- Linear estimators
- From the standard 2-d normal to the general bivariate normal
- Key properties of the bivariate normal distribution
- Higher dimensional joint Gaussian distributions — mean vector, covariance matrix