Correlation and Correlation Structure (11) – Estimation using Random Matrix Theory

In the classical regime, when we have plenty of observations relative to what we need to estimate, we can rely on the sample covariance matrix as a faithful representation of the underlying covariance structure. However, in the high-dimensional settings common to modern data science – where the number of attributes/features is comparable to the number of observations , the sample covariance matrix is a bad estimator. It is not merely noisy; it is misleading. The eigenvalues of such matrices...