Separating signal from noise with random matrix theory
PCA on a return correlation matrix, with Marchenko-Pastur filtering to tell real factor structure from sampling noise. Validated against the closed-form edges and against scikit-learn.
00Overview
An empirical correlation matrix of asset returns is mostly noise: with N assets and T observations, sampling error alone produces a whole spectrum of eigenvalues. This toolkit applies PCA to the correlation matrix and uses Marchenko-Pastur random matrix theory to mark which eigenmodes are real structure and which are noise.
Validation band
01What it does
- PCA on the correlation matrix of standardised returns (a symmetric eigenproblem).
- Closed-form Marchenko-Pastur edges, lambda_pm = sigma^2 (1 +/- sqrt(q))^2.
- Bounded least-squares fit of the MP density to the empirical eigenvalue histogram to estimate sigma.
- Eigenmodes above lambda_plus are labelled signal (the Laloux-Cizeau-Bouchaud-Potters 1999 workflow).
- Reproduces Plerou et al. (1999): most empirical eigenvalues sit inside the MP bulk.
02Validation
Every routine is checked against an independent reference. The MP closed-form edge (sigma=1, N=100, T=200) reproduces lambda_plus ~ 2.9142 to 1e-12; the bulk density integrates to 1 within 1e-3; PCA matches scikit-learn to 1e-10 up to sign and column order; block-diagonal correlation recovers its K signal eigenvalues exactly.