Kullback-Leibler distance as a measure of the information filtered from multivariate data

Michele Tumminello, Fabrizio Lillo, and Rosario N. Mantegna

Phys. Rev. E 76, 031123 – Published 19 September 2007

Abstract

We show that the Kullback-Leibler distance is a good measure of the statistical uncertainty of correlation matrices estimated by using a finite set of data. For correlation matrices of multivariate Gaussian variables we analytically determine the expected values of the Kullback-Leibler distance of a sample correlation matrix from a reference model and we show that the expected values are known also when the specific model is unknown. We propose to make use of the Kullback-Leibler distance to estimate the information extracted from a correlation matrix by correlation filtering procedures. We also show how to use this distance to measure the stability of filtering procedures with respect to statistical uncertainty. We explain the effectiveness of our method by comparing four filtering procedures, two of them being based on spectral analysis and the other two on hierarchical clustering. We compare these techniques as applied both to simulations of factor models and empirical data. We investigate the ability of these filtering procedures in recovering the correlation matrix of models from simulations. We discuss such ability in terms of both the heterogeneity of model parameters and the length of data series. We also show that the two spectral techniques are typically more informative about the sample correlation matrix than techniques based on hierarchical clustering, whereas the latter are more stable with respect to statistical uncertainty.

Received 1 June 2007

DOI:https://doi.org/10.1103/PhysRevE.76.031123

Authors & Affiliations

Michele Tumminello^1,2, Fabrizio Lillo^1,3, and Rosario N. Mantegna¹

¹Dipartimento di Fisica e Tecnologie Relative, Università di Palermo, Viale delle Scienze, I-90128 Palermo, Italy
²CNR-INFM, Unità di Palermo, Palermo, Italy
³Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, New Mexico 87501, USA

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand

Issue

Vol. 76, Iss. 3 — September 2007

Reuse & Permissions

Access Options

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
(Color online) Block diagonal model with degenerate coefficients. Comparison of the four filtered correlation matrices described in the text. In the graph we plot the stability of the filtered matrix ( $x$ axis) against the amount of information about the correlation matrix that is retained in the filtered matrix ( $y$ axis). Small values of $⟨ K (C_{i}^{filt}, C_{j}^{filt}) ⟩$ and $⟨ K (C_{i}, C_{i}^{filt}) ⟩$ correspond to large stability and large amount of information preserved by the filtering, respectively. The analysis is performed for a system of 100 elements divided in 12 orthogonal groups, each one depending on a specific Gaussian factor, i.e., a block diagonal model. Averages have been performed over 1000 independent realizations of the system and error bars correspond to one standard deviation. Error bars are not shown when they are smaller than the symbol size.Reuse & Permissions
Figure 2
(Color online) Hierarchically nested factor model with degenerate coefficients. Comparison of the filtered correlation matrices produced by the four techniques described in the text. The analyzed system is composed by 100 elements following the HNFM with 23 factors obtained in Ref. [20]. Averages have been performed over 1000 independent realizations of the system and error bars correspond to one standard deviation. Error bars are not shown when they are smaller than the symbol size.Reuse & Permissions
Figure 3
(Color online) Block diagonal model with nondegenerate coefficients. Comparison of the four filtered correlation matrices described in the text. The analysis is performed for a system of 100 elements divided in 12 orthogonal groups, each one depending on a specific Gaussian factor, i.e., a block diagonal model. Averages have been performed over 1000 bootstrap replicas of a single realization of the system and error bars correspond to one standard deviation. Error bars are not shown when they are smaller than the symbol size.Reuse & Permissions
Figure 4
(Color online) Hierarchically nested factor model with nondegenerate coefficients. Comparison of the filtered correlation matrices produced by the four techniques described in the text. The analyzed system is composed by 100 elements following the HNFM with 23 factors obtained in Ref. [20]. Averages have been performed over 1000 bootstrap replicas of a single realization of the system and error bars correspond to one standard deviation. Error bars are not shown when they are smaller than the symbol size.Reuse & Permissions
Figure 5
Correlation matrix of a real system composed of 100 stocks traded at NYSE during the period from January 2001 to December 2003. The variable investigated is the daily return of the most capitalized stocks. The length of time series is $T = 748$ for this system. Averages have been performed over 1000 bootstrap replicas of data series and error bars correspond to one standard deviation.Reuse & Permissions
Figure 6
Correlation matrix of a real system composed of 92 stocks traded at LSE during the period from January to December 2002. The variable investigated is the daily return of the most capitalized stocks. The length of time series is $T = 250$ for this system. Averages have been performed over 1000 bootstrap replicas of data series and error bars correspond to one standard deviation.Reuse & Permissions

Physical Review E

covering statistical, nonlinear, biological, and soft matter physics