A Cautionary Note on Incremental Fit Indices Reported by LISREL
Abstract
Abstract. Incremental fit indices (IFIs) are regularly used when assessing the fit of structural equation models. IFIs are based on the comparison of the fit of a target model with that of a null model. For maximum-likelihood estimation, IFIs are usually computed by using the χ2 statistics of the maximum-likelihood fitting function (ML-χ2). However, LISREL recently changed the computation of IFIs. Since version 8.52, IFIs reported by LISREL are based on the χ2 statistics of the reweighted least squares fitting function (RLS-χ2). Although both functions lead to the same maximum-likelihood parameter estimates, the two χ2 statistics reach different values. Because these differences are especially large for null models, IFIs are affected in particular. Consequently, RLS-χ2 based IFIs in combination with conventional cut-off values explored for ML-χ2 based IFIs may lead to a wrong acceptance of models. We demonstrate this point by a confirmatory factor analysis in a sample of 2449 subjects.
References
Arbuckle, J.L. (2003). Amos 5.0 [Computer software]. Chicago, IL: SPSSBentler, P.M. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238– 246Bentler, P.M. (1995). EQS: Structural equations program manual . Encino, CA: Multivariate SoftwareBentler, P.M. , Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588– 606Bollen, K.A. (1986). Sample size and Bentler and Bonetts nonnormed fit index. Psychometrika, 49, 155– 173Bollen, K.A. (1989). Structural equations with latent variables . Wiley: New YorkFan, X. , Thompson, B. , Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56– 83Derogatis, L.R. , Cleary, P.A. (1977). Confirmation of the dimensional structure of the SCL-90-R: A study in construct validation. Journal of Clinical Psychology, 33, 981– 989Hardt, J. , Gerbershagen, H.U. (2001). Cross-validation of the SCL-27: A short psychometric screening instrument for chronic pain patients. European Journal of Pain, 5, 187– 197Hardt, J. , Petrak, F. , Egle, U.T. , Brähler, E. (2004). Die Symptom-Checkliste-27 in Deutschland: Unterschiede in zwei Repräsentativbefragungen der Jahre 1996 und 2003 . [The Symptom Checklist-27 in Germany: Differences between two representative surveys]. Manuscript submitted for publicationHu, L. , Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1– 55Jöreskog, K.G. (2004). On chi-squares for the independence model and fit measures in LISREL . Retrieved from www.ssicentral.com/lisrel/corner.htmJöreskog, K.G. , Sörbom, D. (1996). LISREL 8: User's reference guide . Lincolnwood, IL: Scientific Software InternationalJöreskog, K.G. , Sörbom, D. , duToit, S. , duToit, M. (2000). LISREL 8: New statistical features . Lincolnwood, IL: Scientific Software InternationalLee, S.Y. , Jennrich, R.I. (1979). A study of algorithms for covariance structure analysis with specific comparisons using factor analysis. Psychometrika, 44, 99– 113Marsh, H.W. , Balla, J.R. , Hau, K.-T. (1996). An evaluation of incremental fit indices: A clarification of mathematical and empirical properties. In G.A. Marcoulides & R.E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 315-353). Mahwah, NJ: Lawrence ErlbaumMarsh, H.W. , Balla, J.R. , McDonald, R.P. (1988). Goodness-of-fit indices in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 102, 391– 410McDonald, R.P. , Marsh, H.W. (1990). Choosing a multivariate model: Noncentrality and goodness-of-fit. Psychological Bulletin, 107, 247– 255Muthén, L.K. , Muthén, B.O. (2004). Mplus user's guide (third edition). Los Angeles, CA: Muthén & MuthénRigdon, E. (2003, March 9). Fit indices . Message posted to SEMNET, archived at bama.ua.edu/cgi-bin/wa?A2=ind0303&L=semnet&T=0&F=&S=&P=8783(2002, June). New in LISREL 8.52 . Retrieved from www.ssicentral.com/other/lispatch.htm
Steiger, J.H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173– 180Tucker, L.R. , Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1– 10Widaman, K.F. , Thompson, J.S. (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8, 16– 37
