Parametric analyses of summative scores may lead to conflicting inferences when comparing groups: A simulation study.
DOI:
https://doi.org/10.2340/16501977-1941Keywords:
rating scales, parametric statistics, Rasch analysis, simulation, summative scores, Rasch-based measures.Abstract
OBJECTIVE: To investigate whether using a parametric statistic in comparing groups leads to different conclusions when using summative scores from rating scales compared with using their corresponding Rasch-based measures. METHODS: A Monte Carlo simulation study was designed to examine between-group differences in the change scores derived from summative scores from rating scales, and those derived from their corresponding Rasch-based measures, using 1-way analysis of variance. The degree of inconsistency between the 2 scoring approaches (i.e. summative and Rasch-based) was examined, using varying sample sizes, scale difficulties and person ability conditions. RESULTS: This simulation study revealed scaling artefacts that could arise from using summative scores rather than Rasch-based measures for determining the changes between groups. The group differences in the change scores were statistically significant for summative scores under all test conditions and sample size scenarios. However, none of the group differences in the change scores were significant when using the corresponding Rasch-based measures. CONCLUSION: This study raises questions about the validity of the inference on group differences of summative score changes in parametric analyses. Moreover, it provides a rationale for the use of Rasch-based measures, which can allow valid parametric analyses of rating scale data.Downloads
Downloads
Published
How to Cite
Issue
Section
License
All digitalized JRM contents is available freely online. The Foundation for Rehabilitation Medicine owns the copyright for all material published until volume 40 (2008), as from volume 41 (2009) authors retain copyright to their work and as from volume 49 (2017) the journal has been published Open Access, under CC-BY-NC licences (unless otherwise specified). The CC-BY-NC licenses allow third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for non-commercial purposes, provided proper attribution to the original work.
From 2024, articles are published under the CC-BY licence. This license permits sharing, adapting, and using the material for any purpose, including commercial use, with the condition of providing full attribution to the original publication.