Using the WHODAS 2.0 to Assess Functioning Among Veterans Seeking Compensation for Posttraumatic Stress Disorder

One of the major changes in DSM-5 (1) was removal of the Global Assessment of Functioning (GAF; 2). Previously, the GAF was used to inform treatment needs and determine level of overall functioning. The decision to remove the GAF from DSM-5 was based on the determination that it does not adequately assess functional domains and demonstrates poor reliability (3).

Although DSM-5 does not endorse any replacement for the GAF, it recommends further study of the utility of the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0; 4) for assessment of functioning. As of now, there are no established conventions for interpreting WHODAS 2.0 scores in relation to the clinical significance criterion included in many DSM-5 diagnoses, and WHODAS 2.0 cutoff scores for determining disability have not been determined (5). Therefore, research is needed to evaluate how the WHODAS 2.0 performs in applied mental health care settings and to compare its relationship with other measures of impairment.

Compensation and pension (C&P) disability examinations for posttraumatic stress disorder (PTSD) in the U.S. Department of Veterans Affairs (VA) offer a useful opportunity to evaluate the utility of the WHODAS 2.0. Concerns exist about how well these exams assess functional impairment, given an increasing number of military veterans who receive disability compensation for service-connected PTSD (6) and the costs associated with service-connected disability (7–12). Historically, because of its inclusion in earlier editions of DSM, the GAF was used by VA clinicians to determine disability levels for functional impairment related to service-connected mental disorders and to set accompanying disability compensation levels. Because the GAF was removed from DSM-5, research is needed to establish the utility of other measures of functioning, such as the WHODAS 2.0, in VA PTSD disability examinations. More specifically, research is needed to understand the potential of WHODAS 2.0 in guiding the process of determining the appropriate level of service connection for PTSD.

Accordingly, we examined the performance of the WHODAS 2.0 in the context of C&P examinations for PTSD to determine if it is a suitable replacement for the GAF. We also compared the performance of the WHODAS 2.0 with that of the Inventory of Psychosocial Functioning (IPF; 13), a questionnaire specifically created to assess PTSD-related functional impairment. This comparison allowed us to test whether a measure intended to assess impairment across a variety of mental and general medical disorders (WHODAS 2.0) performs as well as a measure intended to assess impairment specifically associated with PTSD (IPF).

We first examined the relative strength of all three measures as indices of latent functioning by using confirmatory factor analysis (CFA). We hypothesized that the WHODAS 2.0 and the IPF would be stronger indices of functioning than the GAF. Then we examined associations between the WHODAS 2.0, IPF, and GAF and overall PTSD symptom severity, PTSD-related functional impairment determined by clinical interview, and actual C&P disability ratings for PTSD. On the basis of prior research (13,14), we hypothesized that all functioning measures would be strongly related to these variables.

Methods

Participants

To test the hypotheses, we used data from a multisite, cluster-randomized study that compared the outcomes of C&P PTSD disability (service connection) examinations that incorporated the Clinician Administered PTSD Scale (CAPS) and WHODAS 2.0 with outcomes of C&P PTSD exams that did not include these measures between March 17, 2009, and September 29, 2010 (10). Study participants were veterans who were ages 18 or older, were not cognitively impaired, filed a PTSD disability claim and were referred for an initial PTSD disability assessment, and provided written informed consent. Participating C&P examiners also provided written informed consent. For the purposes of this investigation, we analyzed data only from the veterans who were assessed with the CAPS and the WHODAS 2.0 (N=177). Institutional review board approval was obtained from each participating medical center.

Procedures

Prior to data collection, C&P examiners received extensive training on both the CAPS and the WHODAS 2.0, and their first two examinations were reviewed to ensure adequate administration. Veterans were assigned by schedulers to an examiner on the basis of the next available appointment. All study participants completed the IPF.

After obtaining the veteran’s consent, a research assistant accompanied the veteran to a room where a C&P examiner conducted and digitally recorded the examination. Upon concluding the interview, the IPF was administered, and the veteran received $10. Actual service connection disability determinations (yes or no) and service connection ratings (PTSD disability ratings) were obtained for all study participants following their C&P exam.

Measures

The CAPS (15), a structured clinical interview, assesses PTSD diagnostic status, symptom severity, and the symptoms’ impact on social and occupational functioning. Total symptom severity scores (frequency plus intensity) range from 0 to 136. Participants were diagnosed as having PTSD if they met DSM-IV-TR symptom cluster criteria (to be counted, symptoms required ratings of 1 for minimum frequency and 2 for intensity) and had a total score of ≥40.

The WHODAS 2.0 (4) assesses disability in the past 30 days related to general medical and psychiatric disorders across six domains (understanding and communicating, mobility, self-care, getting along with others, life activities, and participation in society) and provides an overall disability score. Higher scores indicate greater disability. We used the 36-item interview version.

The IPF (13) is an 80-item self-report scale that assesses impairment in the past 30 days across seven domains (romantic relationships, family relationships, work, friendships and socializing, parenting, academic pursuits, and self-care) and yields an overall impairment score. Higher scores indicate greater functional impairment. The IPF displays excellent psychometric properties.

The GAF (2) is a clinician-rated global index that ranges from 0 to 100, with higher scores indicating better functioning. In accordance with VA mental health disability examination policies, study examiners rated each veteran they examined by using the GAF. No specific GAF training was included as part of the study; the GAF was administered as it typically would be by examiners.

Data Analyses

We first examined the pattern of intercorrelations among the WHODAS 2.0, IPF, and GAF. The magnitude of the correlations were compared by using t tests to evaluate differences between pairs of dependent correlations (16). We then conducted a CFA with three indicators (GAF score and total scores on the WHODAS 2.0 and IPF; the IPF served as the marker indicator) to examine the relative associations between the three measures of functioning and the latent trait of psychosocial functioning. This approach is well-suited for construct development (17) and provides a useful method for evaluating multiple indices of functioning when it is not evident which instrument is the best proxy for the underlying construct, as was the case in this study. The CFA models the underlying construct and determines the extent to which this latent trait influences observed scores on each measure (17). Each indicator (in this case, total score on each measure) captures a slightly different aspect of the functioning construct and does so with some degree of error; overlap or common variance across the measures can be conceptualized as a reflection of the underlying (in other words, latent) trait and is separable from the measure-specific variance that reflects measurement error. The factor loading for each indicator reflects the extent to which the latent construct influences the observed scores on the indicator (17); indicators with stronger loadings are better reflections of the “true” latent variable.

We tested whether the magnitude of the factor loading for the WHODAS 2.0 was superior compared with the GAF by constraining these loadings to be equal and then evaluating whether doing so decreased model fit (see below for details). We followed this same approach to compare the magnitude of the WHODAS 2.0 and IPF loadings as well as the IPF and GAF loadings. To test the external validity of our hypothesized latent functioning variable, we examined the pattern of correlations between factor scores on the latent functioning variable and two items on the CAPS that indexed PTSD-related functional impairment in the relationship and work domains, as rated by the clinician during administration of the CAPS.

Next we evaluated the clinical utility of the WHODAS 2.0 compared with the IPF and GAF by generating receiver operating characteristic (ROC) curves, comparing the area under the curve (AUC) for each measure, and determining the sensitivity and specificity of each measure in identifying clinically significant functional impairment, as determined by a rating of 2 (“moderate”) or higher on either of the two CAPS impairment items. Finally, we compared the convergent validity of the WHODAS 2.0, IPF, and GAF by evaluating the pattern of correlations between these measures and PTSD severity and actual service connection disability ratings. As before, we compared the magnitude of pairs of correlations by using t tests for dependent correlation coefficients.

The CFA was conducted in Mplus 7.11 (18) by using the robust maximum likelihood estimator. The specified model was just-identified, meaning that there were zero degrees of freedom and model fit was then perfect. Thus we evaluated the appropriateness of this model on the basis of the strength and statistical significance of the factor loadings. We tested the equivalence of the factor loadings in the CFA by constraining loadings to be equal and testing if this was associated with a significant decrease in model fit by using a Wald chi square test via the “model test” command. In the CFA, participants with missing data were included under direct maximum likelihood; a maximum of 4% of data were missing. All other analyses were conducted in SPSS, with pairwise deletion for missing data; for these analyses, a maximum of 6% of data were missing.

Results

Characteristics of the participants are provided in Table 1. Descriptive statistics for all measures and correlations with the three functioning measures are listed in Table 2. The WHODAS 2.0 and IPF were more strongly related to each other (r=.62, p<.001) than to the GAF (r=–.45 and –.52, for the WHODAS 2.0 and IPF, respectively, p<.001). The correlation between the WHODAS 2.0 and the IPF was significantly greater in magnitude compared with the correlation between the WHODAS 2.0 and the GAF (t_dep-r=10.29, df=167, p<.001) and the IPF and the GAF (t_dep-r=11.74, df=167, p<.001).

We next conducted a CFA to compare the relative strengths of the WHODAS 2.0, IPF, and GAF as indicators of functioning (Table 3). The latent functioning construct explained 71% of the variance in IPF scores, 55% of the variance in WHODAS 2.0 scores, and only 37% of the variance in GAF scores. The greater magnitude of the WHODAS 2.0 and IPF loadings was confirmed through additional model testing. In those tests, constraining loadings for the WHODAS 2.0 and the GAF and, separately, for the IPF and GAF to equality resulted in a degraded fit (χ²=121.73, df=1, N=177, p<.001, and χ²=155.66, df=1, N=177, p<.001, respectively). However, constraining the loadings for the WHODAS 2.0 and IPF to equality did not yield a significant result, indicating that the two loadings were equivalent in magnitude. Thus as indicators of functioning, the WHODAS 2.0 and IPF were superior compared with the GAF but did not differ from one another.

To test that the latent variable reflected psychosocial functioning, we evaluated the correlations between factor scores on this variable and clinician-rated CAPS scores on impairment in relationship and work domains. The correlation between factor scores and CAPS severity score was strong for both relationship impairment (r=.64) and work impairment (r=.59) (p<.001 for both). The factor scores also were correlated with overall PTSD severity (r=.66) and PTSD disability ratings (r=.45) (p<.001 for both).

Next we evaluated the sensitivity and specificity of each measure in relationship to functional impairment in the work or relationship domain (score of ≥2 on the relevant CAPS item) (Table 4). All three measures achieved similar AUC values (GAF=.81, WHODAS 2.0=.81, and IPF=.83). [Figures illustrating AUCs for each measure are available as an online supplement to this article.] A cut score of 40 on the WHODAS 2.0 maximized both sensitivity and specificity. Veterans with total scores of ≥40 on the WHODAS 2.0 had higher service-connected disability ratings (mean±SD=47.25±19.42) compared with veterans with lower scores (mean=35.45±15.23) (t=3.11, df=111, p=.002).

Finally, we examined the pattern of correlations between the three functioning measures and overall PTSD symptom severity and PTSD service connection disability ratings (Table 2). The GAF had a significantly stronger association with symptom severity scores compared with the WHODAS 2.0 or the IPF (r=–.72, .53, and .54, respectively, p<.001 for all). The magnitude of the differences in correlation between severity scores and the GAF versus other measures was significant (GAF versus WHODAS 2.0, t_dep-r=14.49, df=164, p<.001; GAF versus IPF, t_dep-r=14.02, df=164, p<.001). Among persons who received a PTSD service connection, disability rating was more strongly related to the GAF (r=–.69, p<.001) compared with the WHODAS 2.0 (r=.29, p=.002) or the IPF (r=.31, p=.001). The magnitude of the differences in correlation between disability ratings and the GAF versus other measures was significant (GAF versus WHODAS 2.0, t_dep-r=10.28, df=109, p<.001; GAF versus IPF, t_dep-r=10.27, df=109, p<.001).

Discussion

The WHODAS 2.0 was a stronger indicator of functioning compared with the GAF. However, the WHODAS 2.0 and the GAF performed similarly in terms of clinical utility, as determined by ROC curves, and ability to identify veterans with clinician-rated impairment. The WHODAS 2.0 was less strongly related to clinician-rated PTSD severity and to PTSD-related service connection ratings compared with the GAF. Although it may seem problematic that the WHODAS 2.0 did not fully outperform the GAF, achieving a better performance than the GAF was difficult because actual PTSD service connection decisions were based, in part, on GAF ratings, per standard protocol. Thus the fact that the WHODAS 2.0 performed as well as the GAF in some arenas (for example, sensitivity and specificity related to identifying persons with clinician-rated impairment) and performed better than the GAF in other areas (for example, as an indicator of latent functioning) reflects a significant psychometric strength. Results underscore prior concerns about the heavy reliance on the GAF by disability evaluators to determine PTSD-related functional impairment, given that this index was the weakest indicator of this construct, relative to the WHODAS 2.0 and IPF.

Our findings suggest that the WHODAS 2.0 may be a better measure of functioning than the GAF among veterans applying for PTSD service connection. An important asset of the WHODAS 2.0 is its relationship with the International Classification of Functioning, Disability, and Health (ICF; 19), an internationally recognized system of classifying the consequences of general medical and mental conditions. The results point to an additional strength of the WHODAS 2.0: there was more variation in the scores on this measure (the same could also be said of the IPF) compared with the GAF, which showed a restricted range, even though all three measures have a possible range of 0 to 100. One possible interpretation of this finding is that the GAF may not be as sensitive as the other two measures to variation in psychosocial functioning. Either purposely or not, GAF users may avoid using either end of the scale. Doing so would artificially restrict the range of scores and weaken performance. In contrast, because the WHODAS 2.0 and IPF have multiple items, they may be more likely to exhibit a greater range of scores.

Although the WHODAS 2.0 may be an improvement over the GAF, some questions remain about its use to assess PTSD-related functional impairment in C&P examinations for PTSD. For example, because the WHODAS 2.0 reflects the content of the ICF, it is designed to assess impairment related to both general medical and mental disorders. As a consequence, many items on the WHODAS 2.0 are not particularly relevant for assessing problems related to PTSD or other mental disorders. In situations such as determining PTSD service connection specifically attributable to a mental condition, it would be inappropriate to base compensation determinations on a measure that is influenced by disability in general medical functioning.

This study constitutes the only existing effort to examine the use of the WHODAS 2.0 in C&P examinations for PTSD. Additional research is needed to replicate our finding that a WHODAS 2.0 score of 40 should serve as the threshold for indicating clinically significant functional impairment for veterans who are applying for PTSD-related disability compensation. The ideal cut score often varies as a function of context and goal. In the context of a VA PTSD examination, it would seem that maximizing both sensitivity and specificity is important in order to compensate veterans with functional impairment and avoid compensating veterans without such impairments.

Comparison of the IPF and the WHODAS 2.0 indicated that the two measures performed equally well as indicators of a latent construct of psychosocial functioning, yielded similar precision in detecting individuals with functional impairment, and showed equivalent patterns of association with total PTSD severity and PTSD service connection ratings. This finding is notable given that the self-reported IPF does not share method variance with the clinician-rated GAF, WHODAS 2.0, PTSD severity scores, or PTSD rating decisions, which would be expected to put the IPF at a disadvantage.

Study limitations included the possibility that veterans who participated may have different characteristics compared with nonparticipants, and our results may not generalize outside the context of PTSD C&P exams. Another limitation was that we studied only three indicators of psychosocial functioning, which did not allow for evaluation of overall model fit in the factor analysis. More generally, additional measures would help to compare the IPF and WHODAS with the broader content domain of psychosocial functioning. Finally, our study’s sample size was relatively small. Additional research with a larger sample is needed to confirm the findings.

A notable study strength was conducting the study within the context of real-time VA PTSD C&P examinations, increasing the generalizability of our results. Also, we used a latent variable to represent core psychosocial functioning across the three measures, thereby removing measurement error.

Conclusions

The WHODAS 2.0 and IPF may be acceptable replacements for the GAF for use in PTSD C&P disability evaluations and determinations. These and other measures should be incorporated into evidence-based clinical practice and disability assessments.