Comparison of the EQ-5D-5L and the EQ-5D-3L using individual patient data from the REFORM trial

REFORM study design and participants

The REFORM trial was a pragmatic multicentre cohort RCT in England and the Republic of Ireland. The design involved the recruitment of an observational cohort from which eligible, consenting participants (≥65 years old) were randomised into the trial to receive either a podiatry intervention (n = 493), including foot and ankle strengthening exercises, foot orthoses, new footwear if required, and a falls prevention leaflet, or usual podiatry treatment plus a falls prevention leaflet (n = 517). Recruitment took place through 37 NHS podiatry clinics in primary or secondary care between October 2012 and August 2014. Participants were ineligible if they were <65 years of age; reported neuropathy, dementia, or another neurological condition; were unable to walk household distances; had a lower limb amputation; or were unwilling to attend their podiatry clinic for a REFORM appointment. Participants in the cohort were eligible for inclusion in the trial if they had had a fall in the past 12 months, or a fall in the past 24 months requiring hospital attention or reported worrying about falling. The same eligibility criteria applied to this substudy. The primary outcome was the incidence rate of self-reported falls per participant in the 12 months following randomisation. Secondary outcomes included: proportion of fallers and those reporting multiple falls, time to first fall, fear of falling, Frenchay Activities Index, Geriatric Depression Scale, foot pain, health related quality of life, and cost-effectiveness. The study (and this sub study) was approved by the East of England National Research Ethics Committee. The REFORM protocol,⁸ and trial results in terms of effectiveness and cost-effectiveness results have been published elsewhere.^9,10

The participants were asked to complete a consent form to indicate they wished to take part in the study; they were also informed about the opportunity to take part in this EQ-5D sub-study. The EQ-5D data used for this analysis were collected from a sample of participants in the REFORM trial (n = 151); hence the dataset was a pre intervention dataset. Each participant was asked to complete a baseline questionnaire which included both the 3L and 5L versions of the UK English language EQ-5D. Participants completed in order: (1) EQ-5D-5L questions and associated visual analogue scale (EQ-VAS); and then (2) EQ-5D-3L and associated EQ-VAS. The 5L version was administered first to avoid participants not using the ‘in-between’ levels 2 and 4 of the EQ-5D-5L.¹¹

Instruments

EuroQol

The EQ-5D comprises of two parts: classification of five dimensions of health (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression) and a visual analogue scale (VAS), which records participant’s overall evaluation of their health on a scale from 100 (best imaginable health) to 0 (worst imaginable health). The 3L classification system considers three severity levels (1 = no problems, 2 = some/moderate problems, 3 = severe problems) which defines 243 health states ranging from 11111 (full health) to 33333 (worst health). In the new EQ-5D-5L version the level of each of the five domains has been increased from 3 levels to 5 levels (1 = no problems, 2 = slight problems, 3 = moderate problems, 4 = severe problems and 5 = unable/extreme problems) and also standardised wording across dimensions. In addition, in the mobility dimension the response option “confined to bed” has been replaced with “unable to walk” as the most severe level. Therefore, the number of possible health states has increased for the 5L to 3,125 ranging from 11111 (full health) to 55555 (worst health).

Other self-reported health outcomes

The self-reported presence of ten long-term health conditions (Diabetes, Arthritis, Osteoporosis, ALS/Lou Gehrig’s disease, Multiple Sclerosis, Parkinson’s disease, Huntingdon’s disease, Alzheimer’s disease, Depression and Dementia) was collected at baseline for REFORM participants. Besides, participants also completed other related health instruments, including the Frenchay Activities Index (FAI)¹² and the Geriatric Depression Scale (GDS).¹³ The FAI is a 15-item instrument to assess a broad range of activities of daily living by recording the frequency with which each item or activity was undertaken over the previous three or six months. The scale provides a summed total score from 15 to 60 (higher scores represent higher activity levels). The GDS is a 15-item scale to assess geriatric depression, with each item representing a negative state of mind, with which respondents either agree or disagree. The GDS score ranges from 0 to 15 (higher scores represent greater levels of depression), with a scores greater than five considered the threshold for mild depression and scores greater than 10 considered the threshold for severe depression.

Data analysis

Feasibility

The feasibility was explored in terms of missing responses. For each EQ-5D version, we assessed the proportion of participants with (i) completely missing responses (all five dimensions missing); (ii) partially missing responses (at least one domain completed); and (iii) missing responses by domain. We also computed the proportion of participants not completing the VAS.

Redistribution analysis

The distribution of the 3L and 5L responses was shown in terms of percentage of each level reported. The redistribution patterns of the responses from the 3L to 5L for each dimension were also reported in terms of percentage. An inconsistent 3L response by the same REFORM participant was defined as being at least two levels away from the 5L response given. The mapping between 3L and 5L responses was recorded, and the mean size of inconsistencies calculated following the redistribution diagram and inconsistency weights proposed by Janssen et al¹⁴ (Table 1). We analysed the redistribution properties of the consistent response pairs as the proportion of the 3L-5L response pairs within each 3L response level (e.g. 1, 2 and 3) and the corresponding mean and median VAS values.

Ceiling effect

Assuming the elderly REFORM participants had at least slight or some problems in one or more dimensions of health, we compared ceiling effects between both systems by estimating the proportion of participants responding with no problems in each dimension and also the proportion of participants reporting problems in every domain (i.e. 11111). We would expect the 5L to have a lower ceiling effect compared with the 3L. We estimated absolute ceiling effect (i.e. difference between the proportions) and the relative change [(ceiling3L – ceiling5L/ceiling3L) ×100].

Discriminatory power

A key feature of any health status measure is the ability to discriminate among people at a single point in time.¹⁵ Discriminatory power was assessed using the Shannon indices. Unlike floor and ceiling effects, the Shannon indices incorporate the frequency distribution across all categories of a health status classification system.¹⁵

The Shannon index is defined as:

where $H^{\prime }$ represents the absolute amount of informativity captured, $C$ is the number of possible categories (levels or permutations in this study), and $p_i$ = $n_i/N$, the proportion of observations in the ith category (i = 1, … , $C$), where $n_i$ is the observed number of scores (responses) in category i and $N$ is the total sample size. The higher the value of $H^{\prime }$, the more information that is being captured by the system.¹⁵

In the case of a homogeneous distribution, where responses are evenly distributed among categories, an optimal amount of information is captured and $H^{\prime }$ reaches its maximum ($H{\prime }_{\mathit{max}}$). This equals ${log}_{2}C$. If the number of categories ($C$) is increased, $H{\prime }_{\mathit{max}}$will increase, however $H^{\prime }$ will only increase if the additional categories are utilised.¹⁵

Shannon’s Evenness Index ($J^{\prime }$) is defined as:

Shannon’s index $H^{\prime }$ can be considered to represent the absolute informativity of a system whereas Shannon’s Evenness index $J^{\prime }$ expresses the relative informativity, regardless of the number of categories.¹⁵

Discriminatory power for the EQ-5D-3L and EQ-5D-5L was assessed by dimension and by instrument as a whole. To calculate Shannon’s indices by dimension, levels are treated as categories, so $C$ here would be the number of levels per instrument and $p_i$ is the proportion of responses of the $i$th level. However, to calculate Shannon’s indices by instrument, permutations are treated as unique categories, where $C$ is the number of permutations ($P_{\mathit{max}})$, $p_i$ is the proportion of the ith permutation and $H{\prime }_{\mathit{max}}$ now equals ${log}_2{P}_{\mathit{max}}$.¹⁵ For the EQ-5D-3L there are 243 possible permutations, or unique health states, and 3125 for the EQ-5D-5L.

As the number of participants in this study is N = 151, the maximum number of unique health states (243, 3125) could never have been reached a priori, and consequently maximum informativity ($H{\prime }_{\mathit{max}}$) and maximum relative informativity ($J^{\prime }$) would never have been reached. Using methods similar to Polinder et al. (2010) the observed number of unique health states in the population are used, not the theoretical possible number of health states.¹⁶

The data shows 25 and 67 unique health states for the EQ-5D-3L and EQ-5D-5L respectively. This gives:

Association between multimorbidity and EQ 5D index score

Utility values for the 3L were derived using the UK EQ-5D-3L value set.⁴ Utility values for the 5L were derived using both the crosswalk by van Hout et al. (2012)⁶ and the English EQ-5D-5L value set.⁵ We compared the value sets scores overall and across the different health conditions.

Characteristics of REFORM sub-sample and EQ-5D profile

The sub-sample comprised a total of 151 participants. Characteristics of respondents were comparable in both samples. The mean age of participants for the sub-sample was 78.3 (SD 6.25) years and 56.2% were women, compared to 78.1 years (SD 7.2) and 61.0% female for the main trial. For the sub-sample, 37% had diabetes, 7% were affected by depression, 17% had osteoporosis and 58% suffered from arthritis; compared to 33% diabetes, 9.6% depression, 15% osteoporosis and 58.6% arthritis for the main trial. Table 2 displays the dimension level responses of REFORM participants to both descriptive systems. The proportion of REFORM participants reporting slight problems was higher for each dimension of the 3L compared to the 5L. For instance, the proportion of participants reporting slight problems with mobility was 61% for 3L compared to 30% for 5L; and similarly, for pain/discomfort (74% for 3L vs 44% for 5L).

Feasibility

The proportion of completed questionnaires (i.e., health dimensions plus VAS score) was higher for the 5L (96.7%) than for the 3L (92.7%). Less than 1% of 5L questionnaires were returned blank compared to 1.3% for the 3L. Unsurprisingly the missing VAS score was higher for the 3L than 5L (4.0% vs 2.0%), as the VAS score appears two times in the questionnaire and participants might have seen this as duplication. The proportion of partially completed questionnaires (i.e., one to four missing domains) was higher for the 3L (3.31%) compared to the 5L (2.64%). Overall missing dimensions were on average 2.2 (1.45%) for the 3L and 2.4 (1.59%) for the 5L. The range of missing values per dimension was similar for both descriptive systems, ranging from 1.32% to 2.65% in mobility for the 5L; and from 1.32% to 1.99% in self-care for the 3L.

Redistribution analysis

Table 3 shows cross tabulations to the 3L and 5L indicating inconsistent responses in italics. On average, the proportion of inconsistent responses was 3.25%. The ‘usual activities’ domain had the highest frequency of inconsistent responses (6.76%) while self-care showed no inconsistency at all.

Redistribution from 3L to 5L is displayed in Table 4. The “some” responses on the 3L are reassigned between levels 2 to 4 (slight, moderate, severe) on the 5L, while the “severe” responses on the 3L are spread between level 4 (severe) and 5 (extreme) on the 5L. For the level 1 in 3L, there was always a higher proportion of 1→1 than 1→2. The most skewed relative frequency distribution was in self-care (94.1/5.9) and the least in pain/discomfort (51.9/48.1). For the level 2 in 3L, the most evenly spread proportion was in mobility (2→2: 36.8/ 2→3: 42.5/2→4: 20.7). Between 54.8% (pain/discomfort) and 69.4% (anxiety/depression) of the participants reporting level 2 (moderate problems) with the 3L answered 2 (slight problems) or 4 (severe problems) with the 5L. Only a few participants reported severe problems in the 3L, among these, participants chose the fourth level (severe) of the 5L in usual activities, the fifth level (extreme) in 5L for anxiety and depression; and were evenly spread between levels 4 and 5 in 5L in pain/discomfort. Generally, the median and the mean VAS scores decreased as participants move from a better to a worse level of health, indicating valid results for the majority of pair combinations of consistent responses. We observed a level of discrepancy in particular for anxiety/depression, where decreasing median VAS values were slightly higher than expected.

Ceiling effect

Self-care and anxiety/depression were the domains showing the highest percentage of “no problems” in both the 3L and the 5L; while Pain/Discomfort was the domain showing the lowest percentage of “no problems” (Table 5). Moving from the 3L to the 5L showed a decrease of “no problem” respondents for all domains; where Pain/Discomfort showed the highest relative reduction in ceiling effect. 17 participants (11.26%) reported full health with the 3L (11111), and when moving to the 5L these participants reported: 41.2 % perfect health; 35.3% scored ‘slight problems’ in one dimension; 11.8% scored ‘slight problems’ in two dimensions; 5.9% scored ‘moderate problems’ in one dimension; and 5.8% ‘slight’ in two dimensions and ‘moderate’ one dimension. Conversely of 9 participants (5.96%) that reported full health on the 5L only 2 participants reported some problems in one dimension of the 3L (pain/discomfort). The percentage of REFORM participants that reported full health decreased from 11.2% with the 3L to 6% with the 5L, indicating an absolute ceiling reduction of 5.2 and a relative ceiling reduction of 47.1% on the full profile.

Discriminatory power

The Shannon’s indices were calculated both by dimension and for each instrument as a whole. The results, by dimension, are shown in Table 6. It can be shown that the absolute informativity ($H^{\prime }$) increased for each dimension when moving from the 3L to the 5L. The relative informativity (J’) increased for all dimensions, except ‘anxiety/depression’.

The Shannon Index and Evenness Index were also calculated for each instrument as a whole. When calculating the Evenness Index (J′) $H_{\mathit{max}}$, varied based on the number of unique health states found in our sample of participants. The results are shown in Table 7. Again, the absolute informativity ($H^{\prime }$) increased when moving from the 3L to the 5L. This is to be expected due to the increase in levels. However, the relative informativity ($J^{\prime }$) also increased, indicating that the extra levels were being used and providing information.

Association between multimorbidity and EQ 5D index score

At the overall level, the EQ-5D-5L scores were significantly higher than both EQ-5D-3L and crosswalk (Table 8). The mean difference between the EQ-5D-5L and EQ-5D-3L values was 0.091 (range -0.345 to 0.505); whilst the mean difference between the EQ-5D-5L and the crosswalk values was 0.082 (range -0.035 to 0.293). As expected, the percentage of states worse than dead was lower for the EQ-5D-5L. Overall long term health problems are associated with a reduction in utility values for all used value sets except among patients with diabetes; even though the decrements associated with specific health problems were different for both value sets.

Underlying data

Full underlying (non-aggregated) data cannot be made publicly available since the ethics approval of this study does not cover openly publishing non-aggregated data.

In order to access this data, it must be requested from the corresponding author. Data requestors will have to provide: i) written description and legally binding confirmation that their data use is within the scope of the study; ii) detailed written description and legally binding confirmation of their actions to be taken to protect the data (e.g. with regard to transfer, storage, back-up, destruction, misuse, and use by other parties), as legally required and to current national and international standards (data protection concept); and iii) legally binding and written confirmation and description that their use of this data is in line with all applicable national and international laws (e.g. the General Data Protection Regulation of the EU).