Using willingness-to-pay to establish patient preferences for cancer testing in primary care

Abstract

Background

Shared decision making is a stated aim of several healthcare systems. In the area of cancer, patients’ views have informed policy on screening and treatment but there is little information about their views on diagnostic testing in relation to symptom severity.

Methods

We used the technique of willingness-to-pay to determine public preferences around diagnostic testing for colorectal, lung, and pancreatic cancer in primary care in the UK. Participants were approached in general practice waiting rooms and asked to complete a two-stage electronic survey that described symptoms of cancer, the likelihood that the symptoms indicate cancer, and information about the appropriate diagnostic test. Part 1 asked for a binary response (yes/no) as to whether they would choose to have a test if it were offered. Part 2 elicited willingness-to-pay values of the tests using a payment scale followed by a bidding exercise, with the aim that these values would provide a strength of preference not detectable using the binary approach.

Results

A large majority of participants chose to be tested for all cancers, with only colonoscopy (colorectal cancer) demonstrating a risk gradient. In the willingness-to-pay exercise participants placed a lower value on an X-ray (lung cancer) than the tests for colorectal or pancreatic cancer and X-ray was the only test where risk was clearly related to the willingness-to-pay value.

Conclusion

Willingness-to-pay values did not enhance the binary responses in the way intended; participants appeared to be motivated differently when responding to the two parts of the questionnaire. More work is needed to understand how participants perceive risk in this context and how they respond to questions about willingness-to-pay. Qualitative methods could provide useful insights.

Background

Shared decision making is a stated aim of several healthcare systems [1, 2]. Involving patients in critical decisions about their care is regarded as not only ethically correct but also as a way of improving quality and “avoiding unwanted and costly medical interventions” [3]. In the UK the philosophy of “no decision about me, without me” has been promoted by the Department of Health [4] and has been applied to many aspects of patient care. However, fully shared decisions can only be made if the asymmetry of information between clinicians and patients is more balanced. Recently, the current imbalance has begun to shift, encouraged by a greater will on the part of clinicians and an increase in readily available information accessible to patients, both from the National Health Service (NHS) and elsewhere.

One notable area where shared decision making has been adopted actively is cancer. Research on patients’ views has informed policy on screening [5] and treatment [6] in accordance with referral guidelines developed by the National Institute for Health and Care Excellence (NICE) [7]. Such information is important, as care plans that incorporate patients’ preferences are more likely to be successful in terms of acceptability and may lead to more efficient use of resources. However, one gap in the evidence is information about patients’ views on testing for cancer with respect to risk: how serious do symptoms have to be, in terms of indicating cancer, for patients to consider a particular test to be worthwhile? The lifetime prevalence of cancer in the UK is more than 30 % [8] and despite falling death rates the fear of cancer is known to be high among the general population [9]. Early diagnosis may improve survival [10] but many early symptoms of cancer, for example, cough, diarrhoea, and headache, far more often indicate a benign condition. General practitioners are faced with the challenge of deciding which patients with such symptoms to refer for diagnostic testing, relying largely on their expertise and limited national guidelines [7]. The risk of failing to investigate a potentially serious symptom has to be weighed against the need to avoid unnecessary anxiety, inconvenience, side-effects and cost from inappropriate investigation.

The study described here is part of a larger study reported more fully elsewhere [11], which used a survey of primary care attenders to investigate preferences for cancer investigation. Three contrasting cancers were chosen as exemplars - colorectal, lung and pancreas – because of their variation in symptoms, type and accessibility of test, treatment, and prognosis. Here, we describe a willingness-to-pay component of the survey, which was designed to enhance the results of the main survey: if the same number of participants opted to be investigated for a particular cancer irrespective of risk level, could the values offered in the willingness-to-pay exercise be used to refine these responses and identify a threshold risk level below which testing was not regarded as worthwhile?

Willingness-to-pay has been used extensively to obtain patient and public valuations for a variety of goods and services in many diverse settings [12]. Despite considerable methodological research into the use of different willingness-to-pay techniques [13] no consensus has emerged as to best practice and it is likely that different methods suit different situations and patient groups [14, 15]. The simplest form of value elicitation is to use an ‘open-ended’ approach whereby the respondent is asked to provide a valuation without any prompting or context; more sophisticated, is a ‘payment scale’ approach where a list of feasible values is offered and the respondent chooses from the list. A bidding approach, which is more refined, has generally come to be preferred to both of these [16, 17]. This method requires the respondent to accept or reject a starting bid (value), which is increased or decreased according to the response and the process continues until a final value is determined. Although this is often a preferred method there is evidence suggesting that responses in a bidding approach tend towards the point at which the bidding starts (starting point bias) [18–20]. This can lead to biased results.

The aim of this study was to develop and administer a willingness-to-pay questionnaire that could be used to elicit the relative values that patients place on diagnostic testing for lung, pancreatic and colorectal cancers. We aimed to identify a risk threshold for each cancer that would indicate when patients choose to be tested in preference to watchful waiting.

Methods

Study design

We developed a vignette-based survey with a willingness-to-pay component to determine the likelihood that patients would choose to be tested for colorectal, lung, and pancreatic cancer, using various levels of risk. The key question of what proportion of the population would choose to be tested at each risk level for each cancer was addressed using a simple 'yes/no' alternative. Those responding ‘yes’ to a test proceeded to a willingness-to-pay exercise with the aim of identifying a strength of preference around the binary choice. Cookson suggests that willingness-to-pay exercises that aim to elicit true, absolute values are unreliable because of “budget constraint bias”, where the value given is inflated because of the close focus placed on a particular service [21]; comparative willingness-to-pay may be a way of avoiding this. This study adopted the latter technique, with the aim of identifying relative values to differentiate between ‘yes’ responses by risk level. The survey was designed specifically for this study and was administered using an electronic touch screen tablet computer (an iPad). The iPad application software was custom built, which gave us considerable scope and flexibility in the design. We obtained ethics approval from the South West (Southmead) National Research Ethics Service committee (ref 11/SW/0055). Participants provided oral informed consent.

Survey design

The survey contained three components. The first section asked for information about participant characteristics, including age, sex, income, education, employment status, ethnicity, experience of cancer (self and family member or close friend), and convenience of the nearest main hospital. Screen shots showing details of the way these questions were asked are included in the Appendix (Fig. 3).

Secondly, we used vignettes to ascertain participants’ attitudes towards testing for cancer. We developed twelve separate vignettes, one for each combination of the three cancers (colorectal, lung, and pancreas) and four different risk levels (1 %, 2 %, 5 %, 10 %). The content of the vignettes was informed by current guidelines and clinical experts on the team. We also undertook qualitative interviews with patients referred for symptoms suspicious of cancer [22], and these were used to validate the vignettes’ depiction of the three diagnostic pathways as experienced by patients. Each vignette contained a description of symptoms, the chance that these might indicate cancer – presented as a percentage, ratio, and pictorially – information about the diagnostic test that would be used, likely treatment, and an indication of the prognosis. The information provided in the vignettes is summarised in Table 1. For each participant, one of the twelve vignettes was generated randomly, thus avoiding any ordering effect, and the respondent was asked to imagine they were in the situation described in the scenario. They were then asked whether they would choose to have the diagnostic test if it were offered.

Table 1 Summary of content of 12 vignettes: one for each combination of cancer and risk level

Following a ‘yes’ response to the question about testing, participants proceeded automatically to the third section of the survey – the willingness-to-pay exercise. The design of this part of the survey was informed by experts in the team and with reference to published costs of tests. The survey and the values included were tested on a sample of participants using the technique of verbal probing [23] to check that respondents interpreted the question correctly, and understood why it was being asked. Two rounds of verbal probing were carried out, the first on 13 participants and the second on five. Further pilot testing ensured that the exercise would not be too burdensome. Feedback from the verbal probing was used to refine the content: data from the first round indicated that respondents did not fully understand the concept of opportunity cost so the wording was changed to convey the idea of sacrifice. Subsequent testing in round 2 showed this change was successful. To mitigate starting point (or anchoring) bias we designed a two-part exercise in which respondents were first presented with a payment scale and the response to that question established the starting point of a bidding process. The starting point was generated randomly from a selection within the range of the scale chosen, and the participants could then bid up or down from the starting point. This mechanism is illustrated in Fig. 1, using an example where a participant selects the payment scale £101 to £300. The starting point for the bidding is randomly selected by the software from £125, £200, £250 and £300; in this example £200 is selected. The participant is then able to bid up as far as £300 or down to below £125, with five possible end points. In total, 18 end points were used, five for each of the lower three bands (£1 to £100, £101 to £300, and £301 to £700) and three for the “more than £700” band. Within each band, the difference between each end point and the one immediately higher increased as the value increased so that proportional differences were roughly similar [24].

Fig. 1

Schema illustrating the two-part willingness to pay exercise. Choice of payment scale leads to a bidding process with the starting point randomly generated from four values within the band

At the beginning of the task, when presenting the payment scales, we used reference goods to help participants think about the value of a diagnostic cancer test (see Fig. 2). We chose a selection of ‘lifestyle’ goods and services, seen as being ‘desirable’ though not essential, and which could conceivably be sacrificed to pay for healthcare. Wording was carefully chosen to encourage them to think in terms of sacrifice – that is, what they might be prepared to give up or go without in order to have a test [25].

Fig. 2

Screen shot of the payment scale exercise showing the use of reference goods

It is known that some individuals find it hard to place a value on healthcare, particularly in the context of a system of universal coverage as in the UK, and moreover some individuals feel it is unethical to expect them to provide a valuation. To accommodate these views the payment scale offered an option of “I would not pay anything for the test”. If this was selected they were then asked a further question about their reason for this view with the choices of “I cannot afford anything extra”, “I do not believe I should pay for healthcare”, and “It is too difficult to put a value on health”.

Throughout the development of the survey we piloted the wording and layout of all components with a patient and public involvement group using the technique of verbal probing [23]. Feedback from these sessions was used to refine and improve the content.

Survey administration

The survey was administered by researchers in general practice waiting areas. We targeted a susceptible population – that is, those at greater risk of cancer who would be likely to seek health care help and advice (GP attenders aged 40 and over). General practices in three geographical areas (Bristol & South Gloucestershire, Devon, and the East of England) were included and practices were purposively sampled to achieve an overall mix of urban and rural, and a range of socio-economic statuses. Data collection took place at 26 practices at different times of the day and week between December 2011 and August 2012.

Participants could complete up to three vignettes, one for each cancer. The first vignette was randomly generated from all 12 possibilities (three cancers, four risk levels), the second from the eight that related to the two remaining cancers, and the third from the four relating to the final cancer.

When data collection was complete we conducted a test-retest exercise in a different practice using a convenience sample of 48 volunteers who agreed to return two weeks later. The random generation of vignettes was removed from the survey for the second stage of this exercise to ensure the two tests were identical.

Analysis

Data were electronically downloaded directly from the iPads at the end of each session. Participant characteristics were explored descriptively and the age/sex profile was compared with that of England as a whole and general practice attenders. Responses to the choice of whether to be tested were analysed descriptively and using logistic regression to establish the extent to which risk played a part in participants’ decisions.

The data from the willingness-to-pay exercise were analysed to explore whether they could be used to inform the strength of preference about the simple 'yes/no' choice. We therefore investigated the extent to which willingness-to-pay values differed according to risk. We explored this descriptively using means and medians, and tested the relationship between the willingness-to-pay values and risk using regression analysis. A one-way analysis of variance, with three degrees of freedom, was conducted to investigate the difference in mean willingness-to-pay across all risk levels, for each cancer.

Analysis was carried out using Stata v13.1 statistical software and Microsoft Office Excel 2013.

Results

Participant characteristics

A total of 3,469 participants took part, completing 6,930 vignettes. The characteristics of the participants are shown in Table 2. The age/sex profile of responders is similar to that of the consulting population in England [26]. As expected, when compared with the general population of England the sample was under-represented in terms of younger men (17 % vs 27 % aged 40–59), and over-represented in respect of older women (15 % vs 11 % aged 60–69), and for all elderly people (29 % vs 24 % aged 70 and over) [27]. The respondents were largely white British and nearly half were retired; 15 % had previously been diagnosed with cancer and 75 % had a family member or close friend who had experienced cancer.

Table 2 Participant characteristics

Binary responses

Detailed results of the responses to the 'yes/no' question about whether to opt for a test have been reported elsewhere [11]. Table 3 shows the number of participants choosing to be investigated at each risk level for each cancer. A large majority (88 %) of participants chose to be referred for a test; this was slightly lower in the low risk (1 %) group and higher in the high risk (10 %) group, but the difference was very small (87 % vs 89 %). Colonoscopy (colorectal cancer) had a lower uptake than chest X-ray (lung cancer) and ultrasound/CT scan (pancreatic cancer) and displayed the greatest risk gradient. These observations were confirmed by the results of the logistic regression analysis, which controlled for patient characteristics.

Table 3 Number (%) choosing to be investigated by cancer and risk level

Willingness-to-pay

The results of the willingness-to-pay exercise are presented in Tables 4, 5 and 6. Table 4 gives the number and percentage of participants who selected each payment scale, by cancer and by risk level and Table 5 gives the values indicated in the bidding exercise. Results for each cancer separately are based on all responses, but because participants could respond to up to three vignettes, results for all cancers together use each participant’s first response so as to reduce differential selection bias. Responses covered the entire payment range offered for all cancers and all risk levels, though 68 % of participants bid up to the highest value within the range of the payment scale chosen.

Table 4 Number and percentage of respondents selecting each willingness-to-pay band by cancer (all responses) and by risk level (first response only)

Table 5 Mean (SD) and median (IQR) willingness-to-pay values, by cancer and risk level

Table 6 Logistic regression: factors influencing the value of willingness-to-pay. Best fit logistic regression models for each cancer separately and all cancers together

The results show that participants placed a lower value on an X-ray for lung cancer than the tests for colorectal or pancreatic cancer; the regression analysis in Table 6, which shows the best-fit logistic regression models, indicate that, controlling for other factors, there was a difference of about £51 (95 % CI: £14 to £87) in the mean willingness-to-pay between an X-ray and a colonoscopy and slightly less between an X-ray and the tests for pancreatic cancer. In general, testing was valued more highly when risk was high than when it was low, though the increase is not monotonic in the case of colorectal and pancreatic cancers (Tables 5 and 6). This may be related to the lack of a clear gradient for these two, as evidenced by testing for a difference in mean willingness-to-pay by risk level (p-values: colorectal 0.71; pancreas 0.91).

Around one fifth of respondents chose “I would not pay anything” when completing the payment scale exercise. Of these, one half said they did not believe they should pay for health care and one third said they could not afford to pay. Comparing the lowest risk level (1 %) with the highest (10 %), more individuals reported not being able to pay at the higher level (32 % vs 26 %) and more said they did not think they should pay at the lower level (59 % vs 52 %).

Factors influencing responses to individual cancers

Previous analysis of responses to the ‘yes/no’ question of whether participants chose to be tested indicated that age was an important factor in all three cancer models [11]. Other variables affecting this decision were travel time to nearest hospital (colorectal and lung), whether a family member or close friend had previously been diagnosed with cancer (colorectal and lung) and income (colorectal and pancreas). The willingness-to-pay exercise indicated a rather different set of variables influencing the values placed on the tests: age did not appear as a factor in any of the three models and neither did travel time (Table 6). Three variables did however contribute to the willingness-to-pay values in all three tests: those with a higher income, particularly those in the highest bracket, were prepared to pay more for testing, as were the more highly educated, and a previous diagnosis of cancer increased the valued placed on a test by between £63 (95 % CI: £13 to £114) (lung) and £73 (95 % CI: £22 to £124) (colorectal). Additionally, the values placed on an X-ray for lung cancer were affected by a family member or close friend having been diagnosed with cancer and employment status. Males were prepared to pay more for testing for pancreatic cancer than were women.

Test-retest

Analysis of the test-retest data suggested a good level of agreement in terms of the binary choice question: 47 (99 %) of the 48 who took part gave the same response as to whether they would choose to be tested. The level of agreement for both parts of the willingness-to-pay element was lower. Forty-three of the 48 respondents entered the willingness-to-pay exercise on both occasions and chose a payment scale. Of these, 25 (58 %) chose the same band on both occasions. Forty-two participants gave two valid willingness-to-pay values from the bidding process and 14 (33 %) gave the same exact value at re-test.

Discussion

Key findings

The aim of this willingness-to-pay exercise was to enhance the responses to a simple 'yes/no' question about testing for cancer, by indicating a strength of preference. We hypothesised that we might be able to identify a threshold level of risk below which patients would prefer to wait and see how symptoms develop before being referred for further investigation. Although the overwhelming majority of respondents opted for testing for the three cancers at all levels of risk included, the responses to the willingness-to-pay exercise did not augment the results of the binary question as anticipated; in fact the results suggest that participants treated the two parts of the survey rather differently. The 'yes/no' component of the survey indicated a risk gradient in the case of colorectal cancer, which was not seen in the willingness-to-pay values given to pay for a colonoscopy, but there was evidence that risk influenced willingness-to-pay values for testing for lung cancer. Furthermore, the participant characteristics that affected the decision to opt for a test were different than those affecting the willingness-to-pay values.

Explanations for the findings

In general, the willingness-to-pay values obtained did not differentiate across risk levels as much as hypothesised. It is possible that the known “fear of cancer” may have contributed to this, indeed this was indicated in the yes/no part of the questionnaire where results suggested that patients would opt for testing even at very low levels of risk.

The willingness-to-pay values obtained suggest that risk is an important consideration when patients are deciding whether to accept the offer of a test for cancer of the lung but not colorectal or pancreas, a finding at odds with those of the binary 'yes/no' decision, where risk was only evident in colorectal cancer. Colorectal cancer involves the most invasive test, which may explain this finding, but it would seem that in the willingness-to-pay exercise participants may have discriminated according to neither the burden of the test nor the prognosis (pancreatic cancer having the worst likely outcome), but possibly their perception of the cost of the test. They valued a chest X-ray more highly if the risk of cancer was high (£365 at 10 % risk) and lower if the risk was low (£305 at 1 % risk) and there was a clear gradient (p-value 0.049). This was not the case with the more expensive tests where there was no evidence of an overall gradient despite the value placed on a CT scan for pancreatic cancer at the 10 % risk level being somewhat higher than the value at 1 %. The tests were described in detail in the vignettes so it is possible that many participants recognised that a chest X-ray is less costly than a colonoscopy and a CT scan. This confirms the belief that willingness-to-pay questions tend not to be sensitive to the size or scope of benefits [28] but is counter to the finding that people tend to state a similar amount for any reduction in risk of death or injury [29].

This raises the question of what people are actually valuing when they answer a willingness-to-pay question. Our hypothesis was that the willingness-to-pay exercise could be used to discriminate between cancers and risk levels more sensitively than a binary choice of whether to be tested or not. However, it would seem that participants viewed the two parts of the questionnaire separately and differently: the initial choice of whether to be tested appears to have been driven by the burden placed on them in undergoing the diagnostic test, illustrated by greater reservation about agreeing to a colonoscopy than the other tests, whereas the willingness-to-pay component appears to reflect people’s perception of the cost burden of the test, as a chest X-ray is by far the least expensive of the three. While the main aim of our study was to make comparisons across cancers and risk levels it is useful to reflect on the absolute levels of the willingness-to-pay values in comparison to those found in similar studies. Marshall et al. [5] compared physician and patient preferences for different methods of screening for colorectal cancer in Canada and the US. The values obtained ranged from US$111 (equivalent to £245 inflation adjusted) to C$232 (£662) depending on the type of test and in another US-based study to determine the value of time and discomfort of a colonoscopy, Jonas et al. [30] reported a mean value of US$263 (£563). In the UK Frew et al. [31] compared different methods of eliciting the willingness-to-pay for a faecal occult blood (FOB) test and a flexible sigmoidoscopy (FS) for colorectal cancer screening with results ranging from £86 (£136, inflation adjusted) for FS to £130 (£205) for FOB. In comparison to this, Nuemann et al. [17] obtained somewhat higher values in their US study for predictive testing: for breast and prostate cancer these ranged from US$508 (£904) for an imperfect test for breast cancer when the risk is 10 % to US$622 (£1007) for a perfect test for prostate cancer when the risk is 25 %. Our results of £305 to £393 across all cancers and risk levels fall towards the lower end of these, closer to the UK study results than those from North America.

Methodological considerations

A variety of willingness-to-pay methods have been used in different settings and different patient groups [12]. In this study we used a bidding approach; furthermore, to mitigate the effect of any possible starting point bias we employed a two-part approach with each participant effectively selecting their own starting point. To our knowledge, this is a novel approach, made possible because of the mode of administration and the electronic nature of the questionnaire. However, in avoiding one methodological difficulty we have arguably introduced another. The bidding exercise was designed to restrict participants to the boundaries of the payment scale they chose in the first part of the willingness-to-pay exercise. This decision was largely driven by concern over ethics: it was considered ‘unfair’ to allow respondents to bid above/below the top/bottom value of the scale they had chosen. In fact, we found that 68 % of participants bid up to the highest value, leaving the unanswered question of what would have happened if they had been allowed to go beyond that value. This, in part, may explain the poor result in the relevant part of the test-retest exercise because once the participant had chosen a different payment scale in the retest, which 40 % did, it was impossible for them to identify the same final value as in the initial exercise.

In designing our study we were conscious of the phenomenon of ‘prominent’ numbers and we felt that the two-part design might mitigate this. In fact, analysing the results of the test-retest exercise we found that all participants who chose exactly the same value on both occasions had chosen a ‘prominent’ number: £0, £100, £300, £700 or £1000. This highlights the need for a better understanding of the role of ‘prominent’ numbers in such studies and has implications for the design of future studies.

We included reference goods in the payment scale exercise to help participants think about the value of a test. The verbal probing exercise did not throw up any consensus concerns about the use of reference goods or the choice of goods but there were some interesting individual comments: “You are given a few ideas to give you the value of the cost”, “I think health is more important than material things” and “At my age we already have most things we need – furniture etc.” However, when asked how easy or hard it was to put a value on the test none of the respondents mentioned that the reference goods helped. Whilst it seems useful to have benchmark reference goods for respondents to use our experiences suggest these are not essential; if they are included they must be chosen carefully and must be relevant to the population being surveyed.

Limitations

The technique of willingness-to-pay is a conceptually attractive method of eliciting valuations and preferences. If responses reflect the true value that a population places on an intervention such as diagnostic testing the value can be compared with cost in a ‘purer’ way than any other outcome and used to make decisions about allocative efficiency. However, the results of this study show that many people find it difficult to think in terms of the value of benefit offered rather than what the intervention involves, and this is likely to be particularly true in a system of universal health insurance such as the NHS in England. While we were unable to use the results of this study as intended, they do reveal interesting unanswered questions which should be explored further. Qualitative methods could be employed to understand more about the thought processes and motivation of respondents as they complete such a survey; although some limited work has been done in this area [24, 32] there is a clear gap in our knowledge that needs to be closed in order to successfully exploit the full potential of willingness-to-pay as a technique for eliciting true valuations.

Conclusion

The willingness-to-pay exercise reported here successfully obtained valuations for cancer testing from a large and diverse sample of the UK consulting population. A risk gradient was found only in the case of an X-ray for lung cancer, with higher values reflecting greater risk. This was inconsistent with responses to the question of whether to be tested or not, which suggested risk affected testing preferences only in the case of colorectal cancer. More investigation is needed to understand how patients perceive and respond to risk in this context, and how best to develop the use of willingness-to-pay techniques, which have the potential to provide good quality evidence which could enhance decision making in the provision of health care services.

Appendix

Fig. 3

Screen shots showing questions about participant characteristics