Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Are Routinely Collected NHS Administrative Records Suitable for Endpoint Identification in Clinical Trials? Evidence from the West of Scotland Coronary Prevention Study

  • Sarah J. E. Barry ,

    sarah.barry@glasgow.ac.uk

    Affiliation Robertson Centre for Biostatistics, University of Glasgow, Glasgow, United Kingdom

  • Eleanor Dinnett,

    Affiliation R&D Department, Greater Glasgow and Clyde Health Board, Glasgow, United Kingdom

  • Sharon Kean,

    Affiliation Robertson Centre for Biostatistics, University of Glasgow, Glasgow, United Kingdom

  • Allan Gaw,

    Affiliation Wellcome Trust/Wolfson Clinical Research Facility, Queen's University Belfast, Belfast, United Kingdom

  • Ian Ford

    Affiliation Robertson Centre for Biostatistics, University of Glasgow, Glasgow, United Kingdom

Abstract

Background

Routinely collected electronic patient records are already widely used in epidemiological research. In this work we investigated the potential for using them to identify endpoints in clinical trials.

Methods

The events recorded in the West of Scotland Coronary Prevention Study (WOSCOPS), a large clinical trial of pravastatin in middle-aged hypercholesterolaemic men in the 1990s, were compared with those in the record-linked deaths and hospitalisations records routinely collected in Scotland.

Results

We matched 99% of fatal study events by date. We showed excellent matching (97%) of the causes of fatal endpoint events and good matching (>80% for first events) of the causes of nonfatal endpoint events with a slightly lower rate of mismatching of record linkage than study events (19% of first study myocardial infarctions (MI) and 4% of first record linkage MIs not matched as MI). We also investigated the matching of non-endpoint events and showed a good level of matching, with >78% of first stroke/TIA events being matched as stroke/TIA. The primary reasons for mismatches were record linkage data recording readmissions for procedures or previous events, differences between the diagnoses in the routinely collected data and the conclusions of the clinical trial expert adjudication committee, events occurring outside Scotland and therefore being missed by record linkage data, miscoding of cardiac events in hospitalisations data as ‘unspecified chest pain’, some general miscoding in the record linkage data and some record linkage errors.

Conclusions

We conclude that routinely collected data could be used for recording cardiovascular endpoints in clinical trials and would give very similar results to rigorously collected clinical trial data, in countries with unified health systems such as Scotland. The endpoint types would need to be carefully thought through and an expert endpoint adjudication committee should be involved.

Introduction

There is increasing interest in the use of routinely collected health records to support clinical trials. In England, the Clinical Practice Research Datalink [1] and in Wales, the Secure Anonymised Information Linkage [2] initiative, support researchers in conducting studies. In Scotland, the Wellcome Trust funded Scottish Health Informatics Programme [3] has created a Scotland-wide research platform for the collation and analysis of electronic patient records. At the same time, there is a desire to reduce the complexity and cost of conducting clinical trials. In the United States, the Clinical Trials Transformation Initiative [4] has been established to drive forward changes in this area. In the United Kingdom, the Academy of Medical Sciences has put forward proposals for improvement of the research environment including increased use of routinely collected health data. Recently, the Medical Research Council and other major funders in the UK have created of a network of four health eResearch Centres as centres of excellence in research using e-health records.

The future success of the UK Biobank [5] project will depend in large part on achieving reliable follow-up for clinical outcomes at reasonable cost. The widely used Scottish Longitudinal Study [6] and the England & Wales (ONS) Longitudinal Study [7] are based on linking census data with routinely collected data such as deaths and hospital admissions, and have led to research in a wide range of areas, from health services research to employment (eg. [8], [9]). Routinely collected data has been linked to nurse/researcher-collected data in the Scottish Health Survey, to follow up individuals for outcomes that would be resource intensive to collect by other means [10], and such data has been used to highlight risk factors to the physical and mental health of the Scottish population [11], [12]. All of these databases are available for research, whether it be for public health or clinical trial follow-up.

An important question is whether routinely collected data, especially data collected for administrative reasons, are fit for purpose as outcomes in clinical trials or epidemiological studies.

In 2007 we were able to extend the original average follow-up of approximately 5 years in the West of Scotland Coronary Prevention Study (WOSCOPS) by approximately 10 years [13] using computer record linkage to nationally held datasets for hospital admission, incident cancers and deaths in Scotland. WOSCOPS was a randomised double-blind placebo-controlled trial of pravastatin in 6595 hypercholesterolaemic men aged 45–64 with no history of myocardial infarction, carried out between 1988 and 1995. The study achieved 100% follow-up of all participants for vital status at the end of the trial. The main endpoints of the trial included definite or suspect coronary heart disease (CHD) death, definite or suspect nonfatal myocardial infarction (MI), other cardiac death, other vascular death or other death. Further details on the design and conduct of the trial may be found elsewhere [14].

The results in the original report of the trial [15] were obtained via intensive and expensive follow-up of participants and detailed recording and adjudication of all clinical outcomes. In the current work we have carried out record linkage follow-up of outcomes during the trial period, and explored the pros and cons of record linkage versus intensive follow-up in the context of cardiovascular endpoints. This research adds detail to an initial assessment of the utility of record linkage data for general serious adverse event reporting [16] and shows that it could also be used to follow patients up on specific endpoints, particularly if an endpoint committee were involved.

Methods

Overview

Approximately 10 years after the end of the trial, WOSCOPS participants were probabilistically matched using name, date of birth and postcode to national datasets for mortality, incident cancer and hospital discharges by the Information Services Division (ISD) of the NHS National Services Scotland, providing computerised record linkage follow-up of all trial participants at low cost.

This paper compares and contrasts the events that were reported by the normal trial processes and the events that were identified by record linkage as occurring during the period of the trial. We focus on deaths, non-fatal MI and non-fatal stroke.

Ethics Statement

The participants in WOSCOPS provided written informed consent for access to their medical records, but at the time this did not explicitly refer to medical records held on a computer. After the end of the trial, the ethics committee of the Royal Infirmary, Glasgow, approved linkage of patient information to the participants' electronic medical records. In addition the Privacy Advisory Committee of National Services Scotland, which acts as the gatekeeper to Scotland's national holdings of patient records and advises ISD, also approved this linkage. Both approvals are necessary before a linkage can be performed. These processes are long established in Scotland as the appropriate pathway for record linkage approvals. Hence, we have strictly adhered to all existing ethical and information governance requirements in Scotland. In addition all data were handled in compliance with the UK data protection regulations.

WOSCOPS Trial Events

During the study, all deaths and all non-fatal coronary events were reviewed by an endpoint adjudication committee. Although not pre-specified as an endpoint, potential stroke or transient ischaemic attack (TIA) events were also documented on a stroke-specific form and were reviewed centrally at the end of the study and classified as stroke, TIA or ‘other’. Imaging was not always available; hence we have not further grouped these events as haemorrhagic or non-haemorraghic. The original WOSCOPS definition of a nonfatal MI included events that were ‘silent’ (only detected via an annual study electrocardiogram without accompanying symptoms) or ‘unrecognised’ (symptoms consistent with an MI detected but not recognised as an MI at the time). Since the events in the routinely collected dataset were, by definition, hospitalisations, we included only hospitalised study events in the analysis, excluding silent events but including unrecognised hospitalised events. Events originally classified as ‘definite’ or ‘suspect’ were merged. Nonfatal events considered plausibly an MI were adjudicated as either a nonfatal MI or as a nonfatal ‘other’ event.

Record Linkage (RL) Events

Death records during the trial provided the date of death, an underlying cause of death and a total of up to four causes coded using the International Classification of Diseases (ICD) codes, version 9. For RL identified deaths, cause of death was taken to be the assigned underlying cause of death.

The Scottish Morbidity Records of hospitalisations contain the dates of admission and discharge from hospital, along with dates of ward transfers and up to six diagnoses for each ward stay, all ICD coded. For analyses involving non-fatal MI, stroke or transient ischaemic attack (TIA), a relevant code in any position of a hospitalisation report was taken as evidence of an event. Hemiplegia was included as a cerebrovascular (stroke/TIA) diagnosis in the RL dataset.

We searched in the RL dataset for all events recorded as MI. In the absence of MI, any code for CHD in any position was taken as evidence of a CHD-related event. In the absence of a CHD code, a cardiovascular (CV) code in any position identified a CV-related outcome, and in the absence of a CV code the event was considered non-cardiovascular. The ICD code groupings for events identified by RL are given in Appendix S1.

Statistical Methods

WOSCOPS and RL events for an individual were considered to be the same event if they had a diagnosis match and the event dates were within at most two weeks of one another. A detailed study of any WOSCOPS events unmatched in the RL data was carried out using the patient casenotes. ‘First events’ were defined as first events of the particular type (eg. MI) during the study period.

The analysis of the original WOSCOPS events was carried out for the fatal and hospitalised nonfatal WOSCOPS events, and similarly for the RL events. These analyses used the log-rank test to calculate the p-values and Cox proportional hazards models to produce the risk reduction estimates and corresponding 95% confidence intervals.

Results

The baseline characteristics of the participants in the WOSCOPS are given in Table 1. Participants were on average 55 years of age and had mean baseline total cholesterol and low density lipoprotein cholesterol levels of 7.0 and 5.0 mmol/l respectively.

The main original results of the WOSCOPS are given in Table 2, alongside results excluding non-hospitalised MI and those obtained using RL outcomes. While the observed risk reduction in the primary endpoint of CHD death or nonfatal MI was slightly greater using the RL data than the original or hospitalised WOSCOPS data, the qualitative conclusion would have been the same regardless of the approach. A similar impact is observed for nonfatal MI alone, while the risk reduction for CHD death is slightly reduced, going from being marginally significant using the original or hospitalised WOSCOPS events to marginally nonsignificant using the RL events.

thumbnail
Table 2. Original WOSCOPS results and corresponding results using hospitalised WOSCOPS and Record Linkage events.

https://doi.org/10.1371/journal.pone.0075379.t002

Fatal Events

There were 241 deaths recorded in WOSCOPS and 240 in the RL dataset. Of these 238 were matched by date. Three WOSCOPS deaths were missing from the RL dataset and further investigation showed that they all occurred outside Scotland (1 Bulgaria, 2 England). Two deaths were found in the RL dataset but not recorded in WOSCOPS and these appeared to be errors in the linkage, since one had a date of death prior to the participant’s randomisation date and thus could not be the same person, while the other had no recorded events in WOSCOPS.

The primary causes of death for the 238 deaths that matched are given in Table 3. The RL data distinguished between 29 ‘other’ and 91 ‘cancer’ deaths (which are all denoted as ‘other’ in the table), with 2 of the ‘other’ deaths being mismatched as CHD and 1 cancer death as ‘other vascular’ in WOSCOPS. There was a high rate of matching on cause of death, with only 7/238 (2.9%) of causes being mismatched. A clinical review of the death certificates for these 7 patients, blinded to the WOSCOPS outcome, re-classified 4 of the events, reducing the mismatch rate to 3/238 (1%).

thumbnail
Table 3. Matching of the causes of death between WOSCOPS and Record Linkage.

https://doi.org/10.1371/journal.pone.0075379.t003

Nonfatal Events

The distribution of RL events conditional on WOSCOPS MI events.

There were 304 cases of nonfatal MI incurring hospitalisation in WOSCOPS, 268 of which were first hospitalised MI events. Table 4 shows the distributions of corresponding RL events for first and subsequent WOSCOPS MI events.

thumbnail
Table 4. Matching of WOSCOPS first and subsequent MI events with corresponding Record Linkage events.

https://doi.org/10.1371/journal.pone.0075379.t004

The matching of MI events is better for first than subsequent events, with 81.4% of WOSCOPS first MI events versus 58.3% of subsequent MI events matching with MI events in RL. The classification rate of WOSCOPS MI events as CHD events (including MI) is similar for first and subsequent events, however, with 89.6% and 91.6% concordance, respectively. Of particular note is that over half of the WOSCOPS MI events not matched with cardiovascular events in RL (15/25 of first and 2/3 of subsequent nonmatched events) were matched with RL events diagnosed as non-cardiac ‘unspecified chest pain’, suggesting that there may be a number of miscodings for such events in hospital records. Of the 9 WOSCOPS MI events that were not found at all in the RL dataset, 7 of the events occurred outside Scotland, 1 patient was admitted to a private hospital (only NHS admissions are recorded in the hospitalisations dataset) and 1 was definitely admitted to hospital according to the casenotes and their absence from the hospitalisations dataset is presumed due to linkage errors.

The distribution of WOSCOPS MI events conditional on RL MI events.

Of the 264 hospitalisations identified in RL as MI, 230 were first events, and 96.0% of these were matched with a WOSCOPS MI event (Table 5). A total of 7 (3.0%) first RL MI events were matched with ‘other’ events in WOSCOPS, all of which had been adjudicated by the WOSCOPS adjudication committee as not being MI events. Of the 10 (29.4%) subsequent events matched with ‘other’ events in WOSCOPS, 9 were similarly adjudicated as not being MI events, while one had MI as the secondary diagnosis to a primary diagnosis of chest pain and the patient casenotes showed it involved a procedure following an earlier (matched) MI and was therefore not a ‘true’ MI admission. Of the 2 (0.9%) first event hospitalisations with an MI diagnosis that were not matched with WOSCOPS MI endpoint events, the patient casenotes showed that one was a day admission for a procedure and the other was admitted with atrial fibrillation and had been confirmed to the trial committee by the patient’s cardiologist as having no evidence of MI. Neither event, therefore, was referred to the endpoint committee.

thumbnail
Table 5. Matching of Record Linkage first and subsequent MI events with corresponding WOSCOPS events.

https://doi.org/10.1371/journal.pone.0075379.t005

Of the 6 (17.6%) subsequent event hospitalisations with an MI diagnosis that were not matched with WOSCOPS MI endpoint events, the patient casenotes show that 3 were readmissions following matched MI events (one involving a procedure), 1 was a transfer from a private hospital following an earlier admission for a MI (and therefore not a true admission), 1 was admitted to Accident & Emergency only and diagnosed with heart failure and 1 was admitted for pulmonary oedema. None of these events were referred to the endpoint committee.

Distribution of RL events conditional on a WOSCOPS stroke or TIA.

There were a total of 94 WOSCOPS events for stroke or TIA, 83 of which were first stroke/TIA events, and the level of matching with RL events is shown in Table 6. Of the WOSCOPS first stroke events, 78.7% were matched with either RL stroke (72.1%) or TIA hospitalisations (6.6%). For WOSCOPS first TIA events, the match rate was similar at 77.2% (54.5% matched as TIA, 22.7% as stroke).

Of the WOSCOPS stroke/TIA first events, 12 were completely unmatched, of which 6 occurred outside Scotland, 1 was an admission to a private hospital, 1 was missing supporting documentation to confirm their admission date, 3 had definite admissions with stroke/TIA diagnoses, suggesting a possible problem with RL, coding or admission dates and 1 had unspecified cerebrovascular disease. There were only 11 instances of subsequent WOSCOPS stroke/TIA events, all of which were matched in RL.

Distribution of WOSCOPS events conditional on a RL stroke or TIA.

Table 7 shows that there were a total of 102 hospitalisations in the RL data for stroke or TIA events, 87 of which were first events. Of the RL first stroke events, 83.1% were matched with either WOSCOPS stroke (75.4%) or TIA (7.7%) events. For RL first TIA events, the matching rate was slightly lower at 72.7% (54.5% as TIA, 18.2% as stroke). There was a higher level of mismatching of a small number of subsequent stroke/TIA events.

thumbnail
Table 7. Matching of the Record Linkage strokes and TIAs.

https://doi.org/10.1371/journal.pone.0075379.t007

The RL stroke/TIA events that were matched with MI endpoints had both MI and stroke/TIA diagnoses for the same hospitalisation. Of the 15 ‘other’ matches and the non-matches for RL first stroke or TIA events in record linkage, 6 were capturing a pre-existing condition according to the patient notes, 2 were capturing a cerebral event other than stroke/TIA, 4 had unclear diagnosis or the diagnosis was clarified later, 1 was not an acute event and 2 were unexplained and may have been due to inaccurate coding or additional information in the medical record. Of the 9 ‘other’ matches and the non-matches for RL subsequent stroke or TIA events, 8 were capturing a pre-existing condition, while for 1 the event was recorded on the case report forms in WOSCOPS but the specific stroke/TIA form was not completed.

Discussion

It is clear that, if WOSCOPS had only considered fatal events and nonfatal events that incurred hospitalisation, the researchers would have drawn very similar conclusions using record linkage data only compared to using adjudicated outcomes. We have shown, for cardiovascular data collected in Scotland, good matching of routinely collected and rigorously collected clinical trial data, in particular for mortality. The matching was also good for MI endpoints, particularly for first events. The quality of matching was less good for the stroke and TIA non-endpoint events.

Due to the imperfect matching of events, our proposed model of conducting a cardiovascular clinical trial using RL data in a unified healthcare system would be to use RL for case finding and to have an expert endpoint committee that would adjudicate based on appropriate source documentation from the patient’s clinical record. Based on our results we believe that this model would be likely to detect most of the MI events that were coded in the hospitalisations dataset as CHD or cardiovascular events. It would also be likely to eliminate most of the spurious ‘events’ that were actually hospital transfers or readmissions for procedures following a previous event. We have also identified that a number of MI or other cardiac events may be miscoded as unspecified chest pain in the hospital admission record, suggesting that including this diagnosis when searching for cardiac events would increase the rate of detection, although possibly at the expense of a significant amount of additional case record review.

Events occurring outside of Scotland were a minor issue. Cross-linking with other nations within the UK would reduce this problem further. Events occurring outside the UK, other than deaths reported in UK death registries, or hospital cases transferred back to UK hospitals, would go undetected. However, the level of under-reporting present in the RL data was low and would be unlikely to have a substantial effect on the outcome of a trial.

The quality of matching of hospitalised stroke and TIA events was poorer since this type of event may be more likely to be misdiagnosed or miscoded. Nevertheless, RL picked up most of the WOSCOPS events, and the larger numbers of events that were recorded in the hospitalisations dataset than in WOSCOPS were mainly due to investigations or procedures for pre-existing conditions or previous events, issues that would be mostly resolved by case record review. It must also be borne in mind that stroke was not a specified endpoint in WOSCOPS and so we would expect a lower level of matching here due to a potentially less rigorous process of recording such events within the trial.

Clearly, in designing a clinical trial using RL data to detect clinical cardiovascular events, choice of endpoint would have to be made carefully. General cardiovascular or coronary hospitalisations are likely to be identified with high sensitivity and specificity. More focussed event types (such as MI or stroke) are likely to benefit from the use of RL for case identification followed by case note extraction and expert review.

In summary, we have shown that the wealth of routinely collected data could be used for cardiovascular outcome detection in a clinical trial carried out in a country with a unified health system, such as Scotland or the wider UK, and would likely lead to a qualitatively similar outcomes to those obtained by the current expensive clinical trial model involving rigorous individual patient follow-up and intensive data collection.

Supporting Information

Appendix S1.

ICD 9 code groupings for each event type.

https://doi.org/10.1371/journal.pone.0075379.s001

(DOCX)

Acknowledgments

Information Services Division of NHS Scotland carried out the record linkage of the datasets.

Author Contributions

Conceived and designed the experiments: IF SK AG. Performed the experiments: SB ED. Analyzed the data: SB ED. Wrote the paper: SB IF. Contributed to revision of manuscript: ED AG SK IF. Final approval of manuscript: SB ED AG SK IF.

References

  1. 1. Clinical Practice Research Datalink website. Available: http://www.cprd.com/intro.asp. Accessed 2013 Aug 20.
  2. 2. Secure Anonymised Information Linkage website. Available: http://hiru.swansea.ac.uk/. Accessed 2013 Aug 20.
  3. 3. Scottish Health Informatics Programme website. Available: http://www.scot-ship.ac.uk/. Accessed 2013 Aug 20.
  4. 4. Clinical Trials Transformation Initiative website. Available: https://www.ctti-clinicaltrials.org/. Accessed 2013 Aug 20.
  5. 5. UK Biobank website. Available: http://www.ukbiobank.ac.uk/. Accessed 2013 Aug 20.
  6. 6. Boyle PJ, Feijten P, Feng ZQ, Hattersley L, Huang ZY, et al. (2009) Cohort Profile: The Scottish Longitudinal Study (SLS). International Journal of Epidemiology 38: 385–392.
  7. 7. Dale A, Creeser R, Dodgeon B, Gleave S, Filakti H (1993) An Introduction to the Opcs Longitudinal-Study. Environment and Planning A 25: 1387–1398.
  8. 8. van Ham M, Manley D (2010) The effect of neighbourhood housing tenure mix on labour market outcomes: a longitudinal investigation of neighbourhood effects. Journal of Economic Geography 10: 257–282.
  9. 9. Clemens T, Boyle P, Popham F (2009) Unemployment, mortality and the problem of health-related selection: Evidence from the Scottish and England & Wales (ONS) Longitudinal Studies Health Statistics Quarterly. 43: 7–13.
  10. 10. Gray L, Batty GD, Craig P, Stewart C, Whyte B, et al. (2010) Cohort Profile: The Scottish Health Surveys Cohort: linkage of study participants to routinely collected records for mortality, hospital discharge, cancer and offspring birth characteristics in three nationwide studies. International Journal of Epidemiology 39: 345–350.
  11. 11. Lawder R, Harding O, Stockton D, Fischbacher C, Brewster DH, et al.. (2010) Is the Scottish population living dangerously? Prevalence of multiple risk factors: the Scottish Health Survey 2003. Bmc Public Health 10.
  12. 12. Hamer M, Stamatakis E, Batty GD (2010) Objectively Assessed Secondhand Smoke Exposure and Mental Health in Adults Cross-sectional and Prospective Evidence From the Scottish Health Survey. Archives of General Psychiatry 67: 850–855.
  13. 13. Ford I, Murray H, Packard CJ, Shepherd J, Macfarlane PW, et al. (2007) Long-term follow-up of the West of Scotland Coronary Prevention Study. New England Journal of Medicine 357: 1477–1486.
  14. 14. Ford I (1992) A Coronary Primary Prevention Study of Scottish Men Aged 45–64 Years - Trial Design. Journal of Clinical Epidemiology 45: 849–860.
  15. 15. Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer AR, et al. (1995) Prevention of Coronary Heart-Disease with Pravastatin in Men with Hypercholesterolemia. New England Journal of Medicine 333: 1301–1307.
  16. 16. Ford I (1995) Computerised record linkage: Compared with traditional patient follow-up methods in clinical trials and illustrated in a prospective epidemiological study. Journal of Clinical Epidemiology 48: 1441–1452.