Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Self-Report of Healthcare Utilization among Community-Dwelling Older Persons: A Prospective Cohort Study

  • Marlies T. van Dalen,

    Affiliation Department of General Practice, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Jacqueline J. Suijker ,

    j.j.suijker@amc.uva.nl

    Affiliation Department of General Practice, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Janet MacNeil-Vroomen,

    Affiliation Department of Internal medicine, Geriatric section, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Marjon van Rijn,

    Affiliation Department of Internal medicine, Geriatric section, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Eric P. Moll van Charante,

    Affiliation Department of General Practice, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Sophia E. de Rooij,

    Affiliation Department of Internal medicine, Geriatric section, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

  • Bianca M. Buurman

    Affiliation Department of Internal medicine, Geriatric section, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

Abstract

Background

Self-reported data are often used for estimates on healthcare utilization in cost-effectiveness studies.

Objective

To analyze older adults’ self-report of healthcare utilization compared to data obtained from the general practitioners’ (GP) electronic medical record (EMR) and to study the differences in healthcare utilization between those who completed the study, those who did not respond, and those lost to follow-up.

Methods

A prospective cohort study was conducted among community-dwelling persons aged 70 years and above, without dementia and not living in a nursing home. Self-reporting questionnaires were compared to healthcare utilization data extracted from the EMR at the GP-office.

Results

Overall, 790 persons completed questionnaires at baseline, median age 75 years (IQR 72–80), 55.8% had no disabilities in (instrumental) activities of daily living. Correlations between self-report data and EMR data on healthcare utilization were substantial for ‘hospitalizations’ and ‘GP home visits’ at 12 months intraclass correlation coefficient 0.63 (95% CI; 0.58–0.68). Compared to the EMR, self-reported healthcare utilization was generally slightly over-reported. Non-respondents received more GP home visits (p<0.05). Of the participants who died or were institutionalized 62.2% received 2 or more home visits (p<0.001) and 18.9% had 2 or more hospital admissions (p<0.001) versus respectively 18.6% and 3.9% of the participants who completed the study. Of the participants lost to follow-up for other reasons 33.0% received 2 or more home visits (p<0.01) versus 18.6 of the participants who completed the study.

Conclusions

Self-report of hospitalizations and GP home visits in a broadly ‘healthy’ community-dwelling older population seems adequate and efficient. However, as people become older and more functionally impaired, collecting healthcare utilization data from the EMR should be considered to avoid measurement bias, particularly if the data will be used to support economic evaluation.

Introduction

Self-reported data on healthcare utilization are often used for estimates of healthcare utilization in cost-effectiveness studies [1][3]. Large national studies use surveys to routinely collect these self-reported data [4][9]. Self-report is mostly an effective and less time-consuming mode of collecting data on the utilization of healthcare resources compared to collecting data from medical records or administrative claims data. However, while persons at older age and with more disabilities under-report their healthcare utilization [10], [11], research on community-dwelling older persons heavily relies on data solely gathered through self-reported questionnaires [12], which may result in underestimation of health-care cost among older persons.

Various other factors are associated with errors in self-rated outcomes [11], [13][15]. For example, inaccuracy increases with longer recall periods [10], [11], [13], [14], [16], [17] and when the frequency of events increases, patients tend to under-report more frequently [10], [14], [16][19]. Several studies have compared the accuracy of self-reported with administrative data on healthcare utilization among older persons [11], [13], [22], but they were mostly based on cross-sectional designs.

In longitudinal studies on frail older persons, it is important to note that attrition may be directly related to the primary outcome [20]. Those who do not respond at follow-up are generally older [21][25],less educated [8], [25], have lower socioeconomic status [23], live alone [25], have more functional impairments [8], [21], [25], [26], suffer from more comorbidities [21], [22], and are more inaccurate in self-reporting costs compared with participants who completed the study [27]. However, others found no difference between respondents and those who were lost to follow-up, and assumed that attrition was non-selective [9], [28], [29]. However, more research is needed to investigate the relationship between loss to follow-up and healthcare utilization, to study the potential bias in self-reported healthcare utilization data in studies with a longitudinal design.

The objective of this prospective study is therefore to analyze the agreement between older adults’ self-report of healthcare utilization and data obtained from the primary care electronic medical record (EMR) and to study the differences in the healthcare utilization between those lost to follow-up and those who completed the study.

Methods

Design and Setting

A prospective cohort study was conducted in seven general practices with a total of 1113 eligible persons aged 70 years and over in and around Amsterdam, the Netherlands. These practices had a mixed population in terms of sex, age, and socio-economic status (SES). The cohort was followed up for 12 months between October 2008 and December 2009.

Health-care in the Netherlands

In the Dutch health-care system the general practitioner (GP) is the only freely accessible medical professional and people are used to visit their GP if they have a health problem. The GP is the gate-keeper in the healthcare system, controlling access to specialized medical care, and virtually all non-institutionalized citizens are registered with a GP. Therefore the total practice population represents the general population, and information about the wider population is automatically available. For these reasons, in the Netherlands, general practice is the optimal setting for providing information on the populations use of healthcare services [30].

In the Netherlands GPs are financed by a fixed rate based on an average of two office visits per patient per year.

Ethics Statement

The study was approved by the Medical Ethics Committee of the Academic Medical Center, University of Amsterdam, the Netherlands (protocol ID MEC 10/182).

Study Population

All community-dwelling persons aged 70 years and over who were registered with one of the participating general practices were selected from the EMR by their GP. Persons were excluded if, according to their GP, they were terminally ill, suffered from dementia, did not understand Dutch, planned to move or spend a long time abroad, or lived in a nursing home. Eligible persons received a letter from their GP with information about the study, along with a written informed consent form, a self-report questionnaire, and a pre-paid envelope. They were invited to fill out the questionnaire themselves, and if they needed help, an informal caregiver was allowed to assist (this assistance was noted on the questionnaire). The recruitment of participants is described in detail elsewhere [31]. All participants were asked to provide written informed consent for data collection and participation in the study on receipt of the study information.

Measurements

Self-reported data.

A self-report questionnaire was sent at baseline, and after 3, 6, and 12 months. It comprised demographic data, comorbidities, physical functioning (modified Katz-ADL index score), self-perceived health status, psychological and social functioning (Rand-36), health-related quality of life (EuroQol), and healthcare utilization data [12]. The self-reported data are part of a national database and are publically available [12]. Healthcare utilization data were specified in 1) GP home visit during office hours in the last 3 months and 2) hospital admission in the last 12 months. The number of contacts and the reason for each contact were recorded. Office visits were not gathered since this information would not be of additive value in a cost-effective analysis. Furthermore, the original study objective was to evaluate acute/un-planned healthcare utilization of community dwelling older persons, which did not include office visits.

Electronic medical records.

EMR data were used to calculate healthcare utilization. Healthcare utilization data from October 2008 to December 2009 were extracted from the EMR at the GP’s office by two independent researchers (MvD and JS). First, billing data were searched for home visits. Hospital admissions were then extracted from the patient’s history, and the correspondence section in the EMR was searched for discharge letters. All hospital admissions were recorded. No difference was made between an admission for one day without an overnight stay and an admission of one or more overnight stays. The number of hospitalization days was not recorded. If a GP contact was followed by a hospital admission on the same day, only the hospital admission was recorded, because it was difficult to discriminate between the number of home visits on the same day before hospitalization or between in or out-of-hours GP-home visits.

Linking self-reported data to the EMR.

The self-reported data on healthcare utilization were linked to the data derived from the EMR using the date on which the questionnaire was filled out. To analyze the agreement between self-report data and EMR data for GP home visits, questionnaires returned after 3, 6, and 12 months were used. To analyze the agreement between self-report data and EMR data for hospital admission, questionnaires returned after 12 months were used.

Non-respondents at Baseline

Previous studies suggested that non-respondents are more often cognitively and functionally impaired and have a higher mortality rate than responding participants, and therefore are often considered a high-risk population [7], [32], [33]. Therefore, non-respondents may use healthcare services more frequently.

Non-response is a collective term for persons (called non-respondents or non-participants) who are invited to participate in a study but do not do so [34]. To explore whether non-respondents were more often functionally and/or cognitively impaired and used healthcare services more frequently, a sample of randomly selected non-respondents and a sample of randomly selected participants were invited to participate in a single home visit conducted by a trained research nurse, on behalf of their own GP. Informed consent was obtained before the interview took place. During this home visit, the same baseline self-report questionnaire was conducted along with a Mini-Mental State Examination (MMSE) [35] in both groups. The healthcare utilization computed from the EMR over the year 2009 was compared between both groups.

Respondents Lost to Follow-up

Self-reported baseline characteristics and healthcare utilization computed from the EMR over the year 2009 were compared between participants who completed follow-up and those who were lost to follow-up after 12 months. Participants who died or were institutionalized were analyzed separately, with the assumption that this group was prone to higher levels of healthcare utilization in the period before death or institutionalization [36]. Older persons lost to follow-up for other reasons were defined as those declining further participation, not able to be contacted at follow-up or lost to follow-up for other reasons.

Statistical Methods

Baseline descriptive statistics were used. Differences in baseline characteristics and healthcare utilization between participants and non-respondents, and between participants who completed follow-up and those lost to follow-up were estimated using a Mann-Whitney U test for continuous, nonparametric data or a Chi-square test for ordinal variables. Overall, a p-value of <0.05 was considered statistically significant.

Data on healthcare utilization were assessed as the number of contacts with healthcare services. The number of self-reported home visits and hospital admissions were compared with the number of home visits and hospital admissions derived from the EMR. To evaluate the agreement between self-report of healthcare utilization and data from the EMR, 3 groups were identified: (1) Under-reporting (patient reported less utilization in comparison to data retrieved from the EMR), (2) Exact agreement (self-reported and the EMR agreed on the volume of utilization), (3) Over-reporting (participant reported more utilization than could be retrieved from the EMR).

Levels of agreement between self-reports and the EMRs were assessed for each follow-up time using absolute agreement and intraclass correlation coefficients (ICCs)[37]. The benchmark for determining the closeness of the comparison of the ICC was based on Landis and Koch’s scale [38]. The strength of agreement was <0.00 (poor), 0.00–0.20 (slight), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial) and 0.81–1.00 (almost perfect) [38].

Finally, healthcare utilization and participation time in the study were plotted against age and number of disabilities in (instrumental) activities of daily living ((I)ADL). Analyses were performed with the Statistical Package for the Social Sciences (SPSS) version 18.0.

Results

Eligible Persons and Baseline Characteristics

The cohort comprised 1113 persons. Reasons for exclusion and non-response are shown in figure 1. In total, 790 (76.3%) participants returned the baseline questionnaire and were included. At baseline, the median age was 75 years (IQR 72–80), 56.7% were female, 55.8% participants had no (I)ADL disabilities and 16.2% reported memory problems (Table 1). A total of 644 (81.5%) participants returned the questionnaire after 12 months. The number of persons lost to follow-up due to death or institutionalization ranged from 9.1% at 6 months to 11.0% at 12 months, calculated over the total number of persons lost to follow-up. Figure 2 shows the mean healthcare utilizaton in 2009 extracted from the EMR and the mean time not in the study with increasing age. Figure 3 demonstrates the mean healthcare utilizaton in 2009 extracted from the EMR and the mean time not in the study with increasing number of (I)ADL disabilities.

thumbnail
Figure 1. Flowchart.

*Non-respondents were those we were unable to contact at baseline. Those who are denoted as ‘unable to contact’ are those persons not responding at follow-up. † These numbers represent a variable group of persons since persons who did not respond at three months follow up, might respond at a later follow-up moment.

https://doi.org/10.1371/journal.pone.0093372.g001

thumbnail
Figure 2. Mean healthcare utilizaton in 2009 extracted from the EMR and the mean time not in the study with increasing age (years) (n = 790).

Mean healthcare consumption with 95%CI.

https://doi.org/10.1371/journal.pone.0093372.g002

thumbnail
Figure 3. Mean healthcare utilizaton in 2009 extracted from the EMR and the mean time not in the study with increasing number of (I)ADL disabilities (n = 790).

Mean healthcare consumption with 95%CI.

https://doi.org/10.1371/journal.pone.0093372.g003

Agreement between Self-reported Healthcare Utilization and EMRs

Figure 4 shows all persons included in the analysis based on possible data linkage. The agreement between both data sources was 86.1% for home visits, and 86.9% for hospital admission. The strength of agreement (ICC) between both data sources at 12 months was 0.63 (95% CI; 0.58–0.68) for both home visits and for hospitalization, substantial in both cases. Agreement between both data sources after three and six months was similar (data not shown). In general, participants slightly over-reported their healthcare utilization. Table 2 and 3 show the healthcare utilization (GP home visits and hospital admission) according self-reported data and EMR at twelve months. Table 4 shows the prevalence and measures of agreement of healthcare utilization according self-reported data and EMR at twelve months. Characteristics of persons under- or over-reporting versus agreement between self-report and EMR, are displayed in Table S1.

thumbnail
Figure 4. Flowchart data retrieval.

*The period of data retrieval did not cover the recall period.

https://doi.org/10.1371/journal.pone.0093372.g004

thumbnail
Table 2. The numbers of self-reported GP-homevisits and GP-homevisits according to the Electronic Medical Record at 12 months.

https://doi.org/10.1371/journal.pone.0093372.t002

thumbnail
Table 3. The numbers of self-reported hospitalization and hospitalization according to the Electronic Medical Record at 12 months.

https://doi.org/10.1371/journal.pone.0093372.t003

thumbnail
Table 4. Agreement between self-reported and Electronic Medical Record healthcare utilization at 12 months.

https://doi.org/10.1371/journal.pone.0093372.t004

Non-respondents

For the non-respondent analysis, 103 home visits were conducted. Of those persons, 32 were non-respondents and 71 responded to our surveys. Non-respondents had a lower socioeconomic status (SES) (p<0.05), a lower scores on the MMSE (p<0.01) and received more GP home visits compared to respondents (p<.05). There were no differences in hospitalization between both groups (p = 0.788) (Table S2).

Healthcare Utilization of Participants Compared to those Lost to Follow-up

At 12 months, those who died or were institutionalized were older (p<0.001), more disabled in (I)ADL (p<0.001) and had more other risk factors for functional decline at baseline. Those lost to follow-up for other reasons had more often a non-Caucasian ethnic background (p<0.001), a lower socioeconomic status (p = 0.002), were more disabled in (I)ADL (p<0.001) and were less able to participate (p<0.001) (Table 5).

thumbnail
Table 5. Baseline characteristics of participants compared to those lost to follow-up (percentages).

https://doi.org/10.1371/journal.pone.0093372.t005

Of all 790 persons included in this study, 19.5% were hospitalized and 32.7% received home visits in the year 2009 according to the EMR. Table 6 displays the healthcare utilization of respondents, those who died or were institutionalized, and those lost to follow-up for other reasons. Persons who died or were institutionalized had significantly higher overall levels of healthcare utilization compared to respondents and those lost to follow-up for other reasons (p<0.001) for both GP home visits and hospitalizations. This finding was already apparent after 3 months’ follow-up (data not shown). Those lost to follow-up for other reasons had more home visits by their GP after 12 months (p = 0.003). This finding was consistent, though not significant, with those at 3 and 6 months follow-up.

thumbnail
Table 6. Healthcare utilization of participants compared to those lost to follow-up.

https://doi.org/10.1371/journal.pone.0093372.t006

Discussion

Main Findings

In this prospective cohort study among 790 community-dwelling older persons, correlations between self-report data and EMR data on healthcare utilization were substantial for ‘hospitalizations’ and ‘GP home visits’ at 12 months. Healthcare utilization was slightly over-reported compared to data based on the EMR. Non-respondents had a lower socioeconomic status, a lower MMSE score at baseline, and received more GP home visits. Participants who died or were institutionalized were older and more often functionally impaired, and utilized more healthcare services. Participants lost to follow-up for other reasons only received more home visits by GPs. This study showed that with increasing age and disabilities in daily functioning, healthcare utilization increased and participation time in the study decreased.

Comparison with the Literature

Our findings of over-reporting of healthcare utilization compared to the EMRs contradict some studies [16], [17], but are consistent with others [11], [15]. Raina et al. found that patients aged 65 years and older (n = 1,038) over-reported hospitalizations over 12 months in 3.3% of the cases and under-reported hospitalizations in 3.1% [15]. In our study these rates were 8.0% and 5.2%, respectively. Raina et al. observed a patient agreement about hospitalization in 13.5% at baseline interviews, compared to 10.4% after 12 months in our study. Wolinsky et al. found comparable measures for misreporting hospitalizations [11].

We did not find any literature on healthcare utilization among non-respondents.

In our study, 18.5% of persons were lost to follow-up. Participants who had died or were institutionalized were older, more often disabled, more often depressed, reported more often memory problems, had worsened health status compared to a year ago and were less able to participate at baseline. Participants lost to follow-up for other reasons more often had a non-Caucasian ethnic background, had a lower socioeconomic status, were more often disabled and were less able to participate. These findings are in accordance with the literature [8], [21][26], although some authors found no difference between responders and those lost to follow-up [28] or assumed nonselective loss to follow-up [9], [29].

Older persons who died or were institutionalized appeared to have significantly higher levels of healthcare utilization. Those lost to follow-up for other reasons received more home visits, though no differences were found in hospitalizations. Hoogendoorn et al. compared self-report versus care provider registration of healthcare utilizations and its impact on cost and cost-utility. They found that the degree of under-reporting was independently associated with loss to follow-up and total costs, showing that those lost to follow-up had larger differences in costs between registrations and self-reports compared with full participants [27].

Strengths and Limitations

This study has several limitations. Firstly, using the EMR as the reference standard is not without its own drawbacks [10], [16], [18], [39]. For example, a substantial proportion (9.5%) of patients consume healthcare outside the urban healthcare system, suggesting that this use may not be captured in the EMR [10]. GPs also tend to under-report repeated consultations for the same problem [40]. However, Dendukuri et al. showed that claims databases had the greatest validity [41]. Some persons were repeatedly hospitalized, though the GP was sometimes notified only once. Furthermore, GP contacts on the same day prior to hospitalization were often omitted when extracting healthcare utilization from the EMR, resulting in a potential underestimation of GP home visits. Moreover, when a patient is dismissed from a hospital, it can take some time before the hospitalization can be found in the EMR.

Secondly, as a measure of healthcare utilization, only hospitalizations and GP home visits were taken into account. When this study was first conducted, information on emergency department (ED) visits and contact with GPs during out-of-hours were also collected from participants. However, the GP was often not separately informed about ED visits, especially if a patient was hospitalized afterwards, and therefore this contact was not retrieved from the EMR. However, it is important to note that 70% of ED visits by older people is followed by a hospital admission [42]. Furthermore, it turned out that the prevalence of out-of-hours GP contacts in our population was very low; therefore, these contacts were left out of the analyses too.

Thirdly, participants may have found it difficult to discriminate between home visits paid by GPs and registered nurses, or between visits during in or out-of-hours, yielding lower agreement measures. Furthermore, GP home visits were not counted in the EMR if they were on the same day as a hospital admission, leading to an underestimation of the costs made by GPs and to an apparent overestimation of self-reported home-visits.

Most studies on the agreement of self-reports have cross-sectionally examined populations. The strength of our study lies in the prospective design and our additional non-respondent analysis, comparing healthcare utilization patterns of respondents with non-respondents and those who died, institutionalized or otherwise lost to follow-up.

Implications

In our study, correlations between self-report and EMR data for GP home visits and hospitalizations were nearly identical. The impact of that difference on, for example, cost-effectiveness studies should be taken into consideration. For example, Hoogendoorn et al. found that despite the almost perfect agreement on hospitalizations and hospital days, the cost difference was highest for this type of care, approximately 390 euros [27].

This research also shows that the inclusion of data from older persons lost to follow-up during prospective cohort studies is important. Those who die or are institutionalized tend to use more healthcare services than those remaining in the study. Those lost to follow-up for other reasons also tend to place a larger burden on more intensive primary healthcare services than those remaining in the study. Therefore, the more substantial the loss to follow-up becomes, the more need there may be to consider data-collection on health services utilization from the EMR.

Conclusion

Self-report of hospitalizations and GP home visits in a broadly ‘healthy’ community-dwelling older population seems adequate and efficient. However, as people become older and more functionally impaired, collecting healthcare utilization data from the EMR should be considered to avoid measurement bias, particularly if the data will be used to support economic evaluation.

What is New?

  • Self-report of hospitalizations and GP home visits in a broadly ‘healthy’ community-dwelling older population seems adequate and efficient.
  • However, as people become older and more functionally impaired, collecting healthcare utilization data from the electronic medical records should be considered to avoid measurement bias, particularly if the data will be used to support economic evaluation.
  • Older persons who died or were institutionalized during follow-up tended to use more healthcare services than participants.
  • Those who were lost to follow-up for other reasons tended to receive more GP home visits than those remaining in the study.

Supporting Information

Table S1.

Patient characteristics versus reporting agreement.

https://doi.org/10.1371/journal.pone.0093372.s001

(DOC)

Acknowledgments

We are grateful to all participants, participating GPs and research assistance: Jose Popma. The data reported in this article were collected as part of a prospective cohort study to prevent functional decline in community-dwelling older persons.

Author Contributions

Conceived and designed the experiments: MTD JJS JM MR EPMC SER BMB. Performed the experiments: MTD JJS JM MR EPMC SER BMB. Analyzed the data: MTD JJS JM MR EPMC SER BMB. Contributed reagents/materials/analysis tools: MTD JJS JM MR EPMC SER BMB. Wrote the paper: MTD JJS JM MR EPMC SER BMB.

References

  1. 1. Oostenbrink JB, Koopmanschap MA, Rutten FF (2002) Standardisation of costs: the Dutch Manual for Costing in economic evaluations. Pharmacoeconomics 20: 443–454.
  2. 2. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL (2005) Methods for the Economic Evaluation of health Care Programmes. Oxford: Oxford University Press.
  3. 3. Goossens ME, Rutten-van Molken MP, Vlaeyen JW, van der Linden SM (2000) The cost diary: a method to measure direct and indirect costs in cost-effectiveness research. J Clin Epidemiol 53: 688–695.
  4. 4. E Buskens, de Rooij SE, Olde Rikkert MGM, Styerberg EW, Izaks G, et al. (2008) Minimale Data Set aanvulling doelmatigheid. Available http://www.nationaalprogrammaouderenzorg.nl/.fileadmin/npoz.nl/documenten/toolkits/mds-aanvulling-doelmatigheid.pdf. Accessed 3 November 2008.
  5. 5. Huisman M, Poppelaars J, van der Horst M, Beekman AT, Brug J, et al. (2011) Cohort profile: the Longitudinal Aging Study Amsterdam. Int J Epidemiol 40: 868–876.
  6. 6. Gill TM, Desai MM, Gahbauer EA, Holford TR, Williams CS (2001) Restricted activity among community-living older persons: incidence, precipitants, and health care utilization. Ann Intern Med 135: 313–321.
  7. 7. Hebert R, Bravo G, Korner-Bitensky N, Voyer L (1996) Predictive validity of a postal questionnaire for screening community-dwelling elderly individuals at risk of functional decline. Age Ageing 25: 159–167.
  8. 8. Daniels R, van Rossum E, Beurskens A, van den Heuvel W, de Witte L (2012) The predictive validity of three self-report screening instruments for identifying frail older people in the community. BMC Public Health 12: 69.
  9. 9. Byles JE, Tavener M, O’Connell RL, Nair BR, Higginbotham NH, et al. (2004) Randomised controlled trial of health assessments for older Australian veterans and war widows. Med J Aust 181: 186–190.
  10. 10. Wallihan DB, Stump TE, Callahan CM (1999) Accuracy of self-reported health services use and patterns of care among urban older adults. Med Care 37: 662–670.
  11. 11. Wolinsky FD, Miller TR, An H, Geweke JF, Wallace RB, et al. (2007) Hospital episodes and physician visits: the concordance between self-reports and medicare claims. Med Care 45: 300–307.
  12. 12. Lutomski JE, Baars MAE, Schalk BWM, Boter H, Buurman BM, et al. (2013) The development of The Older Persons and Informal Caregivers Survey Minimum DataSet (TOPICS-MDS): A large-scale data sharing initiative. PLoS ONE 8(12): e81673.
  13. 13. Bhandari A, Wagner T (2006) Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev 63: 217–235.
  14. 14. Roberts RO, Bergstralh EJ, Schmidt L, Jacobsen SJ (1996) Comparison of self-reported and medical record health care utilization measures. J Clin Epidemiol 49: 989–995.
  15. 15. Raina P, Torrance-Rynard V, Wong M, Woodward C (2002) Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res 37: 751–774.
  16. 16. Cleary PD, Jette AM (1984) The validity of self-reported physician utilization measures. Med Care 22: 796–803.
  17. 17. Bellon JA, Lardelli P, Luna JD, Delgado A (2000) Validity of self reported utilisation of primary health care services in an urban population in Spain. J Epidemiol Community Health 54: 544–551.
  18. 18. Ritter PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, et al. (2001) Self-reports of health care utilization compared to provider records. J Clin Epidemiol 54: 136–141.
  19. 19. Glandon GL, Counte MA, Tancredi D (1992) An analysis of physician utilization by elderly persons: systematic differences between self-report and archival information. J Gerontol 47: S245–S252.
  20. 20. Ferrucci L, Guralnik JM, Studenski S, Fried LP, Cutler GB Jr, et al. (2004) Designing randomized, controlled trials aimed at preventing or delaying functional decline and disability in frail, older persons: a consensus report. J Am Geriatr Soc 52: 625–634.
  21. 21. Sommers LS, Marton KI, Barbaccia JC, Randolph J (2000) Physician, nurse, and social worker collaboration in primary care for chronically ill seniors. Arch Intern Med 160: 1825–1833.
  22. 22. van Hout HP, Jansen AP, van Marwijk HW, Pronk M, Frijters DF, et al. (2010) Prevention of adverse health trajectories in a vulnerable elderly population through nurse home visits: a randomized controlled trial [ISRCTN05358495]. J Gerontol A Biol Sci Med Sci 65: 734–742.
  23. 23. Tas U, Verhagen AP, Bierma-Zeinstra SM, Hofman A, Odding E, et al. (2007) Incidence and risk factors of disability in the elderly: the Rotterdam Study. Prev Med 44: 272–278.
  24. 24. Stalenhoef PA, Diederiks JP, Knottnerus JA, Kester AD, Crebolder HF (2002) A risk model for the prediction of recurrent falls in community-dwelling elderly: a prospective cohort study. J Clin Epidemiol 55: 1088–1094.
  25. 25. Mihelic AH, Crimmins EM (1997) Loss to folow-up in a sample of Americans 70 years of age and older: The LSOA 1984–1990. J Gerontol B Psychol Sci Soc Sci 52: S37–S48.
  26. 26. Binder EF, Schechtman KB, Ehsani AA, Steger-May K, Brown M, et al. (2002) Effects of exercise training on frailty in community-dwelling older adults: results of a randomized, controlled trial. J Am Geriatr Soc 50: 1921–1928.
  27. 27. Hoogendoorn M, van Wetering CR, Schols AM, Rutten-van Molken MP (2009) Self-report versus care provider registration of healthcare utilization: impact on cost and cost-utility. Int J Technol Assess Health Care 25: 588–595.
  28. 28. Hogan DB, MacDonald FA, Betts J, Bricker S, Ebly EM, et al. (2001) A randomized controlled trial of a community-based consultation service to prevent falls. CMAJ 165: 537–543.
  29. 29. Tas U, Steyerberg EW, Bierma-Zeinstra SM, Hofman A, Koes BW, et al. (2011) Age, gender and disability predict future disability in older people: the Rotterdam Study. BMC Geriatr 11 22 22.
  30. 30. Schellevis FG, Westert GP, de Bakker DH (2005) The actual role of general practice in the Dutch health-care system: results of the Second Dutch National Survey of General Practice. Journal of Public Health 13: 265–269.
  31. 31. Suijker JJ, Buurman BM, Ter Riet G, van Rijn M, de Haan RJ, et al. (2012) Comprehensive geriatric assessment, multifactorial interventions and nurse-led care coordination to prevent functional decline in community-dwelling older persons: protocol of a cluster randomized trial. BMC Health Serv Res 12: 85.
  32. 32. Bowns I, Challis D, Tong MS (1991) Case finding in elderly people: validation of a postal questionnaire. Br J Gen Pract 41: 100–104.
  33. 33. Hebert R, Bravo G, Korner-Bitensky N, Voyer L (1996) Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol 49: 373–381.
  34. 34. Kjoller M, Thoning H (2005) Characteristics of non-response in the Danish Health Interview Surveys, 1987–1994. Eur J Public Health 15: 528–535.
  35. 35. Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12: 189–198.
  36. 36. Hebert R, Brayne C, Spiegelhalter D (1997) Incidence of functional decline and improvement in a community-dwelling, very elderly population. Am J Epidemiol 145: 935–944.
  37. 37. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86: 420–428.
  38. 38. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159–174.
  39. 39. Carsjo K, Thorslund M, Warneryd B (1994) The validity of survey data on utilization of health and social services among the very old. J Gerontol 49: S156–S164.
  40. 40. Jordan K, Jinks C, Croft P (2006) Health care utilization: measurement using primary care records and patient recall both showed bias. J Clin Epidemiol 59: 791–797.
  41. 41. Dendukuri N, McCusker J, Bellavance F, Cardin S, Verdon J, et al. (2005) Comparing the validity of different sources of information on emergency department visits: a latent class analysis. Med Care 43: 266–275.
  42. 42. Buurman BM, van den Berg W, Korevaar JC, Milisen K, de Haan RJ, et al. (2011) Risk for poor outcomes in older patients discharged from an emergency department: feasibility of four screening instruments. Eur J Emerg Med 18: 215–220.