Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective

BMC Medical Informatics and Decision Making volume 15, Article number: 67 (2015) Cite this article

Abstract

Background

With the introduction and implementation of a variety of government programs and policies to encourage adoption of electronic medical records (EMRs), EMRs are being increasingly adopted in North America. We sought to evaluate the completeness of a variety of EMR fields to determine if family physicians were comprehensively using their EMRs and the suitability of use of the data for secondary purposes in Ontario, Canada.

Methods

We examined EMR data from a convenience sample of family physicians distributed throughout Ontario within the Electronic Medical Record Administrative data Linked Database (EMRALD) as extracted in the summer of 2012. We identified all physicians with at least one year of EMR use. Measures were developed and rates of physician documentation of clinical encounters, electronic prescriptions, laboratory tests, blood pressure and weight, referrals, consultation letters, and all fields in the cumulative patient profile were calculated as a function of physician and patient time since starting on the EMR.

Results

Of the 167 physicians with at least one year of EMR use, we identified 186,237 patients. Overall, the fields with the highest level of completeness were for visit documentations and prescriptions (>70 %). Improvements were observed with increasing trends of completeness overtime for almost all EMR fields according to increasing physician time on EMR. Assessment of the influence of patient time on EMR demonstrated an increasing likelihood of the population of EMR fields overtime, with the largest improvements occurring between the first and second years.

Conclusions

All of the data fields examined appear to be reasonably complete within the first year of adoption with the biggest increase occurring the first to second year. Using all of the basic functions of the EMR appears to be occurring in the current environment of EMR adoption in Ontario. Thus the data appears to be suitable for secondary use.

Peer Review reports

Background

Although North American countries have previously been lagging behind in the uptake of electronic medical records [EMRs] into their clinical practice [1], in recent years, with the introduction of government policies and mandates, physicians have been increasingly adopting EMRs into their clinical practice [2–5]. Canada Health Infoway strives towards the goal of one electronic health record for all Canadians, however in Canada, healthcare is organized at the provincial level and therefore each province has its own EMR adoption program and policies. Ontario was the first Canadian province to develop an EMR adoption program over a decade ago and although was initially slow in uptake [6], approximately 80 % of family physicians in Ontario are now on, or planning to go on an EMR in the near future [5]. Ontario’s EMR adoption program has targeted family physicians and provides physicians subsidization for adopting an approved EMR software into their clinical practice for capture of clinical activity that occurs within the family physician office and relevant information pertaining to the family physician management of patient health care. There are additional financial incentives if physicians can demonstrate that they are using their EMR to schedule patient appointments, enter encounter notes, enter problem lists, record prescriptions, generate automatic alerts/reminders to support care delivery and receive laboratory results electronically [7].

Government motivation to encourage EMR use includes expectations of both improvements in efficiency and quality of care. However evidence to support this notion is limited and whether or not EMR use results in improved patient care and outcomes is unclear [8–11]. Advanced features such as reminders, clinical decision support, and practice-based surveillance are considered to be amongst the most effective tools within the EMR to improve care [12–14] but cannot be used without having completed fields for the variables which trigger these tools. Understanding the extent to which physicians are using their EMRs is more important for improving care than simply assessing the presence or absence of an EMR. Previous studies that have looked at using electronic health record data for quality measurement [15] or comparative effectiveness research [16] have recognized the complexity of this type of evaluation and have provided conceptual frameworks on how to assess the data and issues to consider. Although useful in concept they do not provide real world analysis of the completeness of data. In addition a recent literature review assessing the reliability and validity of electronic health record quality measures found that most studies have been performed on data from large academic sites [17].

Few studies have looked at the time necessary to have reasonably complete EMR records. One study looked at data reliability but was confined to assessments for preventative services (pap smears, mammograms and influenza vaccinations). They found that data extracted from EMRs had limited reliability in the initial phase of EMR implementation but that during the second year after the introduction of the EMR, data reliability improved substantially [18].

It is unknown if the current government sponsored programs for EMR adoption including financial subsidies and peer leader support, are sufficient for supporting successful EMR implementation and it is unknown if physicians are comprehensively using their EMRs for all aspects of patient care. Evaluating the data quality of EMRs is not only important for policy makers and health administrators, but for researchers who seek to use EMRs for secondary purposes, such as research. Therefore, we evaluated the data population of a variety of EMR fields as a measure for completeness of EMR usage to assess if current policies are sufficient for EMR implementation and the suitability of the EMR data for secondary purposes. We also assessed the duration of time after EMR implementation to have complete EMR records and provide benchmarks for evaluation of the completeness of a variety of EMR fields.

Methods

Data source

We performed a retrospective review of data within the Electronic Medical Record Administrative data Linked Database (EMRALD) at the Institute for Clinical Evaluative Sciences [ICES]. EMRALD contains data from family physicians in Ontario that use Practice Solutions® EMR, the market leading EMR software vendor in Ontario, Canada [19]. All clinically relevant information from the EMRs are extracted. ICES is a ‘prescribed entity’ under provincial privacy legislation which provides the legal authority to collect individual level health information as it has the policies and procedures in place to protect patient privacy and confidentiality [20].

Study participants

Data were extracted in the summer of 2012 from physicians who had been using their EMR for at least one year from both urban and rural locations distributed throughout Ontario. All physicians had a list of patients ‘rostered’ (enrolled) under them for which both the physician and the patient have a signed agreement with the provincial government identifying the physician that is responsible for the patient’s primary care service delivery [21]. The characteristics of physicians included in this study were compared to all family physicians in Ontario using the ICES Physician Database [IPDB].

Patient population

Only patients that had a valid health insurance number, date of birth, and were rostered to the contributing physicians were included. Active patients were defined as rostered patients that had a physician visit within the one year prior to the date of data extraction or in the one year time interval examined. The characteristics of patients included in this study were compared to all patients rostered in Ontario and all Ontario residents. Neighborhood income quintiles and rural residence were determined through linking postal codes to Canadian census data [22]. General assessments of the burden of comorbidities were calculated using number of Ambulatory Care Groups (ACGs) [23] and presence of chronic conditions were determined using previously validated administrative data algorithms for identifying patients with specific disease conditions [24–27].

Measures of EMR utilization

Measures were developed to evaluate the extent to which physicians were using the EMR to record their patient clinical encounters (visit documentation), blood pressures, weights, laboratory tests, prescriptions, referrals, specialist consultation letters and the population of all the fields of the cumulative patient profile (CPP). These EMR fields were grouped into practice style independent fields where the ideal situation would be 100 % completion (completed for every patient) and practice style dependent fields in which a lower level of utilization may not necessarily represent poor quality of EMR use but outlying physicians most likely represent poor users of the EMR for these functions (Table 1).

Table 1 Fields for EMR usage assessed and their measures
Full size table

Blood pressures and weights in the EMR are automatically captured in structured variable fields if entered using the nomenclature bp: and wt: respectively. The advantage for clinicians of having structured variable fields is that the measures can be searched and graphed to see changes over time. The advantage for researchers is that these measures can easily be identified and analysed. Physicians that have low recordings of these measures may not have been doing these measures, not recording these measures, or not recording these measures with the proper nomenclature such that the measurement populated the structured variable field.

Laboratory results can come into the EMR either through an automated electronic feed from the laboratory, manually entered in, or scanned into the EMR as a report. Only laboratory results entered into the EMR in a structured variable field, as what occurs in the instance of an automated electronic feed, are readily accessible both for analysis and for clinical care. Information stored in a structured variable field can be quickly searched and graphed to look at trends over time whereas information in scanned documents tends not to be searchable.

Some physicians who are not fully using their EMRs may opt to write their prescriptions on a prescription pad or a referral on a paper referral form. However doing so would result in incomplete EMR patient records and may not allow for use of advanced EMR functions. Thus we assessed the percent of active patients that had a prescription or referral letter. Physicians with really low numbers of ‘active’ patients with a prescription or a referral letter were thought to be using paper to perform these functions rather than their EMR.

Currently there is no province-wide program that facilitates automatic electronic transfer of consultation letters from specialists and hospital discharge summaries. Mechanisms to electronically transfer hospital discharge summaries are now starting to roll out but at the time of the study these mechanisms were not in place. We therefore looked at the number of consult letters compared to the number of rostered patients, as an approximate measure, to see if these external documents that were coming into physician offices were being entered into the EMR.

The CPP includes the categories: family history, medical history including history of past health and problem list (a list of current active problems), allergies, immunizations, active treatments, risk factors and personal traits. Within the EMR there are two separate fields for the history of past health and for a list of active problems. These two fields were grouped together since physicians do not use the two separate fields in a consistent manner.

Measuring time on EMR

Completeness of EMR fields by physician time on EMR was assessed by looking at each year since initiation of the EMR. The initiation date of the EMR was defined as the earliest date with at least 10 progress notes with a corresponding bill for a visit that were recorded on the same day. Only physicians that had a full year of data in the relevant year were included in the denominators for assessment. For example only the patients of the physicians that had at least 4 years of data were included in the assessment of the completeness of the EMR fields for the physician time on EMR 4 year category. The analysis for patient time on EMR was done by assessing both the EMR fields and the CPP fields in the one year prior to the most recent date of data extraction for patients that had a record in the EMR for <1 year, 1-2 years, 2-3 years and >3 years. Patient time on the EMR was calculated from first date of a progress note and a physician bill on the same date to the date of the most recent data extraction. The analysis for the EMR fields for the patient time on EMR were confined to just those patients that were ‘active’ (had a visit in the last year since extraction).

Statistical analysis

Physician characteristics by duration of EMR use were descriptively analyzed. We estimated the average completeness by duration of EMR use for each field with 95 % confidence intervals [CI] constructed around the average frequency of data completeness. Analyses were performed using SAS version 9.2 [SAS Institute, Cary, North Carolina].

Setting benchmarks

We looked to identify benchmarks of completeness for each of the EMR fields, as benchmarks of this type have not previously been reported. The EMRALD team including several family physicians/EMR users discussed and agreed upon benchmarks for optimal completeness. The rationale behind these benchmarks included considerations for clinical use and secondary use of EMR data.

For the practice style independent fields we agreed that visit documentation (documentation for the purposes of clinical record keeping) and allergies were clinically the most important for patient safety and thus a benchmark of 95 % data completeness was set. For the other practice style independent fields a benchmark of 80 % completeness was agreed to be sufficient, feasible and clinically meaningful. For practice style dependent fields we recognized that physicians may simply prescribe fewer medications, order fewer lab tests, or refer their patients less than their colleagues. Thus it was decided that benchmarks for these parameters was best suited to be data driven, to identify outliers with really low population of these fields potentially indicating systemic EMR issues rather than physician practice style issues. The benchmark for these fields was set at the mean for the measure less one standard deviation.

Physicians were plotted on frequency distribution graphs and benchmarks were drawn to gain an understanding of where physicians lay in terms of meeting benchmarks.

This study received ethics approval from the Sunnybrook Health Science Centre Research Ethics Board. Individual level physician consent was not required for this study. As a prescribed entity under Ontario’s Personal Health Information Protection Act, ICES is authorized to collect and use personal health information from health organizations and clinics without consent for the purposes of health system evaluation and monitoring. Additionally, ICES is prohibited, under its agreements with data providers, from contacting individuals whose information has been entrusted to ICES. This contractual obligation restricts any opportunity to seek individuals’ consent for use of their information.

Results

Although the patients included in this study were more from higher income quintiles and living in rural locations, the age and sex, presence of chronic conditions and measures of comorbidity were similar to rostered patients in Ontario (Table 2). There were differences in all the characteristics when comparing EMRALD participating physicians to the rest of the primary care physicians in Ontario (Table 3). Physician characteristics by duration of EMR use are reported in Table 4. Of the 167 physicians with at least one year of EMR use, we identified 186,237 patients. The number of physicians, and the corresponding number of patients, decreased with increasing physician time of EMR use.

Table 2 Comparison of EMRALD study cohort patients, rostered patients in Ontario and all residents of Ontario as of March 31, 2012
Full size table
Table 3 Comparison of study cohort physicians and all primary care physicians in Ontario as of March 31, 2012
Full size table
Table 4 Physician characteristics by duration of EMR use
Full size table

For the physician time on EMR analysis, the completeness by physician’s duration of EMR use for each field (visit documentation, blood pressures, weights, laboratory tests, prescriptions, referrals and consultation letters) are illustrated in Fig. 1. Overall, the fields with the highest level of completeness for the average proportion of the number of patients were for visit documentation and for prescriptions (≥70 %). By duration of physician EMR use, the average completion of visit documentation increased from 69 % [95 % CI 65-73] in Year 1 to 88 % [95 % CI 84-92] by Year 5. Over time, increasing trends of completeness for all fields were observed, except for laboratory tests and consultation letters.

Fig. 1
figure 1

Average completeness by physician’s duration of EMR use for the fields: visit documentation, blood pressures, weights, laboratory tests, prescriptions, referrals and consultation letters

Full size image

For patient time on the EMR there was an increasing population of blood pressure, laboratory tests and prescriptions over time. No increases were observed for documentation of patient weights, referrals and consultations for patients that had been on the EMR for over 3 years (Fig. 2). For population of the CPP fields, there was an increasing trend for completeness of all fields except for allergies and personal traits for patients with >3 years of patient data on the EMR (Fig. 3). The largest improvements occurred between one and two years.

Fig. 2
figure 2

Population of various fields as a function of patient duration on the EMR

Full size image
Fig. 3
figure 3

Population of the cumulative patient profile [CPP] fields as a function of patient duration on the EMR

Full size image

With respect to the frequency distributions for completeness of the fields and the proportion of physicians meeting benchmarks, we found that for fields where benchmarks were set to look for outliers (laboratory tests, prescriptions, referrals and consultation letters) that the majority of physicians exceeded the benchmarks or threshold cut-offs and that outliers or patients with relatively low completion of these fields could easily be identified (Fig. 4). For benchmarks that were set such that the higher the completion the more comprehensive the record we were able to identify areas where completion rates were suboptimal and could be improved upon. The poorest completion of these fields was weight recording, family history and risk factors as evidenced by the majority of physicians falling below the set benchmark (Figs. 4 and 5).

Fig. 4
figure 4

Distribution of physicians for the various EMR fields

Full size image
Fig. 5
figure 5

Distribution of physicians for the cumulative patient profile fields

Full size image

Discussion

We developed measures to evaluate the extent to which physicians are using the EMR for patient care in Ontario. Overall it appears that physicians are making use of their EMRs within the first year and certainly by the second year post-adoption. Thus, it appears that the current levels of support to use the basic EMR functions are sufficient in achieving adoption of the EMR.

Increased physician and patient duration on the EMR improved data completeness, with the biggest improvements seen between the first and second year. This is similar to a previous study that found recording of preventative screening tests for cancer were not adequately captured in the first year of starting an EMR but improved substantially when looking in the two year window post EMR adoption [18]. Another study found that having two or more years of electronic health record experience was independently associated with reported benefits of having an electronic health record [9]. This time period required for completeness is also relevant to secondary users of the data when making assessment of the quality of care provided or ‘fit for purpose’ assessments, as incomplete EMR data may impede the use of clinical decision support embedded within the EMR, or result in inaccurate analysis of data for quality of care assessment or other types of analysis [28, 29]. Although duration of time on the EMR has not been shown to result in higher performance on quality indicators [30], our study illustrates that both physician and patient time on EMR influence the likelihood of data completeness. The results of this study have led us to develop data quality checks within the EMRALD database to look for outlying physicians to exclude from our studies if a particular field of the EMR is going to be utilized in a study analysis. Our results presented here should have applicability in other settings as EMRALD participants are not just academic sites but draw largely from community practising family physicians.

One of the important features of advanced EMR functions is to automatically generate feedback and reminders at the point of care and to ensure that they are a part of the healthcare providers workflow [31]. Most of these clinical aids rely on completion of EMR data fields in order to accurately report back to the physician. The first step to using advanced functions of the EMR, such as reminders or clinical decision support, is having completed fields for the variables which trigger these tools. While we studied completeness of fields as a measure for EMR usage, we recognize that completeness is only one dimension of data quality [32]. However, assessments of other dimensions of data quality cannot occur without a first assessment of completeness and previous studies looking at the suitability of using EMR data to recruit patients into clinical trials have found data completeness to be an essential component in the assessment of using EMR data for secondary purposes [33]. Whether these completeness measures would hold true with other EMR systems is a potential limitation but we provide real world measures that can be used for comparison with data from other EMR systems.

A previous study has shown that the recording of the presence of some conditions is less frequent in the EMR compared to self-report [34]. Another limitation of this study is that although we were able to determine if there was information recorded in each of the CPP fields we were unable to determine the completeness or quality of recording within each field. Other limitations include the inability to measure the time duration from installment of the EMR to actual usage of the EMR, but given that government funding support is time limited and user fees are charged by EMR vendors from time of installation, it is unlikely that the time from installation to the time of actual use are of significant duration. Another limitation is that we were unable to assess the impact of the local practice environment and/or the presence of allied health professionals on completeness of EMR fields. It is unknown who in a practice was responsible for completing fields such as the CPP or entering in blood pressures. Unfortunately we could also not tell when CPP fields were entered, as we only receive a snap shot of the CPP as it stands on the date of extraction and each item in the CPP is not time stamped. We could also not assess the quality of the data that was entered for example, recording of dates of disease onset. Last, although our patient characteristics were similar to rostered patients in Ontario in terms of presence of chronic diseases and co-morbid conditions, our EMRALD sample of patients were from higher income quintiles and had proportionally more patients living in rural areas compared to the Ontario population. In terms of income quintiles it is not known if this bias is because patients of higher income quintiles may be more likely to seek health care at the primary care level. With respect to a higher proportion of patients residing in rural areas, this proportion constantly changes as new physicians contribute to EMRALD and recruitment is on an ongoing basis. Similarly, our physician characteristics did not exactly match the rest of the family physicians in Ontario. Nonetheless our results showed consistent improvements with completion of fields with increasing duration of time on the EMR despite heterogeneity of physician characteristics.

Conclusions

In this paper we outline a pragmatic process which we have used to assess completeness of a variety of EMR fields to determine if physicians are adequately using their EMRs and to assess suitability for secondary use. Certainly we know that current EMRs in Ontario are deficient in documenting elements such as hospitalizations and emergency room visits [35]. However, using these described methods in a broad range of EMR fields should be sufficient to allow use of primary care EMR data in a variety of different types of studies as we have employed these methods for data quality checking and have been able to perform a wide range of studies with EMRALD. The types of studies we’ve done have included: assessment of wait times from family physician referral to specialist [36], audit and feedback of quality indicators for chronic disease management [37], validation of administrative data algorithms [38, 39] and within EMR algorithms [40–42] to identify patients with a variety of medical conditions.

Additionally the methods that we have developed here can be used to identify physicians who perform poor on measures of data completeness and are in need of further assistance. Programs could be developed to identify those who fall below their peers and offer support to increase data quality and completeness to get them to more optimal usage. Since we did not examine the use of advanced functions of EMR systems that have been indicated to be important for EMR benefits realization, future work could examine the best point after adoption for them to be implemented, and their impact on patient management, performance on quality indicators and patient outcomes. All users of EMR data, including physicians, administrators, researchers, and policy makers should be acutely aware of the need for understanding data quality and completeness prior to utilizing it for purposes secondary to direct patient care. Furthermore, analysis of both patient time and physician time on EMR data are also important considerations for using EMRs for research as there is a greater likelihood of detecting data with increasing time of a patient’s contribution of data to the database.

References

  1. Schoen C, Osborn R, Doty MM, Squires D, Peugh J, Applebaum S. A survey of primary care physicians in eleven countries, 2009: perspectives on care, costs, and experiences. Health Aff (Millwood). 2009;28(6):w1171–83.

    Article  Google Scholar 

  2. Patel V, Jamoom E, Hsiao CJ, Furukawa MF, Buntin M. Variation in electronic health record adoption and readiness for meaningful use: 2008-2011. J Gen Intern Med. 2013;28(7):957–64.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Xierali IM, Hsiao CJ, Puffer JC, Green LA, Rinaldo JC, Bazemore AW, et al. The rise of electronic health record adoption among family physicians. Ann Fam Med. 2013;11(1):14–9.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Canada of Family Physicans of Canada, Canadian Medical Association, Royal College of PHysicians and Surgeons of Canada. National Physician Survey 2007. Results. 2007 [accessed 2014 December 11]; Available from: http://nationalphysiciansurvey.ca/surveys/2007-survey/.

  5. College of Family Physicians of Canada CMA, Royal College of Physicians and Surgeons of Canada. National Physician Survey 2014. Results. 2014. [accessed 2014 December 11]; Available from: http://nationalphysiciansurvey.ca/surveys/2014-survey/survey-results-2/.

  6. Keshavjee K. EMR Implementation in Ontario: A position paper to increase EMR implementation in Ontario. Intel of Canada White Paper; 2007.

  7. Ontario MD EMR Enhanced Use Program. [accessed 2014 December 11]; Available from: https://www.ontariomd.ca/portal/server.pt/community/emr_funding/emr_enhanced_use_program/.

  8. Jang J, Yu SH, Kim CB, Moon Y, Kim S. The effects of an electronic medical record on the completeness of documentation in the anesthesia record. Int J Med Inform. 2013;82(8):702–7.

    Article  PubMed  Google Scholar 

  9. King J, Patel V, Jamoom EW, Furukawa MF. Clinical benefits of electronic health record use: national findings. Health Serv Res. 2014;49(1 Pt 2):392–404.

    Article  PubMed  Google Scholar 

  10. Kern LM, Barrón Y, Dhopeshwarkar RV, Edwards A, Kaushal R, Investigators H. Electronic health records and ambulatory quality of care. J Gen Intern Med. 2013;28(4):496–503.

    Article  PubMed  Google Scholar 

  11. Klumpp TR. Electronic medical records and quality of cancer care. Curr Oncol Rep. 2013;15(6):588–94.

    Article  PubMed  Google Scholar 

  12. Tolar M, Balka E. Beyond individual patient care: enhanced use of EMR data in a primary care setting. Stud Health Technol Inform. 2011;164:143–7.

    PubMed  Google Scholar 

  13. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38.

    Article  CAS  PubMed  Google Scholar 

  14. Abramson EL, Malhotra S, Osorio SN, Edwards A, Cheriff A, Cole C, et al. A long-term follow-up evaluation of electronic health record prescribing safety. J Am Med Inform Assoc. 2013;20(e1):e52–8.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kahn MG, Raebel MA, Glanz JM, Riedlinger K, Steiner JF. A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research. Med Care. 2012;50(Suppl):S21–9.

    Article  PubMed  Google Scholar 

  16. Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013;51(8 Suppl 3):S30–7.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Chan KS, Fowles JB, Weiner JP. Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev. 2010;67(5):503–27.

    Article  PubMed  Google Scholar 

  18. Greiver M, Barnsley J, Glazier RH, Harvey BJ, Moineddin R. Measuring data reliability for preventive services in electronic medical records. BMC Health Serv Res. 2012;12:116.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Vendor and Offering Market Share Reports. [accessed 2014 December 11]; Avaliable from: https://www.ontariomd.ca/portal/server.pt/community/emr_offerings/vendor_market_share/.

  20. Privacy at ICES. [accessed 2015 July 30]; Available from: http://www.ices.on.ca/Data-and-Privacy/Privacy%20at%20ICES.

  21. Ministry of Health and Long-Term Care M. Health Force Ontario: Family Practice Models. 2013 [accessed 2015 July 30]; Available from: http://www.healthforceontario.ca/en/Home/Physicians/Training_%7C_Practising_in_Ontario/Physician_Roles/Family_Practice_Models.

  22. Wilkins R. Automated geographic coding based on the Statistics Canada postal code conversion files, including postal codes through March 2009. Ottawa: Ontario, Canada; 2009.

    Google Scholar 

  23. The John Hopkins University Bloomberg School of Public Health HSRDC. The John Hopkins Adjusted Clinical Groups® (ACG) Case-Mix System Version 6.0 Release Notes. The Johns Hopkins University, 2003 April. Report No.: Contract No.: June 30.

  24. Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512–6.

    Article  PubMed  Google Scholar 

  25. Tu K, Campbell N, Chen Z, Cauch-Dudek KJ, McAlister FA. Accuracy of administrative databases in identifying patients with hypertension. Open Med. 2007;1(1):E18–26.

    PubMed  PubMed Central  Google Scholar 

  26. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J. 2009;16(6):183–8.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying individuals with physcian diagnosed COPD in health administrative databases. COPD. 2009;6(5):388–94.

    Article  CAS  PubMed  Google Scholar 

  28. Liaw ST, Chen HY, Maneze D, Taggart J, Dennis S, Vagholkar S, et al. Health reform: is routinely collected electronic information fit for purpose? Emerg Med Australas. 2012;24(1):57–63.

    Article  PubMed  Google Scholar 

  29. Burge F, Lawson B, Van Aarsen K, Putnam W. Assessing the feasibility of extracting clinical information to create quality indicators from primary healthcare practice EMRs. Healthc Q. 2013;16(3):34–41.

    Article  PubMed  Google Scholar 

  30. Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, et al. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16(4):457–64.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.

  32. Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20:144–151.

  33. Köpcke F, Trinczek B, Majeed RW, Schreiweis B, Wenk J, Leusch T, Ganslandt T, Ohmann C, Bergh B, Röhrig R, Dugas M, Prokosch HU. Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence. BMC Med Inform Decis Mak 2013;13:37.

  34. Violán C, Foguet-Boreu Q, Hermosilla-Pérez E, Valderas JM, Bolíbar B, Fabregas-Escurriola M, et al. Comparison of the information provided by electronic health records data and a population health survey to estimate prevalence of selected health conditions and multimorbidity. BMC Public Health. 2013;13(1):251.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Tu K, Mitiku TF, Ivers NM, Guo H, Lu H, Jaakkimainen L, et al. Evaluation of Electronic Medical Record Administrative data Linked Database (EMRALD). Am J Manag Care. 2014;20(1):e15–21.

    PubMed  Google Scholar 

  36. Jaakkimainen L, Tu K, Barnsley J, Salkeld E, Lu H, Glazier R, et al. Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Fam Pract. 2014;15(1):16.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Ivers NM, Tu K, Young J, Francis JJ, Barnsley J, Shah BR, et al. Feedback GAP: pragmatic, cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci. 2013;8:142.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Tu K, Wang M, Jaakkimainen RL, Butt D, Ivers NM, Young J, et al. Assessing the validity of using administrative data to identify patients with epilepsy. Epilepsia. 2014;55(2):335–43.

    Article  PubMed  Google Scholar 

  39. Tu K, Wang M, Young J, Green D, Ivers NM, Butt D, et al. Validity of administrative data for identifying patients who have had a stroke or transient ischemic attack using EMRALD as a reference standard. Can J Cardiol. 2013;29(11):1388–94.

    Article  PubMed  Google Scholar 

  40. Tu K, Manuel D, Lam K, Kavanagh D, Mitiku TF, Guo H. Diabetics can be identified in an electronic medical record using laboratory tests and prescriptions. J Clin Epidemiol. 2011;64(4):431–5.

    Article  PubMed  Google Scholar 

  41. Ivers N, Pylypenko B, Tu K. Identifying patients with ischemic heart disease in an electronic medical record. J Prim Care Community Health. 2011;2(1):49–53.

    Article  PubMed  Google Scholar 

  42. Krysko KM, Ivers NM, Young J, O’Connor P, Tu K. Identifying individuals with multiple sclerosis in an electronic medical record. Mult Scler. 2015;21(2):217–24.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was funded by a service agreement with Canada Health Infoway.

This study was supported by the Institute for Clinical Evaluative Sciences [ICES], which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care [MOHLTC]. The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.

Dr. Karen Tu is supported by a Canadian Institutes of Health Research Fellowship Award in Primary Care and a Research Scholar Award from the Department of Family and Community Medicine, University of Toronto.

Dr. Noah Ivers is supported by New Investigator Awards from the Canadian Institutes of Health Research the Department of Family and Community Medicine, University of Toronto.

Dr. Debra Butt and Dr. Liisa Jaakkimainen are supported by a Research Scholar Award from the Department of Family and Community Medicine, University of Toronto.

Author information

Authors and Affiliations

  1. Institute for Clinical Evaluative Sciences, G1 06, 2075 Bayview Avenue, Toronto, ON, M4N 3M5, Canada

    Karen Tu, Jessica Widdifield, Jacqueline Young, William Oud, Noah M. Ivers & Liisa Jaakkimainen

  2. Department of Family and Community Medicine, University of Toronto, Toronto, Canada

    Karen Tu, Noah M. Ivers, Debra A. Butt & Liisa Jaakkimainen

  3. University Health Network-Toronto Western Family Health Team, Toronto, Canada

    Karen Tu

  4. Department of Family and Community Medicine-The Scarborough Hospital, Toronto, Canada

    Debra A. Butt

  5. Women’s College Research Institute and Family Practice Health Centre, Women’s College Hospital, Toronto, Canada

    Noah M. Ivers

  6. Canada Health Infoway, Toronto, Canada

    Chad A. Leaver

  7. Department of Family and Community Medicine-Sunnybrook Health Sciences Centre, Toronto, Canada

    Liisa Jaakkimainen

Authors
  1. Karen Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jessica Widdifield
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacqueline Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William Oud
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Noah M. Ivers
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Debra A. Butt
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chad A. Leaver
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Liisa Jaakkimainen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karen Tu.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors contributed to the conception and design of the study and interpretation of the data. KT was responsible for acquisition of the data, JY was responsible for analysis of the data. KT, WO and JW were responsible for the drafting of the article and all authors revised it critically for important intellectual content and provided final approval of the version to be submitted.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tu, K., Widdifield, J., Young, J. et al. Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective. BMC Med Inform Decis Mak 15, 67 (2015). https://0-doi-org.brum.beds.ac.uk/10.1186/s12911-015-0195-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12911-015-0195-x

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us