Evaluation of medication errors with implementation of electronic health record technology in the medical intensive care unit

Introduction

Patients in the intensive care unit (ICU) have an increased risk for medication errors and adverse drug events compared with non-ICU patients due to multifactorial causes, including complex drug regimens and complicated disease states.^1–4 The medication error rate reported in the literature ranges from 1.2 to 947 per 1,000 patient days in the medical ICU (MICU).^5–7 One single-center, prospective, observational study on errors in the MICU and coronary care unit reported 78% of serious errors involved medications.⁸ Other studies have reported medication error rates between 4.5 and 16 errors per 100 medication orders.^5,9 The focus to improve patient safety and quality of health care delivery created incentives through Medicare and Medicaid in 2009 via the Health Information Technology for Economic and Clinical Health (HITECH) Act as part of the American Recovery and Reinvestment Act.¹⁰ Since 2015, eligible practices that did not have certified electronic health record (EHR) systems were subject to a 1% reduction in Medicare payments, with an increase to a maximum of up to 5% reduction in payment for each subsequent year without certified programs.¹¹ In addition, the Institute of Medicine and Leapfrog, an independent national organization that advocates to improve the quality of health care in the USA, published initiatives to endorse the implementation of EHR to improve patient care, especially computerized physician order entry (CPOE).¹² In fact, after the establishment of the HITECH Act, the percentage of hospitals adopting EHR technology increased from 12% to 44% during 2008–2012, particularly with implementation of CPOE technology, which increased 167%.^13,14

Studies evaluating EHR and CPOE implementation in the ICU on medication error rates have focused on prescribing errors and varied in regards to patient population, methodology, definition of error rate, and study period. Furthermore, administration and dispensing errors with CPOE implementation have not been evaluated. Weant et al reported that medication errors increased by 450% in the first month after CPOE implementation in neurosurgical ICU, but the number of errors that caused patient harm decreased.¹⁵ The use of CPOE without decision support in a general ICU in the UK demonstrated significant reduction in overall number of errors attributed to prescribing when compared to hand written prescribing.¹⁶ Both studies observed a change in the types of prescribing errors, with an increase in wrong doses and omission of required drugs after CPOE implementation.^9,15–20

The purpose of this study was to evaluate the impact on medication errors in the short and long term before and after a comprehensive EHR system with CPOE was implemented at a large academic medical center.

Materials and methods

Study design

This prospective, observational study was conducted at a large academic medical center in an urban setting during four study periods: August–September 2010 (preimplementation; period I), January–February 2011 (2 months postimplementation; period II), August–September 2012 (21 months postimplementation; period III), and January–February 2013 (25 months postimplementation; period IV). The study periods were selected to evaluate changes over short and long periods of time post-EHR implementation, as well as based on research staff availability. The study was reviewed by Emory University Institutional Review Board as well as the hospital Research Oversight Committee, and was exempted as a quality improvement study. Charts of patients admitted to the MICU service during the study periods were included for evaluation. If patients were admitted to the MICU service for <24 hours, or readmitted to the MICU service after being transferred or discharged off the service those records were excluded from the study.

Prior to EHR implementation, the institution utilized a paper-based system, including hand-written orders for medications, laboratory, and diagnostic tests. Although results were available electronically, all provider notes, flowsheets, and medication administration records (MAR) were paper-based. The transition to a new EHR system, including CPOE, electronic pharmacy order verification, and barcode-scanning medication administration technology occurred on October 31, 2010. After that all patient-related health information was processed and stored electronically. The institution implemented Epic Systems Corporation as its EHR software. Hospital wide training sessions were required for every health care provider prior to the initiation of the EHR with CPOE system. Inpatient physicians were required to complete 4 hours of training, pharmacists 2 days, and nurses 3 days of training. For 3 weeks during the implementation process, “super-users” were available in all clinical areas at all times.

Super-users were health care providers who underwent additional training to assist colleagues when needed to deliver effective patient care. For the purposes of this study, a chart review for each patient was conducted by a clinical pharmacist or pharmacy student during all study periods until ICU discharge or death to assess for medication errors. If a patient was transferred out of the ICU, the clinical pharmacist reviewed the transfer orders as well. Patients’ demographics included: sex, age, race, ICU admitting diagnosis, ICU length of stay, and Acute Physiology and Chronic Health Evaluation (APACHE) II scores. APACHE II is a severity of disease classification system that uses a point score based upon initial values of 12 routine physiologic measurements, age, and previous health status to provide a general measure of severity of disease. An increasing score (range 0–71) is closely correlated with the subsequent risk of hospital death.²¹ While the next versions of APACHE scoring system (III and IV) are available, at the time of the study version II was commonly used. Medication errors were categorized by definitions established by the National Coordinating Council for Medication Error Reporting and Prevention Taxonomy (Figure 1).²² An agreed upon list of organization-specific operational definitions of medication errors was developed to ensure consistency among reviewers (Table 1).

Figure 1 Definition of severity of errors. Notes: Reproduced from National Coordinating Council for Medication Error Reporting and Prevention. NCC MERP index for categorizing medication errors. Available from: http://www.nccmerp.org/. Accessed April 30, 2017. © 2001 National Coordinating Council for Medication Error Reporting and Prevention. All Rights Reserved.22

Table 1 Medication error classifications and operational definitions Notes: *Need to review documented pharmacy interventions to find these errors. Abbreviations: BP, blood pressure; MAR, medication administration records; pt, patient; OR, operating room; RN, registered nurse; PRN, pro re nata; aPTT, activated partial thromboplastin time; DTI, direct thrombin inhibitor; XL, extended release; IV, intravenous; IVPB, intravenous piggy-bag; NG, nasogastric; IM, intramuscular; MD, Doctor of Medicine; CHF, congestive heart failure; ACEI, angiotensin-converting-enzyme inhibitor; DVT, deep venous thrombosis; ABX, antibiotics; w/o, without.

Outcomes

The main outcome was the rate of medication errors before and after EHR implementation, defined as number of medication errors per 1,000 patient days. Other outcomes evaluated were the type of medication errors, the origin of the error (i.e., prescribing, dispensing or administration), and the severity of medication errors pre- and post-EHR implementation. In addition, the percent change in errors not detected and reached a patient, and errors that were discovered and resolved prior to reaching the patient (i.e., “near-misses”) were evaluated before and after the EHR implementation.

Statistics

Continuous variables (length of stay, patient age, APACHE score) were compared between the four time periods using analysis of variance tests. Chi-square tests were used to compare categorical variables between groups or in a few cases a Fisher’s exact test was used as an alternative due to small numbers. Instead of absolute numbers, the main outcome (medication errors) was compared univariately as a rate (errors per 1,000 ICU days) to account for different ICU days between time periods. The rate per 1,000 days was calculated by dividing the number of medication errors by the number of patient ICU days recorded during that period and then multiplied by 1,000. Each of the Chi-square tests compared the rate of one study period to the reference period of August–September 2010. To account for possible confounding, multivariable models compared the difference in rates between each study period compared to the reference period, controlling for APACHE score, patient age and race. Only APACHE score was found to be important, however, so the final model reported only includes the score and the study period. Negative binomial regression models were used to account for over-dispersion. p<0.05 indicated statistical significance. All data were entered in a Microsoft^© Access database and statistical analysis was performed using SAS software.

Results

A total of 673 patients were evaluated during the study periods. The median age was similar among the four study periods as well as the sex and primary ICU diagnoses (Table 2). Pulmonary conditions were the most common indication for ICU admission during all study periods. Patients in period II had a higher median APACHE II score compared to period I (24 vs 18; p<0.01), whereas there was no difference in period III (18 vs 18; p=0.939) and period IV was significantly lower (17 vs 18; p=0.013). Although there was no significant difference between periods in the overall ICU length of stay (LOS), there was an increase in median ICU LOS in period II compared to period I (5 vs 4; p=0.027).

Table 2 Patient demographics Notes: #p-value compared to first period; *p<0.05, **p<0.01. Abbreviations: APACHE II, Acute Physiology and Chronic Health Evaluation II; CNS, central nervous system; CV, cardiovascular; GI, gastrointestinal; ICU LOC, Intensive Care Unit length of stay; IQR, interquartile range.

For the primary outcome, the overall medication error rate was significantly higher in periods II and III compared to period I (p<0.0001 and p=0.0023, respectively) and significantly lower in period IV (p=0.008, Table 3). The most common origin of errors during period I was prescribing errors (44%), which significantly decreased in all post-EHR implementation study periods (39.1%, 18.5%, 18.5%, p<0.001). Interestingly, administration errors increased during all the three post-EHR implementation periods compared to the pre-EHR implementation period (Table 4). The different types of medication errors are listed in Table 5. During the pre-EHR implementation phase, the most common types of errors were MAR discrepancies (18.8%), incomplete orders (15.3%), and missed doses when the medication was available (14.1%). In the post-EHR implementation periods, these errors were significantly decreased (p<0.0001). The most common types of errors during the post-EHR implementation phases were missed doses because of not being available from pharmacy (35.3%) and medications administered at the wrong time (18.9%), defined as either 1 hour early or later than the scheduled time. Although the number of errors that reached a patient increased in all post-EHR implementation periods, the errors were not severe, did not require an intervention, and did not result in harm. The percent of errors in severity category B (“near-misses”) decreased in all post-EHR periods but increased in period IV compared to periods II and III. The percent of errors in category D and E was lower in all post-EHR implementation periods compared to period I (Table 6).

Table 3 Medication error rate Notes: #p-value compared to first period; *p<0.05, **p<0.01. Period I (Aug to Oct 2010), Period II (Jan to Feb 2011), Period III (Aug to Sept 2012), Period IV (Jan to Feb 2013). Abbreviation: ICU, intensive care unit.

Table 4 Medication errors by type Notes: Refer Table 1 for definition of errors. #p-value compared to first period; **p<0.01.

Table 5 Medication error classification Notes: Refer Table 1 for definition of errors. *p<0.05, **p<0.01. Period I (Aug to Oct 2010), Period II (Jan to Feb 2011), Period III (Aug to Sept 2012), Period IV (Jan to Feb 2013). Abbreviations: MAR, medication administration records; NA, not applicable.

Table 6 Severity of errors Notes: Refer Figure 1 for definition of severity. **p<0.01. Period I (Aug to Oct 2010), Period II (Jan to Feb 2011), Period III (Aug to Sept 2012), Period IV (Jan to Feb 2013).

The final multivariate analysis revealed that for every 1 point increase in APACHE score, the medication error rate was found to increase by 1.3% (multiplicative effect of 1.013, 95% CI of 1.007–1.020). Periods II and III had slightly higher error rates than the baseline period but these differences were not found to be statistically significant (p-values of 0.37 and 0.59, respectively). Period IV was found to be statistically different than the baseline period; the error rate was 24% lower than the baseline period (0.761, 95% CI [0.65–0.89]) controlling for APACHE II scores (Table 7). Models controlling for age and race (not shown) found those factors were not significant or important confounders and so were left out of the final model.

Table 7 Multivariate analysis: regression results@ Notes: Period II (Jan to Feb 2011), Period III (Aug to Sept 2012), Period IV (Jan to Feb 2013). Models controlling for age and race (not shown) found those factors were not significant or important confounders and so were left out of the final model. Multivariate analysis revealed that for every 1 point increase in APACHE score, the medication error rate was found to increase by 1.3% (multiplicative effect of 1.013, 95% CI of [1.007–1.020]). Period IV was found to be statistically different than the baseline period; the error rate was 24% lower than the baseline period (0.761, 95% CI [0.65–0.89]) controlling for APACHE II scores. @Based on a negative binomial regression model. $Multiplicative effect on the mean error rate. **p <0.01. Abbreviation: APACHE II, Acute Physiology and Chronic Health Evaluation II.

Discussion

Based on the results of this study, implementation of EHR revealed an increase in medication errors immediately postimplementation but, importantly, significant reduction in more severe errors. However, additional analysis controlling for severity of illness found that there was no increase in medication errors immediately postimplementation. However, there was a reduction in medication error after 2 years. This study reveals several unique findings. First, the increase in medication errors found early in the roll out period could be related to challenges related to a lack of familiarity with a new system and also the increased detectability because of electronic charting highlighting the timing of drug administration that is computerized instead of nursing charting. Second, there was decrease in prescribing errors likely related to more legible orders. Finally, there was a significant decrease in medication errors over a long period of time that has not been previously reported. During the timeframe of this study, the use of an EHR was fairly new but was becoming more widely implemented. The findings of this study are likely due to improvement in the implementation process. This study was conducted after several previous studies revealed issues with EHR implementation. Our favorable results are likely due to awareness of implementation process challenges that were addressed proactively during the system change. The fact that it took 2 years to see a significant reduction in medication errors brings to the forefront the need for a constant, multidisciplinary evaluation of EHR system to optimize its use. All providers at the institution underwent multiple hours of classroom and online training, and training was tailored to each provider type, which may have increased the effectiveness of the training sessions. The major difference from this implementation from older studies was the technical support by specific health care providers as “super users”. There were specialty trained nurses, pharmacists, and physicians who were readily available to assist each specific user in their real time needs, instead of an information technician who may not be familiar with the workflow needs. The availability of super-users who could help as needed during active clinical care may have helped to decrease provider errors and mitigate any delays in patient care associated with a new care system, while also increasing acceptance of the new EHR system. As various types of EHR are being implemented, this is a key feature that is most generalizable that will likely provide the most benefit.

The type, severity, and origin of errors post-EHR implementation decreased significantly especially in the prescribing and dispensing error rates. However, during the study period administration errors increased. These may be attributed to prebuilt drug orders and order sets that limit the prescribers’ ability to commit prescribing errors as well as barcode-scanning technology that allows for improved documentation of administration times and therefore increased detection of administration errors. The decrease in prescribing errors in this study is similar to the findings in a previously published study from the UK that demonstrated a significantly lower overall number of errors attributed to prescribing when compared to hand written orders in a general ICU population.¹⁶ This study also corroborates the findings from other studies that administration errors contribute significantly to errors in the medication use process. These studies reported that administration errors are the second most common source of errors after prescribing errors in the ICU.^23,24 In fact, 40% of all errors that reached a patient occurred during the administration stage.⁸ These findings emphasize, as George et al recognized, the role of nurses at promoting patient safety by preventing medication errors in the ICU as well as the need to focus efforts to help nurses achieve these goals.²⁵

Undoubtedly, the role of technology to minimize the risk of errors throughout the entire medication use process has been increasing and continues to evolve as emphasis is placed on quality and patient safety; however, the overall benefits when evaluating direct clinical patient outcomes, such as mortality and length of stay, remain conflicting.^26–33 This may be the result of varying study designs as there is a variability in the duration of study periods, patient population, and definition of an error, and as such the need to conduct more well-designed studies in this area is evident. The study conducted at the same institution, around the same time, demonstrated that EHR implementation was associated with reduced severity-adjusted mortality and supported the hypothesis that EHR use may improve outcomes in critically ill patients.³⁴

This study has several strengths. It was conducted over several years therefore we are able to demonstrate the evolution of errors over time related to the implementation and utilization of EHR. In addition, we evaluated the whole medication ordering process; therefore, we were able to provide a comprehensive picture of where the errors occur as well as the shifts in errors related to EHR implementation. The prospective evaluation of charts and MARs daily also allowed for the maximum detection of errors, as medication-related errors tend to be underreported.

Our study has several limitations as well. One limitation is instrumentation bias during the chart review process as much of the data collected after EHR implementation was electronic versus manual prior to EHR. In addition, we did implement a review board to adjudicate each error for severity and categorization, therefore, bias in the classification of the errors could occur, despite the development of a list of operational definitions. Finally, the study was over four discrete time periods rather than continuous data.

Although many studies conducted in adult patients in the ICU have reported that a reduction of medication errors and preventable adverse drug events occurred after implementing technology, many other factors, including human error and system failures contribute to the complexity in the medication use process that result in a medication error. Technology does not eliminate medication errors. Implementing EHR technology significantly changes the origin, types, and severity of errors in the entire medication use process.

Conclusion

This study demonstrates that implementation of EHR did not result in an increase of medication errors after accounting for severity of illness; however, a true reduction in medication errors did not happen until ~2 years later. The origin, types, and severity of errors significantly changed post-EHR implementation

Acknowledgments

The abstract of this paper was presented at the Southeaster Residency Conference (SERC) on April 28–29, 2011 as a platform presentation with interim findings. The title of the presentation was: Evaluation of Medication Errors with Implementation of Electronic Health Records in the Medical Intensive Care Unit. The presentation’s abstract was published in the Conferences Program book on April 28–29, 2011 (no citation provided). Please refer to following link for the abstract publication (p. 70): http://sercpharm.org/wp-content/uploads/2013/10/2011_Abstracts.pdf. This work was completed at Grady Health System in Atlanta, GA, USA.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Cullen DJ, Sweitzer BJ, Bates DW, Burdick E, Edmondson A, Leape LL. Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units. Crit Care Med. 1997;25(8):1289–1297.
2.		Valentin A, Capuzzo M, Guidet B, Moreno RP, Dolanski L, et al. Patient safety in intensive care: results from the multinational Sentinel Events Evaluation (SEE) study. Intensive Care Med. 2006;32(10):1591–1598. Epub 2006 Jul 28.
3.		Valentin A, Capuzzo M, Guidet B, Moreno R, Metnitz B, et al. Errors in administration of parenteral drugs in intensive care units: multinational prospective study. BMJ. 2009;338:b814.
4.		Kane-Gill SL, Jacobi J, Rothschild JM. Adverse drug events in the intensive care units: risk factors, impact, and the role of team care. Crit Care Med. 2010;38(Suppl 6): S83–S89.
5.		Rothschild JM, Landrigan CP, Cronin JW, et al. The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med. 2005;33(8):1694–1700.
6.		Bates D, Cullen D, Laird N, et al. Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995;274(1):29–43.
7.		Kane-Gill S, Weber RJ. Principles and practices of medication safety in the ICU. Crit Care Clin. 2006;22(2):273–290.
8.		Camiré E, Moyen E, Stelfox HT. Medication errors in critical care: risk factors, prevention and disclosure. CMAJ. 2009;18(9):936-943.
9.		Kopp BJ, Erstad BL, Allen ME, Theodorou AA, Priestly G. Medication errors and adverse drug events in an intensive care unit: direct observation approach for detection. Crit Care Med. 2006;34(2):415–425.
10.		US Department of Health & Human Services. Health IT Legislation and Regulations. Available at: http://www.healthit.gov/policy-researchers-implementers/hitech-act. Accessed January 9, 2015.
11.		Centers for Medicare & Medicaid Services. Electronic Health Records (EHR) Incentive Programs. Available at: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/index.html. Accessed March 15, 2017.
12.		The Leapfrog Group. Medication Safety. Available at: http://www.leapfroggroup.org/ratings-reports/medication-safety. Accessed March 15, 2017.
13.		Charles D, King J, Patel V, et al. Adoption of Electronic Health Record Systems among U.S. Non-federal Acute Care Hospitals: 2008–2012, ONC Data Brief, no 9. Washington, DC: Office of the National Coordinator for Health Information Technology. March 2013. Available from: https://www.healthit.gov/sites/default/files/oncdatabrief9final.pdf. Accessed March 15, 2017.
14.		Charles D, King J, Furukawa MF, et al. Hospital Adoption of Electronic Health Record Technology to Meet Meaningful Use Objectives: 2008–2012, ONC Data Brief, no. 10. Washington, DC: Office of the National Coordinator for Health Information Technology. March 2013. Available from: https://www.healthit.gov/sites/default/files/oncdatabrief10final.pdf. Accessed March 15, 2017.
15.		Weant KA, Cook AM, Armitstead JA. Medication-error reporting and pharmacy resident experience during implementation of computerized prescriber order entry. Am J Health Syst Pharm. 2007;64(5):526–530.
16.		Shulman R, Singer M, Goldstone J, Bellingan G. Medication errors: a prospective cohort study of hand-written and computerised physician order entry in the intensive care unit. Crit Care. 2005;9(5):R516–R521.
17.		Wetterneck TB, Walker JM, Blosky MA et al. Factors contributing to an increase in duplicate medication order errors after CPOE implementation. J Am Med Inform Assoc. 2011;18(6):774–782.
18.		Armada ER, Villamañán E, López-de-Sá E, et al. Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. J Crit Care. 2014;29(2):188-193.
19.		Maslove DM, Rizk N, Lowe HJ. Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med. 2011;26(3):165–171.
20.		Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care. 2006;10(1):R21.
21.		Knaus WA, Draper EA, Wagner DP. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829.
22.		National Coordinating Council for Medication Error Reporting and Prevention. NCC MERP index for categorizing medication errors. Available from: http://www.nccmerp.org/. Accessed April 30, 2017.
23.		Tissot E, Cornette C, Demoly P, Jacquet M, Barale F, Capellier G. Medication errors at the administration stage in an intensive care unit. Intensive Care Med. 1999;25(4):353–359.
24.		Benkirane R, R-Abouqal R, Hamieur CC, et al. Incidence of adverse drug events and medication errors in intensive care units: a prospective multicenter study. J Patient Saf. 2009;5(1):16–22.
25.		George EL, Henneman EA, Tasota FJ. Nursing implications for prevention of adverse drug events in the intensive care unit. Crit Care Med. 2010;38(Suppl 6):S136–144.
26.		McDowell SE, Ferner HS, Ferner RE. The pathophysiology of medication errors: how and where they arise. Br J Clin Pharmacol. 2009;67(6):605–613.
27.		Keene A, Ashton L, Shure D, Napoleone D, Katyal C, Bellin E. Mortality before and after initiation of a computerized physician order entry system in a critically ill pediatric population. Pediatr Crit Care Med. 2007;8(3):268–271.
28.		Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116(6):1506–1512.
29.		Zikos D, Diomidous M, Mpletsa V. The effect of an electronic documentation system on the trauma patient’s length of stay in an emergency department. J Emerg Nurs. 2014;40(5):469–475.
30.		Spalding SC, Mayer PH, Ginde AA, Lowenstein SR, Yaron M. Impact of computerized physician order entry on ED patient length of stay. Am J Emerg Med. 2011;29(2):207–211.
31.		Longhurst CA, Parast L, Sandborg CI, et al. Decrease in hospital-wide mortality rate after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2010;126(1):14–21.
32.		Brunette DD, Tersteeg J, Brown N, et al. Implementation of computerized physician order entry for critical patients in an academic emergency department is not associated with a change in mortality rate. West J Emerg Med. 2013;14(2):114–120.
33.		Del Beccaro MA, Jeffries HE, Eisenberg MA, Harry ED. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics. 2006;118(1):290–295
34.		Han JE, Rabinovich M, Abraham P et al. Effect of electronic health record implementation in critical care on survival and medication errors. Am J Med Sci. 2016;351(6):576–581.