http://orcid.org/0000-0001-8455-8332
Figures
Abstract
Background
The study objectives were to describe the incidence and the nature of patient safety incidents (PSIs) in primary care general practice settings, and to explore the association between these incidents and practice or organizational characteristics.
Methods
GPs, randomly selected from a national influenza surveillance network (n = 800) across France, prospectively reported any incidents observed each day over a one-week period between May and July 2013. An incident was an event or circumstance that could have resulted, or did result, in harm to a patient, which the GP would not wish to recur. Primary outcome was the incidence of PSIs which was determined by counting reports per total number of patient encounters. Reports were categorized using existing taxonomies. The association with practice and organizational characteristics was calculated using a negative binomial regression model.
Results
127 GPs (participation rate 79%) reported 317 incidents of which 270 were deemed to be a posteriori judged preventable, among 12,348 encounters. 77% had no consequences for the patient. The incidence of reported PSIs was 26 per 1000 patient encounters per week (95% CI [23‰ -28‰]). Incidents were three times more frequently related to the organization of healthcare than to knowledge and skills of health professionals, and especially to the workflow in the GPs’ offices and to the communication between providers and with patients. Among GP characteristics, three were related with an increased incidence in the final multivariable model: length of consultation higher than 15 minutes, method of receiving radiological results (by fax compared to paper or email), and being in a multidisciplinary clinic compared with sole practitioners.
Conclusions
Patient safety incidents (PSIs) occurred in mean once every two days in the sampled GPs and 2% of them were associated with a definite possibility for harm. Studying the association between organizational features of general practices and PSIs remains a major challenge and one of the most important issues for safety in primary care.
Citation: Michel P, Brami J, Chanelière M, Kret M, Mosnier A, Dupie I, et al. (2017) Patient safety incidents are common in primary care: A national prospective active incident reporting survey. PLoS ONE 12(2): e0165455. https://doi.org/10.1371/journal.pone.0165455
Editor: Nguyen Tien Huy, Institute of Tropical Medicine (NEKKEN), Nagasaki University, JAPAN
Received: January 21, 2016; Accepted: January 17, 2017; Published: February 14, 2017
Copyright: © 2017 Michel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The funds were provided by the Direction Générale de l'Offre de Soins, from the French Ministry of Health. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Adverse event surveys, set up in hospital settings in developed and developing countries using large representative samples [1,2], have showed convergent results: around 10% of the inpatients experiment an adverse event during their stay and that rate does not appear to decrease with time, apart from a recent exception in the Netherlands [3]. In primary care, studies of PSIs, less numerous than in hospitals, have been set up in the USA [4–7], England [8,9], Australia [10], Germany [11], Canada [12], Israel [13], the Netherlands [14] or France [15,16]. They provide heterogeneous results [17] for a number of reasons. The national and international public health and patient safety agenda is still largely focused on hospitals [18] [19]. Secondly, the concept of harm is heterogeneous because the level of harm caused by PSIs in primary care is less severe than within hospitals, and several harm definitions have been used. In addition, the organizational structure of primary health care in many countries (including France) is one where each primary care practice (and especially general practice) works as an independent business with its own organizational culture and dynamic. Finally, a limited amount of information held in practice systems hinders the use of the medical record review method in primary care [20, 21]. In France, no national representative data on patient safety in primary care was available despite the number of encounters with patients being much higher in primary care than in hospitals (one million GPs consultations per day compared with approximately 400,000 hospital inpatients per day [22]).
In 2013, the first national program for patient safety was launched with four components (engaging and empowering patients for patient safety; training and educating for patient safety; generalizing reporting and learning systems of adverse events; and developing research on patient safety). The French Ministry of Health immediately mandated a group of researchers to undertake a national survey with the aim of measuring PSIs in the primary care general practice setting, in order to have an initial evaluation of patient safety in primary care. The aims were to investigate the nature of reported PSIs, their frequency of being reported and the association between the incidence of reported PSIs and characteristics of the participating GPs’ practice and organization.
Materials and methods
The research team was an expert group of eight academic GPs with a special interest in patient safety and of two epidemiologists. The ESPRIT study (Etude nationale en Soins PRImaires sur les événemenTs indésirables—national primary care study on PSIs) was an incidence study with PSIs collection by GPs participating in an influenza surveillance network (GROG Network). The study monitored incidents reported on a voluntary and anonymised basis in a random sample of general practices [23].
Definition of incident
An incident was defined following a two-stage process, including focus groups and consensus among primary care experts and GPs’ representatives, as "an event or circumstance that could have resulted, or did result, in harm to a patient, and which should not be repeated again”. This process has been previously described [24]. The definition of PSI is similar to that used previously by other research in primary care, including the WHO [25], the US Applied Strategies for Improving Patient Safety (ASIPS) [6], and an international pilot study of primary care error reporting involving six countries [26]. This definition encourages reporters to focus on a posteriori judged preventable incidents that may have an association with patient harm.
In order to classify reported incidents, two taxonomies were selected by the primary care research team, being the TAPS (Threats to Australian Patient Safety Study) version of the International Taxonomy of Medical Error in Primary Care [27], which provides a reproducible and internationally recognized codification structured around the major categories of systems versus knowledge based incidents, and the Tempos classification [28], for its current use in French general practice. This classification describes events in terms of (unsafe) dynamic control of multiple parallel constraints by the GP and is based on a framework integrating five time scales termed 'tempos' requiring parallel tasking and processing by GPs: the disease's tempo (unexpected rapid evolutions, slow reaction to treatment); the office's tempo (day-to-day agenda and interruptions); the patient's tempo (time to express symptoms, compliance, emotion); the system’s tempo (time for appointments, exams, and feedback); and the time to access to knowledge (the doctor’s tempo).
Sampling
The population was a nationwide network of over 800 primary care physicians from all over mainland France (S1 appendix). They are all general practitioners in activity, and had volunteered to report cases during winter 2013 as members of a sentinel network for influenza surveillance. The sample was constituted of GPs randomly selected from these physicians. They were then invited by telephone to participate in the study. Based on a UK study that had used a similar method, we calculated that 2665 patient encounters was required to measure an estimated 75 PSIs per thousand encounters with a precision of 10 per thousand. [22]. As the average number of encounters per day was estimated to be 20, and the average number of working days was 4.3 days per week, we needed a minimum of 31 GPs. As we anticipated a 50% response rate amongst invited GPs, our conservative estimate was that a minimum sample of 100 physicians would be needed. We sampled 120 GPs, adjusted by gender to reflect the national distribution (40% female and 60% male). If a GP declined to participate or did not respond after six calls, the next GP on the list was called.
The participant GPs were trained on the protocol and the data collection method via a written procedure posted by mail. They were specifically trained to the definition of patient safety incident (PSI): the network coordinator organized individual or collective sessions by phone with each of the participating doctors. A video was available [23]. The main objective was to reach a shared understanding on what constituted an incident, as defined in the previous paragraph.
Data collection and tools
Over the data collection period, participant GPs completed a daily incident reporting form on a web based tool, which was developed and tested by the research group based on an extensive literature review. Both content and face validity were determined by piloting the reporting form with physicians from the network. The reporting form contained 25 questions of which 10 were open-ended questions for the description of the incidents, their contributing factors and the consequences (no noticeable clinical consequence, temporary or reversible harm, definite possibility for harm), both for the patient and for the GPs office organization (S2 Appendix). In cases where multiple incidents were observed by a GP for the same patient encounter, the GP was asked to report on the PSI that they felt had the most potential for severe harm. Each participant GP chose a specific week within the six week data collection period in May-June 2013 and completed the forms during their working hours. Participating GPs completed a questionnaire detailing their daily count of patient encounters, in terms of office consultations and visits at home or in nursing homes.
Two other questionnaires were completed by the GPs: (i) a GP profile (socio-demographic and professional characteristics–number of continuous professional development sessions followed last year, duration of an encounter, number of encounters in a week) and (ii) a questionnaire on the organization of the office (appointment slots for emergency presentations, existence and type of secretary, formalized protocols for emergent calls, computerized records, electronic prescribing software, decision support tool for management of chronic conditions, use of prescription software alarm, method of receiving laboratory results, method of receiving radiological results, digital archiving processes, resident supervision, and type of exercise).
A secured internet gateway was used to conduct the survey.
Quality control
During electronic data entry, most of the fields were mandatory to complete in order to move to the next screen, and this avoided information being missed. Real-time checks for coherence of the data entry were conducted (coherence of dates, missing information). A weekly review of all incident reports was undertaken by one member of the expert group. Information that was missing and could potentially be useful for further analysis was determined, and GPs were called back to provide missing detail when needed.
At the end of the data collection, a two-day meeting was organized for the expert group and each incident was reviewed by five pairs of experts, in order to: (i) validate their inclusion (conformity with the definition), (ii) score the preventability (would it not have occurred if the patient had received ordinary standards of care, appropriate at the time of the study), (iii) sort them into the two classifications (TAPS taxonomy and Tempos). In cases of discrepancy, the two reviewers tried to reach a consensus and, failing that, the entire expert group discussed the report in order to assign a consensus classification.
Reproducibility analysis
The Kappa coefficient (κ) was calculated within each pair of experts for four variables: validation of reports as incidents according to the definition, preventability, classification according to TAPS taxonomy [26] (at the level 3, most detailed descriptor) and to the Tempo’s method (principal tempo among the five defined above) [28].
Data analysis
Incidence of reported PSIs was defined as the ratio of the number of encounters with at least one PSI and the total number of encounters included during the study period. The frequency of incidents and of a posteriori judged preventable incidents and the frequency by type of incident were calculated.
The association between the number of incidents and the organizational characteristics was based on a negative binomial regression model, which can be used to model count data with over-dispersion [29]. The deviance test was calculated at each step of the Poisson model in order to verify that over-dispersion was present: if present, the negative binomial regression was used. The number of encounters was an offset variable. Dependent variables were all variables from the organizational questionnaire and the practice characteristics of the physicians (supervision of residents and isolated or grouped practice). Discrete variables were used for continuous variables (length of GPs consultations, annual number of continuous professional development full-day and late-afternoon sessions), because the linear assumption was not met. Modeling was done according to the Hosmer-Lemeshow procedure, by stepwise increments with a threshold significance of 0.25 in univariable analysis for inclusion of variables, and stepwise decrements with a threshold significance of 0.05 for exclusion of variables in multivariable analysis.
Results
GPs main characteristics
Out of the 160 GPs who were contacted, 27 (17%) refused to participate (Fig 1). Subsequently, out of the 133 included in the study, six (5%) withdrew during the data collection. Thus, 127 GPs participated (79%) for a total of 649 days; 51 (40%) were female and 76 male, working in 19 out of the 22 French regions. The selected GPs mean age was 54 years, 3 years older than that of the French GPs population; they worked more frequently in a group (60% versus 48% respectively) and in rural areas (17% versus 15% respectively); 60% of the respondents were in charge of supervising residents, which is at odds with the proportion of French GPs population in charge of residents (only 15%) (Fig 1).
Incidence, type and consequences of incidents
Over the reporting period, there was a total of 12,348 encounters (office visits and home visits), and 694 reports made by participant GPs. Almost half of these were ruled out as fitting the study definition of a patient safety incident, mainly because they were problems that could not have resulted in harm to the patient. Therefore a total of 317 PSIs were reported, of which 270 were determined to be preventable. The overall incidence was 26 PSIs per 1000 encounters per week [CI 23‰; 28‰], and 22 a posteriori judged preventable PSIs per 1000 encounters per week [CI 19‰; 24‰]. As the participant GPs had a mean of 20 encounters per day, this corresponded to approximately one PSI being reported every two days for these GPs. The frequency of PSIs related to consultations undertaken outside of the practice and to consultations occurring at the practice were similar (Table 1).
Out of 317 incidents, 244 (77%) had no noticeable clinical consequence for the patient at the time of data collection; in 21% there was a temporary or reversible harm (for example urinary tract infection in relation to a delay in the treatment; diabetes fainting in relation to delay in test results management) and in 2% a definite possibility for harm (S3 Appendix).
According to classification using the TAPS taxonomy, 204 a posteriori judged preventable incidents (76%) were errors relating to the processes of healthcare (type 1) and 66 (24%) were deficiencies in the knowledge and skills of health professionals (Table 2).
A principal tempo was defined in 256 out of the 270 a posteriori judged preventable incidents: 44% related to the office's tempo, 24% to the doctor tempo, 18% to the system tempo, 10% to the patient tempo and 3% to the disease tempo (Table 3).
The Kappa coefficient between the two experts undertaking the incident validation was 0.54 (25% positive match; 70% consensus after initial disagreement and 5% were arrived at with consultation of the expert group). The concordance was moderate. The reproducibility of preventability assessment was 0.53, 0.45, 0.49 and 0.74 of classification according to TAPS taxonomy at the most detailed level and according to each of the five tempos, respectively.
Organizational characteristics of GPs offices
Forty percent of the GPs had isolated solo practice, 56% were in group practices, and 4% worked in multi-disciplinary offices. The median number of encounters (office and home visits) during the week was 91 [first and third quartiles (Q1-Q3): 70–119]. The median length of office visits was 17 minutes [Q1-Q3: 15–20 minutes]. Sixty-three percent of GPs were in practices where consultation appointments were not made (patients walked into the clinic and waited for the next available doctor).
The number of continuous professional development sessions undertaken by reporting GPs per year was described in terms of the number of full-day sessions (median 4, Q1-Q3: 1–6) and of late-afternoon (3 hours) sessions (median 8, Q1-Q3: 3–10).
Seventy-nine percent of the GPs had a secretary; either located at the GP’s office (69%), or remotely located via telephone (23%), or both (8%). Among the offices that had a secretary, 79% had formal protocols for emergency presentations. Ninety-six percent used a computerized medical record, 75% systematically used electronic prescribing software with decision support, and 58% used electronic decision support tools for the management of chronic and complex conditions. Thirty-three percent systematically disabled their prescription software alarms. The GPs received pathology results by secure email (71%), postal mail (25%) or by fax (4%), and radiology results by postal mail (72%), by email (26%) and by fax (2%). Sixty one percent of doctors used a digital archive containing their entire records, and 21% with partial records (Table 4).
Links between organizational characteristics and frequency of incidents
The median count of reported PSIs during the week of data collection was three, ranging from zero to 19 (Q1-Q3: 1–5).
During the univariable analysis, five variables were found to be statistically associated with the PSIs incidence with a threshold significance of 0.25 (Table 4). During multivariable analysis, two variables were excluded because they were found to be not statistically significant: number of continuous professional development full-day session attendance and resident supervision. The final model contained three variables (Table 5). The relative risk of reporting a PSI when receiving laboratory results by fax was 2.71 (p<0.02) compared with by paper or email. For GPs in a multidisciplinary primary care clinic compared with sole practitioners, the relative risk of reporting a PSI was 2.26 (p = 0.03). An increase of one minute per patient encounter resulted in a relative risk of reporting a PSI of 1.05 (p = 0.02).
Discussion
Key results
GPs reported 26 PSIs per 1000 encounters or a mean of one PSI every two days in their practice with a large variability between GPs. Incidents were three times more frequently related to the organization of healthcare than to knowledge and skills of health professionals, and especially to the workflow in the GPs office and to the communication between providers and patients. Their very diverse nature is illustrated by examples in S3 Appendix.
Limitations
More than half of the 694 incidents reported by GPs were deemed not to be PSIs according to the study’s definition after review by the expert group. This suggests that the level of understanding of the definition of PSI by participant GPs was not high, despite the time and effort that was spent in preparation of the study participants before with patient safety concepts (focus groups, video recording, e-learning). This issue has been described in previous studies [30], and should be given particular attention by researchers who try to set up a primary care reporting system for PSIs.
There was a large proportion of PSIs reported (244 out of 317, i.e., 77%) which did not have long-term consequences for the patient, as found in other primary care research [14,31]. These incidents are less threatening as they present less medico-legal risk for participants to report than incidents which caused harm to the patient. In addition, the most frequent type of PSI found in claims-based studies, delayed diagnoses, is missed: they usually escape the short term and are difficult to identify by GPs in their daily practice [7, 15]. Hence, it is worth asking whether in fact more serious PSIs in terms of patient harm were under-represented.
There was an unexpectedly large range of reported PSIs per participant. The number of incident reports made by individual GPs varied between 0 and 19 reports, despite the methods of collection being identical and the sample of doctors chosen from a population of GPs who were all familiar with data collection tasks. This may reflect different levels of awareness, a factor that may explain two of the three associations found in this study between organizational factors and level of reporting.
The kappa statistics of the review process for PSIs and preventability assessment were modest. To our knowledge, there is no data in the literature on the reproducibility of such process regarding PSIs in primary care but a large literature for adverse event (AE) detection with record review in hospital settings. The latest review showed a moderate inter-rater reliability for AE assessment (κ = 0.40–0.57) in 10 studies, although the inter-rater reliability was substantial (κ = 0.61–0.80) in four others; the κ for all studies assessing preventable AEs ranged from 0.19 to 0.76. All methods use an implicit review style, meaning that the AE assessment relies on expert judgment instead of using well defined criteria on a checklist [32]. Our results on the reproducibility of the classification into the two taxonomies were similar to the previous studies: the concordance amongst the coders in the validation study of the TAPS taxonomy was 0.66 [27], and the one in the Brami’s tempo study was 0.68 [28].
A limitation of our study findings is that we did not investigate patient factors that were associated with PSIs. Previous literature investigating adverse events in primary care from a large UK study has shown that factors such as older patient age and an increased number of co-morbidities has been associated with an increased risk of adverse events, and being known for longer by a practice with a reduced risk [33]. As we did not collect this information, we are unable to comment on patient factors that may have contributed to PSIs in our data.
Generalizability
The study participants were not necessarily representative of French GPs as they were already involved in an influenza surveillance network to report incidents. Studies using random samples of GPs in France are frequently impaired by low participation rates; usually less than 50% agree to participate [34]. We thought that a sample of randomly-chosen physicians within a population that was potentially biased towards reporting was preferable to a sample from the whole GP population in order to improve the likelihood of having PSIs reported. Moreover, the quality of data collection was key as we needed sufficient information on the PSIs, their causes and consequences for the secondary assessment of the PSIs by experts. The population used for our sample may have contributed to an optimization of the quality of data collection, without this necessarily affecting the results: there is no indication that GPs engaged in flu surveillance are any more aware of practice and organizational issues in relation to patient safety. On the other hand, we cannot rule out that this sample of GPs was more concerned by collective and organizational issues, including patient safety. We provided a strong supportive infrastructure with training, online entry, real-time coherence tests during data entry and weekly assessment of the reports by an expert. Other research has recently been undertaken that tries to improve on these methodological issues, however the reporting frequency was not increased: despite a supportive infrastructure, 124 voluntary respondents only made 264 reports in 36 months (<1 report per person per year) [32]. Therefore we reduced the data collection period to a brief time period (a week).
This selection bias being discussed, we believe that the generalizability is acceptable, as the results are based on over 12,000 observations from a randomly selected sample with over 100 clusters and a robust data collection process.
Interpretation
By extrapolation, our results suggests that in the fiscal year 2013 (one million GPs encounters/day), between 23,000 and 28,000 PSIs occurred, of which between 18,000 and 21,500 were without long-term consequence for the patient. This is a total of between 19,000 and 24,000 a posteriori judged preventable incidents per year. But in terms of disability or death, 520 (2%) could have serious consequences.
These results are in the same range as the ones of another large observational French study [16] (ECOGEN), conducted in 2012, which is however not representative of the situation in France. In this study, the frequency of AE was estimated at one per GP per day. The incidence was also in the range of the other international large-scale, prospective study based on a similar incident definition [6, 9, 12, 35, 36, 37, 38, 39]. With regard to the tempos classification, our results were different from those that were previously published [28], which were extracted from patients' insurance claims.
The organizational factors in Primary Care may be key to explain the variations found in the frequency of reported PSIs per participant. Longer consultation lengths may be interpreted in different ways: GPs are more likely to detect incidents when they take more time. GPs with an older patient clientele, with multi-morbidity and polypharmacy, may spend more time with their patients. GPs with longer consultation lengths presumably deal with more care processes and therefore experience difficulties (prioritization challenges, memory or attention problems, etc.) which can ultimately result in patient safety problems [12,40]. Lastly, GPs facing organizational problems may spend more time to solve it. Regarding the factor “type of exercise”, there is evidence in the primary care literature supporting the inclusion of multidisciplinary teams to improve the disclosure of adverse events [41]. Any hypothesis for explaining the third factor (receiving laboratory results by fax compared with by paper or email) would be hazardous. This association may be related to a third factor that was not measured, or is related with multiple testing. In addition, the number of physician receiving laboratory results by fax was low (5), reflecting a rare situation which will in addition disappear in the next years.
These findings confirm that the use of PSI reports is insufficient to determine the true underlying proportions of different types of patient safety threats occurring at a population level in Primary Care [42, 43]. Yet, to date, incident reporting remains the most common method; actually the only one affordable at reasonable cost, for counting and classifying PSIs in primary care settings [44]. Record review is not feasible and less appropriate because the information system is not mature and because primary care does not operate in time-limited events like a hospital admission and records may be dispersed across many provider. One potential way forward to improve the knowledge content of studies based on reporting systems would be to undertake more qualitative work on the data [45].
There is a growing call for patient safety incident reporting to move from being a research tool to being embedded in Primary Care health systems. Zwart et al explored the feasibility of a local incident reporting system as a tool for safety management and organizational learning in five primary healthcare centers: they showed a greater uptake of incident reporting by primary care clinicians when reports were dealt with by local groups of ‘experts’ and the local clinicians were involved in designing improvement efforts [36,46]. The issue of increasing PSI reporting is challenging because of the considerable fragmentation of the workforce and the far less structure in its organization compared to hospitals, compounded by a lack of clear governance and patient safety leadership [47]. In France, multidisciplinary mortality and morbidity review meetings have commenced in Primary Care [48]. This remains to be evaluated, along with the role of PSI reporting systems in increasing safety culture and behavior changes and ultimately making healthcare safer for patients outside of the hospital setting.
Conclusions
The sheer size of patients passing through our primary care systems on a daily basis makes the absolute numbers of patients being avoidably harmed a highly significant issue for improvement in this health care setting. Studying the association between organizational features of GPs and PSIs remains a major challenge. However this is probably one of the most important issues for safety in primary care.
Supporting information
S1 Appendix. Brief description of the French health care system.
https://doi.org/10.1371/journal.pone.0165455.s002
(DOCX)
S2 Appendix. Patient safety incident questionnaire.
https://doi.org/10.1371/journal.pone.0165455.s003
(DOCX)
S3 Appendix. Summary examples of patient safety incident reports.
https://doi.org/10.1371/journal.pone.0165455.s004
(DOCX)
Acknowledgments
We would like to warmly thank all 127 general practitioners who dedicated their time to participate in this research.
Author Contributions
- Conceptualization: PM JLQ MM AM.
- Data curation: PM MK.
- Formal analysis: MK.
- Funding acquisition: PM.
- Investigation: PM JB MC AM ID AHC MKG CM FV.
- Methodology: PM AM JLQ MM.
- Project administration: PM.
- Supervision: PM.
- Validation: PM AM.
- Visualization: PM JB.
- Writing – original draft: PM JB MK.
- Writing – review & editing: MM.
References
- 1. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008;17: 216–23 pmid:18519629
- 2. Wilson RM, Michel P, Olsen S, Gibberd W, Vincent C, El-Assady R et al. Patient Safety in Developing Countries: Estimating the scale and nature of harm to hospitalised patients. BMJ. 2012 Mar 13;344:e832. pmid:22416061
- 3. Baines RJ, Langelaan M, de Bruijne MC, Spreeuwenberg P, Wagner C. How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time BMJ Qual Saf. 2015 Sep;24(9): 561–71. pmid:26150548
- 4. Phillips RL, Dovey SM, Graham D, Elder NC, Hickner JM. Learning from different lenses: reports of medical errors in primary care by clinicians, staff, and patients: a project of the American Academy of Family Physicians National Research Network. J Patient Saf. 2006;2: 140–6
- 5. Elder NC, Vonder Meulen M, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med. 2004;2: 125–9. pmid:15083851
- 6. Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: a report from the ASIPS collaborative. Ann Fam Med. 2004;2: 327–32. pmid:15335131
- 7. Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med. 2006;145: 488–96. pmid:17015866
- 8. Kostopoulou O, Delaney B. Confidential reporting of patient safety events in primary care: results from a multilevel classification of cognitive and system factors. Qual Saf Health Care. 2007;16: 95–100 pmid:17403753
- 9. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003;12: 443–7. pmid:14645760
- 10. Makeham MAB, Kidd MR, Saltman DC, Mira M, Bridges-Webb C, Cooper C et al. The Threats to Australian Patient Safety (TAPS) study: incidence of reported errors in general practice. Med J Aust. 2006;185: 95–8. pmid:16842067
- 11. Hoffmann B, Beyer M, Rohe J Gensichen J, Gerlach FM. “Every error counts”: a web-based incident reporting and learning system for general practice. Qual Saf Health Care. 2008;17: 307–12 pmid:18678731
- 12. O’Beirne M, Sterling PD, Zwicker K, Hebert P, Norton PG. Safety incidents in family medicine. BMJ Qual Saf. 2011;20: 1005–1010. pmid:21893612
- 13. Wilf-Miron R, Lewenhoff I, Benyamini Z, Aviram A. From aviation to medicine: applying concepts of aviation safety to risk management in ambulatory care. Qual Saf Health Care. 2003;12: 35–9 pmid:12571343
- 14. Gaal S, Verstappen W, Wolters R, Lankveld H, van Weel C, Wensing M. Prevalence and consequences of patient safety incidents in general practice in the Netherlands: a retrospective medical record review study. Implement Sci. 2011;6: 37. pmid:21470418
- 15. Amalberti R, Bons-Letouzey C, Sicot C. [Patient safety threats in primary care: a still largely unknow area]. Responsabilité. 2009;8: 5–12
- 16. Letrilliart L, Supper I, Schuers M, Darmon D, Boulet P, Favre M et al. [ECOGEN study: reasons for encounter and consultation results in primary care]. Exercer. 2014;114: 148–57
- 17. Panesar SS, deSilva D, Carson-Stevens A, Cresswell KM, Salvilla SA, Slight SP etal. A systematic review. BMJ Qual Saf. 2015 Dec 29.
- 18.
Institute of Medicine. To err is human: building a safer health system. Washington DC: National Academies Press; 2000.
- 19.
Expert Group on Learning from Adverse Events in the NHS. An organization with a memory [Internet]. London: Stationery Office; 2000 [cited 2016 April 20]. http://www.aagbi.org/sites/default/files/An%20organisation%20with%20a%20memory.pdf.
- 20.
Esmail A. Measuring and monitoring safety: a primary care perspective. London: The Health Foundation; 2013 [cited 2016 April 20]. http://www.health.org.uk/publications/measuring-and-monitoring-safety-a-primary-care-perspective/#.
- 21. Wachter RM. Is Ambulatory Patient Safety Just Like Hospital Safety, Only without the “Stat”? Ann Intern Med 2006;145: 547–9 pmid:17015874
- 22.
French social security. Social Security Financing Act 2015. Paris: French social security; 2015 [cited 2016 April 20]. http://lertim.timone.univ-mrs.fr/Ecoles/infoSante/2013/supports_ppt/Lundi_AM/Corte-DPolton.pdf.
- 23. Michel P, Mosnier A, Kret M, Chanelière M, Dupie I, Haeringer-Cholet A et al. [ESPRIT 2013: a French epidemiological survey on medical errors associated to primary care]. Bulletin Epidémiologique Hebdomadaire. 2014;24–25:410–16. Available: http://www.invs.sante.fr/beh/2014/24-25/pdf/2014_24-25.pdf
- 24. Keriel-Gascou M, Brami J, Chaneliere M, Haeringer-Cholet A, Larrieu C, Villebrun F et al. [Which definition and taxonomy of incident to use for a French reporting system in primary care settings?]. Rev Epidemiol Sante Publique. 2014;62: 41–52. pmid:24439084
- 25.
World Health Organization. The conceptual framework for the International Classification for Patient Safety Version 1.1 [internet]. Geneva: World Health organization; 2009 [cited 2016 April 20]. http://www.who.int/patientsafety/implementation/taxonomy/icps_technical_report_en.pdf.
- 26. Makeham MAB, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Austr. 2002;177: 68–72
- 27. Makeham MAB, Stromer S, Bridges-Webb C, Mira M, Saltman DC, Cooper C et al. Patient safety events reported in general practice: a taxonomy. Qual Saf Health Care. 2008;17; 53–57 pmid:18245220
- 28. Amalberti R, Brami J. 'Tempos' management in primary care: a key factor for classifying adverse events, and improving quality and safety. BMJ Qual Saf. 2012;21: 729–36. pmid:22927486
- 29.
Hosmer D. W., Lemeshow S. Applied logistic regression. 2nd edition. John Wiley and Sons 2000
- 30. Sarkar U. Tip of the iceberg: patient safety incidents in primary care. BMJ Qual Saf. 2015(0): 1–3.
- 31. van Dulmen SA, Tacken MA, Staal JB, Gaal S, Wensing M, Nijhuis-van der Sanden MW. Patient safety in primary allied health care: what can we learn from incidents in a Dutch exploratory cohort study? Med Care. Dec 2011;49(12): 1089–1096. pmid:22002643
- 32. Hanskamp-Sebregts M, Zegers M, Vincent C, van Gurp PJ, de Vet HCW, Wollersheim H. Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review. BMJ Open 2016;6:e011078. pmid:27550650
- 33. Tsang C, Bottle A, Majeed A, Aylin P. Adverse events recorded in English primary care: Observational study using the General Practice Research Database. Br J Gen Pract 2013;63: e534–e542. pmid:23972194
- 34. Gay B, Couffinhal T, Saillour-Glénisson F, Sevin F, Kret M, Michel P. [Clinical features, care pathways and behaviors of patients with high level of cardiovascular risk: transversal study in south-west region of France]. Presse Med. 2013;42: e400–8 pmid:23958209
- 35. Zwart DLM, Heddema WS, Vermeulen MI, van Rensen EL, Verheij TJ, Kalkman CJ. Lessons learnt from incidents reported by postgraduate trainees in Dutch general practice. A prospective cohort study. BMJ Qual Saf. 2011;20: 857–62. pmid:21602562
- 36. Zwart DLM, Steerneman AHM, van Rensen ELJ, Kalkman CJ, Verheij TJ. Feasibility of centre-based incident reporting in primary healthcare: the SPIEGEL study. BMJ Qual Saf. 2011;20: 121–7; pmid:21209136
- 37. Harmsen M, Gaal S, van Dulmen S, de Feijter E, Giesen P, Jacobs A et al. Patient safety in Dutch primary care: a study protocol. Implement Sci. 2010;5: 50 pmid:20584268
- 38. Shaw R, Drever F, Hughes H, Osborn S, Williams S. Adverse events and near miss reporting in the NHS. Qual Saf Health Care. 2005;14: 279–83. pmid:16076793
- 39. Rosser W, Dovey S, Bordman R, White D, Crighton E, Drummond N. Medical errors in primary care: results of an international study of family practice. Can Fam Physician. 2005;51: 386–7. pmid:16926931
- 40. Dwamena F, Holmes-Rovner M, Gaulden CM, Jorgenson S, Sadigh G, Sikorskii A et al. Interventions for providers to promote a patient-centred approach in clinical consultations. Cochrane Database Syst Rev 2012;12: CD003267 pmid:23235595
- 41. O'Connor E, Coates HM, Yardley IE, Wu AW. Disclosure of patient safety incidents: a comprehensive review. Int J Qual Health Care. Oct 2010;22: 371–379. pmid:20709703
- 42. Panagioti M, Giles S, Hernan A, Cheraghi-Sohi S Lawton R. Contributory factors to patient safety incidents in primary care: protocol for a systematic review. Systematic Reviews. 2015;4: 63. pmid:25947009
- 43. Dovey SM, Wallis KA. Incident reporting in primary care: epidemiology or culture change? BMJ Qual Saf. 2011;20: 1001–1003 pmid:22016378
- 44.
Makeham M, Dovey S, Runciman W, Larizgoitia I, on behalf of the Methods & Measures Working Group of the WHO World Alliance for Patient Safety. WHO World Alliance for Patient Safety. Methods and measures used in primary care patient safety research: results of a literature review [internet]. Geneva: World Health Organization, 2007 [cited 2016 April 20]. http://www.who.int/patientsafety/research/methods_measures/makeham_dovey_full.pdf
- 45. Rees P, Edwards A, Panesar S, Powell C, Carter B, Williams H et al. Safety Incidents in the Primary Care Office Setting. Pediatrics. 2015;135: 1027–35 pmid:25941305
- 46. Zwart DLM, Van Rensen ELJ, Kalkman CJ, Verheij TJ. Central or local incident reporting? A comparative study. Br J Gen Pract. 2011;61: 183–187
- 47. Gandhi TK, Lee TH. Patient Safety beyond the Hospital. N Engl J Med. 363;11: 1001–3 pmid:20825311
- 48. Chaneliere M, Comte C, Keriel-Gascou M, Siranyan V, Colin C. [Morbidity and mortality conference in general practice: about an experiment in Rhône-Alpes]. Presse Med. 2013;42: e363–70 pmid:23958210