Figures
Abstract
Background
The Emergency, Triage, Assessment and Treatment plus Admission care (ETAT+) course, a comprehensive advanced pediatric life support course, was introduced in Rwanda in 2010 to facilitate the achievement of the fourth Millennium Development Goal. The impact of the course on improving healthcare workers (HCWs) knowledge and practical skills related to providing emergency care to severely ill newborns and children in Rwanda has not been studied.
Objective
To evaluate the impact of the ETAT+ course on HCWs knowledge and practical skills, and to identify factors associated with greater improvement in knowledge and skills.
Methods
We used a one group, pre-post test study using data collected during ETAT+ course implementation from 2010 to 2013. The paired t-test was used to assess the effect of ETAT+ course on knowledge improvement in participating HCWs. Mixed effects linear and logistic regression models were fitted to explore factors associated with HCWs performance in ETAT+ course knowledge and practical skills assessments, while accounting for clustering of HCWs in hospitals.
Results
374 HCWs were included in the analysis. On average, knowledge scores improved by 22.8/100 (95% confidence interval (CI) 20.5, 25.1). In adjusted models, bilingual (French & English) participants had a greater improvement in knowledge 7.3 (95% CI 4.3, 10.2) and higher odds of passing the practical skills assessment (adjusted odds ratio (aOR) = 2.60; 95% CI 1.25, 5.40) than those who were solely proficient in French. Participants who attended a course outside of their health facility had higher odds of passing the skills assessment (aOR = 2.11; 95% CI 1.01, 4.44) than those who attended one within their health facility.
Conclusions
The current study shows a positive impact of ETAT+ course on improving participants’ knowledge and skills related to managing emergency pediatric and neonatal care conditions. The findings regarding key factors influencing ETAT+ course outcomes demonstrate the importance of considering key contextual factors (e.g., language barriers) that might affect HCWs performance in this type of continuous medical education.
Citation: Hategekimana C, Shoveller J, Tuyisenge L, Kenyon C, Cechetto DF, Lynd LD (2016) Correlates of Performance of Healthcare Workers in Emergency, Triage, Assessment and Treatment plus Admission Care (ETAT+) Course in Rwanda: Context Matters. PLoS ONE 11(3): e0152882. https://doi.org/10.1371/journal.pone.0152882
Editor: David W. Dowdy, Johns Hopkins Bloomberg School of Public Health, UNITED STATES
Received: July 29, 2015; Accepted: March 21, 2016; Published: March 31, 2016
Copyright: © 2016 Hategekimana et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant and summary data underlying the current study findings are within the paper. Nevertheless, as per ethics approval granted by the Research Ethics Board of the University of British Columbia (UBC BREB NUMBER: H1500964) and the authorization by the Rwanda Paediatric Association to use the data, the raw data must not be hosted in a public repository. Thus, access to these raw data would require further approval from both the Research Ethics Board of the University of British Columbia and the Rwanda Paediatric Association. The corresponding authors (larry.lynd@ubc.ca, celekimana@gmail.com) may be contacted to facilitate the process.
Funding: DFC received a grant (Maternal Newborn and Child Health in Rwanda, S-065358) from Foreign Affairs and International Trade Canada (http://www.international.gc.ca/international/index.aspx?lang=eng) that supported the ETAT+ course in Rwanda as well as CH to undertake a graduate training in Canada. The ETAT+ course received further funding from other institutions/agencies including: International Health Partnership Funding Scheme (http://www.thet.org/health-partnership-scheme), Belgian Development Agency (https://www.btcctb.org/en/countries/rwanda), and IntraHealth Rwanda (http://www.intrahealth.org/page/rwanda). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Rwanda has a population of more than 11 million people, of which 48% are younger than 18 years old [1]. There are approximately 410,100 births per year and only about 20 pediatricians were working in the country as of 2011, mostly in national referral hospitals [2–3]. The healthcare workforce in district hospitals–the backbone of the health care system–is primarily comprised of generalist physicians and secondary school trained nurses (A2-level, the lowest level of nursing training available) [4]. These physicians and nurses are often required to handle complicated pediatric and neonatal emergencies in the absence of specialists. Healthcare workers (HCWs) have expressed concerns about gaps in their training and their lack of confidence with regards to dealing with maternal newborn and child emergency conditions [5].
District hospitals are the first-line referral sites in low and middle resource countries, and thus it is critical that they be capable of providing effective, efficient and high quality care to all patients [6, 7]. Therefore, additional training is needed to better prepare HCWs in these facilities to provide effective emergency pediatric and neonatal care. While the implementation of advanced pediatric life support courses has been advocated for in these countries to facilitate the achievement of the fourth Millennium Development Goal (i.e. MDG 4: reducing under-five mortality by two-thirds between 1990 and 2015), many existing pediatric advanced life support management guidelines in limited-resource settings are not standardized. Furthermore, systematic approaches to patient assessment and categorization of disease is rare, and current pediatric advanced life support training courses tend to be designed and implemented without sufficient attention to the local context and its associated resource constraints [8].
In an effort to ameliorate these deficiencies, the Emergency, Triage, Assessment and Treatment plus Admission care (ETAT+) course was developed in East Africa for HCWs caring for acutely ill newborns and children. The ETAT+ course is a locally adapted pediatric advanced life support course that offers intensive, competency-driven training over a five-day period that aims to improve pediatric emergency and admission care in the initial 24 hours of hospitalization [9]. The course covers the recognition and initial management of the most common medical causes of pediatric hospital admission in East Africa (Table 1). It was introduced in Rwanda in 2010 to contribute to the reduction of the mortality rate in children less than five years [10].
Clinical practice guidelines, including ETAT+, are developed to help HCWs deliver the best care to patients by combining the best available evidence on the management of diseases into specific recommendations for care. Nevertheless, evidence-based guidelines are rarely implemented with perfect fidelity under real-world conditions. Therefore, evaluations of the real-world experiences associated with implementing such guidelines are important in terms of identifying potential barriers to successful implementation as well as identifying factors that contribute to their successful adoption and scale-up over time.
With these aims in mind, we analyzed data gathered during the implementation of the ETAT+ course for HCWs in Rwanda to evaluate the performance of HCWs in knowledge acquisition and skills assessments with a goal of gathering evidence that would be used to inform the course tailoring and revisions (e.g., course content; scope and sequencing of training modules; program delivery and teaching methods). The objectives of this study were therefore to: (i) determine the effect of ETAT+ course on improvement in knowledge and skills related to pediatric advanced life support management among HCWs in Rwanda; (ii) identify the components of the ETAT+ course in which HCWs demonstrated the largest and most persistent gaps in knowledge or skills; and (iii) identify key factors associated with HCWs performance on the ETAT+ course knowledge and skills assessments.
Materials and Methods
Overview of ETAT+ course implementation
The ETAT+ course consists of lectures, demonstration and practical sessions and case scenarios using manikins, a hospital audit including a review of selected patient medical records within the specific institution where the course is occurring, and assessment and feedback. The lectures are short and topic specific, and are accompanied by demonstrations [9]. Discussions and hands on practice take place in small groups of 5–7 participants where they work in inter-professional groups with nurses, midwives and physicians all learning together. Course preparation materials are provided to participants prior to the start of the course. These materials include an invitation letter highlighting course venue and expectation, timetable, Rwandan basic pediatric protocols [11], WHO pocket book of hospital care for children [12], and a pre-course knowledge assessment. Attendance for the entire five days is compulsory to receive the certification.
For the pre-course knowledge assessment, all participants are required to complete a standard knowledge assessment of 20 multiple-choice questions (MCQs) before the start of the course with access to the course materials, while post-course knowledge assessment are completed under closed-book exam conditions (Fig 1). The pre-course and post-course MCQs are different but test the same concept. The assessments are taken in English or French depending on language proficiency and preference of participants. The knowledge assessment score is the proportion of correct answers. Upon completion of the course, all participants are required to retake the knowledge assessment. Moreover, the participants are assessed on two of the clinical skills scenarios that are discussed during the course; however, they have no prior knowledge of the clinical skills selected for assessment. Information pertaining to participant characteristics is also collected at the start of the course. ETAT+ course instructors train and assess participants and the senior instructor monitor instructors’ performance during training and assessment.
Participants are also assessed according to standardized criteria on two clinical skills scenarios—one pertaining to neonatal resuscitation and the other focusing on the management of a severely ill child with shock from dehydration. The practical skills assessment is done using an Objective Structured Clinical Examination (OSCE) format with a clinical scenario and a standardized checklist. To pass the skills assessment, participants are required to pass both scenarios. If they fail both scenarios on their first attempt, they are judged to fail and are not allowed to redo the scenarios. However, if participants do not meet all standardized criteria in just one scenario, they are allowed to retake the same clinical skills scenario after receiving feedback. If all standardized criteria are met after the retake, they are deemed to pass the skills assessment; otherwise, they fail the assessment.
The implementation of the ETAT+ course in Rwanda is overseen by the Rwanda Pediatric Association, which also trains ETAT+ course instructors.
Study Design
The current analysis was a quantitative assessment of the improvement in emergency pediatric and neonatal care knowledge and skills immediately after an intensive full five-day of ETAT+ course–held between 2010 and 2013 –for HCWs in Rwanda using one group pre- post-test, and post-test only study designs. The pre-post test design was used to assess the impact of ETAT+ on change in knowledge (i.e. difference between pretest score and post test score), while the post-test only design was used to assess the impact of the course on skills performance as there was no skills assessment conducted prior to the course. Program implementers assumed that pre-test skills assessments would have been futile, as all participants would likely perform very poorly since they were not familiar with standardized criteria required to pass the assessment.
Analytic Sample
As of part of a national strategy to reduce under-five mortality, ETAT+ course has been implemented in many health facilities in Rwanda. This study includes all district hospitals and national referral hospitals in which ETAT+ course was implemented between 2010 and 2013. As of 2010, Rwanda had 40 district hospitals and 4 national referral hospitals (http://moh.gov.rw/fileadmin/templates/HMIS_Docs/MOH_Annual_booklet__2010.pdf). The hospitals were purposefully selected to ensure reasonable regional representation while considering availability of funds. The ETAT+ course participants were also purposefully selected to ensure those working in newborn and child health related departments (e.g. pediatrics, maternity, emergency room) were trained as they were expected to use the knowledge and skills gained when they return to their affiliated hospitals to improve emergency care provided to sick newborns and children.
The current analysis includes HCWs (i.e. nurses, midwives, generalist and specialist physicians) who were working in health facilities in Rwanda and enrolled in an ETAT+ course between 2010 and 2013, and who completed the pre- and post-course knowledge assessments as well as ETAT+ practical skills assessment. This analysis excludes participants who did not complete the full five-day course, final year medical students, and HCWs who were working abroad (i.e. outside Rwanda) but completed the course while in Rwanda.
Study Variables
The outcomes for this study included pre-course knowledge score, post-course knowledge score, change in knowledge, and practical skills score. Pre-course and post-course knowledge assessment scores were defined as the percentage of correct answers in the pre- and post-course knowledge assessments, respectively. Change in knowledge outcome was defined as the difference between pre-course and post-course knowledge scores, while practical skill-based competency was defined as passing both the neonatal resuscitation and shock scenarios. ETAT+ certification requires achieving a score of at least 50% on post-course knowledge assessment and passing the practical skills assessment. These standards reflect the criteria that are required for certification in ETAT+ course [13].
Variables (see Table 2) included in the current analysis were hypothesized to be associated with improvement in knowledge or scores on the practical skills assessment were based on existing literature (e.g., profession, years since graduation/experience) [14] or were identified during the course implementation as potentially important factors (e.g., course location, type of materials, timing of delivery of materials, language proficiency).
Statistical Analysis
Descriptive statistics including frequency (and percentages) for categorical variables and mean (and standard deviation, SD) for continuous variables were computed and reported. Change in knowledge scores was assessed by comparing pre- and post-course scores using a paired t-test. We used mixed effects linear and logistic regression models to assess the correlates of knowledge improvement and practical skills performance among HCWs who participated in the ETAT+ course, while accounting for clustering of HCWs in hospitals. We performed bivariate analyses to investigate the relationship between each explanatory variable and outcomes (i.e., change in knowledge scores and skills performance) to see which variables pass an initial screening at α set to p = 0.2 as model entry significance level. All potential predictors independently associated with outcomes (at an α level of 0.2) during bivariate analysis were retained for further exploration in the multivariate regression models. Backward selection was then applied (p>0.05) to select the most parsimonious models. We achieved model selection by minimizing Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) while keeping p values for covariates below 0.05. For logistic models, associations are reported as odds ratios (OR) and 95% confidence interval (CI) while regression coefficient (ß) and 95% CI are reported for the linear models. All statistical analyses were performed with RStudio version 3.2.2 (RStudio Inc., Boston, MA, USA).
Ethics Statement
This study received approval from the Research Ethics Board of the University of British Columbia and from the Rwanda Pediatric Association. The current study used existing data that were collected during the implementation of the ETAT+ course in Rwanda. As such the consent was not obtained and participants’ information were anonymized and de-identified prior to analysis.
Results
Characteristics of Study Sample
Of 615 individuals who were trained in the ETAT+ course between 2010 and 2013, 241 (39.2%) participants were excluded from the current analysis: 1.8% (n = 11) were HCWs who did not complete the full five-day course; 36.0% (n = 222) were final year medical students but were not yet in practice; and 1.3% (n = 8) were HCWs who were working abroad (i.e. outside Rwanda) but completed the course while in Rwanda. Therefore, the final analytic sample included 374 HCWs from 24 health facilities including three of the four national referral hospitals and 21 of Rwanda’s 40 district hospitals.
The overall study sample was unevenly distributed across profession, with more nurses than doctors (generalists and specialists) (Table 2). Most course participants were from district hospitals, were more than one year post graduation, or attended an ETAT+ course held outside their home healthcare facility. Only 3.2% of participants reported having previously attended an ETAT+ course. Two-thirds of the participants were female and 67.9% were working in an urban hospital, while 53.7% of the participants were working in a pediatric department when they attended the course and 53.2% identified as fluent in French (Table 2).
Knowledge Assessment Performance
Only 77 (20.6%) participants achieved a score of 50% or more on the pre-course knowledge assessment, versus 292 (78.0%) on the post-course knowledge assessment. The overall mean (SD) score on the pre- and post-course knowledge assessment was 35.3 (19.7) and 58.0 (13.4), respectively, resulting in an overall mean change in knowledge assessment scores of 22.8 (20.4–25.1) (p<0.0001). Nearly all components of the knowledge assessment appear to contribute equally to failure of both pre- and post-test (Fig 2). The participants demonstrated greater knowledge gaps in resuscitation and management of shock/dehydration and severe malnutrition compared to the management of infection, and thus the improvement in these three components was greatest, with some improvement demonstrated in the infection management component (Fig 2).
MCQs, multiple choice questions.
Performance in pre-course knowledge assessment varied, with midwives, those who had previously attended ETAT+ course, those who identified as proficient in English, and those who received the course materials more than a week before the start of the course achieving a higher score, on average (Table 3). Likewise, performance on post-course knowledge assessment differed with higher average scores achieved by specialist MDs, those who completed their professional training more recently, those who had attended an ETAT+ course previously, and those who identified as proficient in both French and English (Table 3). Greater overall improvement in knowledge scores was evident among specialist doctors, those who were working in national referral hospitals or in other departments when they attended the course, those who had not attended the course before, those who reported being proficiently bilingual, those who received the course materials one week of the start of the course, and those who attended the course outside of their health facility (Table 3).
Performance on Skills Assessment
The pass rates for neonatal resuscitation and shock management scenarios were 85.0% (n = 318) and 82.4% (n = 308), respectively, while the overall pass rate on the skills assessment (i.e. passing both neonatal resuscitation and shock scenarios) was 78.6% (n = 294). Performance on the skills assessment varied with all participants who had previously attended the course and almost all of those who were one year away from graduation passing the skills assessment (Table 3).
With respect to reasons for failing the practical skills assessment, two components (observing muscle tone and color and calling for help) were most often missed in the neonatal scenario (Fig 3), while checking for malnutrition was the most frequent component missed in the shock scenario (Fig 4).
BVM, bag valve mask.
CRT, capillary refill time; IO, intra-osseous; NP, neutral position; PP, peripheral pulse; TG, temperature gradient; LOC, level of consciousness; RD, respiratory distress.
Factors associated with HCWs’ Performance in the ETAT+ course Knowledge and Skills Assessments
Health facility related characteristics (i.e. type and location of health facility) were not associated with either improvement in knowledge (Table 4) or performance in practical skills assessment (Table 5).
Healthcare provider characteristics.
Participants who had recently graduated from their professional training were more likely to pass the practical skills assessment (aOR = 9.62; 95% CI 1.20, 76.85) than those with more than one year away from graduation (Table 5). At the completion of the ETAT+ course, on average generalist and specialist MDs demonstrated a greater improvement in knowledge (aß = 6.5; 95% CI 3.2, 9.8) and (aß = 13.5; 95% CI 8.1 18.9) respectively, higher than nurses (Table 4). Similarly, profession was associated with performance on practical skills assessment with midwives, generalist and specialist MDs having higher odds of passing than nurses (aOR = 3.12; 95% CI 0.85, 11.37), (aOR = 3.50; 95% CI 1.40, 8.73) and (aOR = 4.87; 95% CI 0.60, 39.32), respectively (Table 5). However, the relationship is statistically significant in the multivariate analysis only for generalist MD.
Compared to participants who identified as being proficient only in French, on average, those who identified as proficient in both English and French (bilingual) demonstrated a greater improvement in knowledge (aß = 7.3; 95% CI 4.3, 10.2) (Table 4). Likewise, bilingual participants had higher odds of passing the practical skills assessment (aOR = 2.60; 95% CI 1.25, 5.40) than those who identified as proficient in French (Table 5). The higher the improvement in knowledge, the higher the odds of passing the practical skills assessment (aOR = 1.01; 95% CI 0.99, 1.02) (Table 5).
Since nurses from national referral and district hospitals may have an A2 (just secondary school diploma and the lowest available qualification), A1 (three years of post-secondary nursing training) or A0 (four years of post-secondary nursing training) certification, and health facilities located in rural areas usually have the highest proportion of A2 nurses, a further stratified analysis was conducted to investigate whether performance of nurses varied by location of health facility. However, we found that nurses from urban and rural hospitals performed similarly.
ETAT+ course organization characteristics.
Of all ETAT+ course organization related factors (Table 2), time of receipt of course materials was most strongly associated with improvement in knowledge in the adjusted model, with participants who received materials within a week of the start of ETAT+ course achieving a greater improvement (aß = 2.1; 95% CI 0.9, 5.3) than those who received materials more than one week (Table 4). Moreover, location of the ETAT+ course appeared to be strongly associated with performance on skills assessment, with participants attending an ETAT+ course outside their health facility (off-site) having higher odds of passing the practical skills assessment (aOR = 2.11; 95% CI 1.01, 4.44) than those who attended a course held within their affiliated health facility (Table 5).
Further analysis to investigate the association between performance and type of course materials stratified by hospital location found that participants who were working in rural hospitals and received printed course materials (hardcopy) had a higher performance in the pre-course knowledge assessment than those who got just e-copy (p<0.0001), but there was no difference in post course knowledge and skills assessment (P>0.05).
Discussion
The current study demonstrates that the ETAT+ course is associated with improvements in both knowledge and skills in course attendees in Rwanda. Overall, the course participants performed better overall on practical skills assessment than on knowledge assessment. Further, the current analysis provides evidence of factors associated with knowledge and practical skills performance including the provision of pre-course materials (timing and format), location of ETAT+ course and HCWs’ language proficiency.
Consistent with previous studies, participating in continuing medical education (CME) courses including ETAT+ is associated with immediate improvement knowledge across a wide variety of professions (e.g. physicians, nurses, midwives, in training healthcare professionals) following the training [9, 15–17]. While the pass rate in the post-course knowledge assessment was high (78.0%), the overall mean post-test score was only 58.0% (SD 13.4), far below the average score of 71% for Rwandan medical students who undertook the same course and assessment [15]. Similarly, Rwandan medical students had a higher performance on the skills assessment than the HCWs (pass rate of 98% versus 78.6%) [15]. This difference might be explained by language barriers and limited access to computers and the internet among HCWs, but also eagerness/ motivation for students as they were trained in the ETAT+ during their pediatric rotation. The ETAT+ related content is also assessed in the pediatric end rotation assessment and in the medical board examination that also likely had an effect on their performance.
Local contextual factors related to the course organization (e.g. format of course materials, time of receipt of course materials, location of ETAT+ course) and HCW’s language proficiency might explain some differences in ETAT+ course performance among Rwandan HCWs. Rwanda shifted its official language from French to English in 2008 [18], and secondary and post-secondary education and CME programs are run primarily in English. In the current study, HCWs who identified as being proficient only in French performed more poorly relative to all other participants which might be explained by challenges that HCWs who were not proficient in English (e.g. unable to adequately prepare or understand course content, if it was taught mainly in English). Use of both English and French during the course implementation might therefore offer advantages to those HCWs who identified as bilingual (French & English) over those who identified as either proficient in French or English, and help explain why bilingual HCWs demonstrated the best overall performance. Relatedly, some participants would ask questions or further clarifications in Kinyarwanda, the Rwandan mother tongue while some other participants could not communicate in Kinyarwanda, especially foreign medical and nursing graduates working in Rwanda. Furthermore, it is possible that bilingual participants might have translated for English- or French-speaking participants. If so, the translating/teaching role might have supported their learning and, therefore, might explain to some point the observed greater performance among bilingual participants. Nevertheless, the effect this might have on performance of HCWs on the ETAT+ course assessment was not assessed.
Healthcare providers working in rural based health facilities and who received printed course materials performed better on the pre-course knowledge assessment than those who were offered electronic copy (usually sent by email). This difference in performance might be explained by lower penetrance of the internet and computer ownership in rural areas compared to urban areas in Rwanda [19]. However, this difference in performance disappeared in the post course assessment and further investigation of an interaction between course materials and location of health facilities was not found to be statistically significant.
Location of the ETAT+ course was an important predictor of performance in ETAT+ course as well. While courses held within health facilities were cost saving (e.g. costs associated with accommodation of participants and course venue were saved), they were associated with a poorer performance, which may be the requirement that HCWs may be required to continue to be involved in some of the work-related activities (e.g. direct patient care, night call) during course time and could have missed material when the program is held within their institution (on-site training).
Also consistent with previous research, tasks such as observing color and tone during neonatal resuscitation or checking for signs of severe malnutrition in a child with shock due to dehydration are more likely to be missed on a manikin than when examining a real child whose nutrition status, tone or color would be apparent clinically [15]. Furthermore, patients’ medical files in many health facilities in Rwanda (e.g. neonatal admission record) have checklists included, and these checklists help minimize the omission of required tasks when examining a patient. Likewise, failing to follow resuscitation sequences exactly as is required to pass the ETAT+ skills assessment may not necessarily result in resuscitation failure in practice [15].
Although nurses constitute the majority of the health workforce in Rwandan district hospitals, they unfortunately performed the most poorly on knowledge and skills assessments. This is alarming since nurses are often required to manage severely ill children and neonates without supervision. While we acknowledge that diversity in course participants (e.g. integration of multiple professions) might improve the learning experience as participants share different experiences and learn how to work together in a team, a different approach for delivering ETAT+ for nurses may improve their performance.
Better performance on the infection component compared to other ETAT+ components in the pre-course knowledge assessment might be explained by existing disease-specific programs focusing on training HCWs on management of malaria and HIV across the country, and the treatment of pneumonia as part of an integrated management of childhood illness program. We believe that integrating pediatric advanced life support courses such as ETAT+ into the ongoing Rwanda continuing professional development program might help regular training and retraining of HCWs and increase their knowledge and skills with regards to managing newborn and child health emergencies. The closer to graduation the better performance, which might suggest that the most-recently trained participants have previously (and recently) studied the ETAT+ content in their professional school training. While refresher training for HCWs in life support management cannot be overemphasized because the decay of knowledge and loss of skills pose a real threat to success of best practice care interventions [20, 21], we can speculate that better performance among HCWs who had previously attended the ETAT+ course (while still in their professional training) reflects some levels of knowledge and skills retention.
The current study was the first to assess factors associated with performance on the ETAT+ course that we are aware of, and thus provides evidence of the local contextual factors that might affect Rwandan HCWs’ performance in ETAT+ course. Despite the study sample size of 374 HCWs from 24 health facilities including three of the four national referral hospitals and 21 of Rwanda’s 40 district hospitals, our findings may not be representative of all HCWs in Rwanda given that participants and hospitals were purposefully selected–i.e. selection bias could not be ruled out. Moreover, given the current study design, single group threats to internal validity are of concerns (e.g. testing and instrumentation threats) [22]. It is possible that the improvement in knowledge observed among participants was not only due to attending the ETAT+ course. For example, taking only the pre-course knowledge assessment might affect how participants do on the post-course knowledge assessment even without participating in the ETAT+ course (testing threat) as they might memorize questions. Including different questions in the post-course knowledge assessment while still testing the same concept might have minimized the testing threat; however, changing questions might have introduced an instrumentation threat.
While program evaluation using specific metrics may be easier and more practical than trying to evaluate patient outcomes, the latter is the ultimate goal. Nevertheless, before investing the time and resources to evaluate patient outcomes, programmatic evaluation was carried out under the premise that lack of positive outcomes at the programmatic level likely means no effect on patient outcomes. Since a positive impact of the course on knowledge change and skills performance was identified among HCWs, future studies should attempt patient level evaluation to determine whether the knowledge and skills acquired translate to practice and contribute to improving patient health outcomes.
In conclusion, the current study findings shows a positive impact of the ETAT+ course on improving participants’ knowledge and skills related to managing emergency pediatric and neonatal conditions. Moreover, the analysis provides evidence of specific factors associated with improvement in both knowledge and practical skills in Rwandan HCWs. These factors are modifiable and can therefore be used to inform improvements in the delivery of the ETAT+ program in Rwanda thereby positively impacting HCWs knowledge and skills development. The findings can inform not only ETAT+ course program implementers but also other CME program implementers and policy and decision makers of key contextual factors that might affect Rwandan HCWs’ performance in CME trainings and thus take them into account when developing and implementing CME in Rwanda and regionally.
Acknowledgments
We would like to thank the Rwanda Pediatric Association for granting permission to conduct the study and healthcare providers and ETAT+ course instructors who took part in this study.
Author Contributions
Conceived and designed the experiments: CH JS LT CK DFC LDL. Performed the experiments: CH JS LT CK DFC LDL. Analyzed the data: CH. Contributed reagents/materials/analysis tools: CH JS LDL. Wrote the paper: CH JS LT CK DFC LDL. Trained Rwandan healthcare providers in ETAT+ and collected data: CH LT.
References
- 1.
National Institute of Statistics of Rwanda. The Fourth Rwanda Population and Housing Census (RPHC4) Thematic Report: Population Size, Structure and Distribution. 2012 January 2014.
- 2.
Ministry of Health of the Republic of Rwanda. Rwanda Annual Health Statistics Booklet 2013. 2014.
- 3.
Ministry of Health of the Republic of Rwanda. Human Resources for Health Strategic Plan, 2011–2016. Kigali, RwandaMarch 2011.
- 4. Binagwaho A, Kyamanywa P, Farmer PE, Nuthulaganti T, Umubyeyi B, Nyemazi JP, et al. The human resources for health program in Rwanda—a new partnership. New England Journal of Medicine. 2013;369(21):2054–9. pmid:24256385
- 5.
REPORT ON QUALITY OF HEALTH CARE IN RWANDA. 2010.
- 6.
Mabey D, Gill G, Parry E, Weber MW, Whitty CJ. Principles of medicine in Africa: Cambridge University Press; 2013.
- 7.
World Health Organization. Hospitals and health for all: report of a WHO Expert Committee on the Role of Hospitals at the First Referral Level [meeting held in Geneva from 9 to 17 December 1985]. 1987.
- 8. Ralston ME, Day LT, Slusher TM, Musa NL, Doss HS. Global paediatric advanced life support: improving child survival in limited-resource settings. The Lancet. 2013;381(9862):256–65.
- 9. Irimu G, Wamae A, Wasunna A, Were F, Ntoburi S, Opiyo N, et al. Developing and introducing evidence based clinical practice guidelines for serious illness in Kenya. Archives of disease in childhood. 2008;93(9):799–804. pmid:18719161
- 10.
Rwanda Pediatric Association. Emergency, Triage, Assessment and Treatment plus Admission care (ETAT+) course. 2010 [cited 2014 December 27]; Available from: http://www.etatcourserwanda.org/about/.
- 11.
Ministry of Health of the Republic of Rwanda. Basic Pediatric Protocols 2011. Available from: http://www.etatcourserwanda.org/wp-content/uploads/2013/06/Rwanda-Pediatric-Protocols_oct-2011.pdf.
- 12. World Health Organization. Hospital care for children. Chart. 2013;13:131.
- 13.
ETAT+. Instructor/Facilitator Guide 2010. Available from: http://www.idoc-africa.org/images/documents/ETAT/et_inst_manual/ETAT_Facilitator_Guide_Sept2010.pdf.
- 14. Curran V, Fleet L, Greene M. An exploratory study of factors influencing resuscitation skills retention and performance among health providers. Journal of Continuing Education in the Health Professions. 2012;32(2):126–33. pmid:22733640
- 15. Tuyisenge L, Kyamanya P, Van Steirteghem S, Becker M, English M, Lissauer T. Knowledge and skills retention following Emergency Triage, Assessment and Treatment plus Admission course for final year medical students in Rwanda: a longitudinal cohort study. Archives of disease in childhood. 2014:archdischild-2014-306078.
- 16. Mayhew M, Ickx P, Newbrander W, Stanekzai H, Alawi S. Long and short Integrated Management of Childhood Illness (IMCI) training courses in Afghanistan: a cross-sectional cohort comparison of post-course knowledge and performance. Int J Health Policy Manag. 2015;4.
- 17. WAISMAN Y, AMIR L, MIMOUNI M. Does the pediatric advanced life support course improve knowledge of pediatric resuscitation? Pediatric emergency care. 2002;18(3):168–70. pmid:12066000
- 18.
National Public Radio. English To Become Official Language In Rwanda. 2008.
- 19.
Ministry of Youth and ICT of the Republic of Rwanda. Rwanda ICT Profile: “Measuring ICT sector performance and Tracking ICT for Development (ICT4D) towards Rwanda Socio-Economic Transformation” 2012.
- 20. Andresen D, Arntz HR, Gräfling W, Hoffmann S, Hofmann D, Kraemer R, et al. Public access resuscitation program including defibrillator training for laypersons: a randomized trial to evaluate the impact of training course duration. Resuscitation. 2008;76(3):419–24. pmid:17976888
- 21. Jabbour M, Osmond MH, Klassen TP. Life support courses: are they effective? Annals of emergency medicine. 1996;28(6):690–8. pmid:8953961
- 22.
Trochim WM. The Research Methods Knowledge Base. 2nd ed2006. Available from http://www.socialresearchmethods.net/kb/intsing.php