Practically prepared? Pre-intern student views following an education package

Background

Medical graduates’ readiness for internship entails an awareness of their abilities (and limitations) in basic practical skills in the clinical environment, as well as their confidence to safely commence internship.¹ It is not surprising that this “readiness” is strongly linked to their previous “hands on experience” as students,^1,2 which is unfortunately opportunistic and, therefore, highly varied.³ Consequently, new graduates commence their internship with vastly differing levels of both knowledge and experience in practical skills, required to safely undertake their new role in patient care.³ Intern knowledge gaps have been particularly identified in the management of the acutely ill patients, prescribing,^4–6 and non-technical skills, including communication skills,⁷ team work, and task management.

To address this variability in readiness for internship, preparatory courses such as “Coda” accelerated learning⁸ “Boot Camp”,⁹ Student assistantships,¹⁰ and “Shadowing” are offered in many medical schools immediately prior to graduation. To ease the transition into internship, Sydney Medical School implemented a pre-internship term in 2000. This comprised activities such as allowing students to “shadow” an intern, perform procedural skills under supervision, gain experiential knowledge in non-technical skills such as team work, and to gain an awareness of hospital protocols crucial to patient safety.

While the PrInt term was seen as useful for internship, we felt there was room for further improvement in core procedural skills and designed a new educational package to cover obvious gaps. The aim of our study was to assess the impact of this new educational package on students’ self-reported ability, confidence, knowledge, and awareness of the key practical skills required for internship.

Methods

The Sydney Medical Program is a four-year graduate entry course with problem-based learning as the key teaching strategy. This study took place at the CCS Royal Prince Alfred Hospital, a teaching hospital for The University of Sydney, Sydney Medical School. In the last four weeks of the medical program, students enter their pre-internship term.

In September 2014, all PrInt students at CCS (n = 53) who had completed their final exams attended a three-day PrInt induction course. The course had been designed and piloted previously as a five-day course in 2011.¹¹ The course described here is a three-day modification of the pilot study.

Education package

Didactic teaching

This three-day intensive teaching program included key situations that new interns were very likely to encounter,¹² including core procedural skills, such as “managing” procedural skills, identifying N/G tube placement on X-ray, recognition of the acutely unwell patient, complex communication skills – presented as ward-based scenarios, interpreting common X-rays, suturing, and effective handover utilizing ISBAR (a mnemonic created to improve safety in the transfer of critical information).¹³

Small group teaching sessions

These were designed to provide students with individualized practice, with real-time feedback from experienced clinicians. Students were also given the opportunity to clarify concerns in performing practical skills and were provided with strategies for identifying and dealing with common clinical challenges. This style of learning encouraged reflection and integration of skills and supported their transition to real life as a junior doctor.¹⁴

Case-based teaching

We used case-based scenarios, known to be associated with errors/adverse events, such as test ordering, prescribing medications, recognition of the deteriorating patient, and specimen collection/labeling. Specific education was delivered by pharmacists in “pharmacy/prescribing skills”, and by emergency physicians in “emergency skills”. In addition, while on their subsequent clinical attachments, they had weekly bedside tutorials conducted by hospital clinicians.

Clinical attachment

This component of the package involved four weeks when the students were assigned to a medical or surgical team, where they “shadowed” the intern and became active members of the team. PrInt students are required to perform various mandatory practical skills under supervision, and their overall performance as part of the team must be deemed “satisfactory” by their PrInt supervisor before they can graduate.¹⁵

Instrument

To develop our instrument, focus groups were conducted at CCS with hospital clinicians. Following data analysis, we generated a novel (unvalidated) six-domain questionnaire. We were mindful of including key procedural skills that interns would be expected to perform. The scores ranged from 1 to 100, and we arbitrarily considered 5 points (i.e., 5%) to be the minimal relevant change. Each of the six domains consisted of additional subscales, giving a total of 40 questions answered by either “Yes” or “No”.

The domains are as follows:

1. Performance of core procedural skills
2. Knowledge and confidence in managing procedural skills
3. Clinical recognition scores (recognition of the acutely ill patient)
4. Pharmacy skills/prescribing skills
5. Communication skills
6. Organizational skills

Students anonymously completed two identical questionnaires – the first, immediately prior to the three-day induction course (in late September 2014), and the second, immediately following four weeks of PrInt (in October 2014). Questionnaires were distributed and collected by a third party not involved with the study.

Statistical analysis

The two questionnaires were compared using multiple regression analysis. We were not able to compare individual student results due to the anonymous nature of the research.

Ethical considerations and participant consent details

Ethics approval was granted by The University of Sydney Ethics committee. Permission to undertake this research has been granted by Human Ethics approval number 2013/211 at The University of Sydney, NSW, Australia. Participants received an accompanying statement informing them that lodgment of forms was considered to be their consent for future use of data.

Results

A total of 41/53 (77%) PrInt students, attached to CCS, completed identical questionnaires prior to their PrInt induction session and 37/53 (70%) at the completion of four weeks’ PrInt. Of the 41 preinduction respondents, 24/41 (59%) were male, 15/41 (37%) were female, and 2/41 (5%) did not report their gender. Respondents’ ages ranged from 23 to 34 years.

The mean scores and mean differences in each domain of the six closed questions concerning both experience and knowledge in key skills are presented in Table 1. Respondents reported significant improvements in knowledge of pharmacy skills and managing procedural skills. However, because of preexisting high baseline scores, there was little improvement with respect to other domains, including communication, organizational, and procedural skills.

Table 1 Report of the practical skills ability of PrInt students in order of magnitude of improvement following pre-internship Abbreviations: LS, least square; CI, confidence interval; PrInt, pre-intern.

Subscale results

The subscale results in the order of magnitude of improvement are demonstrated in Figures 1–6.

Figure 1 Subscale results of PrInt students’ self-reported knowledge and awareness in safe prescribing (pharmacy skills) in 2014 before the educational package and immediately after pre-internship. Note that students’ confidence in commencing patients on warfarin remains low. Abbreviation: PrInt, pre-intern.

Figure 2 Subscale results of PrInt students’ self-reported confidence and awareness in their experience of management of procedural skills in 2014 before their educational package and following PrInt. Abbreviations: N/G, nasogastric; PrInt, pre-intern.

Figure 3 Subscale results demonstrating PrInt students’ self-reported knowledge in clinical recognition skills prior to and following PrInt in 2014. There was reported decline in knowledge in the precipitating causes of hypoglycemia; recognition of diabetic ketoacidosis also declined by 5%, whereas knowledge of the management of risk factors leading to hyponatremia improved. Abbreviation: PrInt, pre-intern.

Figure 4 Subscale results demonstrate the decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) and a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events. Abbreviation: PrInt, pre-intern.

Figure 5 Subscale results demonstrating PrInt students’ self-reported awareness and confidence in organizational preparedness before the education intervention and following PrInt in 2014. There was a decreased awareness of 11% (from 99% to 88%) reported for knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) as did performing an effective verbal consult to a senior colleague by 10% (from 68% to 78%). Abbreviation: PrInt, pre-intern.

Figure 6 Subscale results demonstrating PrInt students’ self-reported performance in a range of procedural skills in 2014 before the educational package and following PrInt. Low pre- and post-scores were recorded for their practical experience of the insertion of a nasogastric tube. Abbreviations: ECG, electrocardiogram; PrInt, pre-intern; BSL, blood sugar level.

Pharmacy skills

The results of students’ perceived knowledge and awareness and knowledge of pharmacy skills are shown in Figure 1. There were statistically significant improvements in five areas: safety awareness in pharmacy skills in writing the prescriptions increased by 58% (from 34% to 92%), knowledge in the use of the medication management plan increased by 51% (from 10% to 61%), confidence in prescribing low molecular weight and standard weight heparin increased by 42% (from 22% to 64%), confidence in commencing patients on warfarin therapy increased by 22% (from 22% to 44%), and recognition and treatment of opiate-induced respiratory depression increased by 16% (from 78% to 94%)

Managing procedural skills

Respondents’ perceptions of their ability to manage situations related to procedural skills are shown in Figure 2. The most significant improvements in students’ confidence were as follows: managing common problems associated with indwelling catheters increased by 39% (from 39% to 78%), intravenous cannulas increased by 37% (from 49% to 86%), identification of correct placement of peripherally inserted central catheter-lines on X-ray increased by 32% (from 36% to 78%), and confidence in managing clinical emergencies on the ward increased by 32% (from 10% to 42%).

Clinical recognition skills (recognition of the acutely unwell patient)

Ten questions were asked to participants regarding their ability in the recognition, awareness, and management of the acutely ill patient. The results are shown in Figure 3. There were significant improvements in the following: students’ perceived confidence in recognizing respiratory acidosis increased by 17% (from 51% to 68%), confidence in managing the fluid status of a patient with hyponatremia increased by 12% (from 49% to 61%), confidence in the assessment and management of patients with oliguria increased by 13% (from 63% to 76%). There was a small decline in knowledge regarding the precipitating causes of hypoglycemia, which decreased by 7% (from 95% to 88%); awareness of the risks of ceasing oxygen for a patient with hypoxemia and hypercapnea also declined by 7% from (from 90 to 83%).

Communication skills

As seen in Figure 4, there was a decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) as well as a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events both of which had relatively high preexisting baseline scores.

Organizational preparedness

Three closed questions asked respondents to self-report their organizational preparedness. Figure 5 demonstrates a decreased awareness of 11% in respondents’ reported knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) and their ability to perform an effective verbal consultation with a senior colleague increased by 10% (from 68% to 78%).

Procedural skills

The results of seven closed questions related to students’ reported performance of procedural skills are shown in Figure 6. Respondents indicated very high baseline levels of performance in cannulation (98%) and venipuncture (98%), and there was no room for further improvement. However, there were significant increases to IDC insertion by 15% (from 78% to 83%), electrocardiogram performance by 13% (from 59% to 72%), and N/G tube insertion by 15% (from 29% to 44%).

Discussion

The results of this study demonstrate that PrInt students perceived a substantial benefit from a combined educational package and a month of experiential learning in their hospital attachment as PrInt students, in particular, improvements in pharmacy skills and management of procedural skills. However, despite the positive overall increase in most domains, two subscales remained low with only 44% of students reporting confidence in commencing patients on warfarin therapy, and while most respondents had attended an emergency on the ward, only 42% felt confident about handling emergencies. This is despite having had extensive experience in simulated cardiopulmonary resuscitation throughout their medical course and advanced life support skills training in the simulation laboratory during year three, and PrInt. This clearly indicates that respondents felt that they had not had enough practical experience.

While students reported increased performance of N/G tube insertion by 15% (from 39% to 44%), the score remained low, suggesting that the students had low exposure to the procedure in both year three and PrInt. The 15% increase (from 66% to 81%) in taking BSLs was attributed to “hands on” use of the BSL equipment in the PrInt education course.

Students reported a moderate level of confidence in their ability to recognize the acutely ill patient (clinical recognition skills). Interestingly, there was a slight decline in students’ reported ability to manage and treat diabetic patients, as well as in communicating with other members of the medical staff following PrInt (Figure 4). The likely explanation for this is that students had overestimated their knowledge in the aforementioned skills prior to engaging in further education and learning on the wards. This finding is in accordance with Kruger and Dunning who wrote that “those who are unskilled may over estimate their abilities as they are unaware of their own incompetence”.¹⁶

Our findings are similar to those of other recent researchers. Coombes et al in their questionnaire study on pharmacy skills found that medical graduates when commencing internship had little confidence in commencing patients’ warfarin, believing initiation of anticoagulants as being “too great a responsibility”.¹⁷ They found that students also felt less confident in the management of patients with diabetes.¹⁷ A study by Jones et al also supports our findings in that medical graduates rated themselves well prepared in venipuncture and working in a team but least prepared with N/G tube insertion.¹⁸ Wall et al also found that medical graduates are well prepared for working in a team, but in contrast to our findings, students reported to be less prepared in practical skills and pharmacy skills.¹⁹

Our students reported lack of confidence in managing an emergency is a well-known, worldwide concern.^20,3,21 Clearly, while simulation can shorten the learning curve, these programs do not replace real “hands on experience”.³

Clinical attachment

While it is necessary for new interns to be equipped with sufficient knowledge and ability to perform practical skills, safe internship is also dependent on non-technical skills, such as working environment and ward protocols. Previous cohort studies have found that graduate entry PBL medical courses along with “shadowing“ were associated with improvements in preparedness for internship.¹ This was described by Wenger as “legitimate peripheral participation in a community of practice”,²² and this occurs when students work under supervision and integrate their clinical knowledge into practice while learning how the hospital community operates.²³ Conversely, it has been reported that “shadowing” can be limiting as an observational role, leaving the student feeling demotivated and frustrated.²⁴ Ideally, “shadowing” should be in the same hospital that they will be allocated to as interns (i.e., their new work environment).²³ While PrInt placements are currently held at the “parent” hospital where they were students, unfortunately their intern placement is usually not in their “parent” hospital. To address this issue, all new interns in NSW (Australia) now attend a short orientation, which includes “shadowing” the outgoing intern for one week, prior to commencing their new independent clinical role as interns.

Limitations

This study has several limitations. The use of a control group would have defined if specific gains were due to the education package immediately before PrInt or experiential learning during the four weeks of PrInt.

We acknowledge that this is a relatively small, single-center cohort study. Moreover, the questions in this study pertain to self-reported knowledge and confidence, and in the absence of observational feedback, there are obvious concerns of its reliability and validity.²⁵ Instructional courses may improve the learner’s perception of confidence, including their competence in performing the skill. However, while students reported that their knowledge had increased, it may not be an accurate indicator of their actual skills, knowledge, or performance ability.²⁶ A further limitation to this study was its failure to find out how many times a procedure had been performed and whether any feedback had been given.

Further investigation

Results from our findings are important to patient safety, and we are currently conducting further research to determine whether our educational intervention has helped interns to safely care for patients. Medication safety is of particular interest as interns initiate medications, including those which are of high risk, such as warfarin,²⁷ without the counter-signing by an experienced clinician. Students’ lack of confidence in the management of an emergency on the ward is of real concern, which suggests that extensive simulation training throughout their course is not sufficient to overcome this problem. The low frequency of N/G tube insertion is a further concern, which highlights the need for collaboration with nursing staff, who are more likely to perform this skill.

Recommendations

We recommend that a short, intensive transition course be implemented in the curriculum prior to students commencing their PrInt clinical attachment. In Sydney Medical School, the week before PrInt (so called “conference week”) is an ideal time to present the learning package, as students are not involved in other teaching, nor exams, and can shift their focus to their procedural skills, and patient safety issues, prior to transiting their first clinically independent role as interns.

Conclusion

Our results indicate that students do perceive positive benefits from an educational package specifically aimed at improving their practical skills immediately prior to internship. The need for intensive education prior to internship is also highlighted, to address the variation in students’ experiences during clinical rotations to enable students’ awareness of their abilities and limitations prior to internship.

Abbreviations

PrInt, pre-intern; N/G, nasogastric tube; CCS, Central Clinical School; ISBAR, identify yourself, situation, background, assessment, and recommendation; BSLs, blood sugar levels.

Acknowledgment

The authors wish to thank the pre-intern students of 2014 who participated in this study.

Author contributions

Susan McKenzie is a Clinical Lecturer at Sydney Medical School – Central and is currently a PhD candidate. Susan is responsible for the study concept, design, analysis, and interpretation of the data and drafting of the manuscript. Craig Mellis is a pediatric chest physician, former Associate Dean and Head of Central Clinical School and is now Emeritus Professor at Sydney Medical School. Professor Mellis is responsible for the analysis and interpretation of data and critical revision of manuscript for important intellectual content. Both authors have read and approved the final version of the manuscript.

Disclosure

The authors report no competing conflicts of interest in this work.

References

1.		Cave J, Woolf K, Jones A, Dacre J. Easing the transition from student to doctor: How can medical schools help prepare their graduates for starting work? Med Teach. 2009;31(5):403–408.
2.		Illing J, Morrow G, Kergon C, et al. How prepared are medical graduates to begin practice? A comparison of three diverse UK medical schools (unpublished report). Final Project Report for the GMC Education Committee. London: General Medical Council; 2008. Available from: http://www.gmc-uk.org/FINAL_How_prepared_are_medical_graduates_to_begin_practice_September_08.pdf_29697834.pdf. Accessed January 3, 2017.
3.		Boots RJ, Egerton W, McKeering H, Winter H. They just don’t get enough! Variable intern experience in bedside procedural skills. Intern Med J. 2009;39(4):222–227.
4.		Bleakley A, Brennan N. Does undergraduate curriculum design make a difference to readiness to practice as a junior doctor? Med Teach. 2011;33(6)459–467.
5.		Tallentire VR, Smith SE, Wylde K, Cameron HS. Are medical graduates ready to face the challenges of Foundation training? Postgrad Med J. 2011;87(1031):590–595.
6.		Hilmer SN, Seale JP, Le Couteur DG, Crampton R, Liddle C. Do medical courses adequately prepare interns for safe and effective prescribing in New South Wales public hospitals? Intern Med J. 2009;39(7):428–434.
7.		Matheson C, Matheson D. How well prepared are medical students for their first year as doctors? The views of consultants and specialist registrars in two teaching hospitals. Postgrad Med J. 2009;85(1009):582–589.
8.		Laack TA, Newman JS, Goyal DG, Torsher LC. A 1-week simulated internship course helps prepare medical students for transition to residency. Simul Healthc. 2010;5(3):127–132.
9.		Blackmore C, Austin J, Lopushinsky SR, Donnon T. Effects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysis. J Grad Med Educ. 2014;6(4):643–652.
10.		Braniff C, Spence RA, Stevenson M, Boohan M, Watson P. Assistantship improves medical students’ perception of their preparedness for starting work. Med Teach. 2016;38(1):51–58.
11.		CETI. Preparing for Safe Practice as an Intern (PSPI) Program. [Curriculum guide]. Sydney: Health Education and Training Institute (HETI); 2012. Available from: http://www.heti.nsw.gov.au/Global/HETI-Resources/team-health/preparation-for-safe-practice-intern-pspi.pdf.
12.		Graham IS, Gleason AJ, Keogh GW, et al. Australian Curriculum Framework for junior doctors. Med J Aust. 2007;186(7 Suppl):S14–S19.
13.		Pettitt BJ. Medical student concerns and fears before their third-year surgical clerkship. Am J Surg. 2005;189(4):492–496.
14.		Brennan N, Corrigan O, Allard J, et al. The transition from medical student to junior doctor: today’s experiences of tomorrow’s doctors. Med Educ. 2010;44(5):449–458.
15.		Sydney Medical School, UoS. Pre-Internship (PRINT) Stage 3 Handbook. Vol Version 142014. Sydney: Sydney Medical School, UoS; 2016.
16.		Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6):1121–1134.
17.		Coombes ID, Mitchell CA, Stowasser DA. Safe medication practice: attitudes of medical students about to begin their intern year. Med Educ. 2008;42(4):427–431.
18.		Jones A, McArdle PJ, O’Neill PA. How well prepared are graduates for the role of pre-registration house officer? A comparison of the perceptions of new graduates and educational supervisors. Med Educ. 2001;35(6):578–584.
19.		Wall D, Bolshaw A, Carolan J. From undergraduate medical education to pre-registration house officer year: how prepared are students? Med Teach. 2006;28(5):435–439.
20.		Wakeford R, Roberts S. An evaluation of medical students’ practical experience upon qualification. Med Teach. 1982;4(4):140–143.
21.		Tallentire VR, Smith SE, Skinner J, Cameron HS. Understanding the behaviour of newly qualified doctors in acute care contexts. Med Euc. 2011;45:995–1005.
22.		Wenger E. Communities of practice and social learning systems: the career of a concept. In: Blackmore C, editor. Social Learning Systems and Communities of Practice. London: Springer London; 2010:179–198.
23.		Jones A, Willis SC, McArdle PJ, O’Neill PA. Learning the house officer role: reflections on the value of shadowing a PRHO. Med Teach. 2006;28(3):291–293.
24.		Drinkwater J. Can a medical student be of use while still learning? Clin Teach. 2007;4(4):189–192.
25.		Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med. 1991;66(12):762–769.
26.		Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38(4):358 –367.
27.		Coombes I, Mitchell C, Stowasser D. Safe medication practice tutorials: a practical approach to preparing prescribers. Clin Teach. 2007;4(3):128–134.