A comparison of teaching three common ear, nose, and throat conditions to medical students through video podcasts and written handouts: a pilot study

Background

Ear, nose, and throat (ENT) surgery must compete with many other subjects for space in undergraduate medical education. As highlighted by Mace and Narula in their review of ENT education in 26 universities in the United Kingdom, ENT only receives an average of 1.5 weeks in the medical school curriculum and is often taught as a combined subject with other specialties.¹ Consequently, it is imperative for ENT education to be delivered in an efficient manner and utilize the best methods of delivery available. Medical education is undergoing vast changes. Education no longer follows the traditional model of instructor-centered education, it is now more centered around independent learning, with the emphasis on the learner constructing their own knowledge.²

Reusable learning objects (RLOs) are a new method of delivering medical education that incorporate a spectrum of visual, auditory, and interactive learning resources. A learning object can be defined as a grouping of instructional materials to meet a specified educational objective. A digital learning object is one which can be electronically stored and may utilize text, graphic, animations, audio, and video to support and enhance learning. Due to their electronic nature, they can be considered to be reusable since they can be accessed by many users if hosted online.³ RLOs are a resource that are designed to be used in self-directed learning (SDL) and are therefore an ideal tool to help students in modern medical education.

Video podcasts have the added advantage of engaging students in both auditory and spatial learning compared to other SDL tools such as handouts, that allow spatial learning only. There have only been very few studies that have directly compared the effect of podcasts with other SDL tools in terms of learning topics in medicine.^4–6 This pilot study compares the use of podcasts with more traditional handouts in self-directed education across three common ENT conditions namely epistaxis, otitis media, and tonsillitis.

Methods

Ethics approval and recruitment

The Peninsula College of Medicine and Dentistry Ethics Committee approved the study. All second-year medical students at Peninsula Medical School within the Plymouth locality (103 students) were notified of the study via university email account. Attached to the email were the study protocol, a frequently asked questions sheet, and a consent form. Second-year students were chosen as the target population since they have received no formal ENT teaching in their prior curriculum. The students were initially given 1 week to respond to the email. This was extended to 2 weeks with an email reminder at the end of the first week in order to increase the chances of gaining a statistically significant sample size.

Randomization

Students were randomized to the podcasts group or the handout group using a computerized random number generator.

Pre-exposure test

Following randomization, the participants were sent a unique link to a Survey Monkey™ 20 true or false statements for each of the three topics (epistaxis, tonsillitis, and otitis media). The questions were written by the senior author, an ENT consultant from Plymouth Hospitals NHS Trust. An online and remote method was chosen to conduct the entire study because a previous attempt to invite the students to sit an exam under exam conditions onsite at the Medical School had only yielded eight responses. The students were asked to not use any resources while sitting the test and that it should take no longer than 10 minutes. The test would “lock” after one login attempt. Therefore, the test would need to be completed in one sitting. The 41 respondents were given a 2-week period to complete the test in an attempt to reduce the dropout rate.

Exposure to resources

Upon completion of the initial exam, the participants were given access to their allocated learning resource for a 3-week period. Each resource was hosted on the Peninsula Medical School’s virtual learning environment (EMILY). The relevant resource was made visible on the homepage of the student by using their individual login ID. Therefore, nobody apart from the investigators was able to see both resources. Students in the handout group were able to save or print their handouts. Students were requested not to share each other’s resources or use any supporting material including the printed ­handouts when completing the post-exposure test. There were no additional instructions, so the students were able to use their allocated resource as they saw fit.

Post-exposure test

After the completion of the 3-week period, the resources were removed from EMILY and the students were sent an email with a link to the same 60-item Survey Monkey™ test they sat previously. Again, students were given 2 weeks to complete this under the same conditions as before.

Student questionnaire

Qualitative data regarding level of enjoyment of the learning resource were also collected from the podcast group in the form of a 5-point Likert scale questionnaire.

Statistical analysis

Comparisons were made on the students’ performance scores before and after the interventions. The mean and standard deviations of the pre- and post-exposure test results of students in both groups were computed and the normality of data tested using the Shapiro–Wilk test. Mean difference between pre- and post-exposure scores were calculated for each group. The Shapiro–Wilk test showed the data to be normally distributed. These data were then analyzed for a significant difference using a Student’s paired t-test. The null hypothesis was that there was no improvement between pre- and post-test scores in the podcast group when compared with the handout group. The difference was considered statistically significant if P=<0.05. The breakdown of the data by subject allowed us to see whether podcasts were effective across a range of different subjects, or more effective in some and not others.

The data of the Likert questionnaire were analyzed by calculating the mode response for each question.

Results

Figure 1 is a flowchart of the study. Forty-one out of 103 students responded to the email inviting them to participate in the study (39.8%). Thirty out of the 41 students who agreed to participate in the study completed the pre-exposure test (73.2%). Eighteen students out of 41 who agreed to participate in the study completed the post-exposure test (43.9%).

Figure 1 Population flowchart.

Figures 2 and 3 are box plots of test scores pre- and post-exposure. There was a statistically significant improvement in knowledge scores after provision of both podcasts and written handouts. Figure 4 illustrates the average post-exposure test scores for the podcast and written handouts groups. The average improvement from the podcast group was 4.7 (P=0.004, 95% confidence interval =0.07) compared to 5.3 from the handouts group (P=0.015, 95% confidence interval =0.11). There was however no statistically significant difference found between the use of podcasts and written handouts in improving students’ knowledge scores (average post-exposure scores 37.8 vs 38.3, P=0.07). Figures 5 and 6 illustrate the findings of the Likert scale questionnaire responses of students in the ­podcast group. The podcast questionnaire showed in addition, that students valued this as a learning resource and found it to be a more enjoyable method of learning. The dropout rates post-exposure was identical in both groups (n=6). Only 40% (n=6) of the podcast group returned their satisfaction questionnaires.

Figure 2 Box plot of average pre-exposure test scores.

Figure 3 Box plot of average post-exposure test scores.

Figure 4 Average post-exposure test scores.

Figure 5 Students’ opinions on use of video podcasts.

Figure 6 Students’ opinions on usefulness of video podcasts compared to written material.

Discussion

Our study demonstrates that video podcasts are as effective as written handouts in teaching three common ENT conditions. However, the students’ learning experience in the podcast group appeared to be enhanced compared to written information they traditionally use as the podcasts were more enjoyable. The advantages and disadvantages of RLO’s have been summarized in Table 1 which is adapted from a literature review.⁷ Our findings are supported by the limited literature available on the use of podcasts for medical students.

Table 1 Advantages and disadvantages of digital learning objects Notes: Data from Sandars J, Schroter S.7 Abbreviation:RLOs, reusable learning objects.

Cook published a review on the use of web-based learning (WBL) in medical education. He concluded that the advantages of WBL include overcoming barriers of distance and time, economies of scale, and novel instructional methods. The disadvantages include social isolation, up-front costs, and technical problems. WBL instructional designs failed to always abide by educational principles and may sometimes be used in an incorrect educational context or application.⁸

Bhatti et al evaluated the benefit of educating medical students on a common surgical topic (hemorrhoids), through a website and podcast package vs a traditional lecture. Similar to our study, the authors used a pre-intervention questionnaire and an identical post-intervention one. They concluded that using augmented web-based educational tools reduces demands on teaching time with no decrease in quality for selected parts of the curriculum.⁴

Kumar et al implemented a hybrid curriculum that involved both traditional teaching methods and a new online core curriculum that incorporates audio, video, and text using screen capture technology in a surgical intensive care unit clinical placement. The curriculum was hosted on a secure online portal and covered topics that were considered essential to meet the didactic objectives of the rotation. Students in both groups completed a pre-test on day 1 of the rotation and a post-test administered on the second to last day of the rotation. Unlike our study, the authors did not use identical questionnaires but randomly selected ones before and after the intervention. Again, the authors found that using this form of learning enhanced the satisfaction of students on a busy intensive care unit placement and improved their knowledge-based test scores.⁶

A study by Peacock et al came to a different conclusion from our study. They conducted a prospective observational study of all medical students undertaking their first attachment in surgery. Students were randomly allocated to four groups: lecture, handout, website, and patient-based tutorial, all on the management of an inguinal hernia. Although the knowledge scores improved in the lecture and website groups, the learning in the website group was thought to be less engaging for students.⁵

With the evolution of technology and its ever growing degree of accessibility, the potential role of video podcasts in medical education should be recognized and utilized. This can be especially useful in modern medical degree programs which favor a problem-based learning approach. These ­programs often have limited face-to-face teaching time and rely on individual independent learning. The use of podcasts here could help standardize the information which students are given on a particular topic and can be accessed in their own time and at their own pace.

The potential implications for teachers involved in medical education are highlighted in Table 1. However, the usefulness and enjoyment of podcasts from the perspective of the clinical teacher goes beyond the scope of this study.

The main limitation of this pilot relates to the number of participating students as a type II error cannot be excluded. The results of this study can assist us in calculating the sample size required to perform a larger scale study. The dropout rate for students completing voluntary assessments is not unusual. Similarly, it is not unusual to have low response rates for study questionnaires. The students did not sit the test under exam conditions and were able to sit both the pre- and post-exposure test at any point in a 2-week period. The investigators were aware of this limitation at the start of the investigation; however, it was deemed necessary in order to gain a much larger sample size than was previously obtained. Although students were requested not to share resources, there is no way to find out if this has occurred. Providing the students with the same pre- and post-test assessment could have led to a “repeat testing bias”. This risk is mitigated by the fact that students did not have prior experience of ENT when they sat the first test. Moreover, test equating has its challenges and using a different test post-exposure carries the risk of using an assessment that does not have the same degree of difficulty as the pre-test.

Given that there are very few studies that have looked at the effectiveness of video podcasts as teaching tools for medical students,^4–6 we therefore propose a definitive follow-up study that includes more than one medical discipline. This study has shown that RLOs such as video podcasts are as good a resource as handouts in terms of knowledge test scores, but also that they are more enjoyable.

Conclusion

Video podcasts are as good as written handouts in the teaching and learning of three common ENT conditions. Video podcasts appear to enhance the students’ learning experience. Further large-scale studies are required to demonstrate the effectiveness of podcasts in medical education compared to other widely used learning resources and involving multiple disciplines.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Mace AD, Narula AA. Survey of current undergraduate otolaryngology training in the United Kingdom. J Laryngol Olol. 2004; 118(3):217–220.
2.		Bannan-Ritland B. The role of design in research: the integrative learning design framework. Educational Researcher. 2003;32(1):21–24.
3.		Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81(3):207–212.
4.		Bhatti I, Jones K, Richardson L, Foreman D, Lund J, Tierney G. E-learning vs lecture: which is the best approach to surgical teaching? Colorectal Dis. 2011;13(4):459–462.
5.		Peacock O, Watts E, Foreman D, Lund JN, Tierney GM. Evaluation of teaching methods for students on hernias: an observational study. ANZ J Surg. 2013;83(1-2):11–14.
6.		Kumar AB, Hata JS, Bayman EO, Krishnan S. Implementing a hybrid web-based curriculum for an elective medical student clerkship in a busy surgical intensive care unit (ICU): effect on test and satisfaction scores. J Surg Educ. 2013;70(1):109–116.
7.		Sandars J, Schroter S. Web 2.0 technologies for undergraduate and postgraduate medical education: an online survey. Postgrad Med J. 2007;83(9856):759–762.
8.		Cook DA. Web-based learning: pros, cons and controversies. Clin Med (Lond). 2007;7(1):37–42.