Effectiveness of Combined Flipped and Classroom-based Instruction in Teaching Pulmonary Ultrasound to First-year Medical Students

Introduction

The University of Santo Tomas (UST), Faculty of Medicine and Surgery, admits over five hundred first-year medical students yearly. For the past 9 to 10 years, the integration of ultrasound into medical education has been implemented. The UST Faculty of Medicine and Surgery is the first Asian medical school to have started teaching hands-on ultrasound across the first year to third-year levels in 2013. In the Department of Physiology, we have implemented hands-on exposure of the first-year medical students for the past nine years, both in respiratory and cardiovascular systems. The faculty staff spends over 3 hours per section. Since there are four sections in the first-year medicine level, this activity is repeated four times. The amount of time the faculty has been trained to spend on this activity is too demanding.

A flipped learning classroom is a teaching pattern that allows the learner to review a video of a lecture or subject matter, followed by application synthesis and practice in class, which is assessed in the end. Traditional teaching pattern includes homework or self-study of a subject matter, followed by a lecture by the teacher, practice, and finally, assessment. [1] In the advent of flipped classrooms, this teaching pattern may be beneficial in teaching lung ultrasound. In a study by Lee Kang and colleagues, they looked into using a blended curriculum in the form of web-based lectures followed by dynamic ultrasound scanning in small groups and compared it to traditional classroom-based instruction with hands-on training.[2] They concluded that there was no statistical difference in exam scores between the two groups and that they also implied that the web-based teaching model might be less expensive compared to the traditional classroom teaching model because of the lesser time spent during the actual hands-on scanning due to the previous instruction already taught on line.[2]

The flipped classroom model has helped ensure better instruction to students, especially those who need a lot of input or preparation before teaching the actual skill. In Eastern Virginia Medical School, they have guidelines on developing a good flipped classroom. They recommended that the videos prepared be short, optimally 5-7 minutes in duration, and focus on the important objectives presented clearly.[1] In this study, we likewise used a video to comprise the flipped classroom. Summative assessment and narrative tests were deployed also in this study. A narrative test is a step-by-step description of a particular task asked which involves recounting or retelling a story to another person.

Summative assessment, on the other hand, is the assessment of participants used to both assess the effectiveness of the program and the learning of the students. This is in contrast to formative assessment, which is also an assessment for learning but is used to monitor student learning and provide ongoing feedback to learners.

This paper investigated the effectiveness of the addition of flipped classrooms in the classroom based instruction of lung ultrasound in First-year medical students. Secondary objectives included determining the insights of the teachers and the students on the implementation of both interventions.

Materials and Methods

Research Design

This is a mixed quantitative and qualitative study, a concurrent triangulation type of mixed methods research study. The quantitative data and qualitative data were concurrently collected. The findings collected were confirmed, cross-validated, and corroborated to develop a cohesive data analysis. The use of qualitative results intends to support the findings in the quantitative analysis.

Participants of the study

All sections taking up Physiology were tested before and after the intervention. The sample size for the experimental group (flipped classroom) was derived from published studies that have done similar methods.[2,3] The flipped classroom group is composed of at least 135 students, while another group consisting of a minimum of 135 students will belong to the traditional classroom-based instruction alone.

Sample Size Computation:

A minimum of 135 participants in each group was used to achieve at least 80% power of the test, a two sided 5% level of significance. This study is based on the result of Kang et al. (2015) that classroom lecture group vs. online class group had a mean score of 82.2% (95% CI: 79 to 84%) and 79.9% (95% CI: 75 to 82%), respectively.2 G*Power ver 3.1 was used in the sample size calculation. The participants in the flipped classroom group were randomly selected. At least 135 students viewed the learning material before the hands-on lung ultrasound instruction.

Data Gathering

Phase 1: The pre-test summative and narrative

A narrative test was used to recount the step-by step way of doing the lung ultrasound scanning. The students were asked to write down patient instructions, the ultrasound probe and mode to be used, the position of the subject, proper manner of holding the transducer, where to scan and the image that will be obtained. The procedural knowledge and skills on the conduct of lung ultrasound scanning were assessed in the narrative test.

Summative tests were used to evaluate student learning and skills acquisition in the conduct of lung ultrasound and were composed of multiple-choice questions of test items consisting of 20 questions. The procedural knowledge and skills when doing lung ultrasounds were assessed in this summative test.

The pre-test summative and narrative tests were reviewed by face validity testing by two independent reviewers knowledgeable about lung ultrasound instruction. They approved the narrative and summative tests to be administered to the study participants. Reliability was ensured since the post-test was given right after the hands-on lung ultrasound scanning.

Phase 2: The intervention program

The intervention program composed of at least one hundred thirty-five students from a class in Physiology was assigned to view 7-minute video instruction on lung ultrasound scanning, which was commissioned by the Ultrasound Institute at the University of South Carolina; the video is available on youtube- https://www.youtube.com/watch?v=WOlz8-km6hE. They were also given lung ultrasound handouts. Students then practiced and performed ultrasound scanning to a patient model with a knowledgeable facilitator on lung ultrasound. The ultrasound units used were Sonosite ultrasound units model T from the UST Faculty of Medicine and Surgery. In the study’s other arm, at least one thirty-five First Year medical students proceeded right away with ultrasound scanning with their facilitators.

Phase 3: Post instruction Summative test and narrative test

A narrative test for listing the procedure of doing lung ultrasound scanning was given to the study participants.

Summative tests in the conduct of lung ultrasound scanning were composed of 20 items similar to the pretest multiple-choice question items administered during the pretest.

The post-test - summative and narrative tests were reviewed approved by face validity testing. Two independent reviewers knowledgeable on lung ultrasound instruction reviewed and approved the narrative and summative tests to be administered to the study participants.

Phase 4 Rubrics for Narrative Pre Test and Post Test

A rubric for the narrative examination was available to check the pre and post-narrative tests for both combinations of the Flipped classroom and traditional instruction alone. The components of the narrative’s rubrics are as follows: patient instructions, patient positioning, correct probe selection, and narration of technique to come up with the images being asked in the narrative test.[4-6]

Phase 5 Analysis of results using Paired T-test and Student’s T-Test

Means and its standard error (SEM) were used to summarize the scores of the two groups in the summative and narrative tests, while counts and percentages were used to summarize the number of students who got passing scores. The passing score in the summative test is 13 points (out of 20 points), while the passing score in the narrative test is 10 points (out of 20 points).

Paired t-test was used to compare the mean pre and post-test scores. At the same time, Student’s t-test was used to compare the means of the two groups.

Additionally, McNemar’s test compared the percentage of students who passed in pre-and post tests. In contrast, Fisher’s exact test compared the percentage of students who had passing scores in the two groups.

All the statistical tests were performed in SPSS ver 20.0. P-values less than 0.05 indicate significant differences.

A minimum of 135 participants in each group will be used in the study to achieve at least 80% power of the test, a two-sided 5% level of significance. This study is based on the result of Kang et al. (2015) that classroom lecture group vs. online class group had a mean score of 82.2% (95% CI: 79 to 84%) and 79.9% (95% CI: 75 to 82%), respectively. G*Power ver 3.1 was used in the sample size calculation

Data collection, instruments

We obtained baseline procedural knowledge and skills of a total of two hundred eighty-two first-year medical students on the lung ultrasound scanning using pre-test 20-item summative examination, multiple-choice question type of examination, and a pre-test narrative examination on lung ultrasound. One hundred forty-four of the students were randomly assigned to the flipped classroom instruction group, and 138 were randomly assigned to the traditional classroom-based instruction. The lung ultrasound video was viewed by the flipped classroom instruction with the traditional intervention group before the hands-on ultrasound session. In contrast, the traditional instruction group proceeded to the lung scanning with the facilitator and the patient volunteers. After the video instruction and the lung ultrasound scanning with the patient model and facilitator, a post-intervention summative and narrative examination were administered.

The responses to the survey given to the students were summarized, and thematic analysis was done to evaluate the responses. For the thematic analysis, a coding technique utilized was word repetition, keyword, context, cutting, and sorting techniques. The qualitative data as determined through the survey was used to explain the results in the quantitative data. From the coding, themes were developed, and the qualitative data were triangulated to support the data gathered in both types of data. The mixed type of research is a concurrent triangulated type that could concurrently explain the quantitative and qualitative data. It helps answer the research question of the effectiveness of these two instruction strategies in lung ultrasound.

Ethical Considerations and Investigational Review Board

Any information that was obtained from the data collection was kept strictly confidential. The subjects were not identified by name in the data collection form and will not be placed in any future publication of the results. Moreover, the information contained within the data collection form will only be used for this study. Consent for the study was obtained from all the participants of the study. Before conducting the study’s methodology, the research proposal was submitted and approved by the Investigational Review Board of the University of Santo Tomas, Faculty of Medicine and Surgery. This study was performed in accordance with the Declaration of Helsinki. The University of Santo Tomas approved this human study, Faculty of Medicine and Surgery. All adult participants provided written informed consent to participate in this study.

Statistical Analyses

Means and its standard error (SEM) were used to summarize the scores of the two groups in the summative and narrative tests, while counts and percentages were used to summarize the number of students who got passing scores. The passing score in the summative test is 13 points (out of 20 points), while the passing score in the narrative test is 10 points (out of 20 points).

Paired t-test was used to compare the mean pre and post-test scores. At the same time, Student’s t-test was used to compare the means of the two groups. All the statistical tests were performed in SPSS ver 20.0. P-values less than 0.05 indicate significant differences.

Results

A total of 283 students were included in this study, where 139 (49.1%) were given classroom-based instruction only, while 144 (50.9%) were given classroom-based instruction and flipped classroom.

The mean summative test scores of the students who were given flipped classroom and classroom-based instruction significantly increased [Pre-test (Mean ± SEM) : 7.97 ± 0.23 vs Post-test: 14.25 ± 0.17; t143=23.893, p<0.001], and those who were given classroom-based instructions only, likewise significantly increased [Pre-test: 7.99 ± 0.22 vs Post-test: 14.16 ± 0.18; t138=24.376, p<0.001]. The mean increase in the summative test scores of students who were given flipped classroom and classroom-based instruction [6.27 ± 0.26] and those who were given classroom-based instruction only [6.17 ± 0.25] did not significantly differ [t281=0.289, p=0.773].

Meanwhile, the mean narrative test scores of the students who were given flipped classroom and classroom-based instruction significantly increased [Pre-test: 3.76 ± 0.19 vs Post-test: 12.08 ± 0.21; t143=32.075, p<0.001], and those who were given classroom-based instructions only, likewise significantly increased [Pre-test: 3.99 ± 0.21 vs Post test: 12.14 ± 0.24; t138=29.751, p<0.001]. The mean increase in the narrative test scores of students who were given flipped classroom and classroom-based instruction [8.31 ± 0.26] and those who were given classroom-based instruction only [8.15 ± 0.27] did not significantly differ [t281=0.447, p=0.656].

Combining their scores in the summative and narrative tests, the mean total test scores of the students who were given flipped classroom and classroom-based instruction significantly increased [Pre-test: 11.74 ± 0.35 vs Post-test: 26.33 ± 0.35; t143=37.111, p<0.001], and those who were given classroom-based instructions only, likewise significantly increased [Pre-test: 11.99 ± 0.321 vs Post-test: 26.30 ± 0.34; t138=37.180, p<0.001]. The mean increase in the total test scores of students who were given flipped classroom and classroom-based instruction [14.59 ± 0.39] and those who were given classroom-based instruction only [14.32 ± 0.39] did not significantly differ [t281=0.497, p=0.620].

Number of Students with Passing Scores in the Summative and Narrative Tests

Out of the 20-point summative test, 13 points were used as the cut-off to pass. The number of students who passed the summative (MCQ) test and were given flipped classroom and classroom-based instructions significantly increased [(Pre-test to Post-test): 6.3% to 79.9%; p<0.001]. Likewise, the number of students given classroom-based instructions only, significantly increased [4.3% to 79.9%; p<0.001]. However, the number of students in the two groups who passed in the pre-test (p=0.598) and post-test (p=1.000) did not differ.

On the other hand, 10 points (out of 20 points) were used as a cut-off to pass the narrative test. The number of students who passed the narrative test and were given flipped classroom and classroom-based instructions significantly increased [2.1% to 84.0%; p<0.001]. Likewise, the number of students given classroom-based instructions only significantly increased [3.6% to 84.2%; p<0.001]. However, the number of students in the two groups who passed in the pre-test (p=0.495) and post-test (p=1.000) did not differ.

Out of 40 points, the total score used 23 points as the cut-off to pass. The number of students who passed the total score and were given flipped classroom and classroom-based instructions significantly increased [2.1% to 85.4%; p<0.001]. Likewise, the number of students given classroom-based instructions only significantly increased [0.7% to 83.5%; p<0.001]. However, the number of students in the two groups who passed in the pre-test (p=0.623) and post-test (p=0.743) did not differ.

Qualitative Data

From the questionnaire, the students appreciated the classroom-based instruction because of the knowledgeable facilitators’ very concise approach that is understandable and done in real-time. In combination with the traditional instruction, the flipped classroom was likewise helpful and a good introduction before the classroom-based instruction. Some have commented that the flipped classroom in the form of a video shown to the students was too long. The facilitators have noticed that the ease of instruction is not determined by whether the students have viewed the video before the classroom-based instruction. Instead, it would be influenced by the student’s enthusiasm and willingness to learn. The facilitators also added that some of the students, even when not exposed to the flipped classroom, grasped the hands-on instruction very quickly compared to those who benefited from being exposed to the flipped classroom.

Discussion

This study showed a significant increase in the narrative and summative scores in both the flipped classroom combined with traditional classroom-based instruction and the classroom-based instruction alone in teaching lung ultrasound to first-year medical students. All the scores increased and were statistically significant using the paired T-test. However, in the mean scores of the two groups, in both the narrative and summative tests, there was no statistically significant difference as assessed by Student’s T-test. It may be surmised that the intervention of hands-on or classroom-based instruction alone may have been effective enough to increase the students’ test scores. However, using the survey questionnaire given to the students, most of them appreciated the hands-on, classroom-based instruction on teaching lung ultrasound. They commented that the facilitators were knowledgeable and helped them learn and do the lung scanning. Even if they thought that the ultrasound video helped them, some students commented that the video was too long. Most said that more time during classroom instruction would be best to let them learn lung ultrasound scanning. The facilitators, o the other hand, noticed that most of the students in the flipped classroom group had an easier grasp of what they were teaching during the classroom-based instruction, which is the hands-on lung scanning, compared to those who were not exposed to the flipped classroom group. As seen in the studies by Harrison and Harris in 2014, the flipped classroom teaching method entails time and adequate preparation to be effective and may be challenging for the teaching staff.6 In connection with this, however, the facilitators have added that the student’s level of participation and interest would be a more important determinant on whether the students will learn more effectively during the classroom-based instruction regardless of whether they were given the flipped classroom instruction or not.

Conclusion

The combination of the flipped classroom and classroom-based instruction in teaching pulmonary ultrasound had similar outcomes in post-test narrative scores and post-test summative scores. However, both the post-test scores significantly increased compared to the pretest narrative and summative test scores for both groups. The students appreciated the classroom based instruction because of the knowledgeable facilitators. In contrast, some have appreciated that the flipped classroom was advantageous before the classroom-based instruction because it is a good introduction for them. The facilitators have admitted that the ease in teaching the students was not determined by whether they belonged to the flipped classroom group. What was more important was the willingness of the students to learn and their enthusiasm which made them learn more effectively.

Disclosure and Conflict of Interest

There is no conflict of interest or any factor which may inappropriately influence bias in the execution of the research and publication of the manuscript by the author

Acknowledgment

Special thanks to the Faculty of Medicine and Surgery Administrators, Dean Ma. Lourdes Maglinao, Department of Physiology and my chairman Professor Dr. Elizabeth Angelica Roasa, and Department of Internal Medicine of the University of Santo Tomas, Faculty of Medicine and Surgery for the assistance in the completion of the research and publication of the manuscript. Special mention to my research adviser, Professor Marcela Leus, to my research professor Professor Cheryl Peralta, my statistician Asst. Professor Xandro Nieto and my Medical Students.