Intended learning outcomes and content
The intended learning outcomes (ILO) and content were gathered from two resources. First, the previously published frameworks by Society for Academic Emergency Medicine and International Federation for Emergency Medicine curriculums provided the core structure [6, 7]. The UAEU-undergraduate EM curriculum was further revised in 2014 based on EM student milestones [11]. Second, learning needs assessment was applied to detect local needs. The feedback from stakeholders, including medical students, faculty members, EM residents, nurses, and patients were collected via surveys. These included either paper-based surveys, electronic based surveys, or face-to-face interviews which were collected during the 2013–2014 academic year. Feedbacks have been continuously collected via electronic surveys and face to face interviews for quality improvement. The main results of feedbacks and proposed improvements for the curriculum are shown in Additional file 1: Appendix 1. The final version of UAEU-undergraduate EM curriculum includes eleven chief complaint and six procedure topics (Additional file 1: Appendix 2).
Teaching and learning methods
There are five teaching and learning (TL) aspects of our curriculum; classroom activities, skills and simulation laboratory sessions, clinical exposure, feedback, and the online platform. The use of e-learning tools is a substantial part of our TL activities. However, it does not decrease face-to-face time between faculty/tutors and students. Therefore, this format is a technology-enhanced learning [12].
Classroom activities
Classroom activities are divided into two broad groups according to their topics: (a) chief complaint-related and (b) procedural. Case discussions, flipped classroom, and team-based learning (TBL) form the basis for the chief complaint-related activities. The aim of the chief complaint-related activities is mainly to facilitate students’ differential diagnosis on the concept of ruling out the worst-case scenario, critical decision-making processes, and case management. Factual knowledge is not a core objective during chief complaint-related sessions. The students are expected to attend these sessions prepared by studying recommended pre-class materials regarding each chief complaint. This includes instructor-made videos, university’s online library resources, and external web links (video, web page, and online textbooks). Eight out of 11 chief complaint topics are delivered via TBL sessions. Each TBL session is 90 min. The rest three topics are provided as case discussions. The duration of each case discussion session is 60 min. This includes three to four real-life case scenarios. Case discussions are facilitated by clerkship director, core-faculty members, or senior (fourth year) EM residents. Case discussions, flipped classroom, and TBL have been, respectively, implemented during 2013–2014, 2015–2016, and 2016–2017 academic years.
TBL is a variation of flipped classroom [13, 14]. It includes three stages [15, 16]. In stage 1, students prepare individually for the class. In stage 2, they take an individual readiness assurance test (i-RAT). Immediately after the i-RAT, permanent groups of six to seven students take the same test. This is called group or team readiness assurance test (t-RAT). Group members discuss the questions to reach the best answers. Both i-RAT and t-RAT provide information about learning gaps. In stage 3, detected learning gaps are addressed via collaborative group activities called application exercise. In our curriculum, case scenarios are the main method used in this stage.
Lectures presented in the classroom are only related to procedural topics. This includes Extended Focused Assessment with Sonography in Trauma (E-FAST) and Rapid Ultrasonography in Shock and Hypotension (RUSH) protocol. These lectures are given by the clerkship director. The practical sessions of these procedures are held separately.
Skills and simulation laboratory sessions
There are six procedures covered in the curriculum: cardiac arrest and arrhythmia management, airway management, suturing, intraosseous line insertion, E-FAST, and RUSH protocols. Students are expected to prepare for the sessions. We are using classroom teaching only for EFAST and RUSH protocols. Each practical session (100 min) consists an introductory presentation for 10 min followed by supervised practice on high-fidelity manikins, skills training models or simulated patients for 90 min. Additional supervised and non-supervised practice sessions are available as needed (Additional file 1: Appendix 2). We used Gagne et al.’s instructional design for procedure education [17]. These levels are the following: level 1, gaining attention; level 2, informing learner of objectives; level 3, stimulate recall of prior learning; level 4, presenting stimulus; level 5, providing learning guidance; level 6, eliciting performance; level 7, providing feedback; level 8, assessing performance; and level 9, enhancing retention and transfer.
Clinical exposure
Students have to complete 10 clinical shifts. The 9-h shifts are different depending on the location, ED section, ED unit, time, supervision, clinical severity, and learning experience. Students rotate in two main hospitals, each treating over 115,000 emergency patients annually. One of these hospitals provides EM residency program. Students are assigned to adult and pediatric ED sections. They undertake shifts in resuscitation, urgent care, and fast-track units. Students participate in different time of shifts including day time (08:00–17:00), prime time (13:00–22:00), evening time (15:00–24:00), and night time (23:00–08:00) under the supervision of senior EM residents and attending emergency physicians. These include core faculty members. Ideally, a maximum of two students are allowed to be in one unit per shift to ensure enough patient and procedure exposure for each student. Occasionally, this rule was not followed because of the relatively large number of students.
Feedback
Feedback is at the core of UAEU-undergraduate EM curriculum. Several types of feedback from various resources are available during this clerkship. First, self-assessment surveys provide an opportunity for self-reflection. Two surveys, one at the beginning and one at the end of the TL activities, are conducted to induce insights into the students’ strength and weakness regarding ILOs. Additionally, the TL activities are slightly modified to address personal anticipated needs according to the survey results.
Second, brief feedback is an integral part of all classroom and skills practice sessions. These sessions are designed to enable peer and tutor feedback. Tutors provide feedback during the case discussions and skill sessions. Under-achieved areas are detected during discussions and reviewed at the end of the sessions. Furthermore, the highlights of each session are shared on a collaborative online platform (OneNote).
Third, individual and group feedback sessions are held on alternate weeks. These two-way feedback sessions are facilitated by the clerkship director. The duration of each group session is 30 min. The duration of individual feedback session is 20 min per student. Both types of sessions focus on the review of students’ learning activities, clinical performance, TBL assessment results, and areas that need improvement as well as the students’ recommendations to improve the program.
Fourth, the collaborative online platform supports feedback in different ways. The summary of the feedback on the under-achieved areas is published online after each session. Furthermore, the clerkship director weekly creates additional group-specific tasks and case discussions to induce discussion on areas that need improvement. The clerkship director provides his online feedback on the students’ responses for the weekly cases so as to close the learning gap. This feedback is available for the whole group to enhance learning and promote self-reflection.
Online platforms and supportive applications
Online platforms
Undergraduate EM curriculum contains a number of online platforms to enhance learning experiences such as Google Forms, Socrative app, Microsoft OneNote, and SharePoint.
Google forms
Self-assessment surveys, evaluations of student case presentations, tutor case discussions, and faculty teaching performance are conducted via Google Forms [18]. Google Forms provides multiple advantages including easiness to create, share, and analyze with free of charge.
Socrative
Socrative [19] is mainly used for assessments during TBL sessions and during clerkship final evaluation surveys. Socrative is a free online platform. It is compatible with several mobile devices and provides several question types including true-false, multiple choice, and short answer questions. One of the advantages of this platform is the real-time observation of the answers.
OneNote
OneNote [20] provides a private virtual classroom environment. Tutors and invited students can establish a two-way or group communication. The details of the curriculum and ILOs, additional weekly case discussions, case discussion highlights, self-assessment surveys’ results, student case presentations, and students’ selected clinical cases can be shared on OneNote. This platform needs institutional membership to provide online classroom environment.
SharePoint
SharePoint [21] is the electronic logbook (e-Portfolio) medium of UAEU undergraduate EM curriculum. Students need to record every case and procedure they perform during clinical shifts. Additionally, this platform hosts students’ shift evaluation and supervisors’ student evaluation, which need to be completed after each shift. These results are continuously monitored by the clerkship director to feedback the students, supervisors, and staff.
Supportive applications
DartSim
DartSim [22] mimics the functions and audio-visual features of a real defibrillator and monitor, so it enables to imitate a more realistic ED environment. Displayed on the screen, there are vital signs, ECG rhythm strip, buttons for a 12-lead ECG, defibrillator, CPR, pacing, patient X-ray, and drugs options which can be manually managed by students or remotely by the tutor.
Resuscitation!
Resuscitation! [23] provides more than 100 case scenarios for EM-related chief complaints. Scenarios start with only a case vignette. Students lead the scenario to solve the case. The application provides vital signs monitoring, examination, diagnosis, order choices grouped from critical to harmful options. The choices made are recorded by the application for review.
eOSCE
The clerkship director uses Electronic Objective Structured Clinical Evaluation (e-OSCE) desktop program to compose evaluation sheets for OSCE exams [24]. Examiners may evaluate and record the students’ performance using these sheets via e-OSCE application on iOS tablets. The clerkship director can remotely monitor the results on real-time. Once the evaluation forms are downloaded from the cloud, this validated application does not need a continuous internet connection to work [24]. All information collected can be uploaded to the cloud when the Wi-Fi connection is re-established.
Assessments
UAEU undergraduate EM curriculum contains multiple formative and summative assessments. The formative assessments mainly include the answers to the additional weekly cases in OneNote. However, flipped classroom with case discussions, TBL sessions, and skills practice sessions were used as formative feedback tools. There are six summative assessments in the clerkship. Forty percent of summative assessments are from OSCE and final MCQ exam. The details of summative assessments are summarized in Additional file 1: Appendix 3.
Evaluation of the program
Individual and group feedback sessions, clerkship evaluation survey, senior residents’ and core faculty members’ feedback, and summative assessment results are taken into consideration when evaluating the program. The evaluation process is completed between June and July. Required changes are implemented to curriculum before the new academic year. All students’ groups are exposed to the same updated curriculum. There is no change applied during the academic year.
Students’ exposure and need for revision sessions
Students start the clerkship with self-assessment entry survey regarding ILOs. They are exposed to 47 h of TL activities assisted by the clerkship director and tutors in the classroom and skills/simulation laboratory. During the classroom activities, we discuss 30 to 44 cases depending on the needs of the students. These cases are determined by the results of individual and team readiness assurance test of TBL activity. For example, if individuals or groups are lacking to reach an ILO by having wrong answers to questions, then this ILO is stressed during the case discussions of the application exercise step of TBL. We also share 16 additional cases in OneNote (Additional file 1: Appendix 4). The cases in the OneNote are shared to improve the understanding of the ILOs after the flipped classroom and TBL sessions. Groups also present their 13–18 case presentations. Students are free to choose their cases to be presented depending on their level and learning needs. This approach gives them an opportunity to master the knowledge of any topic they want. Students also share the cases they were exposed to during their clinical shifts. Their minimum case discussion exposure in the TL activities is 59 cases during the four weeks clerkship. Students are exposed to 120 MCQs during their TBL sessions before their final exam. Students experience practical skills for 13.5 h before their OSCE. They receive 90-min group feedback session and 20-min individual feedback session. Students’ clinical exposure is 90 h in which they see an average of 68.3 (SD 17.6) patients and perform an average of 46.1 (SD 14.0) procedures [25].
After the curriculum content is covered and all measures are tried to reach ILOs, students take an EM clerkship self-assessment exit survey which includes the same items they did as an entry survey. They re-assess their confidence levels regarding the ILOs. The results of the exit survey are reviewed by the clerkship director. The ILO items which received less than 80% achievement are listed on the revision day topics. Revision topics are discussed with the students one by one under the guidance of the ILOs. At the end of the revision session, students are provided with the information regarding the coming two exams, MCQ and OSCE. After this process, they are considered to be eligible to proceed for both exams.