Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Acute critical illness is routinely treated by highly trained staff in specialized care units. However, the initial resuscitation, the “Golden Hour” (after the initial hour following physiological insult or trauma is most crucial for successful resuscitation) can be impeded by inefficient processes of care delivery [1]. A key factor contributing to this is that critically ill patients continually generate vast quantities of clinical data [2]. This information can overwhelm providers, especially those not specifically trained to work in fast-paced, high-stress environments [3]. It is crucial to develop strategies that streamline the processes of care to minimize clinical misjudgment [4].

Checklists are a validated tool for dealing with such challenges; long adopted by aviation and nuclear industries, they have recently been demonstrated invaluable in standardizing and improving clinical care [5]. Haynes et al. showed a decreased in complications and 30-day mortality after non-cardiac surgery by implementing a surgical safety checklist [6].

However, checklist implementation has been slow in many settings, often due to a perception that the tool disturbs existing workflows. One approach to reduce the disruption is the presence of an additional team member, a checklist “prompter” specifically tasked with ensuring checklist completion. This has been tested during ICU [7] rounds, and it was associated with decreased mortality and shorter length of stay [8].

Proper care and timely interventions are critical in initial resuscitation [1, 9–12] especially in situations with diverse provider background and training [13]. Algorithms like advance cardiac life support (ACLS) and advance trauma life supports (ATLS) have been created to structure the care provided in emergent cases. However, none of these algorithms address the more frequently encountered clinical problems like altered mental state, respiratory distress, syncope, and sepsis with a prescribed structured approach. Furthermore, these packages are based on memorizing and recall rather than providing easy access to informational cues to guide the resuscitation workflow.

It seems evident that providing an interface capable of summarizing this information and providing decision support and validated management algorithms would address a critical need [14]. We have recently designed and developed a novel electronic tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), to apply a standard approach to evaluation and management of the acutely decompensating patient which included process of care workflows and a designated prompter [8]. We hypothesized that a standardized approach to the evaluation and management of acutely decompensating patients using the CERTAIN model would improve clinicians’ performance and satisfaction in a simulated acute care environment.

From 24 participants enrolled in the simulation study, 18 completed both baseline and follow-up assessments; 11 critical care fellows, two residents, two visiting clinicians, and three medical students. Two thirds of the study participants were clinicians with at least 6 months of formal critical care training and were considered with high level of expertise.

In a high-fidelity simulation environment prompting with the CERTAIN improved clinical task completion and decreased omission rates of critical tasks. Clinician satisfaction was high, with majority of participants wanting to incorporate this method to their own clinical practice.

Traditionally, resuscitation teams have been formed and trained to perform under stressful situations where they have to rapidly coordinate evaluation and treatment efforts [17, 18]. However, they have been structured around specific types of illness like trauma or cardiac arrest and largely rely on memory recall [18, 19]. Our experience demonstrated the performance of clinicians under stress is suboptimal at best, with an overall task completion rate of 50% at baseline. This is consistent with the findings of Smith et al. showing a decline in skill retention and loss of ability to perform ACLS and BLS skills to standard level when re-tested at 12 months [20].

As a potential solution, other studies have evaluated memory aids to improve health team performance in other situations. Haynes et al. tested the use of a checklist applied to regular operating room workflow on elective surgeries. His work showed a decrease on preventable surgical-related complications in the operating room [6]. Two studies designed for emergency teams dealing with late-phase resuscitation, one with a smart phone application [21] and the other a traditional checklist [22], had promising results in simulated environments. However, their designs did not include a prompter, which likely reduced their team’s compliance. Prompting with the CERTAIN is aimed to approach critical illness resuscitation earlier in the natural course of the disease, targeting to standardize the care on the so-called Golden hour. Early structured treatment has been shown to give better outcomes in simulated operating room crises [16], sepsis [23], myocardial infarction [24], and other critical illness states and providing a unified approach to decompensation may prevent the need of cardiopulmonary resuscitation efforts.

In a different setting, Weiss et al. tested the usefulness of prompting in critical care practice by implementing the use of checklist with a prompter versus checklist alone during daily ICU rounds. This study showed improvement in compliance with process of care, decreased length of stay, and a decrease in mortality [7]. These findings clearly demonstrate that important role prompting can play in facilitating complex process of care. The present study differs, however, in using a prompter in a higher stress environment with simulated acute medical emergencies. This stress imposes an extra burden on providers which could increase the risk of task omission. In this sense, the choreography of the CERTAIN with a prompter combines the lessons of prior studies on checklists and prompting with leadership “best practices” [25], such as egalitarian leadership [26] and closed-loop communication [27].

Ideally, one of the existing team members should be able to play a role of prompter, without any added cost. However, in resource constraint situations, the team lead should focus on resuscitation and as soon as the time allows review the checklist to see if anything has been missed.

The CERTAIN approach was well received by the participants. However, it is worth noting that software usability limitations were evident in the post-intervention survey. With only 33% of participants feeling the software is easy to use, a combination of improved training and interface may be necessary to make this practical in real high-stress environments. In this simulation study, due to volunteers’ time and schedule constraints, the training was limited to 90 min. Most initial training in clinical resuscitation models (ACLS, BLS, or ATLS) are 12 h courses (usually two full training days) [28].

Another potential limitation could be that, even though the didactic sessions and practice scenarios were focused on tool usability rather than the scenario performance, having these done just prior to final testing could have influenced the performance.

The inferences from our results are further limited due to the simulation nature of this study, as well as the small sample size. Simulation training has been increasingly suggested as a valid research and training tool paired with good outcomes which makes it the ideal scenario to test a new method and clinical software [29]. The spectrum of scenarios encountered by our test subjects was limited to three common types: respiratory distress and hypoxia, hypotension due to severe sepsis/septic shock, and chest pain secondary to acute coronary syndrome. These cases were chosen as they are the most common clinical presentations in hospitalized medical patients [30–33]. Each clinician serving as his/her own control minimized the effect of variability in general medical knowledge. However, the absence of the control group of clinicians who were simply re-tested may limit the ability to discern the effectiveness of CERTAIN prompting vs training.

Prompting with an electronic checklist (CERTAIN) improves clinical performance, subjective perceptions, and confidence of bedside clinicians confronted with typical emergency medical scenarios in high-fidelity simulation environment. Enhanced design/usability and better training are needed to leverage potential benefits of electronic checklist at the bedside of critically ill patients.