Skip to main content
Download PDF

Development and usability testing of a fully immersive VR simulation for REBOA training

International Journal of Emergency Medicine volume 16, Article number: 67 (2023) Cite this article

Abstract

Background

Resuscitative endovascular balloon occlusion of the aorta (REBOA) is a potentially life-saving procedure for bleeding trauma patients. Being a rare and complex procedure performed in extreme situations, repetitive training of REBOA teams is critical. Evidence-based guidelines on how to train REBOA are missing, although simulation-based training has been shown to be effective but can be costly and complex. We aimed to determine the feasibility and acceptance of REBOA training using a fully immersive virtual reality (VR) REBOA simulation, as well as assess the confidence in conducting the REBOA procedure before and after the training.

Methods

Prospective feasibility pilot study of prehospital emergency physicians and paramedics in Bern, Switzerland, from November 2020 until March 2021. Baseline characteristics of trainees, prior training and experience in REBOA and with VR, variables of media use (usability: system usability scale, immersion/presence: Slater-Usoh-Steed, workload: NASA-TLX, user satisfaction: USEQ) as well as confidence prior and after VR training were accessed.

Results

REBOA training in VR was found to be feasible without relevant VR-specific side-effects. Usability (SUS median 77.5, IQR 71.3–85) and sense of presence and immersion (Slater-Usoh-Steed median 4.8, IQR 3.8–5.5) were good, the workload without under-nor overstraining (NASA-TLX median 39, IQR 32.8–50.2) and user satisfaction high (USEQ median 26, IQR 23–29). Confidence of trainees in conducting REBOA increased significantly after training (p < 0.001).

Conclusions

Procedural training of the REBOA procedure in immersive virtual reality is possible with a good acceptance and high usability. REBOA VR training can be an important part of a training curriculum, with the virtual reality-specific advantages of a time- and instructor-independent learning.

Background

Trauma is the leading cause of death in patients under 45 years of age [1]. In particular, non-compressible torso hemorrhage resulting in hemorrhagic shock bears a high mortality and morbidity [2,3,4]. Resuscitative endovascular balloon occlusion of the aorta (REBOA) has recently gained popularity as a potentially life-saving procedure, by allowing quick transitory hemorrhage control for truncal injuries [5,6,7,8,9]. A recent meta-analysis suggests a positive effect of REBOA in non-compressible torso injuries when compared to resuscitative thoracotomy [10], as does a recent propensity score-matched analysis [11]. Different international guidelines on polytrauma management suggest its application in unstable trauma patients who are unresponsive to other resuscitative efforts [12,13,14].

The application of REBOA is also under investigation during cardiopulmonary resuscitation, as it increases coronary and cerebral perfusion pressure [15, 16].

However, because REBOA is a rare and complex procedure performed in extreme situations with the potential to cause great harm to the patient, mastery of this particular skill is critical. High-volume deployment centers show increased survival of REBOA patients when compared to low-utilization centers [17].

REBOA training is usually done by simulation or a combination of simulation with knowledge transfer done with lectures or/and eLearning. Although a recent systematic review confirms the effectiveness of simulation-based training, there is still confusion about optimal course design, effect size, skill transfer, and skill retention [18], and evidence-based guidelines on how to train REBOA are missing. Simulation training is usually done with a high-fidelity endovascular simulator (e.g., Mentice VIST, Mentice, Gothenburg, Sweden) or live animal models [18]. Traditional simulation-based training therefore is very resource intensive, including high costs for training materials (e.g., high-fidelity manikin, REBOA training catheters) and personnel resources.

Virtual reality (VR) is a technology that immerses the user in an artificial 3D environment with the use of a head-mounted device (VR headset). Interaction with the virtual environment takes place via wearable devices (controllers) or even with the user’s own hand (hand tracking). VR simulations have proven to be a useful and effective tool, mainly for training surgical and technical skills [19,20,21,22]. VR simulation training offers a scalable, autonomous (time- and location-independent) experience, especially for settings that prove to be too risky or resource-intense for traditional simulations.

To our knowledge, there is no fully immersive virtual reality simulation for REBOA training so far.

At the “Schutz und Rettung Bern” [23], we recently started a clinical trial on REBOA in non-traumatic cardiac out-of-hospital cardiac arrest [24]. While the actual insertion of the REBOA catheter in this setting is performed by a core team of 4–5 senior emergency physicians, we need to train the regular prehospital teams (prehospital emergency physicians and emergency paramedics) in the basic principles of the REBOA procedure so that they understand the procedure and can assist the core REBOA team. We thus aimed to.

  1. i)

    Develop a fully immersive VR REBOA training simulation

  2. ii)

    Determine the feasibility of the application of the VR REBOA simulation at the local prehospital emergency medical services

  3. iii)

    Evaluate the acceptance of the VR REBOA simulation (usability, simulator sickness, sense of presence and immersion, workload, user satisfaction)

  4. iv)

    Examine the subjective confidence in conducting the REBOA procedure before and after the simulation training.

Methods

Development of the fully immersive VR simulation

To realistically recreate a medical procedure in VR, we have to understand its basic steps, actions, and milestones. The best way to obtain such a breakdown analysis is to consult professionals specialized in this method. In our case, the Subject Matter Experts material, required for the design of a complete storyboard tailored for the REBOA VR training module, was provided by our medical experts (TB, WEH, TCS) to the development team of ORamaVR (Geneva, Switzerland), according to the methodology published in [25].

The simulated environment consisted of an emergency theatre including a virtual patient, who is hemodynamically unstable after a motor vehicle accident with free fluid in the abdomen. Clinical information, vital signs, ultrasound, or X-ray images, as well as information on the next steps are displayed on monitors in the virtual emergency room. A sterile covered table is used to store and prepare the required materials. The insertion of the REBOA catheter is performed step by step on a virtual person. These steps include.

  1. 1.

    Decision on placement depth and measurement for zone 1 REBOA using the clinical information

  2. 2.

    Preparation of femoral arterial access

  3. 3.

    Cannulation of the common femoral artery using ultrasound

  4. 4.

    Guidewire introduction

  5. 5.

    Placement of sheath

  6. 6.

    REBOA catheter preparation

  7. 7.

    REBOA catheter insertion

  8. 8.

    Balloon inflation and confirmation of its effect

  9. 9.

    Fixation of catheter

  10. 10.

    Chest X-ray

Screenshots of the simulation are detailed in Figs. 1 and 2.

Fig. 1
figure 1

Screenshot of the VR REBOA simulation. Preparation of the sheath, adaptable view into the torso

Full size image
Fig. 2
figure 2

Screenshot of the VR REBOA simulation. Patient monitorized in the resuscitation bay, REBOA catheter in situ after successful insertion

Full size image

Two modes of action were available for single-player use: In the tutorial mode visual aids and prompts helped the trainees in providing information on the next procedural step; these prompts were missing in the normal mode.

Study design, setting, and ethical approval

This is a prospective feasibility pilot study involving prehospital emergency physicians and emergency paramedics of the “Schutz und Rettung Bern” [23]. The study was conducted at the University Emergency Department (Universitätsklinik für Notfallmedizin) at the Inselspital, University Hospital, Bern, Switzerland. The study took place from November 2020 until March 2021.

This study was exempt from full ethical review by the local institutional review board (Kantonale Ethikkommission Bern (Ethics Committee Bern), BASEC-No: Req-2020-00970). Written informed consent for study participation was obtained from all participants. Written informed consent from a parent and/or legal guardian is “not applicable”. Consequently, the present study was conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki and its subsequent amendments.

Participants

The local prehospital emergency medical services (“Schutz und Rettung Bern”) are carrying out about 23,000 preclinical medical emergency operations annually. There are 17 emergency physicians and 108 paramedics (50% female) working in a rendezvous system. All participants were offered and attended the training on a voluntary basis and we provided no remuneration. Written informed consent was obtained for the study and publication of the study results. Written informed consent from a parent and/or legal guardian is “not applicable”.

Baseline data

Sociodemographic data (gender, age, profession (physician/paramedic), working experience in years, need to wear eyeglasses, right/left-handedness), prior training and experience in REBOA as well as prior experience with VR, were collected in a survey.

Intervention

Initially, three peer teachers were introduced to the VR set-up and the correct operation of the REBOA VR module by the development and study team of the University Emergency Department (TS, TB, JB) in a 2-h training session, who then passed on their knowledge and were able to train their peers (“teach the teacher”). The REBOA VR simulation station was set up in an empty room at the Schutz und Rettung headquarter Bern (Fig. 3). The hardware consisted of a stand-alone VR headset with two hand-held controllers (Oculus Quest, Oculus VR, Facebook Inc., Menlo Park, CA, USA) and a tablet pc. The REBOA module, version 1.2.6, software platform, developed by ORamaVR (Geneva, Switzerland), was used in the single-player tutorial and normal mode.

Fig. 3
figure 3

Setup of the VR simulation. Setup for the VR REBOA simulation including hand-held controllers, head-mounted device (Oculus Quest), tablet PC, disinfection materials

Full size image

During 5 months, from November 2020 to March 2021, paramedics and preclinical emergency physicians had the opportunity to train with the VR simulation on their own after being instructed by the trained peer instructors during their shifts or whenever was a suitable timeslot for them. The peer instructors had the opportunity to follow the simulation on the tablet PC, and thus to provide additional targeted assistance, apart from the automated feedback by the tutorial mode of the software, if necessary.

Outcome measures

Acceptance of the VR simulation

Evaluation of acceptance of the VR simulation was carried out according to established questionnaires directly after the VR training.

Usability

Usability was assessed using the System Usability Scale (SUS) [26], which is composed of 10 questions with a 5-point Likert attitude scale and the After-Scenario Questionnaire (ASQ) [27], which assesses the ease of task completion, satisfaction with completion time and satisfaction with supporting information on a 7-point Likert scale (total score ranges from 1 = full satisfaction to 7 = poor satisfaction).

Simulator sickness

“Visually-induced motion sickness” was assessed with four items (The VR training caused nausea/headache/blurred vision/dizziness) taken from the Simulator Sickness Questionnaire (SSQ) from Kennedy et al. (Likert scale from 1 = totally disagree to 5 = totally agree) [28].

Sense of presence and immersion

Presence and immersion in the virtual world were determined according to the 6-item questionnaire developed by Slater-Usoh-Steed (total score ranges from 1 = no immersion to 7 = full immersion) [29].

Workload

Perceived subjective workload on a scale from 0 to 100 was assessed using the NASA-Task Load Index (NASA-TLX) as a total score and within 6 subdomains [30]. Overstraining is associated with a total score > 60, and understraining with a total score of < 37 [31].

User satisfaction

The User Satisfaction Evaluation Questionnaire (USEQ) has six questions with a 5-point Likert scale to evaluate user satisfaction (total score ranges from 6 = poor satisfaction to 30 = excellent satisfaction) [32].

Furthermore, free-text comments were collected.

Subjective effectiveness/confidence

Confidence in the correct performance of the REBOA intervention was assessed before and after the training (“I feel confident in conducting the REBOA intervention correctly” (Likert scale from 1 = totally disagree to 5 = totally agree).

Statistical analysis

Data was analyzed using SPSS.

Baseline characteristics are presented as numbers and percentage or median and interquartile range (IQR) using descriptive statistics as appropriate. Pre- and post-simulation comparisons were performed with the Wilcoxon signed rank test. A p value < 0.05 was considered significant.

Results

Development of the VR REBOA simulation

To create the VR training, we used MAGES 4.0, which enables rapid prototyping of shared, collaborative networked medical training in VR [32].

Feasibility

VR REBOA training for paramedics and emergency physicians was found to be feasible. The chosen peer teaching format was well accepted and implemented by the participants and confirmed as a very useful approach. The use of the VR simulation in only one empty room without further equipment was possible and enabled spontaneous practice sessions without scheduled training hours or permanently reserved training rooms and personnel (Fig. 3).

Baseline characteristics of the study population

Baseline characteristics of the study population are detailed in Table 1. Of the 45 participants, 4 (8.9%) were physicians. None of the participants had received prior training in REBOA or had ever carried out the procedure in real life. Participating physicians had limited experience in femoral arterial cannulation. The majority of the participants did not regularly use of video games or VR.

Table 1 Baseline characteristics of the study population (n = 45)
Full size table

Acceptance of the VR REBOA simulation (usability)

Overall, the VR REBOA simulation was well received by the participants (Table 2). Usability measured with the SUS was clearly above the average of 68 (median 77.5, IQR 71.3–85), indicating good usability. The simulation was very well tolerated. Sense of presence and immersion according to Slater-Usoh-Steed was good (median 4.8, IQR 3.8–5.5). Workload as measured in the NASA-TLX was in the desired range (neither under- nor overstraining with a median of 39, IQR 32.8–50.2). User satisfaction measured by the USEQ scored a median of 26 of 30 points (IQR 23–29).

Table 2 Acceptance of the VR REBOA simulation
Full size table

Free-text comments of the participants generally indicated a good acceptance (e.g., “Great way to practice courses of action or scenarios”; “I thought the VR experience was great and was able to get a good look into the REBOA catheter procedure”). However, critical aspects were illuminated as well (e.g. “Interesting experience to have a VR headset on my head for once. But for me this is no substitute for other means of education, as there is too much support needed. Cost/benefit ratio is not right for me.”). The complete free-text comments of the participants are detailed in Supplement 1.

Confidence

Subjective confidence of the participants in using the REBOA procedure correctly before training was low and significantly increased after the VR simulation (Table 3).

Table 3 Confidence
Full size table

Discussion

Summary

Training with a fully immersive VR simulation for REBOA is feasible with a good usability, high satisfaction, and optimal workload during training. We showed that the VR training increased familiarity with the procedure with little VR training-associated side effects.

Feasibility

Our study shows that it is possible to set up and conduct REBOA VR training on a population without prior knowledge of the procedure and in any given location without specific preparations necessary. Compared to a traditional simulation center, this type of implementation does not require any special constructional prerequisites but only an empty playing area.

The study participants found their way easily within the simulation and were able to run through it independently after a brief introduction, although they had no previous experience with VR or other computer games. Since the simulation has a tutorial mode and a realistic game character, the use was easy and intuitive for the majority of the test subjects. Some participants initially needed help from a peer teacher outside of the simulation, which could be provided through observation on an adjunct tablet computer. Since the VR training is an autonomous gaming experience, it does not necessarily require a trained instructor with specific medical knowledge. The peer instructor concept is based on the “teach the teacher” principle and can thus save expensive human resources as well as instructor time. As the system shows and constantly controls the correct execution and sequence of the skills to be learned, the VR simulation can be used time and instructor-independent.

Acceptance

The usability, measured with the System Usability Scale (SUS), was high. To evaluate user satisfaction, the key component of usability from different perspectives, we confirmed good user satisfaction with both ASQ and USEQ. When evaluating a new training method such as VR, it must be taken into account that the evaluation may be subject to the novelty effect, and the usability measurement results may be overestimated [33].

In addition to high satisfaction and good usability in general, participants indicated they experienced a high level of presence and immersion in the VR training without significant side effects. This high level of immersion and presence could be achieved through immersive VR technology with head-mounted displays, which, to the best of our knowledge, we are investigating for the first time for a REBOA training, and by avoiding immersion-interrupting elements, e.g., dialog boxes or drop-down menus whenever possible. It has been argued that a high level of presence and immersion in VR can be an indicator of cognitive engagement with the content of the virtual environment, and thus an important predictor of experiential learning [34, 35].

Confidence

Although the confidence after the VR training had increased significantly in the pre-post-comparison, a relevant number of participants still reported a low confidence.

Since VR training is limited in terms of haptic experience and knowledge transfer is also easier to teach using classic learning methods, such as self-study e-learning, it is recommended to integrate a VR training into a dedicated learning curriculum. VR training is not intended to replace any other training but to supplement it. An example is the out-of-hospital-cardiac-arrest training curriculum by Brede et al. [36]. However, this problem is not specific to our REBOA training setting shown here but has already been demonstrated in other settings. Since our goal was not to train and enable our participants to self-administer the REBOA catheter in the planned clinical trial, we simply wanted to improve their knowledge and understanding of the procedure and enable them to support the REBOA core team in the field.

Another possible application of VR training could be self-guided training to prevent skill decay, a well-known and relevant problem in the teaching of skills [37]. Park et al. showed REBOA skill degradation was most pronounced in surgical trainees who did not receive training for more than 5 months [38]. There are no studies to date on the best way to teach REBOA in the long term and how to minimize skill decay. However, the persistence of knowledge learned in VR over one month was previously demonstrated for procedural teaching [35]. Given the time-, location-, and instructor-independent nature of our REBOA training, it could be an ideal way to support regular self-guided repetition training to prevent skill decay. Further long-term research is needed.

Limitations

These results must be interpreted with some limitations. First, this was a study of a single population with a limited number of participants who may be subject to selection bias due to voluntary participation, thus impacting generalizability. Although technically possible in the present VR simulation, in our study setting the training was conducted by only one person at a time. This lacks the opportunity to train teamwork skills that are essential for effective work in emergency settings. However, procedural skills are the basis of any teamwork and the present simulator was designed to teach these procedural basics. Likewise, a non-haptic VR simulation cannot be a substitute for haptic skills training such as sonography-guided vascular puncture, underscoring the use of VR simulation as a supplement to, rather than a substitute for, a REBOA training curriculum.

In our study, only a subjective measure of the effectiveness of the training was conducted using self-rated confidence pre- and post-training.

Due to the infrequency of the trained procedure, objective outcomes at the patient level are difficult to collect. One potential approach for future studies could be the use of the REBOA rate instrument [39].

Conclusion

The procedural training of the REBOA catheter procedure in immersive virtual reality is possible with a good acceptance and high usability indicated by the trainees. REBOA VR training can be an important part of a training curriculum, with the virtual reality-specific advantages of a time- and instructor-independent learning.

Availability of data and materials

Data used in this study are available upon reasonable request from the corresponding author at the Emergency Department of the University Hospital Bern, Switzerland to researchers eligible under Swiss legislation to work with codified personal health care data. Eligibility will be determined by Cantonal Ethics Committee Bern.

Abbreviations

ASQ:

After-Scenario Questionnaire

IQR:

Interquartile range

Med:

Median

NASA-TLX:

NASA-Task Load Index

REBOA:

Resuscitative endovascular balloon occlusion of the aorta

SSQ:

Simulator sickness questionnaire

SUS:

System Usability Scale

USEQ:

User Satisfaction Evaluation Questionnaire

VR:

Virtual reality

References

  1. Injury Data Visualization Tools | WISQARS | CDC. Available from: https://wisqars.cdc.gov/data/non-fatal/home. Cited 2022 Nov 11.

  2. Kauvar DS, Lefering R, Wade CE. Impact of hemorrhage on trauma outcome: an overview of epidemiology, clinical presentations, and therapeutic considerations. J Trauma. 2006;60(6 Suppl):S3-11.

    PubMed  Google Scholar 

  3. Morrison JJ, Rasmussen TE. Noncompressible torso hemorrhage: a review with contemporary definitions and management strategies. Surg Clin North Am. 2012;92(4):843–58, vii.

    Article  PubMed  Google Scholar 

  4. Kisat M, Morrison JJ, Hashmi ZG, Efron DT, Rasmussen TE, Haider AH. Epidemiology and outcomes of non-compressible torso hemorrhage. J Surg Res. 2013;184(1):414–21.

    Article  PubMed  Google Scholar 

  5. Kinslow K, Shepherd A, McKenney M, Elkbuli A. Resuscitative endovascular balloon occlusion of aorta: a systematic review. Am Surg. 2022;88(2):289–96.

    Article  PubMed  Google Scholar 

  6. Long B, Hafen L, Koyfman A, Gottlieb M. Resuscitative endovascular balloon occlusion of the aorta: a review for emergency clinicians. J Emerg Med. 2019;56(6):687–97.

    Article  PubMed  Google Scholar 

  7. Mazzoli CA, Chiarini V, Coniglio C, Lupi C, Tartaglione M, Gamberini L, et al. Resuscitative endovascular balloon occlusion of the aorta (REBOA) in non-traumatic cardiac arrest: a narrative review of known and potential physiological effects. J Clin Med. 2022;11(3):742.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Aoki M, Abe T. Traumatic cardiac arrest: scoping review of utilization of resuscitative endovascular balloon occlusion of the aorta. Front Med. 2022;9:888225.

    Article  Google Scholar 

  9. Knapp J, Jakob DA, Haltmeier T, Lehmann B, Hautz WE. Resuscitative endovascular balloon occlusion of the aorta in severely injured patients in the emergency trauma room: a case series. Anaesthesiol. 2022;71(8):599–607.

    Article  Google Scholar 

  10. Castellini G, Gianola S, Biffi A, Porcu G, Fabbri A, Ruggieri MP, et al. Resuscitative endovascular balloon occlusion of the aorta (REBOA) in patients with major trauma and uncontrolled haemorrhagic shock: a systematic review with meta-analysis. World J Emerg Surg WJES. 2021;16:41.

    Article  PubMed  Google Scholar 

  11. Cralley AL, Vigneshwar N, Moore EE, Dubose J, Brenner ML, Sauaia A, et al. Zone 1 endovascular balloon occlusion of the aorta vs resuscitative thoracotomy for patient resuscitation after severe hemorrhagic shock. JAMA Surg. 2022. https://doi.org/10.1001/jamasurg.2022.6393. Cited 2023 Jan 2.

  12. Spahn DR, Bouillon B, Cerny V, Duranteau J, Filipescu D, Hunt BJ, et al. The European guideline on management of major bleeding and coagulopathy following trauma: fifth edition. Crit Care Lond Engl. 2019;23(1):98.

    Article  Google Scholar 

  13. Brenner M, Bulger EM, Perina DG, Henry S, Kang CS, Rotondo MF, et al. Joint statement from the American College of Surgeons Committee on Trauma (ACS COT) and the American College of Emergency Physicians (ACEP) regarding the clinical use of Resuscitative Endovascular Balloon Occlusion of the Aorta (REBOA). Trauma Surg Acute Care Open. 2018;3(1):e000154.

    Article  PubMed  PubMed Central  Google Scholar 

  14. AWMF Leitlinienregister. Available from: https://register.awmf.org/de/leitlinien/detail/187-023. Cited 2022 Nov 11.

  15. Levis A, Greif R, Hautz WE, Lehmann LE, Hunziker L, Fehr T, et al. Resuscitative endovascular balloon occlusion of the aorta (REBOA) during cardiopulmonary resuscitation: a pilot study. Resuscitation. 2020;156:27–34.

    Article  PubMed  Google Scholar 

  16. Jang DH, Lee DK, Jo YH, Park SM, Oh YT, Im CW. Resuscitative endovascular occlusion of the aorta (REBOA) as a mechanical method for increasing the coronary perfusion pressure in non-traumatic out-of-hospital cardiac arrest patients. Resuscitation. 2022;179:277–84.

    Article  PubMed  Google Scholar 

  17. Gorman E, Nowak B, Klein M, Inaba K, Morrison J, Scalea T, et al. High resuscitative endovascular balloon occlusion of the aorta procedural volume is associated with improved outcomes: an analysis of the AORTA registry. J Trauma Acute Care Surg. 2021;91(5):781–9.

    Article  CAS  PubMed  Google Scholar 

  18. Engberg M, Taudorf M, Rasmussen NK, Russell L, Lönn L, Konge L. Training and assessment of competence in resuscitative endovascular balloon occlusion of the aorta (REBOA) - a systematic review. Injury. 2020;51(2):147–56.

    Article  PubMed  Google Scholar 

  19. Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, et al. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(1):e12959.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Zhao G, Fan M, Yuan Y, Zhao F, Huang H. The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis. Ann Transl Med. 2021;9(3):252–252.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Tang YM, Chau KY, Kwok APK, Zhu T, Ma X. A systematic review of immersive technology applications for medical practice and education - trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev. 2022;35:100429.

    Article  Google Scholar 

  22. Barteit S, Lanfermann L, Bärnighausen T, Neuhann F, Beiersmann C. Augmented, mixed, and virtual reality-based head-mounted devices for medical education: systematic review. JMIR Serious Games. 2021;9(3):e29080.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Stadt Bern. Sanitätspolizei. Available from: https://www.bern.ch/politik-und-verwaltung/stadtverwaltung/sue/schutz-und-rettung-bern/sanitaetspolizei. Cited 2022 Oct 15.

  24. University Hospital Inselspital, Berne. CPR-REBOA: improving outcome of prehospital cardiopulmonary resuscitation with balloon occlusion of the descending aorta. clinicaltrials.gov; 2021. Report No.: NCT04373122. Available from: https://clinicaltrials.gov/ct2/show/NCT04373122. Cited 2022 Nov 9.

  25. Zikas P, Kateros S, Lydatakis N, Kentros M, Geronikolakis E, Kamarianakis M, et al. Virtual reality medical training for COVID-19 swab testing and proper handling of personal protective equipment: development and usability. Front Virtual Real. 2022;2. Available from: https://www.frontiersin.org/articles/10.3389/frvir.2021.740197s. Cited 2022 Nov 13.

  26. Brooke J. SUS-a quick and dirty usability scale. In: Usability evaluation in industry. London: Taylor and Francis; 1996. p. 189–94.

  27. Lewis J. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. SIGCHI Bull. 1991;23:78–81.

    Article  Google Scholar 

  28. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol. 1993;3(3):203–20.

    Article  Google Scholar 

  29. Usoh M, Catena E, Arman S, Slater M. Using presence questionnaires in reality. Cambridge: MIT Press, journals-info@mit.edu; 2006. Available from: https://www.mitpressjournals.org/doix/abs/10.1162/105474600566989. Cited 2020 Sep 23.

  30. Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, editors. Advances in psychology. North-Holland; 1988. p. 139–83. (Human Mental Workload; vol. 52). Available from: http://www.sciencedirect.com/science/article/pii/S0166411508623869. Cited 2020 Sep 23.

  31. Grier RA. How high is high? A meta-analysis of NASA-TLX global workload scores. Proc Hum Factors Ergon Soc Annu Meet. 2015;59(1):1727–31.

    Article  Google Scholar 

  32. Gil-Gómez JA, Manzano-Hernández P, Albiol-Pérez S, Aula-Valero C, Gil-Gómez H, Lozano-Quilis JA. USEQ: a short questionnaire for satisfaction evaluation of virtual rehabilitation systems. Sensors. 2017;17(7):1589.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Huang W, Roscoe R, Johnson M, Craig S, Arizona State University. Investigating the novelty effect in virtual reality on STEM learning. In: ASU electronic theses and dissertations. Arizona State University; 2020. Available from: http://hdl.handle.net/2286/R.I.57391. Cited 2021 Jun 2.

  34. Baniasadi T, Ayyoubzadeh SM, Mohammadzadeh N. Challenges and practical considerations in applying virtual reality in medical education and treatment. Oman Med J. 2020;35(3):e125.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Birrenbach T, Zbinden J, Papagiannakis G, Exadaktylos AK, Müller M, Hautz WE, et al. Effectiveness and utility of virtual reality simulation as an educational tool for safe performance of COVID-19 diagnostics: prospective, randomized pilot trial. JMIR Serious Games. 2021;9(4):e29586.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Brede JR, Lafrenz T, Krüger AJ, Søvik E, Steffensen T, Kriesi C, et al. Resuscitative endovascular balloon occlusion of the aorta (REBOA) in non-traumatic out-of-hospital cardiac arrest: evaluation of an educational programme. BMJ Open. 2019;9(5):e027980.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Arthur W Jr, Bennett W Jr, Stanush PL, McNelly TL. Factors that influence skill decay and retention: a quantitative review and analysis. Hum Perform. 1998;11(1):57–101.

    Article  Google Scholar 

  38. Park C, Grant J, Garigipati P, Kuhlenschmidt K, Black G, Bhat S, et al. Resuscitative endovascular balloon occlusion of the aorta: simulation improves performance but may require interval training to prevent skill degradation. Eur J Trauma Emerg Surg. 2022;48(3):1955–9.

    Article  PubMed  Google Scholar 

  39. Engberg M, Lönn L, Konge L, Mikkelsen S, Hörer T, Lindgren H, et al. Reliable and valid assessment of procedural skills in resuscitative endovascular balloon occlusion of the aorta. J Trauma Acute Care Surg. 2021;91(4):663–71.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the Touring Club Switzerland for their support of the emergency telemedicine at the University of Bern and the “Schutz und Rettung Bern” ambulance service for their support with the conduction of the study.

Funding

No external funding for the present manuscript.

Author information

Authors and Affiliations

  1. Department of Emergency Medicine, Inselspital, University Hospital Bern, Freiburgstrasse 16C, Bern, CH-3010, Switzerland

    T. Birrenbach, R. Wespi, W. E. Hautz, J. Berger, P. R. Schwab, A. K. Exadaktylos & T. C. Sauter

  2. Schutz und Rettung Bern, Sanitätspolizei Bern, Bern, Switzerland

    P. R. Schwab

  3. ORamaVR SA, Geneva, Switzerland

    G. Papagiannakis

  4. Institute of Computer Science, Foundation for Research and Technology, Hellas, Heraklion, Greece

    G. Papagiannakis

  5. Department of Computer Science, University of Crete, Heraklion, Greece

    G. Papagiannakis

Authors
  1. T. Birrenbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Wespi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. W. E. Hautz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. Berger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P. R. Schwab
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. G. Papagiannakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. A. K. Exadaktylos
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. T. C. Sauter
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TB: conception, design, acquisition, analysis, interpretation, drafting and revising. RW: interpretation, drafting and revising. WEH: conception, design, interpretation, drafting and revising. JB: acquisition, analysis, interpretation, drafting and revising. PRS: conception, drafting and revising. GP: design, acquisition, interpretation, drafting and revising. AKE: conception, design, drafting and revising. TCS: conception, design, acquisition, analysis, interpretation, drafting and revising.

Corresponding author

Correspondence to T. Birrenbach.

Ethics declarations

Ethics approval and consent to participate

This study was approved and classified as a quality evaluation study by the local institutional review board (Ethics Committee Bern, BASEC-No: Req-2020-00970). All methods were carried out in accordance with relevant guidelines and regulations. Written informed consent was obtained from all participants.

Consent for publication

Written informed consent was obtained from the participants for the publication of this study and accompanying images. Written informed consent from patients is “not applicable”. No individual details are revealed in the publication.

Competing interests

WEH has received research funding from the European Union, the Swiss National Science foundation, Zoll foundation, Dräger Medical Germany, Mundipharma Research UK, MDI International Australia, Roche Diagnostics Germany, all outside the submitted work. WEH has provided paid consultancies to AO foundation Switzerland and MDI International Australia, all outside the submitted work. WEH has received financial support for a congress he chaired from EBSCO Germany, Isabel Healthcare UK, Mundipharma Medical Switzerland, VisualDx USA, all outside the submitted work.

TCS holds the endowed professorship of emergency telemedicine at the University of Bern sponsored by the Touring Club Switzerland. The sponsor has no influence on the research or decision to publish.

GP is co-founder, CEO and CTO of ORamaVR. The company had no influence on the study setup and interpretation of the results or decision to publish.

All other authors have nothing to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Some free-text comments from participants. Free-text responses were collected with an open response item. Comments of the participants generally indicated a good acceptance. However, critical aspects were illuminated as well.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Birrenbach, T., Wespi, R., Hautz, W.E. et al. Development and usability testing of a fully immersive VR simulation for REBOA training. Int J Emerg Med 16, 67 (2023). https://doi.org/10.1186/s12245-023-00545-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12245-023-00545-6

Keywords

International Journal of Emergency Medicine

ISSN: 1865-1380

Contact us