ECG interpretation skills of South African Emergency Medicine residents

Abstract

Background

The use and interpretation of electrocardiograms (ECGs) are widely accepted as an essential core skill in Emergency Medicine. It is imperative that emergency physicians are expert in ECG interpretation when they exit their training programme.

Aim

It is unknown whether South African Emergency Medicine trainees are getting the necessary skills in ECG interpretation during the training programme. Currently there are no clear criteria to assess emergency physicians’ competency in ECG interpretation in South Africa.

Methods

A prospective cross-sectional study of Emergency Medicine residents and recently qualified emergency physicians was conducted between August 2008 and February 2009 using a focused questionnaire.

Results

At the time of the study, there were 55 eligible trainees in South Africa. A total of 55 assessments were distributed; 50 were returned (91%) and 49 were fully completed (89%). In this study, we found the overall average score of ECG interpretation was 46.4% [95% confidence interval (CI) 41.5–51.2%]. The junior group had an overall average of 42.2% (95% CI 36.9–47.5%), whereas the senior group managed 52.5% (95% CI 43.4–61.5%).

Conclusion

In this prospective cross-sectional study of Emergency Medicine residents and recently qualified emergency physicians, we found that there was improvement in the interpretation of ECGs with increased seniority. There exists, however, a low level of accuracy for many of the critical ECG diagnoses. The average score of 46.4% obtained in this study is lower than the scores obtained by other international studies from countries where Emergency Medicine is a well-established speciality.

Introduction

The use and interpretation of electrocardiograms (ECGs) is widely accepted as an essential core skill in Emergency Medicine [1]. Electrocardiography is a commonly used procedure for the diagnosis of heart disease—ECG abnormalities may be the first indication of ischaemia, metabolic disturbance, or life-threatening arrhythmia [2]. The benefits of early correct diagnosis and rapid appropriate treatment have clearly been demonstrated in patients with acute myocardial infarction presenting to the Emergency Centre (EC) [3, 4]. It is therefore imperative that Emergency Physicians are expert in ECG interpretation when they exit their training programme. Specialist emergency physicians are known to improve patient outcomes for patients requiring urgent life saving interventions; previous authors have shown that ECG interpretation skills improve throughout the training programme in Emergency Medicine in Australia [1].

South Africa is a developing country and Emergency Medicine is a developing speciality. The first locally trained Emergency Physicians have only recently qualified (October 2007). Presently EC staffing mostly consists of medical officers or junior doctors. In teaching hospitals in the Western Cape and Gauteng, there may be Emergency Medicine residents on duty; in most of the bigger referral hospitals, consultant cover is available on site during the day and telephonically after hours. For the vast majority, the immediate medical management is initiated by the junior doctor without specialist or consultant input. This is especially true after hours; even in training institutions, emergency medicine residents are often the most senior doctors in the EC. The final decision regarding ECG interpretation and patient management often lies with the resident on duty. Accurate ECG interpretation by EM residents is therefore imperative to delivering quality patient care.

Resident teaching programmes at different universities are constantly updated and developed to improve the level of training. The Fellowship of the College of Emergency Medicine [FCEM (SA)] is the exit examination for trainees; during training towards this exam they maintain a logbook. This CRITICAL performance portfolio states the training objectives for a 4-year rotation in Emergency Medicine; it clearly states that a high skill level in ECG interpretation is a requirement [9]. However, it is not known whether South African Emergency Medicine trainees are getting the necessary skills in ECG interpretation during their training programmes.

Currently there are no clear criteria to assess emergency physicians’ competency in ECG interpretation in South Africa. The evaluation of ECG skills is done as part of the general examination process; ECG questions form part of the written and practical evaluation of the FCEM(SA) examination. There are no data regarding the quality of local Emergency Medicine training. It is necessary to determine whether the skill levels of locally trained Emergency Physicians are at the levels required by the College of Emergency Medicine.

The primary outcome of this study was a comparison of the accuracy of ECG interpretation by the senior group to the accuracy of the junior group.

Secondary aims include:

• ◦ Evaluate the accuracy of Emergency Medicine trainees in interpreting individual ECGs.

Methods

Study design

A prospective cross-sectional study of Emergency Medicine residents and recently qualified emergency physicians was conducted between August 2008 and February 2009.

Inclusion and exclusion criteria

All South African universities with training programmes in Emergency Medicine were approached to join the study. All trainees in the training programmes were approached and asked to consent to take part. Only trainees enrolled in a full-time 4-year MMed (or equivalent) rotation or recently qualified (under 1-year) from the programme were included. Doctors who were admitted by peer review onto the specialist register were not approached. Trainees refusing consent were excluded.

Data collection

Collection of data took place at each university’s dedicated Emergency Medicine teaching sessions. The surveys were completed under direct supervision, although all completed forms were rendered anonymous prior to being assessed.

Questionnaire

All participants completed a survey detailing:

• (1) The year of graduation as a doctor.

• (2) Current year of Emergency Medicine training.

• (3) Previous specific ECG training. Participants were asked to indicate in which format they received their previous ECG teaching:

  • ◦ Medical school.

  • ◦ ECG workshops.

  • ◦ Formalized ECG teaching.

  • ◦ Internet-based teaching.

  • ◦ Self study.

  (They could select more than one option.)

• (4) The satisfaction with current ECG teaching: In this question participants were asked if they felt that the current ECG teaching is adequate. Choices ranged from poor to excellent.

• (5) A subjective assessment of training. Trainees were asked how, in their opinion, ECG teaching could be improved. This was an open question; no specific options were given.

After the survey, participants were asked to interpret ten ECGs and to write their interpretations on a standard data form. Marks were awarded for correctly identifying the rate, rhythm and the QRS axis. They were then asked the main diagnosis and also to describe important additional findings on each of the ECGs. Each ECG was also accompanied by an appropriate clinical scenario: these scenarios comprised brief descriptions of the clinical presentation, sex and age, as applicable to every ECG. There was no time limit placed on completing the ECG interpretation. An example ECG and a pre-completed data form were provided to serve as a guide.

All ECGs were selected from an ECG bank, recorded from real patients. All investigators and participants were blinded to the identity of the patients. The ECGs were selected by the lead investigator and deemed clinically significant to the speciality of Emergency Medicine.

All ECGs were reviewed independently by two cardiologists. The cardiologists compiled answers on standard data forms. In the event of disagreement, a third independent cardiologist was approached, and the majority opinion was accepted as the gold standard.

The main diagnosis for the ten ECGs was:

• (1) Antero-septal ST elevation myocardial infarct

• (2) Atrial flutter

• (3) Pericardial effusion/tamponade

• (4) Third degree atrioventricular block

• (5) Inferior ST elevation myocardial infarct

• (6) Ventricular tachycardia

• (7) Hypothermia

• (8) Wolf-Parkinson-White syndrome

• (9) Right bundle branch block with left anterior hemiblock (bifascicular block)

• (10) Hyperkalaemia

Mark allocation

The ECGs were marked independently by two qualified senior emergency physicians. Markers were trained on the main ECG diagnosis and were provided the gold standard answers (from the cardiologists). They were blinded to the identity of the individual participants. Each ECG was scored out of a total of ten. The average of the two scores awarded was used as the final mark. A discrepancy of more than three in any score between the markers would be referred to one of the cardiologists for a final opinion. Each marker was blinded to the other’s answers.

Data analysis

Participants were divided into junior and senior groups based on their progress in emergency medicine training. The junior group consisted of emergency medicine residents in their first and second years of training; the senior group consisted of emergency medicine residents in their third and fourth years of training, and those qualified in the last 12 months.

Data were captured in Microsoft Excel® (Microsoft, Richmond, VA) by the principal researcher (JdJ). Any further analysis was done using SAS® Version 9.1.3. (SAS Institute Inc.) Frequencies and percentages were calculated for categorical data. Means and standard deviations or medians and percentiles were calculated for numerical data. The 95% Confidence intervals were calculated for the mean scores.

Based on years of training, the study group was divided into a junior and senior group. Comparing the categorical data between different groups, the chi-square statistic or Fisher’s exact test was used. Comparing the mean values between groups, the t-test was used. If median values were compared, the Kruskal-Wallis test was used. A significance level of 0.05 was used throughout the analysis.

Ethics

Ethical approval for this study was obtained from the UCT ethics committee. (REC REF 252/2008).

Results

Returns

At the time of the study, there were 55 eligible trainees (or Junior Consultants) in South Africa. A total of 55 assessments were distributed; 50 were returned (91%) and 49 were fully completed (89%).

Seniority

The earliest graduate from medical school was in 1993 and the latest graduates were in 2005. The mean length of time since graduation from medical school was 8 years. Most of the residents (85%) in the senior group graduated from medical school prior to 2002. In the junior group 66% of the residents graduated after 2002. There was a clear statistical difference in year of graduation between the junior and senior groups (p = 0.0067, Fischer’s exact test).

Emergency Medicine training

From the 49 returns received, 29 were in the junior group and comprised 21 first-year residents and 8 second-year residents. The senior group consisted of seven third-year, six fourth-year and seven fifth-year residents/junior consultants.

ECG training

Respondents were asked about their previous ECG training. Details are presented in Table 1.

Table 1 Previous ECG training

Adequacy of ECG training

When asked whether they felt that ECG teaching was adequate on the Emergency Medicine rotation, they responded as follows (Fig. 1):

Fig. 1

The adequacy of ECG teaching as perceived by the residents

• ◦ Seven (14%) felt that the training was poor with no ECG teaching taking place.

• ◦ Twenty-three (47%) felt that they had to mostly rely on self-study.

• ◦ Eight (16%) judged the level of teaching at a moderate standard.

• ◦ Six (12%) scored the ECG training as good, but can improve.

• ◦ Three (6%) felt the ECG teaching was of an excellent standard.

• ◦ Two residents declined to answer.

No statistical difference was noted between the junior and senior groups. (p = 0.39)

Improvement of ECG teaching

In the final question of the survey, respondents were asked how they felt the level of teaching could be improved. Thirty requests were made for more formal lectures; 15 requests for teaching at the bedside were made. ECG workshops as a teaching method were deemed useful by 12 residents. A cardiology rotation or cardiology input was requested by eight; only four were of the impression that Internet-based ECG teaching would be helpful. The junior and senior groups had similar suggestions on how to improve the level of teaching.

Results of ECG interpretation

Overall

In this study, we found the overall average of ECG interpretation was 46.4% [95% confidence interval (CI) 41.5–51.2%]. The junior group had an overall average of 42.2% (95% CI 36.9–47.5%), while the senior group managed 52.5% (95% CI 43.4–61.5%). There was an improvement in ECG interpretation between the junior and senior groups in this study (p = 0.035, t-test).

On interpretation of the main diagnosis alone, we found an overall average of 40.1% (95% CI 34.1–46.1%); for the junior group it was 34.5% (95% CI 28–41%), and the senior group 48.4% (95% CI 37.4–59.4%). There was a significant improvement in identifying the main diagnosis of each ECG between the junior and senior groups (p = 0.02, t-test).

Discussion

In other countries, emergency physicians have been shown to have similar skill levels to cardiologists in emergency ECG interpretation, especially in determining ST-segment elevation [5]. Emergency physicians show a low rate of ECG misinterpretation in patients with chest pain and ST elevation [6]; emergency medicine residents have similar skill levels in ECG interpretation compared to medicine residents (although the overall performance was low for both disciplines—rates of incorrect diagnosis were 58% for complete heart block and 8% for myocardial infarction) [7].

The American Boards of Internal and Emergency Medicine have a general requirement for staff training, initial competency evaluation and maintenance of competency in electrocardiography [8]. Salerne et al. suggested that determination of initial competency in ECG interpretation at the end of residency training should be based on periodic objective assessment and documentation of resident interpretation skills in a clinical context rather than completion of a minimum number of interpretations [8]. The American College of Cardiology and the American Heart Association have criteria that they suggest be used to assess the level of competence in ECG interpretation. These include the following:

• (1) Board certification in cardiology, on the basis of passing an included standard examination of ECG interpretation.

• (2) Documentation of prior interpretation of 500 ECGs under supervision of an expert echocardiographer as an alternative.

• (3) Passage of the American Board of Internal Medicine Institute for clinical evaluation, ECG certifying examination.

Even though there is no clear agreement between the American College of Cardiology/American Heart Association and the American Boards of Internal and Emergency Medicine on the requirements needed to be judged as competent in ECG interpretation, both have policy documents on the subject.

Seniority

In this study, the senior group had generally qualified prior to the junior group: the seniors may therefore have had more exposure to ECG interpretation. They scored higher than the juniors on interpretation of the ECGs; it is not certain if the higher scores obtained by the senior group are due to training in emergency medicine or due to more clinical experience. The data collected in this study suggest that most of the growth in ECG interpretation took place between year 4 and 5 of training. This may suggest that preparation for the emergency medicine exit examination improves ECG interpretation more than experience alone. Further research is needed to demonstrate this.

Previous ECG training

Both the junior and senior groups had previous ECG training in medical school or by self-study methods. Only small numbers reported formal ECG training, ECG workshops and computer-based training. These three areas should be addressed during the Emergency Medicine rotation. Both groups had similar types of previous ECG training, and this should not have an effect on the results.

Adequacy of ECG training

Both groups felt the current level of ECG training was not adequate and that they had to mostly rely on self-study to improve their skills.

Improvement of ECG training

The majority of residents requested more formal ECG teaching, bedside ECG teaching and ECG workshops during the training programme. Only eight requests for a cardiology rotation to improve ECG interpretation were made. This was an interesting finding; there has been a lot of pressure by residents to have a cardiology rotation included in the training programmes.

There are few literature publications describing evidence-based methods of teaching ECGs [10, 11]. In a recent survey of ECG teaching programmes of cardiology trainees [12], most programmes reported spending a total of 11 total hours per month divided among four specific areas:

• ◦ 0 to 4 h of core curriculum didactic lectures,

• ◦ 0 to 5 h of one-on-one tutorials,

• ◦ 0 to 4 h of interactive conferences, and

• ◦ 0 to 2 h of board review conferences.

The same study reported that 42% of fellowships formally tracked the number of ECGs interpreted by their trainees; 58% performed a formal test of ECG competency during the training programme.

There is a scarcity of data demonstrating clear superiority of any specific method of teaching ECG interpretation, but the experiences of recognized experts have led to consistent agreement that the one-on-one method is most beneficial. Lecture-based ECG teaching is anecdotally seen as more of an adjuvant method [13].

General

Previous studies that have looked at the competency of junior doctors to interpret ECGs found from 56% [1] to 60% [8] overall accuracy. Other studies showed accuracies of 36–96% in residents and non-cardiologists [8, 14–17].

In this study, the average score by the junior group was 42.2% (Hoyel found 49.6%1.) The average of the senior group was 52.5%, compared to Hoyle’s 67.5%. However, Hoyle used fourth-year upwards as senior; this differs significantly from our study group, where the senior group consisted of third- and fourth-year students (and junior consultants). It is possible to redefine our study groups; the fourth-year and higher group then scored an average of 58.5%—this is still below the 67.5% in Hoyle’s study (a direct comparison is difficult due to the difference in methodology between the two studies).

A short clinical scenario was given for each ECG in this study. Studies have shown that supplying a correct clinical scenario increases the accuracy of ECG interpretation [18, 19]. A misleading but plausible clinical scenario can decrease the accuracy of ECG interpretation [19].

In our study the focus was not only on the main diagnosis of the ECG, but also the approach to interpreting an ECG. The main diagnosis only contributed a third to the overall mark. If the main diagnosis alone was interpreted, the overall average would have been 40.1% (juniors 34.5% , seniors 48.4%).

Limitations

The small study group available currently in South Africa may have an influence on the final results. The quality of ECG interpretation by individuals could be affected by other factors, like tiredness, night duty and illness.

Supplying a clinical scenario to each ECG should have a beneficial effect on interpretation. If the scenario is misleading but plausible, it may have the opposite effect.

Conclusions

In this prospective cross-sectional study of Emergency Medicine residents and recently qualified emergency physicians, we found that there was improvement in the interpretation of ECGs with increasing seniority. Overall performance, however, was poor in both groups and compared unfavourably to overseas studies.

The biggest improvement in skills was from year 4 to 5, where residents are actively preparing for, or have completed, the exit exam. We can speculate that this may be due to revision for the Fellowship of the College of Emergency Medicine (FCEM) exit exam. There is a plateau in the improvement of ECG interpretation during the middle-year groups.

Changes in training programmes are required, with more emphasis on formal ECG teaching.

Since the completion of this research, a 6-week cardiology rotation has been implemented for all senior residents in the Department of Cardiology at Tygerburg Hospital, Western Cape, South Africa.