Does implementation of a paediatric asthma clinical practice guideline worksheet change clinical practice?

Abstract

Background

Despite the development of evidence-based practice guidelines in many countries for asthma treatment in children, there is limited evidence that using such guidelines improves patient care.

Aims

Our aim was to evaluate whether the implementation of an evidence-based asthma clinical practice guideline (CPG) worksheet changes clinical practice.

Methods

The study was a before and after study of the implementation of a paediatric asthma CPG in a tertiary paediatric emergency department (ED). All children aged 2–16 years who had a diagnosis of asthma were included. Clinical data were obtained by retrospective chart review for time periods before (May to September 2003) and after (May to September 2005) the introduction of the CPG worksheet. Primary outcomes were: use of spacers for salbutamol instead of nebulisers, use of ipratropium and use of corticosteroids. Secondary outcomes were use of an ED action plan, ordering chest X-rays (CXR) and admission rate.

Results

Before implementation, 240 children presented with asthma and after implementation, 286 children presented. The two groups had similar ages, asthma severity, admission respiratory rate (RR) and oxygen saturation. Following implementation there was an increase in spacer use from 17 to 26% [+9%; 95% confidence interval (CI): 2–16%; p = 0.015] and a reduction in ipratropium use from 58 to 44% (−13%; 95% CI: −22 to −5%; p = 0.0029). The proportion of patients treated with corticosteroids did not change. The number of patients with an ED action plan increased. The number of CXR ordered decreased and the hospital admission rate decreased.

Conclusions

The study demonstrates that implementation of an asthma CPG worksheet in a tertiary paediatric ED resulted in modest changes in clinical practice, mainly by increasing clinician adherence to the guidelines.

Introduction

Numerous epidemiological studies investigating asthma worldwide have shown its incidence is increasing, particularly in children [1]. In Australia, there has been an increase in childhood asthma incidence of 0.5% per year since the 1980s, with between 14 and 16% of children having asthma today [2]. Asthma is a dynamic illness. Not only does its prevalence change over time, and between various countries and within communities [3], but the modalities and types of treatments for asthma have changed over time with continually emerging new evidence. As testament to this there have been multiple revisions of asthma guidelines worldwide since their inception in the late 1980s [4–6]. However, there is less information on the implementation of these guidelines and whether implementation improves prescribing practices of clinicians or patient outcomes [7].

A recent systematic review investigated the effect of protocols on the treatment of asthma in the emergency department [7]. Although it generally concluded that protocols improved emergency department treatment of asthma patients, most studies were in adults and over a decade old. There have been significant changes in the management of childhood asthma, particularly in the emergency department, over the last decade with recommendations to use metered inhalers with spacers [8] and decrease the use of ipratropium [9]. The only previous study of an emergency department protocol for paediatric asthma treatment investigated early use of inhaled beta-agonists and corticosteroid use [10], which are now used widely for the treatment of paediatric asthma.

There is general agreement among the health care community that incorporation of effective, evidence-based treatments into guidelines is necessary to modulate clinician behaviour and to ensure uniformity of treatment within different hospitals and between individual doctors. A study by Gabbay et al. [11] showed that clinicians rarely accessed and used evidence from research or guidelines but rather relied on “mindless”, personal guidelines collectively, reinforced by their own experience, interactions with colleagues, opinion leaders, patients, business and system constraints. For highly autonomous physicians and nurses this standardization of medical decision by moving to guidelines and protocols represents a difficult transition [12].

It is therefore important to find optimal methods to improve clinician access to evidence-based medicine, such as the implementation of clinical practice guidelines at the point of care. It follows that such implementation must be critically and regularly assessed to determine whether these improve both adherence to evidence-based practice and patient outcomes. The wording of clinical practice guidelines is also important and one group of researchers found that guidelines which are simple, concrete and specific make implementation more likely [13]. Presentation of guidelines at every patient encounter can work as a continuous reminder and will hopefully eliminate difficulties that occur when traditional educational measures are used to foster and maintain new physician behaviours [14].

The importance of paediatric asthma as a public health problem and the constantly changing treatment strategies for the management of asthma has led to the development of clinical practice guidelines at the local hospital and national levels. Despite the proliferation of these guidelines throughout many countries, there are no studies investigating the implementation of paediatric asthma guidelines in the emergency department, despite the large burden of illness. Some studies have also suggested a poor implementation of asthma guidelines [15].

In Australia the Asthma Clinical Practice Guideline was developed by the Australian Government and the Royal Australasian College of Physicians to attempt to streamline the treatment of paediatric asthma. A project officer was employed to coordinate the implementation of the asthma clinical practice guidelines at 14 hospitals selected for their high level of paediatric presentations. To assess the effectiveness of the implementation in one hospital, the asthma clinical practice guidelines were adapted and developed into a paediatric asthma clinical practice guidelines worksheet (Fig. 1) for use in the emergency department.

Fig. 1

The asthma clinical practice guidelines worksheet

This study aimed to determine if the introduction of the paediatric asthma clinical practice guidelines worksheet into the emergency department improved clinical practice by investigating treatment practices, documentation, diagnostic tests used and intermediate patient outcomes.

Methods

The study was a before and after study with clinical data obtained by a retrospective chart review from two separate time periods. The study was designed to compare the difference in treatment practices between the two time periods based on three primary outcomes. During the period 2003–2005 there were no major changes in the level of staffing or building at the hospital or associated paediatric emergency department. The paediatric emergency department has one full-time paediatrician employed since 2001. Ethics approval was waived by the Hunter New England Research Ethics Committee as the study was designated a quality improvement exercise.

The setting for the study was the John Hunter Children’s Hospital and the John Hunter Hospital emergency department. The emergency department is a mixed department that has over 56,000 annual presentations, with 18,000 of these being children (age < 16). Paediatric patients are seen in a separate dedicated part of the emergency department. The John Hunter Children’s Hospital is one of three paediatric tertiary teaching hospitals in the State of New South Wales and is contained within the main adult hospital. It has 100 paediatric beds and is staffed by 30 general/subspecialty paediatricians.

The asthma clinical practice guidelines were developed as a joint venture between the Australian Government Department of Health and Ageing and the Royal Australasian College of Physicians and was based on the latest recommendations by the National Asthma Council of Australia. An Area Health Service guideline project officer was employed to coordinate the implementation of the asthma clinical practice guidelines at the hospital and in the emergency department. In conjunction with local parties and a local champion, the clinical practice guidelines were adapted into an asthma clinical practice guidelines worksheet (Fig. 1).

The Area Health Service guideline project officer’s role also involved mentoring and regular communication with emergency department staff providing paediatric emergency care. The project officer sought feedback and incorporated changes into the asthma clinical practice guidelines prior to introducing a final clinical practice guidelines worksheet to emergency department staff at educational sessions. The asthma clinical practice guidelines worksheet was implemented in the emergency department during the second half of 2004. Each child presenting to the emergency department with a provisional diagnosis of acute asthma automatically has an asthma clinical practice guidelines worksheet printed and placed in their chart, ready to be utilised by the treating doctor.

All children aged 2–16 years who presented to the emergency department with an outcome diagnosis of asthma were included in the study. The hospital uses the Patient Information Management System (PIMS) computer software [16] for patient tracking. The following search data were entered into the Patient Information Management System to identify participants: emergency department outcome diagnosis of asthma (ICD-9 codes 493.90 to 493.91) and age (2–16 years). Two groups of patients with asthma were identified for comparison: those who presented to hospital before the implementation of the clinical practice guidelines worksheet (1 April 2003 until 30 September 2003) and those who presented after the implementation (1 April 2005 until 30 September 2005). Medical records for both groups of patients were obtained and reviewed by a single investigator. Only one admission for each patient per month was included in the study.

Data were extracted from the patient medical record using a proforma and entered into a spreadsheet. Initially 20 randomly selected medical records from the 2003 group were reviewed to evaluate the availability of the desired data and whether these could be accurately extracted from the medical record. There was only one abstractor so only intra-rater reliability was assessed. After extracting all of the data from both groups, 25 charts were randomly chosen from May 2003. The same data were then extracted from the medical record a second time to determine if the data collected previously were reproducible. There was a 100% agreement between the second data extraction and the first.

The following data were extracted from the medical record: patient age, time of triage, documentation of asthma clinical practice guidelines use, severity as recorded by the treating doctor (mild, moderate, severe or not recorded), respiratory rate (RR) on admission, oxygen saturation on admission, treatment modalities, administration of salbutamol (metered inhaler with a spacer or nebuliser), use of ipratropium, use of corticosteroids, chest radiographs done in the first 8 h, patient disposition (discharged, admitted) and whether an emergency department action plan was issued.

The primary outcomes of interest were: (1) the device used for the administration of salbutamol—spacer or nebuliser, (2) the use of ipratropium by either method of administration and (3) the use of corticosteroids. Five additional secondary outcomes were also extracted and analysed: whether a chest X-ray was performed, whether the patient was admitted to hospital or not, the severity documented in the medical record and based on physiological parameters, use of the asthma clinical practice guidelines documented and whether an action plan was issued to the patient or family on discharge.

Descriptive data are quoted as means and standard deviations (SD), or medians and interquartile ranges (IQR) for non-parametric data. Fisher’s exact test was used for comparison of the primary and secondary outcomes between the two groups. All statistical analysis was done using GraphPad Instat (Version 3.05, 32 bit for Win 95/NT created September 2000).

Results

A systematic search of PIMS identified 265 medical records in 2003 and 301 medical records in 2005 corresponding to the two 6-month periods. Cases were excluded where the diagnosis was incorrect, the medical record could not be found, a patient presented twice in a month (only the first admission was used), no information was written in the clinical record or patients did not wait (Fig. 2). Before the implementation of the clinical practice guidelines worksheet, 240 children presented with a diagnosis of asthma and after the implementation, 286 children presented. The two groups had similar ages, severity as documented in the notes, admission RR and admission oxygen saturation (Table 1).

Fig. 2

Flow chart for cases included in the study

Table 1 Baseline characteristics for both groups showing the median and interquartile ranges or proportion with 95% confidence intervals

Prior to the introduction of the asthma clinical practice guidelines worksheet the use of the asthma guidelines was not documented. After the implementation it was documented in 145 of 286 (50.7%) medical records that the guidelines had been used. Following the implementation there was an increase in the use of spacers from 17 to 26% [absolute difference: +9%; 95% confidence interval (CI): 2–16%; p = 0.015] and a reduction in the use of ipratropium from 58 to 44% (absolute difference: −13%; 95% CI: −22 to −5%; p = 0.0029). There was no change in the proportion of patients treated with corticosteroids: 75% before implementation compared to 70% after implementation (absolute difference: −6%; 95% CI: −2 to +14%; p = 0.1429) (Table 2).

Table 2 Comparison of the outcomes between the two groups—proportions and 95% confidence intervals

Following the implementation of the asthma clinical practice guidelines worksheet there was also a significant increase in the number of patients with an emergency department action plan from 11% (95% CI: 7–15%) up to 23% (95% CI: 18–28%; p = 0.0003). There was also a decrease in the number of chest radiographs that were ordered from 48% (95% CI: 45–52%) down to 31% (95% CI: 26–37%; p = 0.0001) (Table 2). There was substantial drop in the number of admissions to hospital from 55% (95% CI: 48–61%) to 37% (95% CI: 31–43%; p < 0.0001) (Table 2). Recording of the severity of asthma in the medical record was more common after the implementation, with 55% recording mild, moderate or severe asthma, compared to 28% prior to the implementation of the asthma clinical practice guidelines worksheet. The severity of asthma was similar between groups based on the respiratory rate and oxygen saturation as recorded on admission to hospital and the severity score documented by the treating doctor (Table 1).

Discussion

The study found that implementation of the asthma clinical practice guidelines worksheet modestly improved the adherence of clinicians to current evidence-based guidelines with an increase in the use of metered inhalers with spacers for salbutamol and a decrease in the use of ipratropium. Corticosteroid use in the emergency department was not significantly altered by the implementation of the asthma clinical practice guidelines because it was probably already close to 100% when treatments received prior to emergency department were included (data not shown).

There were also changes in the ordering of chest X-rays, use of asthma management plans, medical documentation and the admission rate following implementation of the asthma clinical practice guidelines worksheet. The increase in the dissemination of asthma action plans is an important finding, as written asthma action plans are an integral part of the six-step asthma management plan advocated by the National Asthma Council [4]. In addition, the recording of the severity of asthma in the medical record increased and recording of compliance with the asthma guidelines increased, both suggesting that the clinical practice guidelines worksheet improved clinical documentation. The decrease in chest X-ray use may confer potential health benefits by not exposing children to unnecessary radiation and cost savings. Lastly, the reduction in admissions to hospital by almost a fifth is an important change to clinical practice. The treatment of more patients with asthma in the community is beneficial to both the patient and family and decreases the burden on hospitals with a large potential cost saving and an increase in bed availability.

Consistent with our study results, it has been shown in the literature that the majority of guideline implementation strategies do achieve modest improvements in guideline compliance [17]. The above authors conducted a systematic study of 235 guideline dissemination and implementation strategies and showed that the vast majority (86%) achieve improvements in guideline compliance, but that these improvements are modest, with a median improvement effect of 10%. These findings are in agreement with the findings of our study, which noted improvements of 9 and 14% for two of the primary outcomes examined. It is not known why guideline implementation strategies have to date achieved only modest improvements; even in rigorous and multifaceted implementations where a number of strategies are utilised, the effects are likely to be only modest and not greater than those achieved in single implementation schemes [17].

The study by Grimshaw et al. [17] also compared different types of implementation strategies in the hope of achieving a model for interventions that result in the greatest improvement. Interestingly, reminders and patient-directed implementations, such as the clinical practice guidelines worksheet used in our study, fared slightly better at 14% improvement than educational strategies at 6–8% improvement. Care needs to be exercised in interpretation of these findings because it was also found that considerable variation in the observed improvement exists within intervention classes; for example, the improvements in performance for reminders varied from between −1 to 34%. In addition to this, a confounder to statistical comparison exists in that researchers have probably chosen implementation strategies likely to be most effective within their study context. This was the case in our study, where it was thought that a reminder system in the form of a worksheet presented at every new patient encounter would be the most effective at targeting the multiple, transient staff members involved. Overall it seems likely that the most efficient strategy for successful implementation of a guideline will depend heavily on the specific application.

A problem with the inability to find a generalised, widely applicable model for guideline application is the effort and cost required in tailoring strategies specifically to their application. In the current study, our approach required employment of an area guideline project officer to coordinate implementation, facilitate the teaching of staff, review use of the clinical practice guidelines worksheet and reinforce behaviour of using the clinical practice guidelines worksheet. In addition, the clinical practice guidelines worksheet was “localised” to our hospital by liaison between the project officer and a “local champion” (an emergency specialist and paediatrician staff member of the hospital), and it was also optimised by feedback from staff members. An economic valuation was not carried out at this time, but it would be worthwhile to compare the costs involved in this rigorous intervention with the cost-benefit of the modest effects seen.

Ideally better measures of baseline severity should be used to compare the two groups, but would require more consistent documentation than was observed in the patient charts used in this study. This would also be complicated by the difficulties found in performing spirometry and peak flows on most children under 6 years of age [18]. Changes in clinical practice in subgroups of asthma severity may be important to consider, such as the use of ipratropium decreasing for mild and moderate asthma and increasing for severe asthma. The majority of patients had mild or moderate asthma and changes in this group are more likely to be reflected in the whole group. This limits any conclusion about the effect of implementation of the guidelines for severe asthma.

The study focussed on measures of clinical adherence to the guidelines (use of medical devices and prescribing practices) and less on patient outcomes. The two intermediate outcomes investigated were the hospital admission rate which was significantly reduced and the increase in the use of asthma action plans.

There was no analysis of patients presenting with asthma within the same month, with these presentations simply being excluded from the study. Separate investigation of presentations would be useful in order to give a further idea of whether the asthma management was successful. Patients presenting within the same month were thought to account for less than 2% of presentations (data not shown), indicating much greater numbers would be needed for statistical analysis.

Conclusion

The study supports the implementation of clinical practice guidelines in the emergency department for paediatric asthma by demonstrating that implementation of the asthma clinical practice guidelines worksheet improved both the adherence to the guidelines as well as a number of patient outcomes. After the implementations there was an increase in both the use of metered inhalers with spacers for salbutamol and a decrease in the use of ipratropium. In addition, there was a decrease in patients admitted to hospital, a decrease in the use of chest X-rays, an increase in asthma action plans and improved documentation in the notes. It will be important to evaluate the cost-effectiveness of this and the overall benefit to the care of children with asthma in the community.