- Original Research Article
- Open Access
Outcomes considered most important by emergency physicians when determining disposition of patients with pulmonary embolism
International Journal of Emergency Medicine volume 3, pages 239–264 (2010)
Clinical decision rules for the disposition of patients with pulmonary embolism (PE) are typically validated against an outcome of 30-day mortality or disease recurrence. There is little justification for this time frame, nor is it clear whether this outcome reflects emergency department (ED) decision making.
To determine which outcomes emergency physicians (EP) consider most relevant to disposition decisions.
Survey of attending EPs in geographically diverse US states using acute PE as the diagnostic framework. Responses required single-answer multiple choice, a numerical percentage, rank-ordered responses, or a five-point Likert scale. We distributed the survey via e-mail to 608 EPs.
We received responses from 292 (48%) EPs: 88% board certified, 91% trained in emergency medicine, and 70% work in academics. Respondents reported discharging 1% of patients with PE from the ED, but 21% reported being asked to do so by an admitting service. EPs were more interested in knowing 5-day (in hospital) outcomes [192/265, 72% (95% exact CI = 66%–78%)] than 30-day outcomes [39/261, 15% (95% exact CI = 11%–20%)] or 90-day outcomes [29/263, 11% (95% exact CI = 8%–15%)]. On a Likert scale, 212/241 (88%, 95% exact CI = 83%–92%) agreed or strongly agreed that they considered 5-day (in hospital) clinical deterioration when making a decision to admit or discharge a patient from the ED compared to 184/242 (76%, 95% exact CI = 70%–81%) and 73/242 (30%, 95% exact CI = 24%–36%) for 30 and 90 days, respectively. A wide variety of clinical outcomes beyond death or recurrent PE were considered indicative of clinical deterioration.
Five-day (in hospital) outcomes that incorporate a variety of clinical deterioration events are of interest to EPs when determining the disposition of ED patients with PE. Researchers should consider this when developing and validating clinical decision rules.
With every patient encounter, emergency physicians (EPs) must decide on an appropriate disposition. In some cases, disposition decisions are obvious—clinically unstable patients must be admitted and patients with minor problems may be safely discharged from the Emergency Department (ED). However, for a large number of conditions, determining which patients are safe for outpatient treatment is more complex.
Prospectively validated decision rules are available to help clinicians determine which patients with pneumonia, syncope, transient neurological attacks, pulmonary embolism (PE), and other conditions should be admitted to the hospital and which patients are safe for discharge [1–7]. It is common for these rules to be validated against outcomes such as 30-day mortality or disease recurrence. However, there is little justification for this in the literature. Since few patients are hospitalized for 30 days, it is not clear how this time frame informs the decision to admit or discharge a patient. Narrowly defined outcomes such as death and disease recurrence may not reflect the complexity of the disposition decision either. Moreover, statistical models that predict all-cause mortality may unduly reflect factors with a high fatality rate (e.g., cancer), whether or not they are associated with the diagnosis in question. Given these issues, it is not surprising that physicians tend not to be familiar with clinical decision rules and use them infrequently in practice .
We sought to determine which of three time frames EPs considered most relevant to the disposition of patients with PE. We also sought to determine whether different outcomes (including cardiopulmonary arrest, hypotension, hypoxia, need for respiratory support, need for thrombolysis, development of a cardiac dysrhythmia, and bleeding) were considered more relevant to disposition decisions than others. A better understanding of the clinical decision making of EPs may improve our ability to develop decision rules that are useful to practicing clinicians.
We developed a survey to determine how different outcomes influence an EP’s decision to admit or discharge patients from the ED. The survey was developed by two of the authors (Weston Sacco and Christopher Kabrhel). The survey was designed to be anonymous, with respondents being asked to create a coded unique identifier. Substantive questions required: single-answer multiple choice; yes/no; input of a numerical percentage (0-100%); rank ordering of a series of six responses (from most important to least important); or completion of a five-point Likert scale. Distribution of the survey was approved by the Institutional Review Board of Partners Healthcare Inc.
Demographic information gathered included respondent age, training, and practice setting. We chose to use a diagnosis of PE as the diagnostic framework, and respondents were asked to consider a patient with a diagnosis of acute PE when determining whether an event would represent clinical deterioration. The main goals of the survey were to determine: (1) which of three time frames (5, 30, and 90 days) EPs consider most relevant to determining ED disposition; (2) which outcomes represent clinical deterioration and (3) the degree to which these outcomes are considered relevant to disposition decisions. For questions regarding the most appropriate time frame, respondents were provided the general instruction: “We are trying to determine the most appropriate time frame for considering risk.” For questions assessing clinical deterioration, respondents were provided the general instruction: “We are trying to determine what events or interventions define ‘clinical deterioration.’” In addition, for questions assessing specific clinical events, and whether they represent clinical deterioration, respondents were provided with the following clinical scenario: “You have diagnosed an Emergency Department patient with Pulmonary Embolism (PE). The patient is currently stable, and you are deciding whether to admit the patient or to discharge him/her from the Emergency Department” and the general instruction “We would like you to consider ‘clinical deterioration’ events that would, if predictable, impact the ED disposition of a patient with PE.” Respondents were then asked to complete the sentence: “I would consider a patient to have had a clinical deterioration if….”
We inquired about outcomes occurring within three specific time frames: 5 days, 30 days, and 90 days. We chose 5 days to reflect an average length hospitalization for a patient with PE [9–11]. We chose 30 and 90 days because these time frames have been used to validate the pulmonary embolism severity index in prior studies [7, 12–14].
During development, the survey was piloted serially on six board-certified EPs using an iterative process to assess question clarity and completeness. Physicians in the pilot group were asked to identify any question that was unclear or vague, and to suggest improvements. Suggested changes were incorporated until the physician felt each question was clear and addressed the goal of the survey. Pilot physicians took the survey, with revisions incorporated, two to three times each. The final survey was then uploaded to a commercially available online survey site (www.surveymonkey.com).
The survey was distributed to 608 attending EPs at 16 institutions, including a mix of academic and community practices, located in 13 geographically diverse US states. Potential respondents were contacted three times in order encourage completion of the survey. Informed consent was implied with completion of the survey.
Demographics are presented as simple means and proportions. Comparative analysis was performed by comparing binomial proportions and exact 95% confidence intervals (CI), with non-overlapping CIs considered significantly different. In order to assess generalizability across academic and community practice settings, we performed a sensitivity analysis, limiting our analysis to respondents (n = 86) who described their practice setting as a “community medical center” or “combined academic/community.” All statistical analyses were performed using SAS version 9.1 (SAS Institute, Cary, NC).
We received survey responses from 292 EPs, or 48% of those contacted. Demographic data describing respondents are provided in Table 1. The vast majority of respondents did residency training in Emergency Medicine and were board certified. The majority of respondents worked in academic medical centers, though a sizable percentage worked in combined academic/ community settings.
Respondents reported substantial experience diagnosing PE. Only 3/281 (1%) respondents had not diagnosed a PE within the last year, whereas 71 (25%) had diagnosed 1–5 PEs, 100 (36%) had diagnosed 6–10 PEs, 45 (16%) had diagnosed 11–15 PEs, 36 (13%) had diagnosed 15–20 PEs, and 26 (9%) had diagnosed more than 20 PEs in the past year. Respondents reported discharging a mean of 1% of patients with PE from the ED after diagnosis. Of the 83/281 (30%) respondents who reported that they do sometimes discharge patients with PE from the ED, only 9 (11%) said they use a scoring system to decide which patients with PE are safe for discharge. When asked whether an admitting service had ever suggested discharging a patient with PE from the ED, 59/282 (21%) said “Yes.”
When asked to rank order 5-, 30- and 90-day outcomes, 192/265 (72%, 95% exact CI = 66%–78%) ranked 5-day (in hospital) outcomes “most important,” whereas 39/261 (15%, 95% exact CI = 11%–20%) said 30 days, and 29/263 (11%, 95% exact CI = 8%–15%) said 90 days (Fig. 1). Similarly, on a five-point Likert scale, 212/241 (88%, 95% exact CI = 83%–92%) agreed/strongly agreed that they considered 5-day (in hospital) clinical deterioration when making a decision to admit or discharge a patient from the ED compared to 184/242 (76%, 95% exact CI = 70%–81%) who agreed/strongly agreed that they considered 30-day clinical deterioration, and 73/242 (30%, 95% exact CI = 24%–36%) who agreed/strongly agreed that they considered 90-day clinical deterioration. When respondents were asked to consider a patient who had no clinical deterioration throughout their hospitalization, but subsequently deteriorated after discharge, 148/241 (61%, 95% exact CI = 55%–68%) agreed/strongly agreed that the hospitalization was justified. Only 29/244 (12%, 95% exact CI = 8%–17%) of respondents agreed/strongly agreed that clinical deterioration was “only important if it required treatment,” while 192/244 (79%, 95% exact CI = 73%–84%) disagreed/strongly disagreed with that statement.
We asked respondents which clinical events are indicative of clinical deterioration that would influence the disposition decision for a patient with PE. All respondents [242/242, 100% (95% exact CI = 98%–100%)] said that cardiopulmonary arrest within 5 days represented clinical deterioration, though fewer said so if cardiopulmonary arrest occurred within 30 days [210/241, 87% (95% exact CI = 82%–91%)] or 90 days [127/238, 53% (95% exact CI = 47%–60%)]. Hypotension was considered clinical deterioration by 237/237 (100%, 95% exact CI = 98%–100%) if it required vasopressor therapy, 232/237 (98%, 95% exact CI = 95%–99%) if it required volume resuscitation, and by 168/232 (72%, 95% exact CI = 66%–77%) if no treatment was required. The greatest number of respondents said that if a patient’s room air oxygen saturation became less than 90%, they would consider the patient to have had a clinical deterioration (Fig. 2a). Most respondents said that if a subject required any supplemental oxygen to maintain the SaO2, either at rest or with exercise, it represented clinical deterioration (Fig. 2b). Treatment with thrombolysis was considered clinical deterioration by 214/242 (88%, 95% exact CI = 84%–92%). Bleeding (intracranial, gastrointestinal, retroperitoneal, other major bleeding, or minor bleeding) was considered clinical deterioration by >80% of respondents, regardless of type or whether treatment was required, with the exception of minor bleeding not requiring treatment [46/203, 23% (95% exact CI = 17%–28%)]. Similarly, all cardiac dysrhythmias (bradycardia requiring treatment, reentrant supraventricular tachycardia requiring treatment, atrial fibrillation/flutter, ventricular tachycardia) were considered clinical deterioration by more than 90% of respondents, with the exception of reentrant supraventricular tachycardia not requiring treatment [158/207, 77% (95% exact CI = 71%–82%)] and bradycardia not requiring treatment [80/204, 38% (95% exact CI = 32%–45%)].
Finally, we asked respondents about factors that, while not indicative of clinical deterioration, might influence disposition. On a five-point Likert scale, respondents said they were more/much more likely to admit patients >70 years old [200/240, 83% (95% exact CI = 78%–88%)] and 50–69 years old [179/241, 74% (95% exact CI = 68%–80%)], but not patients 30–49 years old [60/240, 25% (95% exact CI = 19%–31%)] or <30 years old [42/241, 17% (95% exact CI = 13%–23%)]. Other factors that made admission more/much more likely were: lack of family/friend support at home [213/241, 88% (95% exact CI = 84%–92%)]; inability to return to hospital if further problems arise [232/241, 96% (95% exact CI = 93%–98%)]; pregnancy [202/241, 84% (95% exact CI = 79%–88%)]; recent surgery [182/239, 76% (95% exact CI = 70%–81%)]; non-English speaking [138/239, 58% (95% exact CI = 51%–64%)]. Lack of insurance was less influential, with most respondents [131/241, 54% (95% exact CI = 48%–61%)] being neutral on the question.
When we limited our analysis to the 86 respondents who described their practice setting as either a community medical center or a combined academic/community setting, results were similar. When asked to rank order 5-, 30- and 90-day outcomes, 65/83 (78%, 95% exact CI = 68%–87%) ranked 5 day (in hospital) outcomes “most important,” while fewer [4/73, 5% (95% exact CI = 2%–13%)] said 30 days, and 2/69 (3%, 95% exact CI = 0%–10%) said 90 days. The results of all other analyses were similar to responses overall (data not shown).
In our survey, when determining the most appropriate disposition of patients with PE, EPs were more interested in knowing 5-day (in hospital) clinical outcomes than 30- or 90-day outcomes. This sentiment was apparent when the question was asked directly—with 5-day outcomes more than four times more likely to be considered “most important” than 30-day outcomes. It was also apparent when the question was asked indirectly—with only 61% answering that hospitalization was justified if a patient remained stable while hospitalized but deteriorated after discharge. The importance of 5-day (in hospital) outcomes was also apparent in the responses to questions about specific clinical deterioration events. For example, the percentage of EPs who responded that the risk of cardiopulmonary arrest would influence the disposition decision declined from 100% when considering 5-day risk to 87% when considering 30-day risk and 53% when considering 90-day risk. Our results suggest that while 30- and 90-day outcomes have value, EPs find time frames that reflect the length of a typical hospitalization more relevant to their decision making.
Although the importance of in-hospital outcomes is intuitive and apparent from our survey, the use of such short-term clinical outcomes in the medical literature is rare [3, 4, 15]. Studies demonstrating that clinical factors and biomarkers are associated short-term (10-15 day) adverse clinical events after PE have been published [16, 17], though to the authors’ knowledge, there are no outcome studies in the PE literature that use a time frame reflective of a typical hospitalization. Our data suggest that EPs would consider studies of outcomes occurring during an average length hospitalization highly relevant to their disposition decisions.
We also found that EPs consider a broad range of clinical events to be indicative of clinical deterioration and that these events inform the disposition decision. It is our feeling that studies that limit their outcomes to death or disease recurrence oversimplify the disposition decision made by EPs every day. Our results suggest that in addition to outcomes that clearly represent clinical instability (e.g., cardiopulmonary arrest), when determining the best disposition of a patient with PE, EPs consider the need for respiratory support, the risk of developing a dysrhythmia, hypoxia, hypotension, and, bleeding, even if those events do not require treatment. In contrast, 12% of respondents did not consider treatment with thrombolysis indicative of clinical deterioration. We did not ask respondents to explain the rationale for their responses, so the reason for this remains unclear. One possibility is that treatment with thrombolysis was felt to reflect a clinical decision rather than a measure of patient status. This finding warrants further exploration.
We felt that questions related to clinical deterioration would be difficult to answer unless physicians were provided with a diagnostic framework within which to consider their decision. We chose to use a patient with PE as the diagnostic framework. PE is a common diagnosis and is an entity with which most EPs are familiar [18, 19]. PE is usually treated in the hospital, though outpatient treatment has gained traction in Europe and Canada, and patients with deep vein thromboses are commonly treated as outpatients in the US [7, 13, 20–23]. The fact that our respondents reported discharging a mean of 1% of patients with PE from the ED supports this and shows that while ED discharge after PE remains rare, some patients may be considered appropriate for discharge.
Understanding the factors that impact emergency department decision making, especially with regards to patient disposition, is relevant for several reasons. Systems designed to aid clinical decision making, improve patient care, and increase cost-effectiveness are only useful if they are adopted by practitioners. However, studies including ours have shown that clinical decision instruments are infrequently applied to practice . While clinical rules are undoubtedly difficult to use for a variety of reasons, we believe that a clinical rule that poorly or partially reflects the concerns of the physician making the decision is unlikely to be adopted. The need for new outcomes specific to the practice of emergency medicine was highlighted at the Future of Emergency Medicine Research Conference more than a decade ago [24, 25]. Despite this, we believe that our survey is the first to document the importance of a broad range of clinical outcomes, occurring in a clinically relevant time frame, to EP decision making.
There are several limitations of this study methodology that deserve mention. First, the recipients of this survey were chosen because they work in institutions that have a history of working relationships with study investigators. There is disproportionate representation towards academic centers in the eastern portion of the United States. This may limit the generalizability of our results as survey responses may not reflect the opinions of community EPs. However, when we limited our analysis to respondents practicing in community or combined academic/community settings, results were similar. Our response rate was moderate at 48%, but we acknowledge that our results may be biased towards those physicians inclined to complete surveys or otherwise participate in research. We acknowledge that the relevance of our data is particular to the question of disposition of patients with PE and does not necessarily inform disposition decisions for patients with other diagnoses. Also, for clinical decision rules that address different concerns than disposition, such as the likelihood of diagnosis, different outcomes may be relevant. We only compared 5-, 30-, and 90-day outcomes, so it is possible that there is a threshold time frame that would be more important to clinicians than 5 days. It is also possible emergency physicians simply chose the time frame that was temporally closest to their evaluation. However, we chose our time periods to reflect a typical hospitalization for PE (5 days) and time frames (30 and 90 days) used previously in the medical literature to validate clinical decision rules. Each time respondents were asked about a 5-day outcome, it was made clear that this reflected an in-hospital outcome. Thus, we feel that our results reflect physicians’ impressions of outcomes occurring during a typical hospitalization versus those occurring after discharge. While the vast majority of responses to our questions yielded consistent answers, we did find 34 respondents who gave inconsistent responses on questions about the time frame they would most like to know (e.g., choosing 5 days as the “most important” time frame, but strongly disagreeing with the idea of considering a patient’s risk of clinical deterioration while in the hospital). It is likely the labeling of the ordinal system was misinterpreted by some respondents. To assess the impact of this, we performed a sensitivity analysis excluding such inconsistent responses, and again, a greater percentage chose 5-day outcomes as “most important” (data not shown). We also sought to assess the importance of certain clinical outcomes specific to PE. Most of these were amenable to discrete categorization. However, when piloting the survey we found that respiratory parameters were particularly intertwined with other factors (e.g., need for respiratory support is determined by oxygen saturation, which is influenced by exercise and patient comfort). We therefore chose to divide our questions about respiratory parameters into several questions, but acknowledge that the lack of a single descriptor of respiratory clinical deterioration is complex.
Five-day (in hospital) outcomes are of more interest to EPs when determining the disposition of ED patients with PE than longer time frames. EPs also consider a wide variety of events indicative of clinical deterioration. Researchers should consider this when developing and validating clinical decision rules.
Fine MJ et al (1997) A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 336(4):243–250
Quinn J et al (2006) Prospective validation of the San Francisco Syncope Rule to predict patients with serious outcomes. Ann Emerg Med 47(5):448–454
Quinn JV et al (2004) Derivation of the San Francisco Syncope Rule to predict patients with short-term serious outcomes. Ann Emerg Med 43(2):224–232
Rothwell PM et al (2005) A simple score (ABCD) to identify individuals at high early risk of stroke after transient ischaemic attack. Lancet 366(9479):29–36
Tsivgoulis G et al (2006) Validation of the ABCD score in identifying individuals at high early risk of stroke after a transient ischemic attack: a hospital-based case series study. Stroke 37(12):2892–2897
Aujesky D et al (2005) Derivation and validation of a prognostic model for pulmonary embolism. Am J Respir Crit Care Med 172(8):1041–1046
Aujesky D et al (2006) A prediction rule to identify low-risk patients with pulmonary embolism. Arch Intern Med 166(2):169–175
Runyon MS, Richman PB, Kline JA (2007) Emergency medicine practitioner knowledge and use of decision rules for the evaluation of patients with suspected pulmonary embolism: variations by practice setting and training level. Acad Emerg Med 14(1):53–57
Hyers TM et al (2001) Antithrombotic therapy for venous thromboembolic disease. Chest 119(1 Suppl):176S–193S
Aujesky D et al (2008) Length of hospital stay and postdischarge mortality in patients with pulmonary embolism: a statewide perspective. Arch Intern Med 168(7):706–712
Park B et al (2009) Recent trends in clinical outcomes and resource utilization for pulmonary embolism in the United States: findings from the nationwide inpatient sample. Chest 136(4):983–990
Jimenez D et al (2007) Prognostic models for selecting patients with acute pulmonary embolism for initial outpatient therapy. Chest 132(1):24–30
Aujesky D et al (2007) Validation of a clinical prognostic model to identify low-risk patients with pulmonary embolism. J Intern Med 261(6):597–604
Nendaz MR et al (2004) Validation of a risk score identifying patients with acute pulmonary embolism, who are at low risk of clinical adverse outcome. Thromb Haemost 91(6):1232–1236
Sun BC et al (2007) External validation of the San Francisco Syncope Rule. Ann Emerg Med 49(4):420–427, 427 e1-4
Agterof MJ et al (2009) Risk stratification of patients with pulmonary embolism based on pulse rate and D-dimer concentration. Thromb Haemost 102(4):683–687
Lobo JL et al (2009) D-dimer levels and 15-day outcome in acute pulmonary embolism. Findings from the RIETE registry. J Thromb Haemost
Heit JA (2005) Venous thromboembolism: disease burden, outcomes and risk factors. J Thromb Haemost 3(8):1611–1617
Silverstein MD et al (1998) Trends in the incidence of deep vein thrombosis and pulmonary embolism: a 25-year population-based study. Arch Intern Med 158(6):585–593
Janjua M et al (2008) Treatment of acute pulmonary embolism as outpatients or following early discharge. A systematic review. Thromb Haemost 100(5):756–761
Segal JB et al (2007) Management of venous thromboembolism: a systematic review for a practice guideline. Ann Intern Med 146(3):211–222
Stein PD, et al (2009) Early discharge of patients with venous thromboembolism: implications regarding therapy. Clin Appl Thromb Hemost
Yeager BF, Matheny SC (1999) Low-molecular-weight heparin in outpatient treatment of DVT. Am Fam Physician 59(4):945–952
Cairns CB et al (1998) Development of new methods to assess the outcomes of emergency care. Acad Emerg Med 5(2):157–161
Courtney DM et al (2009) Unique characteristics of emergency care research: scope, populations, and infrastructure. Acad Emerg Med 16(10):990–994
Conflicts of interest
The views expressed in this paper are those of the author(s) and not those of the editors, editorial board or publisher.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Kabrhel, C., Sacco, W., Liu, S. et al. Outcomes considered most important by emergency physicians when determining disposition of patients with pulmonary embolism. Int J Emerg Med 3, 239–264 (2010). https://doi.org/10.1007/s12245-010-0206-8
- Clinical decision rule
- Emergency department
- Pulmonary embolism
- Venous thromboembolism