The Association of qSOFA, SOFA, and SIRS with Mortality in Emergency Department Pneumonia

Rationale: Sepsis scores are widely used and influence management decisions. Objective: To determine the association between 30-day mortality with Systemic Inflammatory Response Syndrome (SIRS), Sequential Organ Failure Assessment (SOFA), and quick SOFA (qSOFA) in emergency department patients with pneumonia. Secondary outcomes included the association of sepsis scores with hospital admission and direct ICU admission. Methods: This is a secondary analysis of a pneumonia population conducted in the emergency department of 3 tertiary care medical centers and 4 community hospitals. Adult immunocompetent patients diagnosed with pneumonia were included from 3 twelve-month periods spanning December 2009 to October 2015. We generated area under the receiver operating characteristic curve (AUC) values for each sepsis score for 30 day mortality and secondarily for hospital admission and direct ICU admission. We also created logistic regression models to assess associations of individual score components to the outcomes. Results: We studied 6931 patients with mean (SD) age 58 (20) years, and 30 day all-cause mortality rate 7%. Hospital and ICU admission rate was 63% and 16% respectively. Sepsis by SIRS was present in 70% of patients. Only respiratory rate and white blood count of the SIRS criteria were associated with 30-day mortality (OR=2.42 [1.94, 3.03] and 2.06 [1.68, 2.54] respectively, both p<0.001). Sepsis by qSOFA was present in 20%; all three components were associated with 30-day mortality (systolic blood pressure OR=1.36 [1.10, 1.68], respiratory rate OR=2.14 [1.72, 2.67], and altered mentation OR=6.53 [5.25, 8.09]; all p≤0.005). All six SOFA components were associated with 30-day mortality (all p≤0.001). qSOFA outperformed SIRS for 30-day mortality, (AUC=0.70 vs 0.61, p<0.001), hospital admission (AUC=0.70 vs 0.67, p<0.001), and intensive care unit admission (AUC=0.72 vs 0.64, p<0.001). SOFA significantly outperformed qSOFA for all outcomes except intensive care unit admission (AUC=0.74 vs 0.72, p=0.08). When compared to traditional pneumonia severity scores, the sepsis scores underperformed in prediction of mortality and ICU admission. Conclusions: In emergency department patients with pneumonia, qSOFA outperformed SIRS in relation to 30-day mortality, as well as hospital and ICU admission. SOFA performed better than qSOFA and SIRS for all outcomes except ICU admission. DOI: 10.18297/jri/vol2/iss2/4 Received Date: May 7, 2018 Accepted Date: July 10, 2018 Website: https://ir.library.louisville.edu/jri Copyright: ©2018 the author(s). This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Affiliations: 1Division of Respiratory, Critical Care, and Occupational Pulmonary Medicine, Department of Medicine, University of Utah, Salt Lake City, Utah. 2Division of Pulmonary and Critical Care Medicine, Intermountain Medical Center, Murray, Utah. *Correspondence To: Ian D Mecham, MD Work Address: University of Utah Hospital 26N 1900E 701 Wintrobe Salt Lake City, UT 84132 Work Email: ian.mecham@gmail.com 12 ULJRI Vol 2, (2) 2018 ORIGINAL RESEARCH Materials and Methods Study Design & Population This is a secondary analysis of a large pneumonia database. We studied patients seen in the ED from seven Utah hospitals during 3 twelve-month time periods (December 2009 to November 2010, December 2011 to November 2012, and November 2014 to October 2015). Three of the hospitals are tertiary care centers, and four are community hospitals. The Intermountain Healthcare Institutional Review Board approved this study with waiver of informed consent. The study was funded by the Intermountain Research and Medical Foundation. We collected data from the highly detailed Intermountain electronic medical record. We included consecutive patients ≥18 years of age with pneumonia seen in the ED with at least one set of vital signs measured. We identified patients with pneumonia using the International Statistical Classification of Diseases, 9th edition discharge codes for a diagnosis of pneumonia (480487.1) as either primary diagnosis or secondary diagnosis with respiratory failure or sepsis (581.x, 038.x) as a primary diagnosis. We excluded patients without evidence for pneumonia on initial chest imaging reports, reviewed by physician authors. We previously reported that this method of pneumonia case definition was 68% sensitive and 99% specific when compared to the gold standard of physician review of ED case records in our study population. (6) We also identified additional pneumonia patients by ED physician completion of a real-time electronic clinical decision support tool called ePneumonia, introduced in 2012 at 4 of the study hospitals. (6, 7) We excluded patients who died while in the ED, those who had immunocompromised conditions including human immunodeficiency virus and acquired immunodeficiency syndrome, solid organ transplant, and hematologic malignancies. To exclude patients with recurrent pneumonia, often caused by chronic aspiration or structural lung disease, we included only the first episode in a given 12-month period. Data Collection & Measurements Data elements included age, gender, Charlson comorbidity score, lactate measurement, use of vasopressors, mechanical ventilation, and presence of septic shock by Sepsis-3 criteria. Our primary outcome was 30-day all-cause mortality. Mortality data was obtained from a combination of hospital records, social security records, and the Utah Population Database. Secondary outcomes were hospital admission, direct admission to ICU, hospital length of stay, in-hospital mortality, and secondary hospital admissions within seven days among those discharged home. Calculating SIRS and qSOFA: Each component of SIRS and qSOFA was calculated using the worst values while in the ED, except laboratory values could be up to 4 hours prior to ED admission. We extended the time frame to within 24 hours of ED admission if the white blood cell count (WBC) was missing. We used only the respiratory rate for the SIRS respiratory component, since partial pressure of carbon dioxide was measured in <5% of the study population. We assumed the presence of sepsis under the Sepsis-2 definition as SIRS ≥2. For qSOFA, altered mentation was defined as Glasgow Coma Score (GCS) ≤14, or clinician documentation of disorientation to person, place, or time. We also assumed the presence of sepsis under Sepsis-3 for qSOFA ≥2. Calculating SOFA: We calculated SOFA in accordance with our previously published methods and used the worst values for each component while in the ED plus 4 hours prior. (8) We extended our search to within 24 hours of ED admission where values for platelets, bilirubin, and creatinine were missing in the ED. For the respiratory component, calculation of the partial pressure of arterial oxygen/fraction of inspired oxygen (PaO2/FiO2) ratio was performed from arterial blood gas when available. When blood gas values were not available, we estimated PaO2/FiO2 from pulse oximetry (converting peripheral oxygen saturation SpO2 to PaO2 using the Ellis’s corrected version of the Severinghaus equation). (9-11) We adjusted the resultant PaO2/FiO2 for the usual atmospheric pressure (645 mmHg) of study hospitals (~1400 meters above sea level) in accordance with previously described methods. (12) Where necessary, we estimated FiO2 by the equation liter flow of oxygen/min multiplied by 0.03 plus 0.21. (10) For the cardiovascular component, we converted all vasopressors to norepinephrine equivalent dosing, in accordance with previously published methods. (13) We used the highest charted dose irrespective of length of time it was applied. If a patient received dobutamine at any dose, a score of 3 was given unless the patient received a vasopressor at a dose sufficient to assign a score of 4 or 5. For the renal component, we solely used creatinine, as urine output measurement was unreliable in the ED. Sepsis was defined as present under the Sepsis-3 definition for total SOFA score≥2. Missing Data: We had complete qSOFA data for all patients. The only component missing for SIRS determination was WBC count in 437 patients, all of whom were discharged home from the ED. We imputed the missing WBC count as normal. Patients were included for SOFA analyses if they had at least five of the six components measured, among whom we assigned a score of 0 for any missing components as directed by the Sepsis-3 definition. (14) We omitted the baseline SOFA calculation, as it is incomplete for many patients. Altered mentation and GCS were extracted via electronic query from nurse charting, supplemented by manual review of ED physician and admission notes. In patients missing both GCS and orientation status, we used the following rules: Patients discharged home from the ED were assigned a GCS of 15 and normal mentation (98 patients). We imputed missing GCS using Classification and Regression Trees (CART) for patients with altered orientation based on nurse charting or manual review. We built the CART using cases in which the patient was confused and the GCS was measured (see supplemental material). 421 cases were imputed for the ED time frame using the predicted GCS from the CART. Statistical Methods The area under the receiver operating characteristic curve (AUC) for association of 30-day mortality was obtained for all three scores and compared using a bootstrap approach. We used a similar approach for the secondary outcomes of hospital admission and disposition to ICU among those admitted. Calibration was assessed using Spiegelhalter’s Z-test. The AUC for both CURB-65 (confusion, uremia, respiratory rate, blood pressure, age ≥65) and eCURB (an electronic version of CURB-65 using continuous, weighted variables) are reported for compa


Study Design & Population
This is a secondary analysis of a large pneumonia database. We studied patients seen in the ED from seven Utah hospitals during 3 twelve-month time periods (December 2009 to November 2010, December 2011 to November 2012, and November 2014 to October 2015). Three of the hospitals are tertiary care centers, and four are community hospitals. The Intermountain Healthcare Institutional Review Board approved this study with waiver of informed consent. The study was funded by the Intermountain Research and Medical Foundation.
We collected data from the highly detailed Intermountain electronic medical record. We included consecutive patients ≥18 years of age with pneumonia seen in the ED with at least one set of vital signs measured. We identified patients with pneumonia using the International Statistical Classification of Diseases, 9th edition discharge codes for a diagnosis of pneumonia (480-487.1) as either primary diagnosis or secondary diagnosis with respiratory failure or sepsis (581.x, 038.x) as a primary diagnosis. We excluded patients without evidence for pneumonia on initial chest imaging reports, reviewed by physician authors. We previously reported that this method of pneumonia case definition was 68% sensitive and 99% specific when compared to the gold standard of physician review of ED case records in our study population. (6) We also identified additional pneumonia patients by ED physician completion of a real-time electronic clinical decision support tool called ePneumonia, introduced in 2012 at 4 of the study hospitals. (6,7) We excluded patients who died while in the ED, those who had immunocompromised conditions including human immunodeficiency virus and acquired immunodeficiency syndrome, solid organ transplant, and hematologic malignancies. To exclude patients with recurrent pneumonia, often caused by chronic aspiration or structural lung disease, we included only the first episode in a given 12-month period.

Data Collection & Measurements
Data elements included age, gender, Charlson comorbidity score, lactate measurement, use of vasopressors, mechanical ventilation, and presence of septic shock by Sepsis-3 criteria. Our primary outcome was 30-day all-cause mortality. Mortality data was obtained from a combination of hospital records, social security records, and the Utah Population Database. Secondary outcomes were hospital admission, direct admission to ICU, hospital length of stay, in-hospital mortality, and secondary hospital admissions within seven days among those discharged home.
Calculating SIRS and qSOFA: Each component of SIRS and qSOFA was calculated using the worst values while in the ED, except laboratory values could be up to 4 hours prior to ED admission. We extended the time frame to within 24 hours of ED admission if the white blood cell count (WBC) was missing. We used only the respiratory rate for the SIRS respiratory component, since partial pressure of carbon dioxide was measured in <5% of the study population. We assumed the presence of sepsis under the Sepsis-2 definition as SIRS ≥2. For qSOFA, altered mentation was defined as Glasgow Coma Score (GCS) ≤14, or clinician documentation of disorientation to person, place, or time. We also assumed the presence of sepsis under Sepsis-3 for qSOFA ≥2.
Calculating SOFA: We calculated SOFA in accordance with our previously published methods and used the worst values for each component while in the ED plus 4 hours prior. (8) We extended our search to within 24 hours of ED admission where values for platelets, bilirubin, and creatinine were missing in the ED. For the respiratory component, calculation of the partial pressure of arterial oxygen/fraction of inspired oxygen (PaO 2 /FiO 2 ) ratio was performed from arterial blood gas when available. When blood gas values were not available, we estimated PaO 2 /FiO 2 from pulse oximetry (converting peripheral oxygen saturation SpO 2 to PaO 2 using the Ellis's corrected version of the Severinghaus equation). (9)(10)(11) We adjusted the resultant PaO 2 /FiO 2 for the usual atmospheric pressure (645 mmHg) of study hospitals (~1400 meters above sea level) in accordance with previously described methods. (12) Where necessary, we estimated FiO2 by the equation liter flow of oxygen/min multiplied by 0.03 plus 0.21. (10) For the cardiovascular component, we converted all vasopressors to norepinephrine equivalent dosing, in accordance with previously published methods. (13) We used the highest charted dose irrespective of length of time it was applied. If a patient received dobutamine at any dose, a score of 3 was given unless the patient received a vasopressor at a dose sufficient to assign a score of 4 or 5. For the renal component, we solely used creatinine, as urine output measurement was unreliable in the ED. Sepsis was defined as present under the Sepsis-3 definition for total SOFA score≥2.
Missing Data: We had complete qSOFA data for all patients. The only component missing for SIRS determination was WBC count in 437 patients, all of whom were discharged home from the ED. We imputed the missing WBC count as normal. Patients were included for SOFA analyses if they had at least five of the six components measured, among whom we assigned a score of 0 for any missing components as directed by the Sepsis-3 definition. (14) We omitted the baseline SOFA calculation, as it is incomplete for many patients.
Altered mentation and GCS were extracted via electronic query from nurse charting, supplemented by manual review of ED physician and admission notes. In patients missing both GCS and orientation status, we used the following rules: Patients discharged home from the ED were assigned a GCS of 15 and normal mentation (98 patients). We imputed missing GCS using Classification and Regression Trees (CART) for patients with altered orientation based on nurse charting or manual review. We built the CART using cases in which the patient was confused and the GCS was measured (see supplemental material). 421 cases were imputed for the ED time frame using the predicted GCS from the CART.

Statistical Methods
The area under the receiver operating characteristic curve (AUC) for association of 30-day mortality was obtained for all three scores and compared using a bootstrap approach. We used a similar approach for the secondary outcomes of hospital admission and disposition to ICU among those admitted. Calibration was assessed using Spiegelhalter's Z-test. The AUC for both CURB-65 (confusion, uremia, respiratory rate, blood pressure, age ≥65) and eCURB (an electronic version of CURB-65 using continuous, weighted variables) are reported for comparison for 30-day mortality and hospital admission. (15) The AUC for the Infectious Disease Society of America/ American Thoracic Society (IDSA/ATS) severe communityacquired pneumonia minor criteria was reported for comparison for ICU admission. (16) Logistic regression models were fit for SIRS, qSOFA, and SOFA separately, with the individual score components as the covariates in order to assess each component's contribution to the prediction of outcome. Outcomes again included 30-day mortality, hospital admission, and disposition to ICU among those admitted.
We also described patient characteristics and outcomes overall and for patients with and without sepsis as determined by each of the three scores. Student's T-tests, Wilcoxon rank sum tests, and chi-squared test for proportions were used as appropriate when comparing patients with and without sepsis. All analyses were conducted in R version 3.4.0. (17)

Study population
We initially identified 11,051 patients during the study period. We then excluded 4,120 patients resulting in a final study cohort of 6,931 patients (Figure 1). All study patients had complete values for SIRS and qSOFA, and 6,394 (92%) had at least 5 SOFA data components. Bilirubin was the most common missing value, unmeasured in 25% of patients, of which 70% were discharged home from the ED. Table 1 details the patient characteristics and outcomes according to positive (≥2 criteria) and negative (≤1 criteria) qSOFA. See eTables 1 and 2 for characteristics and outcomes for SIRS and SOFA data. Seventy percent of patients had sepsis by SIRS criteria, and 20% had sepsis by qSOFA. Patients with positive qSOFA were older and had more comorbidities than those with negative qSOFA. Most patients with positive qSOFA also had positive SIRS, while only 27% of patients with positive SIRS also had positive qSOFA. Of patients with positive qSOFA, 91% had organ dysfunction compared to 70% for positive SIRS. Thirty day mortality for patients with positive qSOFA was 15% versus 4% with negative qSOFA (p<0.001). Elevated lactate, vasopressor use, mechanical ventilation, and longer length of stay were significantly more common in patients with positive qSOFA as compared to those without; these were also more frequent in those with positive SIRS.  We compared these results to CURB-65 and eCURB risk scores developed specifically for community-acquired pneumonia. Both outperformed all three sepsis scores with an AUC = 0.80 and 0.83 respectively. Sensitivity and specificity for mortality for qSOFA≥2 were 0.47 and 0.82 respectively (See eTable 3). SIRS≥2 had greater sensitivity at 0.83 but poor specificity for mortality at 0.31.
Regression analysis for 30-day mortality of the components of each sepsis score is shown in More patients with positive qSOFA were admitted to the ICU compared to those with positive SIRS (42% vs 20%). More patients with positive SIRS were sent home than with positive qSOFA (30% vs 14%). Of the patients sent home despite positive qSOFA, 7% were admitted to the hospital within 7 days. Of those sent home despite positive SIRS, 4% were admitted to the hospital within 7 days.   All components for each score were predictive of these outcomes. Central nervous system alteration was the most predictive variable in the qSOFA and SOFA model for hospital admission, and also was a strong predictor for ICU admission along with cardiovascular and respiratory components.

Discussion
Among our population of ED pneumonia patients, qSOFA had better association than SIRS for 30-day mortality and for hospital and ICU admission. SOFA significantly outperformed either score for all outcomes, except compared to qSOFA for predicting ICU admission in which no significant difference was seen. Our results support the original findings for Sepsis-3. (3) While SOFA and qSOFA were largely developed for inpatients, our study suggests the same criteria have utility in the emergency department assessment of pneumonia severity since 37% of our population were discharged home.
Having ≥2 qSOFA criteria effectively identified older patients with a high degree of organ dysfunction. These patients more frequently required advanced support with vasopressors, mechanical ventilation, and remained in the hospital longer. In contrast, SIRS had relatively weaker associations with mortality, hospital admission, or ICU care. Several studies investigating the utility of the Sepsis-3 scores to predict outcomes in patients with suspected infection while in the ED have shown similar results to ours for qSOFA and SIRS. (18,19) Our population differs from these studies in that we only included patients with pneumonia, many of whom likely had viral rather than bacterial pathogens. (20) In our study, mortality prediction was inferior to the pneumoniaspecific score CURB-65/eCURB, which were calculated for comparison. CURB-65 and qSOFA are electronically calculable using routinely obtained elements. However qSOFA omits age and laboratory evaluation, and has different thresholds than CURB-65 for respiratory rate and systolic blood pressure. We unsurprisingly observed that the IDSA/ATS minor criteria for severe community-acquired pneumonia outperformed sepsis scores in predicting need for ICU care in pneumonia patients. Similar findings in pneumonia patients were reported by Ranzani et al. In their study, qSOFA outperformed SIRS for in-hospital mortality though the pneumonia severity index remained the most accurate and useful tool for mortality prediction and clinical decision-making. (21) Combining the results of our studies, traditional pneumonia scores remain more accurate for making decisions while in the ED, though at the expense of additional time and data elements. Our results do not suggest that qSOFA or SOFA should replace traditional pneumonia scoring tools or clinician judgement. However, as more emergency departments embrace qSOFA and SOFA over SIRS, we are reassured to observe that these scores have some value in estimating pneumonia mortality and may assist in ED triage.
Our study confirmed fair performance of SOFA for mortality and ED disposition, outperforming SIRS and qSOFA in almost all outcomes. However, SOFA remains difficult to calculate without electronic assistance and is less useful outside the ICU. (3,22) SOFA ideally relies on prior data which is often unavailable to allow comparison to a baseline SOFA score. Bilirubin is not routinely measured in ED patients with pneumonia, often making SOFA scoring incomplete in ED settings. Despite qSOFA showing better outcome prediction than SIRS, the primary criticism of qSOFA has been concern over its lack of sensitivity for various outcomes. (19,(23)(24)(25) Churpek et al. investigated several early warning scores to predict the composite outcome of death and ICU transfer in admitted patients in the ED and wards. (26) qSOFA outperformed SIRS for the composite outcome; however qSOFA's sensitivity was 54% compared to 91% for SIRS. Our study similarly showed qSOFA was insensitive though relatively specific for all outcomes while SIRS was highly sensitive but poorly specific. This suggests 16 ULJRI Vol 2, (2) 2018 qSOFA to be mostly helpful when positive. Further study looking at thresholds for criteria or adding additional values to qSOFA may improve its utility.
We compared the different components of each sepsis score for prediction of mortality. All qSOFA and SOFA components were associated with 30-day mortality while only 2 of 4 components were predictive in the SIRS model. Altered mentation was most predictive of mortality as well as hospital admission. Cardiopulmonary dysfunction often requires ICU specific therapies of vasopressors and mechanical ventilation, explaining the prominence of hypotension and elevated respiratory rate in the ICU admission decision.
Our study has limitations. We utilized a large patient cohort from multiple hospitals of different sizes. Although our study was retrospective, our database is comparable to many prospective observational studies with minimal missing data. Several retrospective studies examining similar outcomes were limited by their definition of suspected infection, while our criteria selected a population with high specificity for the diagnosis of pneumonia. (3,26,27) Most of our missing data were two of the SOFA components, GCS and bilirubin. Our emergency departments did not routinely obtain a bilirubin or measure the GCS score in patients discharged home or admitted to a ward. Adoption of SOFA would thus require additional testing in our hospitals. We substituted altered mentation or GCS ≤14 for qSOFA, understanding that the original model used a GCS ≤13, which likely led to a relative increase in sensitivity of qSOFA in our study.

Conclusions
In ED patients with pneumonia, qSOFA was better associated with 30-day mortality than SIRS. Secondary outcomes showed qSOFA had superior association with hospital and ICU admission. SOFA performed better than qSOFA and SIRS for all outcomes except compared to qSOFA for ICU admission. CURB-65/eCURB was more predictive of mortality than the sepsis scores as were the IDSA/ATS minor criteria for severe community-acquired pneumonia in predicting need for ICU admission, confirming that well validated pneumonia scores are more accurate for pneumonia patients. Emergency departments adopting Sepsis-3 criteria in their initial triage may have some value when applying them to patients with pneumonia.