Assessment of Pneumonia Severity Indices as Mortality Predictors

Background: The leading cause of infectious disease death in the United States is communityacquired pneumonia (CAP). Several pneumonia severity indices exist and are widely used as tools to assist physicians regarding site of care based on risk of death. However, limited data exists that discerns which of the most commonly used severity scores is the best predictor of mortality across multiple time points. The objective of this study is to determine the best mortality predictor at different time points between four of the most commonly used pneumonia severity scores. Methods: This was a secondary analysis of a prospective, multicenter, population-based, observational study of patients hospitalized with CAP in the city of Louisville, KY. The severity indices used were the American Thoracic Society (ATS) criteria, the Pneumonia Severity Index (PSI), the British Thoracic Society criteria (CURB-65), Quick Sepsis-Related Organ Failure Assessment (QSOFA), and direct ICU admission to represent physician discretion. The accuracy, kappa statistic, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the ability to predict in-hospital, 30-day, 6-month, and 1-year mortality. 95% confidence intervals for each variable were generated by bootstrapping with random sampling and resampling of the subjects 1000 times. In addition, the area under the curve (AUC) was calculated for each severity score and mortality time point. Results: There were 6013 eligible patients included in this analysis with data collected between the years 2014 and 2016. At each time point, the QSOFA had the highest sensitivity and NPV, while the PSI had the highest specificity and PPV. QSOFA had the highest accuracy for in-hospital mortality, 30-day mortality, and 6-month mortality, and the CURB-65 had highest mortality for 1-year mortality. The QSOFA had the highest kappa statistic for in-hospital mortality, the CURB-65 had the highest kappa statistic for 30-day mortality, and the PSI had the highest kappa statistic for 6-month and 1-year mortality. The AUC was highest for the ATS criteria for in-hospital mortality, and was highest for the PSI at the remaining time points. Conclusions: The results of this study show that QSOFA and the PSI are the most reliable severity indices for mortality predictions based on these measures. QSOFA was found, on average, to have the highest accuracy, sensitivity, and NPV. Additionally, PSI was found, on average, to have the highest kappa statistic, specificity, and PPV. The AUC, on average, was best with PSI as the predictor. QSOFA is most capable of making true negative predictions and the PSI is the most capable of making true positive predictions across the four time points. DOI: 10.18297/jri/vol3/iss1/7 Received Date: November 20, 2018 Accepted Date: January 14, 2019 https://ir.library.louisville.edu/jri/vol3/iss1/ Affiliations: 1University of Louisville School of Medicine, Department of Medicine, Division of Infectious Diseases 2University of Louisville School of Public Health and Information Sciences, Department of Epidemiology and Population Health This original article is brought to you for free and open access by ThinkIR: The University of Louisville’s Institutional Repository. It has been accepted for inclusion in The University of Louisville Journal of Respiratory Infections by an authorized editor of ThinkIR. For more information, please contact thinkir@louisville. edu. Recommended Citation: English, Connor L.; Chandler, Thomas; Guinn, Brian E.; Furmanek, Stephen P.; and Ramirez, Julio A. (2019) “Assessment of Pneumonia Severity Indices as Mortality Predictors,” The University of Louisville Journal of Respiratory Infections: Vol. 3 : Iss. 1, Article 7. *Correspondence To: Stephen P. Furmanek, MPH, MS Work Email: stephen.furmanek@louisville.edu ORIGINAL RESEARCH Copyright: © 2019 The author(s). This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


Study design
This was a secondary analysis of a prospective, multicenter, population-based, observational study of patients hospitalized with CAP in the city of Louisville, KY from June 1, 2014 to May 31, 2016 [3].

Subjects
Patients were only eligible for inclusion in this analysis if 1) they were hospitalized for CAP during the study period, 2) pneumonia severity could be assessed for each index, and 3) mortality at each time point was known.

Study Definitions and Measurements
Community-acquired pneumonia: Screenings were conducted for the primary analysis participation for all hospitalized adult patients at participating hospitals. Individuals were asked to participate in the study if they met the following 3 criteria: (1) presence of a new pulmonary infiltrate on chest radiograph and/or chest computed tomography scan at the time of hospitalization, defined by a board-certified radiologist's reading; (2) at least 1 of the following: (a) new cough or increased cough or sputum production, (b) fever >37.8°C (100.0°F) or hypothermia <35.6°C (96.0°F), (c) changes in leukocyte count (leukocytosis: >11000 cells/μL; left shift: >10% band forms/ mL; or leukopenia: <4000 cells/μL); and (3) no alternative diagnosis at the time of hospital discharge that justified the presence of criteria 1 and 2.

American Thoracic Society (ATS) Guidelines
List of major and minor criteria for ICU admission intended primarily for use by emergency medicine physicians, hospitalists, and primary care practitioners in response to confusion regarding differences between guidelines for the ATS and the Infectious Diseases Society of America [4].

Pneumonia Severity Index (PSI)
A clinical prediction rule that considers patient demographics, comorbidities, physical examination findings, vital signs, and essential laboratory findings to categorize pneumonia severity into five risk classes, with categories IV and V having the highest probability of mortality [5].

British
Thoracic Society criteria (CURB-65) Severity assessment tool that identifies severe CAP and high risk mortality in the presence of two or more of the following features: mental confusion, respiratory rate > 30/min, diastolic blood pressure < 60 mmHg, and blood urea > 7 mmol/l [6].

Quick Sepsis-Related Organ Failure Assessment (QSOFA)
Identifies patients with a suspected infection at high risk for in-hospital mortality outside of the ICU, based on the following criteria: Respiratory rate ≥ 22/min, mental confusion, and systolic blood pressure < 100 mm Hg [7].

Definition of Pneumonia Severity
Pneumonia severity was then assessed using 4 severity indices: American Thoracic Society (ATS) criteria [3], the Pneumonia Severity Index (PSI) [4], the British Thoracic Society criteria (CURB-65) [5], Quick Sepsis-Related Organ Failure Assessment (QSOFA) [6], and direct ICU admission to represent physician discretion. The measurements associated with each index can be found in Table 1.
Severity for the ATS criteria was defined as the presence one major criteria and/or three minor criteria as defined in the guidelines [3]. Severity for the PSI was defined as risk class IV or V. Severity for the CURB-65 was defined as a score of 4 or 5. Severity for the QSOFA was defined as a score of 3. x Sex x Sodium x Thrombocytopenia x Vasopressors x

Human Subjects Protection
The study was approved by the Institutional Review Board (IRB) at the University of Louisville Human Subjects Research Protection Program Office (IRB number 11.0613) and by the research offices at each participating hospital. The study was exempt from informed consent.

Statistical Analysis
Descriptive statistics were performed. For each severity index, severity as defined was used to determine accuracy, the kappa statistic, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the ability to predict in-hospital, 30-day, 6-month, and 1-year mortality. To obtain 95% confidence intervals for each variable, bootstrapping with random sampling and resampling of the subjects 1000 times was performed. In addition, the area under the curve (AUC) was calculated for each severity score using the severity as defined and each mortality time point.

Results
There were 6013 patients eligible for this analysis. Patient characteristics, including the number of patients who met severity as defined, and mortality, are shown in

Discussion
This study indicates that QSOFA and the PSI are the most reliable severity indices for mortality predictions over the short and long-term. QSOFA is the most reliable severity index for accuracy, sensitivity, and NPV whereas the PSI is the most reliable severity index for the kappa statistic, specificity, and PPV. QSOFA is most capable of making true negative predictions and PSI is most capable of making true positive predictions. From the AUC values, we can see that the PSI is the most consistent, while QSOFA was always the poorest performer in predicting outcomes at every time point.
One important clinical implication is that these severity indices consistently outperform a physician's discretion when severity is determined. However, these indices are used almost exclusively in research settings and not in a clinical setting. This shows a need for an index that can predict mortality during the course of a patient's hospital stay to better direct medical interventions and improve health outcomes related to pneumonia.
This data supports the study and creation of these indices. However, the agreement between severity and mortality is poor across all indices at each time point, as evidenced by the low Kappa statistics and AUC values. No AUC went above 0.7, indicating that alone these severity indexes at a dichotomous cut-point do not exhibit strong predictive power for predicting mortality. Also, accuracy decreases as the time increases. This shows a need for an index that can better predict long-term mortality with higher accuracy.
Funding Source: Study was supported by the Division of Infectious Diseases, University of Louisville, Kentucky. Table 3 Statistical measures for severity indices with in-hospital mortality as the outcome. Table 5 Statistical measures for severity indices with 6-month mortality as the outcome. Table 6 Statistical measures for severity indices with 1-year mortality as the outcome. Table 7 Area under the curve (AUC) for mortality outcomes with severity indices as the predictor.