Level of Recall Bias Regarding Pneumococcal Vaccination History among Adults Hospitalized with Community-Acquired Pneumonia: Results from the University of Louisville Pneumonia Study

Background: Recall bias is likely to occur in vaccine effectiveness studies using self-reported vaccination history. The validity of patient-reported vaccination status for adults is not well defined. The objective of this study was to evaluate the validity of self-reported pneumococcal vaccination history among patients hospitalized with community-acquired pneumonia (CAP). Methods: Prospective ancillary study of a population-based observational study of hospitalized patients with CAP in the city of Louisville. To be included in the analysis, patients had to (i) be reached by phone 30-days after discharge from the hospital and (ii) report that they remembered whether or not they received a pneumococcal vaccine in the past five years. The vaccination history was classified as 1) Subjective: patient recollection, or 2) Objective: vaccination records from insurance companies or primary care physicians. Results: A total of 2,787 patients who recalled their vaccination history were included in the analysis. Subjective vaccination history was documented to be inaccurate in 1,023 (37%) patients. Conclusions: Our study indicates that in adult patients, self-reported data regarding pneumococcal vaccination is likely to be inaccurate in one out of three patients. This level of recall bias may incorporate a fatal flaw in vaccine effectiveness studies. DOI: 10.18297/jri/vol1/iss4/3 Received Date: April 19, 2017 Accepted Date: May 4, 2017 Website: https://ir.library.louisville.edu/jri Affiliations: 1University of Louisville Division of Infectious Diseases, Louisville, KY 40205 2University of Louisville School of Public Health and Information Sciences Department of Epidemiology and Population Health, Louisville, KY 40205 3Pfizer, Inc.


Introduction
Vaccines are one of the most important public health interventions for the prevention of infectious diseases [1]. Recent vaccines introduced into the US market, such as the pneumococcal conjugated vaccine and the rotavirus vaccine, continue to demonstrate a significant public health impact by reducing hospitalizations and death [2,3]. For a new vaccine to be licensed in the US by the Food and Drug Administration (FDA), a trial needs to prove that the vaccine is able to prevent the disease for which it was developed. To test the efficacy of a new vaccine, studies are performed under well-controlled conditions. These studies are usually double-blind, randomized control trials (RCT). One of the primary advantages of vaccine efficacy studies using the RCT methodology is that randomization controls for confounding bias. On the other hand, one of the primary disadvantages of the RCT methodology is the exclusion of many patients at risk of acquiring the disease of interest. For example, these studies are likely to exclude patients who are immunocompromised, since these patients may not produce enough antibodies after vaccination. Once a vaccine has proven efficacy for a selected group of patients, the vaccine is further evaluated in a more "real life" study. These studies, including all patients at risk of the disease of interest, will produce data regarding vaccine effectiveness.
From an FDA regulatory point of view, vaccine efficacy will be tested in a phase III RCT and vaccine effectiveness in a phase IV post-license study. One of the most common study designs used to evaluate vaccine effectiveness is a retrospective case-control "test-negative" study, where the rate of vaccination among a population of patients with the disease is compared to the rate of vaccination among a population of patients without the disease [4].
One important challenge with this approach is that in clinical practice, vaccine history data are collected primarily via selfreport and some patients do not have an accurate recollection of their vaccine history. Inaccurate recollection of vaccine history will classify patients in the inappropriate study group, incorporating a systematic error in the study, defined as recall bias. Some level of recall bias is expected to occur in most studies when patients are asked to recall their vaccination history. If the level of recall bias is significant, the results of the vaccine effectiveness study may be invalid. Adult patients hospitalized with community-acquired pneumonia (CAP)with community-acquired pneumonia (CAP) may be asked to recall their pneumococcal vaccination history in studies evaluating effectiveness of pneumococcal vaccination for the prevention of hospitalizations.
The level of recall bias in adults regarding their vaccination history has not been evaluated.
We designed this study to define the level of inappropriate recollection of pneumococcal vaccination in adults hospitalized with CAP.

Study Design & Study Patients:
This was an ancillary study performed during a population-based prospective observational study of all adults hospitalized with CAP in the city of Louisville, Kentucky. Patients were included in the primary study if they had signs and symptoms of lower respiratory infection, an infiltrate on chest x-ray, and a final diagnosis of CAP. Patients were included in the vaccine recall ancillary study if they were able to be reached by phone 30-days after discharge from the hospital. Once patients were contacted, they were asked if they recalled their pneumococcal vaccination history. Patients with no recollection of vaccination history were excluded from the study. All participants provided written, informed consent.

Pneumococcal vaccination record verification:
Patients with recollection of vaccination history were asked if they received pneumococcal vaccination (polysaccharide or conjugated). Patients were also asked to provide the name of his/her current and prior primary care physician and pharmacy where patient could have received the vaccine, and current and prior health insurance coverage. These primary care physicians and local pharmacies were contacted for verification of pneumococcal vaccine administration.
In the city of Louisville, pneumococcal vaccination is only provided to individuals with health insurance, since it is the insurance company that covers the cost of the vaccine. Vaccination record was verified with the insurance companies for all patients included in the study.
All vaccination records were obtained corresponding with the five years prior to hospitalization due to CAP.

Study Definitions:
The vaccination history was classified as 1) Subjective: Patient Recollection, obtained from the patient at the time of the telephone interview or 2) Objective: Vaccination Record, obtained by contacting the insurance company, patient's primary care physician, or local pharmacy. The objective history was considered the gold standard.
Data Management & Quality: Data collection, management, and data quality control were performed by the same research team involved in the primary study.
Statistical Analysis: Categorical data were described using frequencies and percentages. Medians with interquartile ranges (IQRs) were used to describe continuous data. P-values were calculated to define differences between those with and without accurate recall of vaccine history using Chi-squared or Fisher's exact tests for categorical variables and the Mann-Whitney U-test for continuous variables. Measures of diagnostic accuracy were calculated using the objective vaccine history as the gold standard and the Kappa statistic was calculated to define the agreement between the two data collection methods.
Human Subjects Protection: Participants in the primary University of Louisville Pneumonia Study provided their consent for inclusion in the vaccine recall ancillary study.

Results
From a total of 3,378 patients who were contacted by telephone 30 days after hospital discharge due to an episode of CAP, 591 patients were excluded because they did not recall vaccination history. A total of 2,787 patients who recalled their vaccination history were included in the analysis.
Subjective vaccination history: Based on the self-reported vaccination history, 1,998 patients indicated that they received the pneumococcal vaccine prior to hospitalization, and 789 patients indicated that they did not receive pneumococcal vaccination prior to hospitalization.
Objective vaccination history: Based on insurance records or primary care physician records, 1,149 patients received the pneumococcal vaccination prior to hospitalization, and 1,638 patients did not receive pneumococcal vaccination prior to hospitalization.
Agreement between the two data collection methods were as follows: 1,062 patients subjectively recalled obtaining the vaccine and were verified via objective methods, and 702 patients recalled not having received the vaccine and were objectively documented to have not received it. In 936 patients subjectively recalled obtaining the vaccine were documented to not have received the vaccine via objective methods, and 87 patients recalled not having had the vaccine but were objectively defined as having received it. As depicted in Figure 1, from the total of 2,787 patients, accurate recollection was documented in 1,764 patients and inaccurate recollection was documented in 1,023 patients (37%). The diagnostic accuracy of subjective vaccination history with objective vaccination history is depicted in Table 1. The characteristics of the patients with accurate versus inaccurate recollection is depicted in Table 2.

Discussion
In this study of hospitalized adult patients with CAP, we documented that recall of pneumococcal vaccination history is inaccurate for 37% of the patients. Our study indicates that in adult patients, the self-report data regarding vaccination history is likely to be inaccurate in one out of three patients. Recall tended to be worse among older, sicker adults. Previous studies have reported similar discordance between self-reported and verified vaccination status across a variety of age groups and vaccines [5].
The results of our study have vaccine research implications. In research studies of vaccine efficacy, the administration of the vaccine is one of the study interventions; hence, misclassification of patients is very unlikely. In vaccine effectiveness studies, defining the vaccine status can be challenging. When vaccination status is ascertained from adult patients, the potential for recall bias is significant, to the point that a fatal flaw may be incorporated into the study [6].
The results of our study also have clinical implications. The majority of health care workers obtained vaccination history during a medical interview with the patient. Our study indicates that verbal questioning, as a method to obtain vaccination history for adults, should be considered unreliable. In clinical practice, this may lead to under-vaccination and lack of protection, or overvaccination and potential for vaccine hyporesponsiveness. In addition, a health care worker may look at the patient's electronic medical record to define vaccination history. If the information in the medical record was obtained from a prior patient interrogation, the possibility of inaccurate information will persist. Nearly 20% of all hospitalized patients with CAP enrolled in the main population-based study were excluded from this ancillary study due to inability to recall vaccination history. This lack of recollection intensifies the magnitude of using patient self-reported data as a reliable source for information about their immunization status.
Since pneumococcal vaccine administration in the city Louisville only occurs in those individuals with medical insurance, we believe that we were able to accurately capture vaccination records by identifying the medical coverage that patients had at the time of vaccine administration. However, it is possible that, due to a lack of recollection from the patients regarding all the insurance coverage over the prior 5 years from enrollment to this study, some information may have been missed in relationship with vaccine administration.
We evaluated only self-reported history of pneumococcal vaccination. Our findings of 37% of patients having inaccurate recollection is likely to be similar to the recollection with other vaccines. In the US, we have now 13 vaccines recommended for adults aged 19 years and older [7]. Considering the number of vaccines under clinical research, it is very likely that the number of vaccines for adults will continue to expand. The need for adults to remember an expanded number of vaccines will make recollection of vaccine history more challenging. If the patient received all medical care under a single health care system, the vaccine history may be obtained objectively from the electronic medical record. However, for the majority of adults, medical care is fragmented, and objective information regarding vaccination history may be difficult to obtain. Until a national vaccination registry is available, it will be important for adults to have a vaccination card or record where accurate vaccination history may be readily accessible.
Funding Source: This study was funded by Pfizer, Inc.