Impact of Pooling Samples on Analytic Sensitivity of a Real-Time Reverse Transcriptase PCR Assay for SARS-CoV-2

During the COVID-19 pandemic, laboratories experienced periods of shortages for certain critical materials required to meet the high demand for SARS-CoV-2 testing. The U.S. Food & Drug Administration provided a template for molecular diagnostic testing, including guidance for a specimen pooling process in order to evaluate performance of the SARS-CoV-2 nucleic acid amplification assay. This study aimed to evaluate the testing of pooled specimens consisting of four nasopharyngeal swab specimens using the Luminex ARIES ® nucleic acid amplification platform. Results indicated that there was a loss of analytic sensitivity with pooled nasopharyngeal swab samples, demonstrating that this approach should be bal- anced against material shortages and the clinical utility of a less sensitive assay.


Introduction
When the World Health Organization (WHO) declared COVID-19 a pandemic in March 2020 [1], the demand for laboratory testing increased dramatically. This high demand for testing placed considerable pressure on healthcare providers and especially clinical laboratories to collect, process, and test the recommended respiratory samples, with nasopharyngeal swabs (NP) being the "gold standard." Simultaneously, device manufacturers needed to rapidly deploy viral transport media, instruments, kits, and reagents to their customers under the U.S. Food & Drug Administration (FDA) emergency use authorization (EUA) guidelines. Due to these factors, competition for limited supplies led to periodic shortages of real-time reverse transcriptionpolymerase chain reaction (RT-PCR) materials for clinical laboratories. The placement of clinical laboratories on allotment of viral transport media, test kits, reagents, swabs, and other critical supplies challenged the workflow, limiting the number of tests that could be performed. The University of Louisville (UofL) Infectious Diseases Laboratory experienced shortages in carrier RNA, primers and probes, aerosol barrier tips for micropipettes, and vendor-specific RT-PCR master mix and cassettes at various time points in 2020 due to the pandemic. One possible solution to the material shortages was to consider pooling of respiratory samples if the positivity rate in the target population was less than 5-6% as recommended by U.S. FDA. [2] In this way, true savings of materials, time, and labor costs would be realized.
Following the FDA EUA guidance for molecular diagnostic testing (updated on July 28, 2020), the UofL Infectious Diseases Laboratory evaluated the positivity rate for SARS-CoV-2 in various populations being tested by Luminex ARIES ® individual real-time RT-PCR assay for SARS-CoV-2. The impact on the sensitivity of the assay if four respiratory samples were pooled prior to testing was determined; results of pooled tests were compared to individual real-time RT-PCR assay results for SARS-CoV-2 on the Luminex ARIES ® platform. [3] The aim of this study was to determine whether pooling of NP specimens was a viable option to reduce the total number of testing components required for real-time PCR detection of SARS-CoV-2.

Materials and Methods
Real-time RT-PCR assay for SARS-CoV-2 The UofL Infectious Diseases Laboratory validated a TaqMan real-time RT-PCR assay for SARS-CoV-2 with NP specimens tested individually. The reagents, in-strument, and assay settings for pooled testing were the same as for individual sample testing as previously described for the Luminex ARIES ® . [3,4] The limit of detection (LOD) was the cycle threshold (Ct) value at which 100% of the individually tested NP specimens were positive in triplicate, and this final LOD concentration was confirmed by testing 24 individual replicates. Briefly, the established LOD Ct value for N1 was 33.6±0.56 (range 33.04-34.16) and for N3 was 32.3±0.36 (range 31.94-32.76) for individual NP specimen PCR, which was equal to 1.5 tissue culture infective dose-50% (TCID 50 )/mL of SARS-CoV-2. The positive control material used was heat-inactivated viral culture fluid (CFHI-0.5 mL, Zeptometrix, Buffalo, NY). Because SARS-CoV-2 could be detected in clinical samples (confirmed as positive by another method) beyond the LOD of the assay as determined using a quantitated commercial positive control (listed above), the amplification Ct cutoff value for nucleocapsid gene N1 was 38, the Ct cutoff value for nucleocapsid gene N3 was 37, and the Ct cutoff value for RNase P was 45, to avoid missing any positive specimens at this early point in the pandemic (March 2020). The Ct cutoffs for each target were determined using the LOD data and from 60 confirmed positive and negative clinical specimens. Furthermore, if either N1 or N3 was detected, and the RNase P internal control was detected, the test was repeated in duplicate. If either one of the nucleocapsid targets was repeatedly detected, then the result was reported as "presumptive positive" for patient management purposes and for the benefit to public health.

Sample pooling process
Groups of four NP specimens were placed in a rack in a biosafety cabinet. Each group of four was labeled as Pool A, Pool B, Pool C, etc., until all the pools were created. The unique identification of each sample in the pool was recorded on worksheets. Each specimen was vortexed prior to removing 500 µL, which was placed in a sterile 15 mL tube for a final volume of 2 mL per pool. Each pool was vortexed again; then, 195 µL of the pool was transferred to a microcentrifuge tube containing 5 µL of 1 µg/1 µL carrier RNA in AVE buffer (QI-AGEN, Germantown, MD). Next, 200 µL of this mixture was placed in an ARIES ® cassette connected with ReadyMix ® tubes containing 5 µL of primer/probe set for SARS-CoV-2 real-time RT-PCR testing. Individual samples were stored in the refrigerator until pool testing was complete. If a pool resulted positive, then each individual sample in the pool was retested. Groups of three NP specimens were prepared in the same manner, with the total "n-1 pool" equaling 1.5 mL. Because deidentified samples were used and no individually identifiable information was obtained, this method validation study was deemed exempt from institutional review board (IRB) requirements.

Choice of population for pooling of specimens
The UofL Infectious Diseases Laboratory created reports in REDCap ® , the Laboratory's information management system [5,6], to determine positivity rates from different populations being tested for SARS-CoV-2. [7] At the time of the evaluation, the health care worker and hospitalized patient populations had a positivity rate greater than 6% (data not shown), so the laboratory chose to pool NP swabs from patients of the UofL Division of Infectious Diseases Travel Clinic. These patients were generally healthy and asymptomatic, only being tested for the purposes of travel, workplace surveillance, or elective surgery; thus, positivity rates were well below 6% for this population. Therefore, the UofL Travel Clinic population met the FDA criteria to perform the SARS-CoV-2 pool test validation.

Analytical validation
All steps were followed as outlined in the FDA EUA guidance, updated on July 28, 2020. [2] Briefly, 100 Ct values for N1 and N3 targets from individually tested SARS-CoV-2-positive NP specimens were examined to determine the number of specimens with Ct scores close to the Limit of Detection. Next, 80 negative NP specimens previously tested by the ARIES ® real-time RT-PCR assay and individually negative for SARS-CoV-2 were used to create pools of 4 to verify that they remained negative when tested in pools. To compare the results of pooled specimens to individual test results, 48 positive NP specimens were analyzed by combining 1 positive specimen with 3 known negative specimens ("n pools," or 4 specimens total). Per the FDA EUA guidance, 25% of the positive specimens should be within 2-4 Ct values of the established N1 and N3 cutoff values. Finally, 26 samples were analyzed by combining 1 positive specimen with 2 known negative specimens ("n-1 pools," or 3 specimens total).

Analysis of data
Microsoft Excel was used to determine the positive and negative agreement, comparing the performance of tests for pooled samples vs. expected results from individual tests. The MedCalc Diagnostic Calculator was used to determine analytic sensitivity and specificity. [8] Agreement between pooling and undiluted or "neat" samples, based on uncategorized Ct value thresholds, was assessed through unweighted and weighted Cohen's Kappa. Simple linear regression analysis was performed to evaluate the shift in Ct values for N1 and N3 for the positive patient samples diluted with the negative patient samples ("n pool," or 4 total samples). Cohen's Kappa and linear regressions were performed using R version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria). All values reported as frequency/total (%). All values reported as frequency/total (%).

Results
We focused our pooling validation efforts on patients of the UofL Division of Infectious Diseases Travel Clinic, typically healthy people being screened for COVID-19. This group of specimens gave our population a combined positivity rate of 1.78% (2/112), which was acceptable for the specimen pooling evaluation. When 80 SARS-CoV-2 negative NP specimens were tested both individually (80 results) and in pools of 4 (20 results), all real-time RT-PCR results remained negative for SARS-CoV-2. The 100% negative agreement translated to no false positives, or 100% analytical specificity.
To determine the percentage of positive agreement for "n pools", 48 previously positive SARS-CoV-2 speci-mens (having a range of Ct values from 15.5 to 36.7 as required by FDA) were re-tested both individually and in pools with three negative samples ("n pool," or four samples). Per the FDA EUA Guidance, 25% of the 48 specimens were to be within 2-4 Ct values for N1 and N3 targets when tested individually. Eight of these 48 previously positive NP specimens were not detected in the "n pool" sample ( Table 1). All eight Ct values were near the established LOD when tested individually. Six of these eight "false negative" specimens had Ct values above the LOD range (already weaker positives in the individual assay), but below the cutoff values established for the individual real-time RT-PCR assay, while the Ct value for one sample was within the established LOD range, and one sample had a Ct value below the established LOD range. Thus, for NP specimens with Ct values near the LOD when tested individually, pooled testing may give false negative results.
To determine the percentage of positive agreement in "n-1 pools" as recommended by the FDA EUA guidance, 26 previously positive SARS-CoV-2 specimens (with a range of Ct values from 17.3 to 36.9 as required by FDA) were re-tested both individually and in pools with two negative samples ("n-1 pool", three samples). Per the FDA guidance, 25% of the 26 specimens were to be within 2-4 Ct values of the Ct cutoff values for N1 and N3 when tested individually. Three of these 26 positive samples were not detected in the "n-1 pool" sample. The results of this analysis are shown in Table  2. Of the three samples not detected, all had Ct values above the LOD (weaker positives) established for this assay, but below the assay cutoff values. As expected, we found that the sensitivity increased with smaller pools of three rather than four specimens, improving the percent positive agreement to 88.5% from 83.3%.
The next steps in the data analysis process were to use statistical methods to determine the level of positive detection agreement for each target (N1 and N3) for pools of four NP specimens. As the groups could be viewed as ranked/ordinal, weighted Cohen's Kappa  was appropriate to use for analysis. Both gene targets indicated a statistically significant (determined by the confidence interval) level of agreement between pooled and individual testing; only the magnitude of agreement was different. In either case, the level of agreement could be described as fair (unweighted) to moderate (weighted) (Tables 3 and 4).
The final step in the data analysis process was to use linear regression to calculate the average difference between individual and "n pool" specimen testing with regard to detection of N1 and N3 targets. In this case, the line of best fit represents the shift in Ct values for N1 or N3 from individual to "n pool" testing of NP specimens. The linear regression analysis demonstrated a mean difference of 2.016 and a P-value <0.001 for N1. This indicated that N1 Ct values from pools were on average 2.016 higher than the N1 Ct values from individual testing (Figure 1). Similarly, linear regression analysis of Ct values for N3 demonstrated a mean difference of 2.272 and a P-value <0.001. This indicated that N3 Ct values from pools were on average 2.272 higher than the N3 Ct values from individual testing (Figure 2).

Discussion
The aim of this study was to evaluate the impact of pooling four NP specimens prior to testing on the realtime RT-PCR assay's sensitivity. The FDA guidance recommended pooling samples when the target population's positivity rate was less than 5-6% for true savings to be realized. For example, if we chose to pool 100 samples from a population with a 10% positivity rate, then 25 pools of four samples each would be created. Assuming that ten patients of the 100 were evenly distributed among the 25 pools, then ten of the 25 pools would need to be retested individually to find the individual positive samples. Thus, the original 25 pooled tests plus 40 individual tests (from the ten positive pools) give a total of 65 tests instead of 100 individual tests. While this represents savings on materials, when compared to the staff time and labor spent creating pools and breaking down pools, the savings are not significant. Performing the same analysis with a 5% positivity rate would reduce the total number of tests from 100 to 45, which is significant.
After a target population with a less than 5-6% positivity rate was found, NP specimens were tested individually and in pools of four specimens first, then pools of three, following U.S. FDA EUA guidance for method validation. Positive predictive agreement for pools of four NP specimens was 83%, while pools of three NP  specimens improved to 88%. Linear regression data analysis showed that for each gene target (N1 and N3), there was a shift in Ct values between 2 and 3 when comparing individual and pooled NP specimen testing on the ARIES ® instrument. Based on these results, the UofL Infectious Diseases Laboratory would choose to implement pooling under the following conditions: (1) when the target population has a rolling weekly average of less than 5-6%; (2) when the persons being tested are asymptomatic; and (3) when required kits, reagents, or consumables are limiting for the real-time RT-PCR assay for SARS-CoV-2.
This study provided new information about the ARIES ® SARS CoV-2 assay with regard to the utility of pooling when laboratory testing resources are limited. One strength of the study was that a standardized, rigorous evaluation was performed following the criteria set forth in the U.S. FDA EUA guidelines updated on July 28, 2020. As a result, all of the data analysis indicated that an expected shift of 2-3 Ct values occurred with pooling. After the evaluation was completed, a total of 49 specimens were tested in 17 pools, consisting of 2-4 NP specimens from asymptomatic populations, using the validated SARS-CoV-2 pool assay. No pools had detectable N1 or N3 RNA, but the internal control RNaseP was detected in each pool. As a result, we demonstrated that pooling of four NP specimens (or fewer) was a viable alternative to individual testing whenever laboratory supply chains are inter-rupted. Furthermore, the benefits of the ARIES ® instrument were maintained, using a sample-to-answer protocol with a closed-system, reducing the risk of crosscontamination.
Some limitations of the study were identified. The realtime RT-PCR laboratory-developed test for SARS-CoV-2 using the Luminex ARIES ® system is a multiplex assay, targeting N1, N3, and the internal control human RNaseP. Using a multiplex assay potentially reduced the analytic sensitivity when compared to a single target assay. In a single target assay, there is no primer competition nor any potential "bleed over" between fluorescent channels that may reduce sensitivity. In addition, since the test uses a sample-to-answer instrument, there is no separate RNA extraction or cDNA synthesis step prior to the amplification step, possibly affecting analytic sensitivity compared to multi-step procedures. For example, using the ARIES ® system, a fixed volume of the sample must be added to the cartridge. With a separate extraction step, the amount of nucleic acid can be quantified and normalized prior to reverse transcription and PCR amplification to account for samples with low viral loads. One study demonstrated that up to 30 samples could be pooled if RNA was extracted prior to the RT-PCR step. [9] Finally, during the time frame in which this study was performed, testing demand was high, and resources were limited, so a full validation with data analysis for pools of three NP specimens was not pursued.
While the loss in analytical sensitivity was expected with pooling of four NP specimens, further study of the correlation of Ct values with clinical signs, symptoms, and transmissibility is warranted. Two studies early in the pandemic indicated that specimens tested by realtime RT PCR assays and having Ct values higher than 33-34 [10] or 35 [11] were unlikely to have cultivable virus and thus were less likely to be infectious. Repeating these viral cultivation studies with pooled specimens would provide additional data regarding the impact of pooling specimens on clinical sensitivity and patient management decisions.