A Comparison Efficacy Study of Commercial Nasopharyngeal Swabs versus a Novel 3D-Printed Swab for the Detection of SARS-CoV-2

Introduction: The large volume of diagnostic tests required by the response to the COVID-19 pandemic resulted in a shortage of commercial nasopharyngeal swabs. In an effort to alleviate the shortage, swabs created by 3D printing may be a solution. Methods: We designed and produced 3D printed swabs and sought to compare their ability to detect SARS-CoV-2 in patients admitted for COVID-19 or who were suspected of having COVID-19. Results: A total of 30 patients were swabbed with a commercial and a 3D printed swab. Results matched in 27 of 30 patients (90%). Two patients were discordant with a positive commercial swab and a negative 3D printed swab and another was discordant because the 3D printed swab was positive and the commercial swab was negative. The sensitivity was 89%, specificity was 92% and Cohen’s kappa coefficient was 0.80. Conclusion: The 3D printed swabs performed acceptably compared to the commercial swab and may be considered for use in lieu of a commercial swab.


Introduction
During the coronavirus-19 (COVID-19) pandemic, materials were in short supply including personal protective equipment (PPE), ventilators, viral transport medium, and nasopharyngeal swabs.The shortages were compounded by the increased demand in "hotspots" like New York and Louisiana.Creative solutions were approved for PPE and viral transport medium by the Centers for Disease Control and Prevention (CDC).[1] Emergency use authorizations were granted by the Food & Drug Administration for ventilators and the analysis of alternative body fluids for the diagnosis of COVID-19.[2] However, a nasopharyngeal specimen remains the standard diagnostic method for COVID-19, especially in asymptomatic patients.
Working to alleviate the need for an increased demand of nasopharyngeal swabs, and using 3D technology available to make models for dental implants out of resin, we designed and created a 3D printed swab resembling a commercial flocked nasopharyngeal swab.Certain physical and handling parameters were included in the design, but it was unknown if it would actually extract SARS-CoV-2 from a mucus membrane of someone's nasopharynx and elute the virus into the transport media so that it could be detected by reverse transcription-polymerase chain reaction (RT-PCR) analysis.If a manufacturer does not specify a certain swab to be used with their kit, then a new swab should have in-house validation first.[3] The objective of this study was to compare the detectability of SARS-CoV-2 from the nasopharynx of a commercial swab to our novel 3D printed swab.

Setting and Population
This was a cohort prospective study of non-consecutive patients and healthcare workers with a recent diagnosis of, or suspicion for, COVID-19.(IRB # 20.0334)It was performed at two acute care hospitals of UofL Health (an affiliate of the University of Louisville School of Medicine) and required Institutional Review Board approved informed consent.There was a recently implemented policy at the time of the study that was relevant; all patients admitted to the hospital were screened upon admission.Recently identified COVID-19 positive populations, or those who tested negative but who had signs or symptoms consistent with COVID-19, were candidates for enrollment.

Sample collection
A commercial swab and a 3D printed swab were inserted into the nasopharynx of each subject in succession during the same encounter per standard methods (insertion for 10 seconds with some rotation) and then inserted into a viral transport media tube with a sealed cap for transport to the laboratory on campus.The laboratory was CLIA certified under an emergency use authorization by the FDA and had been used to perform approximately 4,500 samples prior to initiating the present study.Internal validation of the laboratory had been performed with the established CLIA certified University of Louisville Infectious Diseases Research Laboratory.A goal of testing 30 COVID-19 positive people was established per standard recommendations for validation with at least 90% agreement.[4]

Data Collection and Management
All data for this study were de-identified and kept in a REDCap database.Demographic information was col-lected for each patient.The primary information collected was the result of each swab used in each subject.If a subject had been previously positive, then the time from the previous test to the present test was also recorded.

3D Printed Swab Design and Production
Guidance for the swab design for 3D printing was provided by an on-line research collective that provided insight in to testing and evaluation of different swab designs and materials.[5] A design similar to a typical commercial swab was chosen (Figure 1).Clinicians required that the material be flexible so it would not break off during use, that the collection end be comfortable for the patient and not cause any trauma to the tissues, and that there be a notch between 80-100 mm from the tip in order to break the swab off into the transport tube containing media.An open lattice design with a domed tip was selected (Figure 2).Test swabs were produced from Envisiontec (Envisiontec, Inc., Dearborn, MI) E1 guide soft material, and 3D printed using direct light projection with a 3D printer (NewPro3D, North Vancouver, BC).After the printing phase, swabs were washed in 99% alcohol for 10 min, washed again in new 99% isopropyl alcohol, dried for 30 minutes at 38º Celsius, and then cured with an UV curing unit at 100% for 10 minutes.All swabs were individually inspected, prepared for sterilization, and subjected to internal testing for collection ability and physical properties.UV-Vis spectroscopy was run to ensure that the swab material did not affect the transport media.The commercially available FLOQSwab (COPAN Diagnostics, Brescia, Italy) was used as a reference swab for comparison to the 3D printed swabs.

Definitions
The CDC has defined that a positive result means that two viral RNA targets (N1 and N2) were detected in the sample.[6] The PCR cycle threshold (Ct) values were set at 38; meaning below 38 was detected and above 38 was not detected.If neither target was detected, then the result interpretation was negative.If only one target was detected, then the result was 'inconclusive' and the test was run again from the same sample.An internal control to detect human DNA (RNaseP) was included to ensure cells were obtained from the swabbing procedure and to detect any sample inhibition.If no human or viral fragments were identified, then the result was deemed 'invalid' and the CDC recommends recollecting another specimen.A recollection was not performed for the present study and so if a sample was invalid, the matched pair were excluded.

Statistics
Comparative proportional analyses were performed to assess sensitivity, specificity, positive predictive value, and negative predictive value.Because only two values may be compared with these tests-positive or negative-and because indeterminate values were not positive, patients with two indeterminate tests were considered negative.Additionally, Cohen's Kappa Statistic was used to measure agreement of diagnostic results between the 3D printed and commercial swab groups.[7] Ct values for N1 and N2 were plotted against each to produce paired plots.

Results
There were 32 total enrolled subjects in the study.Two subjects had an invalid swab, so the matched pairs were excluded leaving 30 patients.Demographic and clinical information is included in Table 1.All subjects had already been diagnosed with COVID-19 except one who had tested negative despite bilateral infiltrates with a right pleural effusion, and who was ultimately diagnosed with metastatic cancer.The median time from the admission surveillance swab to enrollment for all subjects was 2 days.
Results matched in 27 of 30 patients (90%).Among the patients included, 15 matched as positive and 12 matched as negative.Two were discordant because the commercial swabs were positive (considered true positives) and the 3D swabs were negative (considered false negatives).Another one was discordant because the commercial swab was negative (considered true negative) and the 3D swab was positive (considered false positive).The sensitivity was 89%, specificity was 92%, positive predictive value was 94% and the negative predictive value was 86%.Cohen's kappa coefficient was 0.80.
For subjects whose swabs were either positive or who had one swab test positive, the Ct values were plotted against each other (Figure 3).For indeterminate pairs with only one Ct value, the complement sample was attributed the least positive value of 39.Values closer to the identity line (45º one-to-one line) correspond to more similarity between the swabs.

Discussion
The primary finding of this study was that the sensitivity and specificity were acceptable for the 3D printed swab with respect to a commercial swab.Swabbing patients and identifying virus requires a series of steps performed in the present study including capturing the virus from someone's nasopharynx with a swab, transferring the swab to transport media, and identifying viral RNA targets using PCR analysis.All results are important, positive or negative, thus it is important to be as accurate as possible.A false negative may result in a patient being taken out of isolation and exposing others, while a false positive may, and likely would, result in someone staying in quarantine or isolation inappropriately, and even going on to die alone.
When calculating sensitivity, an experiment (e.g., 3D printed swab) is measured against a reference standard (e.g., commercial swab).A commercial swab is certainly not 100% accurate as false negatives do occur.
In the present study, one discordant sample was positive with the 3D printed swab, but negative with the commercial swab.This was labeled as a false positive 3D printed swab result because the commercial swab was the reference, but in actuality it was a true positive result while the commercial swab was falsely negative because it occurred in a patient who had tested positive prior to being enrolled and was being considered positive clinically.Globally, false negative tests during the COVID-19 pandemic were ignored in patients with consistent signs, symptoms and radiographical evidence of COVID-19.False negative results may be related to swab technique, transport or laboratory error.The implication of the sensitivity and specificity generated with the present data are either to proceed with mass production to fill the void created by the COVID-19 pandemic or redesign a better 3D printed swab.At this time, several thousand 3D printed swabs have been created using our design for locally conducted COVID-19 evaluation.
Nasopharyngeal swabs are considered a class I device, which is not regulated by the FDA, but using a swab that is not specified by a manufacturer to be used with their kit is not recommended if it has not been shown to be accurate.Our data were similar to another comparison study between a commercial nasopharyngeal swab, a repurposed urogenital cleaning swab and four printed 3D prototypes with Cohen's kappa coefficients ranging from 0.85 to 0.90.[8] There are other prototypes of printed 3D swabs being tested.Manufacturing companies (Carbon, Redwood City, CA; Formlabs, Somerville, MA; Markforged, Watertown, MA;) currently manufacture and distribute 3D printed nasopharyngeal swabs for the purpose of detecting COVID-19, but efficiency data were not provided.
Limitations of the study include the commercial swab serving as a gold standard reference, which it is certainly not in actual practice.Potential reasons for discordant results may have been the variability of col- lection between samplings.Although swabbing once should not preclude virus from being collected by a subsequent swab, the depth that a patient allows a swab inserted does vary.Sometimes, swabs have blood on them and in differing amounts, which may interfere with PCR analysis.The study was strengthened by consistency in the swab printing process and consistency in swab inspection criteria after printing and before steam autoclave sterilization.It was also strengthened by matching swab technique, using similar viral transport media and running pairs together in the same batch for RT-PCR analysis.
Future studies may address testing a different swab de-sign, a different body site (e.g., nasal versus nasopharyngeal) or a different body fluid (e.g., nasopharyngeal versus saliva).Advantages of each would be higher viral capture, more comfort and more convenience, respectively.

Conclusion
A 3D manufactured swab with locally designed specifications was found to have a sensitivity and specificity of 89% and 92%, respectively.The 3D printed swab tested may be considered appropriate for use independently now, but future optimization studies would be beneficial.

Figure 1 .
Figure 1.The comparison of design of a commercially available, flocked, nasopharyngeal swab (above) to a 3D-printed one (below).

Figure 2 .
Figure 2. The tip of a) a nasopharyngeal commercial swab and b) a 3D-printed swab.

Figure 3 .
Figure 3. Scatterplot of cycle threshold values for a) N1 and b) N2 for all samples with at least one swab in a pair that was positive.For discordant pairs, the swab without a Ct value was attributed a value of 39-the closest value to negative.

Table 1 .
Demographic and clinical information for patients diagnosed with or suspected for COVID-19 who were swabbed with a commercial nasopharyngeal swab and a 3D printed swab.