key: cord-0743871-f5no10eq authors: Nagura-Ikeda, Mayu; Imai, Kazuo; Tabata, Sakiko; Miyoshi, Kazuyasu; Murahara, Nami; Mizuno, Tsukasa; Horiuchi, Midori; Kato, Kento; Imoto, Yoshitaka; Iwata, Maki; Mimura, Satoshi; Ito, Toshimitsu; Tamura, Kaku; Kato, Yasuyuki title: Clinical Evaluation of Self-Collected Saliva by Quantitative Reverse Transcription-PCR (RT-qPCR), Direct RT-qPCR, Reverse Transcription–Loop-Mediated Isothermal Amplification, and a Rapid Antigen Test To Diagnose COVID-19 date: 2020-08-24 journal: J Clin Microbiol DOI: 10.1128/jcm.01438-20 sha: 1c7bfd0e91bb22a581cc369e4843d8e1b64c0019 doc_id: 743871 cord_uid: f5no10eq The clinical performances of six molecular diagnostic tests and a rapid antigen test for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) were clinically evaluated for the diagnosis of coronavirus disease 2019 (COVID-19) in self-collected saliva. Saliva samples from 103 patients with laboratory-confirmed COVID-19 (15 asymptomatic and 88 symptomatic) were collected on the day of hospital admission. SARS-CoV-2 RNA in saliva was detected using a quantitative reverse transcription-PCR (RT-qPCR) laboratory-developed test (LDT), a cobas SARS-CoV-2 high-throughput system, three direct RT-qPCR kits, and reverse transcription–loop-mediated isothermal amplification (RT-LAMP). The viral antigen was detected by a rapid antigen immunochromatographic assay. Of the 103 samples, viral RNA was detected in 50.5 to 81.6% of the specimens by molecular diagnostic tests, and an antigen was detected in 11.7% of the specimens by the rapid antigen test. Viral RNA was detected at significantly higher percentages (65.6 to 93.4%) in specimens collected within 9 days of symptom onset than in specimens collected after at least 10 days of symptoms (22.2 to 66.7%) and in specimens collected from asymptomatic patients (40.0 to 66.7%). Self-collected saliva is an alternative specimen option for diagnosing COVID-19. The RT-qPCR LDT, a cobas SARS-CoV-2 high-throughput system, direct RT-qPCR kits (except for one commercial kit), and RT-LAMP showed sufficient sensitivities in clinical use to be selectively used in clinical settings and facilities. The rapid antigen test alone is not recommended for an initial COVID-19 diagnosis because of its low sensitivity. (LDT), including RT-qPCR, a high-throughput RT-qPCR system (fully automated from RNA extraction to reporting of results without the need for highly skilled laboratory technicians), and direct rapid RNA extraction-free RT-qPCR kits (using a modified RT-qPCR master mix), have been widely used worldwide (3) . Other molecular diagnostic methods, such as reverse transcription-loop-mediated isothermal amplification (RT-LAMP), have also been reported as useful for diagnosing COVID-19 in settings of point-of care testing (4, 5) . Recently, a SARS-CoV-2 rapid antigen test (RAT) (Espline SARS-CoV-2; Fuji Rebio Inc., Tokyo, Japan), which combines immunochromatography with an enzyme immunoassay to detect the viral nucleocapsid (N) protein, has been approved by the Japanese government (6) . The RAT is beginning to be used for diagnosing COVID-19 in clinical settings because it does not require special equipment, it does not have a time-consuming protocol, and highly skilled laboratory technicians are not essential. Although these diagnostic tests are useful in the identification of patients with COVID-19, the process of collecting upper and lower respiratory tract specimens increases the risk of exposure to viral droplets, and there is a patient burden (7) . Therefore, an alternative specimen, which can be self-collected, to diagnose COVID-19 is desirable for the clinical management of COVID-19 during this pandemic era (7) . Previously reported sensitivities of RT-qPCR assays were lower for upper respiratory specimens (32 to 61% for pharyngeal swabs and 63 to 73% for nasopharyngeal swabs) than they were for lower respiratory specimens (72 to 93% for sputum and 93 to 100% for bronchoalveolar lavage fluid) (8) (9) (10) . Recently, several reports highlighted the clinical usefulness of RT-qPCR analysis of saliva specimens (11) (12) (13) (14) (15) (16) (17) . Saliva specimens can be easily collected by the patients themselves by spitting into a collection tube; thus, using saliva specimens can reduce the burden on a patient, reduce the risk of exposure to viral droplets for medical workers, and reduce the time (1.38-fold shorter) and cost (2.09-fold lower) of the testing procedure compared to those for nasopharyngeal swabs (18) . However, the clinical usefulness of saliva specimens for diagnosing COVID-19 remains controversial because the reported diagnostic sensitivity varies widely between 69.2 and 100%, and it has yet to be thoroughly evaluated due to small sample sizes and a lack of detailed clinical information (11) (12) (13) (14) (15) (16) (17) 19) . Here, we describe the clinical performance of various molecular diagnostic methods, including the RT-qPCR LDT, the cobas SARS-CoV-2 high-throughput system, 3 direct RT-qPCR kits, and RT-LAMP, and a commercial SARS-CoV-2 RAT used on self-collected saliva specimens in diagnosing COVID-19. Patients and sample collection. Patients with COVID-19 were enrolled in this study after being referred to the Self-Defense Forces Central Hospital in Japan for isolation and treatment under the Infectious Disease Control Law in effect from 11 February to 13 May 2020. All patients were examined for SARS-CoV-2 by RT-qPCR using pharyngeal and nasopharyngeal swabs collected at public health institutes or hospitals in accordance with the nationally recommended method in Japan (20) . Asymptomatic patients were tested by RT-qPCR because of mass screening due to an outbreak or family cluster. Patient information was retrospectively collected from the hospital electronic medical records. On the day of admission, a sterile tube was provided for the patients, and they were requested to self-collect saliva specimens (ϳ500 l) by spitting into the tube. The saliva specimens were collected without restriction on timing or food intake. All samples were stored at Ϫ80°C until sample preparation. All sample preparation and sample analysis were conducted by SRL, Inc. (Tokyo, Japan). Sample preparation. Saliva specimens were diluted with 1-to 5-fold (average, 4-fold) phosphatebuffered saline for an endpoint volume of 2,000 l. This reduced the viscosity of the sample, allowing for easier pipetting. The suspension was centrifuged at 2,000 ϫ g for 5 min at 4°C, and the supernatant was used in the RT-qPCR LDT, cobas SARS-CoV-2 test, RT-LAMP, and RAT on the same day. Residual supernatant was frozen at -80°C for 4 days until used in the three direct RT-qPCR kit assays. Detection of viral RNA by the RT-qPCR LDT using the standard protocol. The RT-qPCR LDT was performed according to the National Institute of Infectious Diseases (NIID) protocol, which is nationally recommended for SARS-CoV-2 detection in Japan (20) . Viral RNA was extracted from 140-l saliva specimens using a QIAsymphony RNA kit (Qiagen, Hilden, Germany) according to the manufacturer's instructions. RT-qPCR amplification of the SARS-CoV-2 N protein gene was performed using the Quan-tiTect probe RT-PCR kit (Qiagen) with the following sets of primers and probes: the N-1 set, the forward primer 5=-CAC ATT GGC ACC CGC AAT C-3=, the reverse primer 5=-GAG GAA CGA GAA GAG GCT TG-3=, and the probe 5=-FAM-ACT TCC TCA AGG AAC AAC ATT GCC A-TAMRA-3= (where FAM is 6-carboxyfluorescein and TAMRA is 6-carboxytetramethylrhodamine), and the N-2 set, the forward primer 5=-AAA TTT TGG GGA CCA GGA AC-3=, the reverse primer 5=-TGG CAG CTG TGT AGG TCA AC-3=, and the probe 5=-FAM-ATG TCG CGC ATT GGC ATG GA-TAMRA-3= (20) . A positive result with either or both of the primer and probe sets indicated the presence of viral RNA. Detection of viral RNA by direct RT-qPCR methods without RNA extraction. Direct RT-qPCR methods without RNA extraction were performed using three commercial kits. Method A used a SARS-CoV-2 direct detection RT-qPCR kit (TaKaRa Bio Inc., Kusatsu, Japan), method B used an Ampdirect 2019 novel coronavirus detection kit (Shimadzu Corporation, Kyoto, Japan), and method C used a SARS-CoV-2 detection kit (Toyobo, Osaka, Japan) according to the manufacturers' instructions. Kits for methods A and B were used with the primer sets recommended by the Centers for Disease Control and Prevention (CDC) (21) . The kit for method C was used with the same primer sets as those used in the RT-qPCR LDT method. These methods are semiquantitative. A positive result, indicating the presence of SARS-CoV-2 RNA, was determined according to the cycle threshold (C T ) value. C T values of Ͻ40 and 45 obtained using methods A and B, respectively, with the primer or probe set for SARS-CoV-2 indicated virus presence. For method C, C T values of Ͻ45 obtained with both the primer and the probe sets for SARS-CoV-2 alongside the internal control indicated a positive result. The experimental details of the three direct RT-qPCR methods are presented below. For method A, each sample (8 l) was mixed with sample preparation buffer (2 l) and incubated for 5 min at room temperature and then at 95°C for 5 min. The reaction mix contained 35 l of reaction buffer, including the primers and probe, and was added to 10 l of a prepared sample. Thermal cycling included reverse transcription at 52°C for 5 min and 95°C for 10 s and then 40 cycles of denaturation at 95°C for 5 s and annealing/extension at 60°C for 30 s. For method B, each sample (5 l) was mixed with sample preparation buffer (5 l) and incubated at 90°C for 5 min. The reaction mix contained 15 l of reaction buffer, including the primers and probe, and was added to 10 l of a prepared sample. Thermal cycling included reverse transcription at 42°C for 10 min and 95°C for 1 min, 45 cycles of denaturation at 95°C for 5 s, and annealing/extension at 60°C for 30 s. For method C, each sample (3 l) was mixed with sample preparation reagent (3 l) and incubated at 95°C for 5 min. The reaction mix contained 40 l reaction buffer, including the primers and probe, and was added to 6 l of a prepared sample. Thermal cycling included reverse transcription at 42°C for 5 min, initial denaturation at 95°C for 10 s, 45 cycles of denaturation at 95°C for 5 s, and annealing/extension at 55°C for 10 s. Detection of viral RNA by an automated RT-qPCR device. The cobas SARS-CoV-2 test (Roche, Basel, Switzerland) (22) was performed on the automated cobas 8800 RT-qPCR system (Roche) (3). Specimens (600 l) were loaded onto the cobas 8800 with cobas SARS-CoV-2 master mix containing an internal RNA control, primers, and probes targeting the specific SARS-CoV-2 open reading frame 1 (ORF1) gene (target 1) and the envelope (E) gene (target 2). A cobas 8800 result positive for the presence of SARS-CoV-2 RNA was defined as "detected" if targets 1 and 2 were detected or "presumptively positive" if target 1 was not detected but target 2 was detected. Detection of viral RNA by RT-LAMP. RT-LAMP detection of SARS-CoV-2 was performed using a Loopamp 2019-SARS-CoV-2 detection reagent kit (Eiken Chemical, Tokyo, Japan) according to the manufacturer's instructions. The reaction was conducted at 62.5°C for 35 min with the turbiditymeasuring real-time device LoopampEXIA (Eiken Chemical). A positive result for the LAMP reaction was determined automatically by LoopampEXIA based on turbidity. Detection of SARS-CoV-2 viral antigen by a rapid antigen test. An RAT was performed using Espline SARS-CoV-2 (Fuji Rebio Inc.) according to the manufacturer's instructions. In brief, the sample for analysis was obtained by dipping a swab, which was provided with the RAT kit, into the saliva specimen and then into the sample preparation mixture provided by the kit. The mixture (200 l) was added to the sample port of the antigen assay. Subsequently, 2 drops of buffer were added, and the results were interpreted after a 30-min incubation. Definitions. The saliva sample collection day was defined as day 1. Symptomatic cases were subdivided into two groups (23) . Severe symptomatic cases were defined as patients showing clinical symptoms of pneumonia (dyspnea, tachypnea, a saturation of percutaneous oxygen [SpO 2 ] level of Ͻ93%, and the need for oxygen therapy). Other symptomatic cases were classified as mild cases. The clinical signs and symptoms included all symptoms which occurred and were observed before saliva collection. A singleton test was conducted for each method. Ethical statement. Written informed consent was obtained from each enrolled patient at the Self-Defense Forces Central Hospital. This study was reviewed and approved by the Self-Defense Forces Central Hospital (approval number 02-024) and the International University of Health and Welfare (20-Im-002-2). Statistical analysis. Continuous variables with a normal distribution are expressed as means (Ϯ standard deviations [SD] ) and, with a nonnormal distribution, as medians (interquartile ranges [IQR]) and were compared using Student's t test and the Wilcoxon rank sum test for parametric and nonparametric data, respectively. Categorical variables were expressed as numbers (percentages) and compared by the 2 test or Fisher's exact test. The Kruskal-Wallis test was used for nonparametric analysis with over three independent samples. Linear regression analysis was used to assess the relationship between each molecular diagnostic method. A two-sided P value of Ͻ0.05 was considered statistically significant. All statistical analyses were calculated using R (v3. 4 In this study, seven diagnostic tests for COVID-19 were compared across 103 saliva specimens, self-collected by 103 patients (Table 1 , and see Table S1 in the supplemental material). Among the molecular diagnostic tests, the RT-qPCR LDT showed the highest sensitivity in analyzing the 103 saliva samples (81.6%), followed in order by the cobas SARS-CoV-2 test (80.6%), direct RT-qPCR method B (78.6%), method A (76.7%), RT-LAMP (70.9%), and method C (50.5%). Only 12 patients (11.7%) tested positive using the RAT. Correlation of molecular diagnostic tests and the antigen test. The mean C T values for the target 1 primer set (28.3 Ϯ 2.3) for the cobas SARS-CoV-2 test were lower than those for N-1 (32.8 Ϯ 4.1) and N-2 (30.1 Ϯ 4.4) for the RT-qPCR LDT and target 2 (30.0 Ϯ 3.2) for cobas SARS-COV-2 (Fig. 1) . A significant correlation was observed between RT-LAMP detection time and the C T value of target 2 in the cobas SARS-CoV-2 test (P Ͻ 0.001) (Fig. 2A) . The C T value of target 2 in the cobas SARS-CoV-2 test was significantly lower in saliva samples that tested positive by the RAT than in samples that tested negative (25.4 Ϯ 1.8 versus 30.8 Ϯ 2.7, respectively; P Ͻ 0.001) (Fig. 2B) . Effect of collection time on test sensitivity. On the day of admission, 15 patients (14.6%) who did not display any symptoms were classified as asymptomatic, whereas 88 patients (85.4%) were classified as COVID-19 symptomatic. Of the 88 symptomatic patients, saliva specimens were collected from 61 patients (69.3%) during the early phase of onset, defined as within 9 days of symptom onset (Table 1) . Among these 61 early-phase patients, saliva samples were taken from 37 of them within 6 days of symptom onset (Table S1 ). Saliva samples were taken from the other 27 symptomatic patients (30.7%) after 10 days of symptoms, defined as the late phase of onset (Table 1) . Samples from early-phase, late-phase, and asymptomatic patients tested positive by molecular diagnostic tests at percentages of 65.6 to 93.4%, 22.3 to 66.7%, and 40.0 to 66.7%, respectively. The detection of viral RNA in saliva was significantly higher in symptomatic patients who collected their saliva within 9 days of symptom onset than in saliva collected after 10 days of symptoms and in saliva from asymptomatic patients Early phase (<9 days) (n ‫؍‬ 61) (P Ͻ 0.01). There were no significant differences in prevalences of positive results by the RAT among the three groups. Among the original asymptomatic patients, four patients (26.7%) developed symptoms during the hospitalization period, and all four patients tested positive using each of the methods except for direct RT-qPCR method C and the RAT. The period from the day of saliva collection to symptom onset was 2 to 4 days. Effect of clinical background on the prevalence of viral RNA in saliva. The baseline clinical characteristics of the 103 patients enrolled in this study are presented in Table 2 . Briefly, patient age ranged from 18 to 87 years (median, 46 years; IQR, 38 to 63 years), and 66 (64.1%) patients were male. The time from symptom onset to sample collection was 1 to 14 days (median, 7 days; IQR, 6 to 10 days). The time from the initial RT-qPCR-positive test to sample collection was 1 to 8 days (median, 4 days; IQR, 3 to 5 days). Of the 88 symptomatic patients, 72 (81.8%) and 16 (18.2%) were classified as having mild and severe COVID-19, respectively. Table 2 ). There were no significant differences in distribution by age or disease activity between patients in whose samples viral RNA was detected or undetected (P Ͼ 0.05). A summary of clinical symptoms and disease severity is shown for 88 symptomatic patients in Table 3 . The disease symptom coughing was observed in 45 patients, and 41 of the 45 (91.1%) tested positive by the RT-qPCR LDT, while 4 of the 45 patients Here, we present evidence for the clinical usefulness of saliva specimens in diagnosing COVID-19. Previous studies reported that the sensitivities of RT-qPCR in analyzing saliva specimens for COVID-19 were 69.2 to 100%, compared with the initial diagnosis from throat and nasopharyngeal swabs initially collected from hospitalized patients (11) (12) (13) (14) (15) (16) (17) 19) . The difference in sensitivity probably reflects differences in the clinical backgrounds of subjects and in timing of sampling in each study. In a comparison using RT-qPCR of nasopharyngeal swabs, Becker et al. reported the lowest sensitivity (69.2%) when clinical saliva samples that were collected in the late phase of onset were used (19) . On the other hand, Azzi et al. reported the highest sensitivity with saliva samples (100%) among hospitalized patients with severe and very severe disease (12) . In our study, the detection of viral RNA in saliva was significantly higher in samples collected in the early phase of symptom onset (within 9 days) than in samples collected in the late phase of symptom onset (over 10 days). Since the viral load of SARS-CoV-2 in saliva has been shown to decline from symptom onset (13) , saliva specimens should be collected during the early phase of symptom onset to increase sensitivity. The difference in saliva flow rate may affect the viral load in saliva and be associated with the difference in diagnostic sensitivity between males and females. We did not observe significant differences in disease severity or clinical symptoms between patients in whose saliva viral RNA was and was not detected; however, the prevalence of severe disease and the symptom of coughing were frequently observed in patients in whose saliva viral RNA was detected. Regarding disease activity, the presence of viral RNA was detected in more than 50% of the asymptomatic patients and the patients before the onset of symptoms. These findings support previous studies reporting the presence of viral RNA in the saliva of both symptomatic and asymptomatic patients (16) . Therefore, our findings revealed that saliva, collected in the early phase of symptom onset, is a reliable and practical source for the screening and diagnosing of COVID-19. The clinical performances of direct RT-qPCR kits and RT-LAMP and any correlation with RT-qPCR results were not well evaluated because of the small number of clinical specimens collected from patients in previous studies (4, 5, 24) . The sensitivity of RT-LAMP for SARS-CoV-2 using upper and lower respiratory tract specimens has been reported as equivalent to that of RT-qPCR (4, 5, 24) . However, our results indicate that the sensitivity of RT-LAMP is inferior to that of the RT-qPCR LDT and the cobas SARS-CoV-2 test for COVID-19 in saliva specimens. Direct RT-qPCR kits without an RNA extraction process can reduce the time, cost, and human resources needed to conduct the assay. However, we showed that there is a large difference in sensitivity among the direct RT-qPCR kits. It is necessary to pay attention to the false-negative results of RT-LAMP and direct RT-qPCR kits, especially when testing saliva samples. In clinical settings with limited medical and human resources, using RT-LAMP and direct RT-qPCR kits are options for screening and diagnosing COVID-19 because of their simplicity. In comparison with molecular diagnostic tests, the SARS-CoV-2 RAT of saliva specimens showed low sensitivity. The sensitivity of the RAT is still unclear, not only when saliva samples but also when nasopharyngeal swab specimens are used (6) . The experiment to compare the sensitivities of RT-qPCR and the RAT, prior to approval as an in vitro diagnostic test by the Japanese government, showed that the sensitivity of the RAT was 66.7% (16/24 patients) for nasopharyngeal swabs. Furthermore, lowsensitivity specimens contained a low viral copy number (50% sensitivity [6/12 patients] for specimens containing Ͻ100 copies/test) (6) . Our findings also suggest that the RAT requires a high viral copy number to achieve positive results. The RAT kit was not originally compatible with saliva specimens and their viscosity; the freeze-thaw and centrifugation processes may have affected sensitivity. Improvements in sample preparation may increase its sensitivity. Our study had several limitations. First, the saliva specimens were collected from patients 3 days (median) after receiving their first positive RT-qPCR result from an analysis of upper respiratory specimens. Directly comparing the sensitivities of tests using saliva and other upper or lower respiratory specimens is difficult in our study design because the viral loads in the clinical specimens vary with time (14) . Second, although the high specificity of RT-qPCR and RT-LAMP for SARS-CoV-2 has been confirmed (5, 20, (24) (25) (26) (27) , the specificities should be analyzed by also using saliva from non-COVID-19 patients. Further studies are warranted to determine the usefulness of saliva specimens for screening and diagnosing COVID-19. Conclusions. Self-collected saliva in the early phase of symptom onset is an alternative specimen option for diagnosing COVID-19. The RT-qPCR LDT, the cobas SARS-CoV-2 high-throughput system, direct RT-qPCR kits (except for one commercial kit), and RT-LAMP showed different sensitivities for detecting viral RNA in saliva specimens, but each can be selectively used according to the clinical setting and facilities if close attention is paid to any false-negative results. The rapid SARS-CoV-2 antigen test alone is not recommended for use at this time due to its low sensitivity. Supplemental material is available online only. SUPPLEMENTAL FILE 1, XLSX file, 0.02 MB. Clinical features of patients infected with 2019 novel coronavirus in Wuhan Interpreting diagnostic tests for SARS-CoV-2 The cobas 6800/8800 system: a new era of automation in molecular diagnostics Rapid detection of COVID-19 coronavirus using a reverse transcriptional loop-mediated isothermal amplification (RT-LAMP) diagnostic platform Evaluation of rapid diagnosis of novel coronavirus disease (COVID-19) using loop-mediated isothermal amplification Approval of in vitro diagnostics for the novel coronavirus infection Self-collection: an appropriate alternative during the SARS-CoV-2 pandemic Detection of SARS-CoV-2 in different types of clinical specimens Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections Improved molecular diagnosis of COVID-19 by the novel, highly sensitive and specific COVID-19-RdRp/Hel real-time reverse transcription-PCR assay validated in vitro and with clinical specimens Saliva sample as a non-invasive specimen for the diagnosis of coronavirus disease-2019 (COVID-19): a cross-sectional study Saliva is a reliable tool to detect SARS-CoV-2 Consistent detection of 2019 novel coronavirus in saliva Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study Saliva as an alternate specimen source for detection of SARS-CoV-2 in symptomatic patients using Cepheid Xpert Xpress SARS-CoV-2 Self-collected oral fluid and nasal swabs demonstrate comparable sensitivity to clinician collected nasopharyngeal swabs for Covid-19 detection Saliva is more sensitive for SARS-CoV-2 detection in COVID-19 patients than nasopharyngeal swabs Saliva as a diagnostic specimen for testing respiratory virus by a point-of-care molecular assay: a diagnostic validity study Saliva is less sensitive than nasopharyngeal swabs for COVID-19 detection in the community setting Development of genetic diagnostic methods for novel coronavirus 2019 (nCoV-2019) in Japan Research use only 2019-novel coronavirus (2019-nCoV) real-time RT-PCR primers and probes US Food and Drug Administration. 2020. cobas® SARS-CoV-2. US Food and Drug Administration Therapeutic and triage strategies for 2019 novel coronavirus disease in fever clinics Rapid and visual detection of 2019 novel coronavirus (SARS-CoV-2) by a reverse transcription loop-mediated isothermal amplification assay Comparison of commercially available and laboratory developed assays for in vitro detection of SARS-CoV-2 in clinical laboratories Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system An assessment of real-time RT-PCR kits for SARS-CoV-2 detection We thank the clinical laboratory technicians at the Self-Defense Forces Central Hospital for sample collecting, everyone involved in the COVID-19 Task Force at the Self-Defense Forces Central Hospital, and members who were assembled from other institutes of the Japan Self-Defense Forces. We declare that we have no conflicts of interests. This work was supported by MHLW Research on Emerging and Re-emerging Infectious Diseases and Immunization Program grant number JPMH20HA2002.