key: cord-1038794-g2oc32dm
authors: Klink, Thomas; Rankin, Danielle A.; Piya, Bhinnata; Spieker, Andrew J.; Faouri, Samir; Shehabi, Asem; Williams, John V.; Khuri-Bulos, Najwa; Halasa, Natasha B.
title: Evaluating the diagnostic accuracy of the WHO Severe Acute Respiratory Infection (SARI) criteria in Middle Eastern children under two years over three respiratory seasons
date: 2020-04-30
journal: PLoS One
DOI: 10.1371/journal.pone.0232188
sha: 7b2e58c9fb494bf80226905af4db48accf619bcc
doc_id: 1038794
cord_uid: g2oc32dm

OBJECTIVE: The World Health Organization created the Severe Acute Respiratory Infection (SARI) criteria in 2011 to monitor influenza (flu)-related hospitalization. Many studies have since used the SARI case definition as inclusion criteria for surveillance studies. We sought to determine the sensitivity, specificity, positive predictive value, and negative predictive value of the SARI criteria for detecting ten different respiratory viruses in a Middle Eastern pediatric cohort. MATERIALS AND METHODS: The data for this study comes from a prospective acute respiratory surveillance study of hospitalized children <2 years in Amman, Jordan from March 16, 2010 to March 31, 2013. Participants were recruited if they had a fever and/or respiratory symptoms. Nasal and throat swabs were obtained and tested by real-time RT-PCR for eleven viruses. Subjects meeting SARI criteria were determined post-hoc. Sensitivity, specificity, positive predictive value, and negative predictive value of the SARI case definition for detecting ten different viruses were calculated and results were stratified by age. RESULTS: Of the 3,175 patients enrolled, 3,164 were eligible for this study, with a median age of 3.5 months, 60.4% male, and 82% virus-positive (44% RSV and 3.8% flu). The sensitivity and specificity of the SARI criteria for detecting virus-positive patients were 44% and 77.9%, respectively. Sensitivity of SARI criteria for any virus was lowest in children <3 months at 22.4%. Removing fever as a criterion improved the sensitivity by 65.3% for detecting RSV in children <3 months; whereas when cough was removed, the sensitivity improved by 45.5% for detecting flu in same age group. CONCLUSIONS: The SARI criteria have poor sensitivity for detecting RSV, flu, and other respiratory viruses—particularly in children <3 months. Researchers and policy makers should use caution if using the criteria to estimate burden of disease in children.

a1111111111 a1111111111 a1111111111 a1111111111 a1111111111

Respiratory infections are the second leading cause of global years of life lost in all ages, and the leading cause of mortality in children under five years [1] . In 2011, the World Health Organization (WHO) created a case definition for Severe Acute Respiratory Infection (SARI) in an attempt to standardize global surveillance of hospitalization related to influenza (flu)-allowing national health authorities to interpret their data in an international context [2] . Flu is particularly difficult to surveil because its clinical presentation is often indistinguishable from other respiratory viruses [3] . With that in mind, the WHO designed the SARI criteria to strike a balance between sensitivity and specificity, while also noting that the case definition is not necessarily intended to capture all cases but to describe trends over time [4] .

Several studies, including at least nine in the Eastern Mediterranean region, have been published since 2011 using the SARI case definition as inclusion criteria to report a combination of clinical characteristics, risk factors, viral burden, or outcomes in adult and pediatric populations for flu and other respiratory viruses [5] [6] [7] [8] [9] [10] [11] [12] [13] . Only a handful of studies have evaluated the effectiveness of the criteria by including both SARI-positive and SARI-negative patients, allowing them to calculate the diagnostic accuracy of the criteria for detecting flu and respiratory syncytial virus (RSV) [14] [15] [16] [17] [18] [19] . The diagnostic accuracy of the criteria for other individual respiratory viruses remains unknown. Additionally, only one of these studies stratified age to include a group of children less than three months old, and studies of this type from the Eastern Mediterranean region are lacking [17] .

Moreover, those studies that have included both SARI-positive and SARI-negative pediatric patients have indicated that the SARI criteria are less sensitive in younger pediatric patients, but only one such study stratified by the youngest children [14, [16] [17] [18] . This means that studies using the SARI case definition as inclusion criteria may underestimate the burden of disease in this age group [5] . Therefore, we sought to evaluate the diagnostic accuracy of the SARI criteria for eleven respiratory viruses in a large hospitalized pediatric cohort in Amman, Jordan in children less than two years old who presented with fever and/or respiratory symptoms.

We conducted a prospective active surveillance study of acute respiratory infections in hospitalized children <2 years in Amman, Jordan. Participants were recruited over a three-year period (March 16, 2010-March 31, 2013) within 48 hours of hospital admission for fever and/ or respiratory symptoms with one of the following admission diagnoses: ARI, apnea, asthma  exacerbation, bronchiolitis, bronchopneumonia, croup, cystic fibrosis exacerbation, febrile seizure, fever without localizing signs, respiratory distress, pneumonia, pneumonitis, pertussis,  pertussis-like cough, rule out sepsis, upper respiratory infection (URI) , or other. Children were excluded only if they had chemotherapy-associated neutropenia and/or were newborns who had never been discharged [20] . Trained local staff obtained written informed consent from parents or guardians of all participants. The study was approved by the Institutional Review Boards of the University of Jordan, the Jordanian Ministry of Health, and Vanderbilt University.

Al-Bashir Hospital is a 185 pediatric bed (120 pediatric and 65 neonatal intensive care unit) government-run hospital that serves the Jordanian capital, Amman, a city with >2 million inhabitants. The catchment area is densely populated, low-income, and includes a Palestinian refugee camp. Due to government policy, children <6 years are provided free care at Al-Bashir regardless of insurance status. During the study period, there were 11,230 hospitalizations of children <2 years.

Throat and nasal swabs from all participants were obtained by trained research staff. Demographic, social, and medical histories were obtained by standardized questionnaires, which were a component of the interview portion of the case report form. All interviews were conducted in Arabic to parents and recorded in English. Following patient discharge, clinical outcome data and antibiotic use during the hospitalization data were systematically collected from the medical record. All data were entered into a secure REDCap TM (Research Electronic Data Capture, Vanderbilt University, Nashville, TN, USA) database. Extracted information is explained in detail in a previous publication [20] .

Throat and nasal swabs were combined in transport medium (M4RT 1 , Remel, USA) aliquoted into MagMax TM Lysis/Binding Solution Concentrate (Life Technologies, USA), snap frozen, stored at -80˚C, and shipped to Nashville, TN. Original and lysis buffers were tested by real-time RT-PCR for eleven respiratory viruses: RSV, human metapneumovirus (HMPV), human rhinovirus (HRV), flu A, B, and C, parainfluenza virus (PIV) 1, 2, and 3, adenovirus, and Middle East respiratory syndrome coronavirus (MERS-CoV) [21] . Cycle threshold (Ct) values were determined, with lower Ct corresponding with higher viral load.

The WHO SARI case definition includes three elements: hospitalization with ARI with symptom onset in the past 10 days, history of fever or measured fever �38 C˚, and cough [4] . In order to capture this definition in our subjects, we used information obtained from the questionnaire and extracted from the medical chart. Days of symptoms were captured in the questionnaire. Fever was captured with at least one of the following: an admission or discharge diagnosis of fever, measured temperature �38 C˚at admission, or reported history of fever as a symptom of current illness. Cough was recorded if it was reported as a symptom or was captured as an admission or discharge diagnosis.

Descriptive statistics were reported as frequencies or mean, median and interquartile range (IQR) where appropriate. Odds ratios for covariates contributing to SARI status were calculated using a series of simple logistic regressions, followed by multiple logistic regressions, controlling for age and sex. Holm-Bonferroni adjustments were made to account for multiple analyses with the same dependent variable. Covariates included age, sex, birthweight, prematurity, medical history, smoke exposure [nargila (smoking pipe) or cigarettes], vitamin D level, breastfeeding, length of illness, admission diagnoses of pneumonia, bronchopneumonia, bronchiolitis, sepsis, or febrile seizure, viral detection, and markers of illness severity including antibiotics before or after hospitalization, length of stay, ICU admission, oxygen use, mechanical ventilation, and death. All analyses were completed using STATA version 15.1. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the SARI criteria and modifications excluding either fever or cough were calculated for each respiratory virus tested. Parainfluenza viruses 1, 2, and 3 were combined into one category, as was flu A, B, and C. This was done for simplicity and relatively low numbers of detections of each subtype. For NPV and PPV, we assumed the prevalence found in our study to be accurate for hospitalized children with fever or respiratory symptoms. For each virus, the prevalence of cough and fever was also calculated. These data were then stratified by age into three groups: less than 3 months, 3-5 months, and 6-23 months. These groups were chosen to compare results to a previous study; and were based off a visual analysis of the age of virus-positive patients meeting SARI criteria [17] .

Between March 16, 2010, and March 31, 2013, there were 3,793 eligible hospitalized infants and 3,175 (83.7%) were enrolled as previously described [20] . Seven of these were excluded: four admitted with the diagnosis of meningitis and three who were older than two years. Of the remaining 3,168 subjects, four were excluded for the purpose of this analysis because they did not have information on the SARI criterion of illness duration. Therefore, 3,164 subjects (83.4% of eligible patients) are included in our analyses.

The median age for enrolled subjects was 3.5 months (Table 1) . Most were male (60.4%), had household exposure to smoke or nargila (76.6%), and reported exclusive breastfeeding (60.6%). Viral pathogens were detected in 2,581 (81.5%) subjects, and 315 (10.1%) had at least one underlying medical condition.

Of the 3,164 subjects included, 1,261 (39.9%) met SARI criteria ( Table 1) . When compared to their SARI-negative counterparts, SARI-positive subjects tended to be older, were less likely to have a history of premature birth, and had a shorter duration of illness prior to hospitalization. No significant differences were detected in sex, underlying medical conditions, smoke exposure, vitamin D levels, birth weight, or breastfeeding history.

SARI-positive patients were more likely to be diagnosed with pneumonia (OR: 1.77; 95% CI: 1.41-2.22; p-value: <0.001) and bronchopneumonia (OR: 3.83; 95% CI: 3.21-4.55; p-value: <0.001). Alternatively, SARI-negative subjects were more likely to be diagnosed with rule out sepsis (OR: 0.37; 95% CI: 0.20-0.47; p-value: <0.001) and febrile seizure (OR: 0.07; 95% CI: 0.03-0.14; p-value: <0.001). No significance was found in the prevalence of bronchiolitis between the two groups (OR: 0.96; 95% CI: 0.79-1.17; p-value: 0.687). SARI-positive subjects were more likely to receive antibiotics both before and during hospitalization. No differences were detected between length of stay, ICU admission, oxygen use, mechanical ventilation, or death (Table 1) .

Virus detection was more common in SARI-positive patients compared to SARI-negative subjects (OR: 2.84; 95% CI: 2.27-3.56; p-value:<0.001). Including co-detections, SARI-positive 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East subjects were more likely to have RSV, and HMPV (Table 1) . Co-detections were also more prevalent in SARI-positive patients. An analysis of single virus detections only yielded one significant difference with the analysis that included co-detections: SARI-negative patients were more likely to have a single virus infection of HRV, whereas there was no significant difference between groups when co-detections were included following Holm-Bonferroni adjustment ( Table 1 ). There were no differences seen in Ct values by SARI status. MERS-CoV was not detected in any sample.

Overall, the sensitivity of the SARI criteria including co-detections for detecting virus-positive patients was 44%, with a specificity of 78%, PPV of 89.8%, and NPV of 24% (Figs 1 and 2) . The difference in sensitivity and specificity between single and co-detections for a particular virus was within 10%, but differences were larger changes for PPV and NPV depending on the prevalence of the virus in the study. SARI criteria showed the highest sensitivity for single HMPV detections (64.8%). The sensitivity of the SARI criteria did not reach above 60% for any other virus. Specificity of the criteria was greatest for RSV detections including RSV and co-detections (66.9%). Specificity was between 58.2% and 63.2% for all other viruses. PPV was greatest for RSV detections at 53.6%, including co-detections, and lowest for single-flu detections at 2.2%. Conversely, NPV was greatest for single-flu detections at 98.8% and lowest for rhinovirus detection, including codetections, at 59%.

The prevalence of fever was highest in patients with single-flu detection at 90%; whereas those with RSV, including co-detections, had a fever rate of 52.4%. The cough rate was highest in those with single-hMPV and single-RSV detections at 96.1% and 96% respectively. The cough rate was lowest in those with single-adenovirus detections at 56.2%.

Modifying the SARI criteria by removing fever or cough changed the diagnostic accuracy for each virus (Fig 2) . Sensitivity of the SARI-NoF and SARI-NoC criteria was greater for detecting each virus than the original SARI criteria. SARI-NoF had the greatest gains in sensitivity with an increase of 35�1% in detecting any of the viruses, including a 44.1% increase in RSV detection. Conversely, SARI-NoC had a 13.2% increase in sensitivity for detecting any of the viruses and a 2.7% increase for RSV detection.

Specificity for detecting all viruses was decreased for both SARI-NoF and SARI-NoC criteria compared to the original criteria. The specificity of the SARI-NoF criteria for detecting any of the viruses was decreased by 17.8%. The largest decrease was for adenovirus with a change of -32.4%. The specificity for the SARI-NoC criteria decreased by 43.3% for detecting any of the viruses.

The changes in PPV were more modest for the modified criteria. The PPV of the SARI-NoF criteria for any of the viruses decreased by only 0.1%. The PPV for the SARI-NoC criteria for any of the viruses decreased by 10.4%. The NPV of the SARI-NoF criteria increased by 15.5% for any of the viruses; whereas, the SARI-NoC criteria saw a decrease of 8.5% in the same category.

The proportion of virus-positive patients meeting SARI criteria by their age yielded a clear trend, with the youngest patients being less likely to meet criteria (Figs 1 and 3A) . Data were stratified into three age groups (<3 months, 3-5 months, 6-23 months) to compare the SARI criteria to the modified criteria in virus-positive patients (Fig 3A) , showing that the SARI-NoF criteria had the highest sensitivity in all age groups. Differences were also observed with regards to the presence of fever in virus-positive patients across the groups. Virus-positive patients had a reported or measured fever in 44.4%, 60.6%, and 75.5% in the less than 3 months, 3-5 months, 6-23 months groups, respectively (Fig 3B) . Fever was more common in the virus-negative patients for each age group ( Table 2 ).

The sensitivity, specificity, PPV, and NPV of both the original and modified criteria for each virus was also stratified by age group (Figs 4-6) . The SARI criteria had the lowest sensitivity in the youngest age group at 22.4% for detecting any of the viruses. Its specificity in this category, 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East however, was 90.6%. The SARI criteria had the highest sensitivity in the oldest age group at 64.3% for detecting any of the viruses. The specificity in this age category was 57.7%. The SARI-NoF criteria saw the greatest gains in sensitivity amongst the youngest age groups compared to the original criteria with an increase of 46.6% for detecting virus-positive patients (Figs 4-6 ).

Our study found that for ARI surveillance, the SARI criteria fails to capture more than 50% of children with respiratory virus infections (RVI), and it performs even more poorly for children <3 months. Therefore, it is important to be cautious when using the SARI criteria for surveillance studies, especially if trying to determine the burden of illness to select viruses to influence policy decisions (e.g. vaccine implementation and/or effectiveness). If understanding the true burden of disease is desired, a higher sensitivity is important. While the SARI case definition was initially intended for flu surveillance, several studies have since used it for surveillance of other respiratory viruses [16] [17] [18] [22] [23] [24] . Our study provides researchers and public health officials the data on diagnostic accuracy of the criteria for detecting ten different respiratory viruses and would caution SARI use for true RVI burden in children.

During our study period, the SARI criteria correctly identified a little over half of the subjects who were flu-positive. Comparisons to previous studies are nuanced due to differing age stratifications, but this result is similar to the 52% sensitivity reported by the Amini (2017) 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East study in Quebec for children <1 year; however, it is lower than the 79.2% found in that same study for its one to four-year-old age group [14] . Similarly, two studies in Kenya that reported 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East results for participants aged two months to four years found sensitivities of 84.3% and 89.6% [15, 24] . The specificity of the criteria for detecting flu in our study (60.6%) also differs from what was found in the aforementioned studies (13.1%-29.5%). The age distribution differences likely explain these discrepancies as our cohort consists only of children under two years, and the SARI criteria were the least sensitive (15.9%) and most specific (80.5%) for detecting flu in our patients under three months, who make up 45.4% of our study population. The sensitivities and specificities of the criteria for our patients aged 3-5 months (72.2%, 50.4%) and 6-23 months (75.4%, 39.9%) are more closely aligned with the findings of the previous studies. Therefore, if policy makers are deciding if flu vaccine should be administered to pregnant women or infants based on the SARI definition, they would be underestimating the true burden of flu illness.

Our findings for the diagnostic accuracy of the criteria for detecting RSV also have some differences from previous studies, including one study that included similar age group stratifications-allowing for a more direct comparison [17] . For children <3 months, our study showed a sensitivity of 25.2%, lower than the 55% found by Rha (2018) in South Africa [17] . However, our 74% sensitivity in the 6-23 months age group more closely matched their groups of 6-11 and 12-23 months with sensitivities of 77% and 81%, respectively. The latter result is also in line with the findings of Nyawanda (2016), which reported sensitivities for children <1 year at 79.4%, and one to four years at 86.2%. Our finding for the specificity of the criteria was greater than these two previous studies across all age groups but differed most from Rha for children less than three months (85% vs 54%). Similar to Rha, however, the specificity of the criteria in our study for our oldest age group (6-23 months, 47.7%) was nearly half that of our 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East youngest age group (85.1%). One potential explanation for differences between our studies is the use of singleplex RT-PCR in our study versus the multiplex RT-PCR that was used in Rha, with singleplex historically being more sensitive. Given that RSV vaccines are in the pipeline, it is important to also capture all cases of RSV to determine the true burden of illness to document if the vaccine is effective at impacting disease. Thus, if the SARI definition is used for ARI surveillance studies, it will once again underestimate the burden of RSV illness, and therefore altering the inclusion criteria to include either fever and/or cough may be more accurate. The differences in diagnostic accuracy of the SARI criteria by age is the most significant finding of this study. The SARI criteria performed poorly with regards to sensitivity for the detection of any respiratory virus in children under three months. Only 22.4% of children in this age group with a respiratory virus were captured by the case definition. This increased to 64.3% in children aged 6-23 months. Conversely, the criteria are very specific in the youngest age group for detecting any respiratory virus compared to the oldest age group (90.6% vs. 64.3%).

We further dissected by symptoms and found that having the combination of both fever and cough was the reason for low sensitivities and having either one or the other could capture more RVI cases. Specifically, the lower rates of fever-and to a lesser extent, cough-in patients <3 months who are respiratory virus-positive account for the poor sensitivity of the SARI criteria in this age group. Only 44.4% of the virus-positive patients <3 months had a reported or measured fever, compared to 75.5% of their 6-23 months counterparts. Consistent with what has been found in previous studies [25] , removing fever from the SARI criteria greatly increased the sensitivity of detecting any respiratory virus in children <3 months, particularly for RSV, which then increased by 65.3% [18] . Notably, both the PPV and NPV for detecting RSV also increased with this change. In general, for this youngest age group, removing fever as a criterion increases sensitivity more than it decreases specificity for each virus tested, and it results in minimal changes to PPV and NPV. This result would seem to advise caution when mandating that fever must be included as a criterion for this age group.

Our study has several strengths. It is the first to report the diagnostic accuracy of the WHO SARI criteria for detecting multiple viruses, including but not limited to flu and RSV. It is also the first study of this kind in the Middle East. Our prospective study took place over three years, included very young children, and included both fever and/or respiratory criteria, which allowed us to compare SARI-positive to SARI-negative children. Our study also has a number of limitations. It only includes hospitalized children <2 years who had ARI symptoms from one large hospital in Amman, Jordan. This limits its generalizability to other locations, or to study populations that include all hospitalized patients. Additionally, the 2011 SARI criteria were developed during our study period. While our study design allowed us to capture all the necessary elements to determine which patients met SARI criteria, it involved combining together the data as opposed to a straightforward data collection tool. Part of the data used in this study also included a parental survey of symptoms-introducing a source of reporting bias.

The SARI criteria were initially developed by the WHO in 2011 as part of its recommendations for global flu surveillance. The ultimate goal of such surveillance is "to minimize the impact of 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East the disease by providing useful information to public health authorities so they may better plan appropriate control and intervention measures, allocate health resources, and make case management recommendations" [4] . However, our study found that more than half of the RVI cases would have been missed using the SARI criteria. Therefore, it is critical to know how well these criteria perform in different patient populations and geographic settings. Our study shows that use of the criteria in children <3 months significantly underestimates the burden of disease for RSV, flu, and other viruses. Therefore, removing fever as a mandatory criterion in this age group, particularly for RSV, would greatly increase sensitivity with an acceptable decrease in specificity. Overall, more studies need to be conducted in the Middle East to gather more information on this finding; however, it is advisable that policymakers are cautious when using the SARI criteria. 

Diagnostic accuracy of the WHO severe acute respiratory infection (SARI) criteria in the Middle East

Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study

Revision of clinical case definitions: flu-like illness and severe acute respiratory infection

Clinical signs and symptoms predicting flu infection

World Health Organization. Global Epidemiological Surveillance Standards for Flu

Flu hospitalization epidemiology from a severe acute respiratory infection surveillance system in Jordan

The burden of fluassociated hospitalizations in Oman

Circulation of respiratory syncytial virus in Morocco during 2014-2016: Findings from a sentinel-based virological surveillance system for flu

Estimation of flu and severe acute respiratory illness incidence (Burden) in three provinces of the Islamic Republic of Iran

Viral etiology, seasonality and severity of hospitalized patients with severe acute respiratory infections in the Eastern Mediterranean Region

Morbidity, mortality, and seasonality of flu hospitalizations in Egypt

Incidence of flu virus-associated severe acute respiratory infection in Damanhour district

Flu-associated severe acute respiratory infections in 2 sentinel sites in Lebanon

Characteristics of severe acute respiratory infection associated hospitalization in Yemen

Evaluation of the new world health organization case definition of severe acute respiratory infection for flu surveillance during the peak weeks of two flu seasons in Quebec

Comparison of severe acute respiratory illness (sari) and clinical pneumonia case definitions for the detection of flu virus infections among hospitalized patients, western Kenya

Evaluation of case definitions to detect respiratory syncytial virus infection in hospitalized children below 5 years in Rural Western Kenya

Performance of Surveillance Case Definitions in Detecting Respiratory Syncytial Virus Infection Among Young Children Hospitalized with Severe Respiratory Illness-South Africa

Evaluation of case definitions for estimation of respiratory syncytial virus associated hospitalizations among children in a rural community of northern India

What are the most sensitive and specific sign and symptom combinations for flu in patients hospitalized with acute respiratory illness? Results from western Kenya

Natural history and epidemiology of respiratory syncytial virus infection in the Middle East: Hospital surveillance for children under age two in Jordan

Human metapneumovirus in hospitalized patients in Amman, Jordan

Human metapneumovirusassociated severe acute respiratory illness hospitalisation in HIV-infected and HIV-uninfected South African children and adults

Detection of non-flu viruses in acute respiratory infections in children under five-year-old in

Etiology and incidence of viral acute respiratory infections among refugees aged 5 years and older in Hagadera Camp

Performance of case definitions used for flu surveillance among hospitalized patients in a rural area of India