key: cord-0940731-1c0zl3hp authors: Nunez, I.; Caro-Vega, Y.; Belaunzaran-Zamudio, P. F. title: Importance of adequate COVID-19 case definitions in the SARS-CoV-2 pandemic date: 2021-06-15 journal: nan DOI: 10.1101/2021.06.13.21258845 sha: 7d42f2799b3ab018acd22228c49c07e9c6008396 doc_id: 940731 cord_uid: 1c0zl3hp Background: Epidemiologic case definitions serve a myriad of purposes during a pandemic, including contact tracing and monitoring disease trends. It is unknown how any COVID-19 case definition fares against the current gold standard of molecular or antigen tests. Methods: We calculated the diagnostic properties of five COVID-19 definitions (three of the Mexican government and two of the WHO) using open data of suspected COVID-19 cases in Mexico City from March 24th 2020 until January 31st 2021. Results: All 1,632,420 people included in the analysis met the WHO suspected case definition (sensitivity 100%, specificity 0%). The WHO probable case definition was met by 1.4%, while the first and second Mexican suspected case had sensitivities of 61 and 62% and specificities of 58 and 62%, respectively. Confirmed case by epidemiological contact had a low sensitivity (33%) but slightly higher specificity (77%). Conclusions: Case definitions should maximize sensitivity, especially in a high-transmission area such as Mexico City. The WHO suspected case definition has the potential for detecting most symptomatic cases. We underline the need for routine evaluation of case definitions as new evidence arises to maximize their usefulness. Epidemiologic case definitions for a disease are vital in surveillance during epidemics. In this context, the aim of a case definition is to be highly sensitive as to miss the fewest true cases of the disease. In low-income countries, case definitions are invaluable to make decisions on isolation, contact tracing, and monitoring disease trends, since definitive tests might be scarce, unavailable, and highly expensive [1] . During the COVID-19 pandemic the World Health Organization (WHO) released its recommended version of case definitions for suspected, probable and definitive COVID-19 cases, which have been periodically updated [2] . Countries have also released their own definitions with irregular updating [3, 4] , even though case definitions must be updated according to new scientific evidence as to increase their diagnostic value [1] [2] [3] [4] . with the aim of determining who should be tested. Only one in ten ambulatory suspected COVID-19 patients would be tested, as well as all hospitalized ones. An update in the case definition was published in august 2020, with only minor changes but the testing strategy remained the same [3] [4] . Considering this deliberate under testing, suspected case definitions become especially important to account for disease undercounting, initiate contact tracing (which has not been a feature of Mexico's pandemic response, but is elsewhere), and starting individual treatment. An important caveat is that the people who design these definitions are in many cases not the same people that have to apply them on the field. Thus, a dissociation could occur between intended and actual use. Mexico City is a large metropolitan area with 9,209,944 habitants [5] . In this analysis we calculated the diagnostic properties of the definitions of COVID-19 cases of the Mexican All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Ministry of Health (MoH) and those of the WHO, to determine their adequacy for epidemiological monitoring purposes. We used open data from the Mexico City government for reported cases of suspected COVID-19 between March 24th 2020 and January 31st 2021 [6]. We calculated the diagnostic properties (sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio), as well as post-test probabilities of five different epidemiological COVID-19 case definitions: three issued by the MoH for COVID-19 surveillance purposes (suspected case, updated suspected case definition, and suspected case "confirmed" by epidemiological linkage to a laboratory confirmed case) and two WHO recommended definitions (suspected and probable) [2] [3] [4] . A comparison between these definitions is provided in suspected case definition of the MoH was issued in March 2020 and included anybody seeking care for at least one of the following symptoms starting within the 7 previous days: cough, dyspnea, fever or headache; with at least one of the following: myalgia, arthralgia, sore throat, thoracic pain, rhinorrhea, polypnea or conjunctivitis [3] . COVID-19 suspected case definition was updated by the MoH in August 2020 (adding chills, anosmia and dysgeusia and expanding the period of symptoms onset from 7 to 10 days) [4] . The case definition for COVID-19 confirmed by epidemiological linkage to a laboratory confirmed case (anyone meeting the COVID-19 suspected case criteria that have had contact with a laboratory confirmed case within the previous 14 days) [4] . We substituted contact with "confirmed case" with contact with an "individual with respiratory symptoms", as only information on this variable was available [6] . The MoH COVID-19 case definition is the All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 15, 2021. ; https://doi.org/10.1101/2021.06.13.21258845 doi: medRxiv preprint same case definition used for surveillance activities of seasonal Influenza thus, tests are only performed to symptomatic patients but only one in ten ambulatory patients are tested, while all hospitalized are [3] . There are no pre-established criteria or even guidelines on which ambulatory patients test for SARS-CoV-2, and decisions about testing depends heavily in clinical judgement and tests availability on sentinel sites. The revised WHO COVID-19 case definitions for suspected (which includes a set of different options of clinical and epidemiological criteria, as shown in Table A .1) and probable cases (which requires the presence of the clinical criteria in suspected cases (acute onset of fever and cough OR acute onset of any three or more of the following symptoms: fever, cough, general weakness/fatigue, headache, myalgia, sore throat, coryza, dyspnea, anorexia/nausea/vomiting, diarrhea, altered mental status in combination) in combination with chest imaging showing findings suggestive of COVID-19 disease, which we replaced with the variable "clinical diagnosis of pneumonia", since no chest imaging variable was available in the database [2] [3] [4] . Since all tested patients in Mexico are symptomatic, the diagnostic gold standard for our calculation of the diagnostic properties of case definitions was having either a positive molecular test (real-time reverse-transcriptase polymerase chain reaction; RT-PCR) or a positive antigen test. Post-test probabilities of COVID-19 were calculated using the daily proportion of positive molecular or antigen tests and graphed using 7-day rolling means. Data is freely available at the official Mexico City government COVID-19 website and the R code used for the analysis will be made freely available with the final version of the article [6]. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Post-test probability varied greatly according to the pre-test probability and the definition utilized, with a mean probability of 44% (SD 10) for the first MoH case definition, 46% (SD 10) for the updated definition, 43% (SD 10) for the MoH definition of confirmed case by epidemiological contact, 35% (SD 9) for the WHO definition of suspected case, and 61% (SD 10) for the WHO definition of probable case. Post-test probability along the study period for each definition is shown in Figure 1 . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Considering that theoretically it should have a sensitivity of 100%, it is fortunate that it is not being used as planned, as almost 40% of currently observed cases would be missed. A suspected case definition that is not met by many confirmed cases is not useful, for epidemiologic purposes or otherwise. Our analysis underlines the importance of this, as Mexico is a country that tests a small percentage of symptomatic people. In our context, suspected cases based on symptoms should include all but asymptomatic individuals as the WHO suspected case definition does, and be formally counted and included in epidemiologic surveillance, as most do not have access to confirmatory tests. Thus, the WHO suspected case definition appears to be the most useful in our study population (which has heavy community transmission), and guiding testing decisions in Mexico City using it would reduce sub-estimation case count. We consider it reasonable that results would be similar if we replicated the analysis country wide, unfortunately we do not have the data to do so. As only symptomatic people are being tested, clinical judgement remains key and patients should be retested in case of a negative result if prevalence remains high [8] . Point-of-care All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Our results differ to those of a previous study with a smaller sample size where a higher sensitivity for the original MoH suspected case definition was observed (87%) [9] . Their study population was highly selected, as it included patients from a single healthcare system from a single city that had exhaustive clinical information, including a complete medical history, which could explain the higher sensitivity. Our study has several limitations. We used repurposed data that did not have information on several variables, such as anosmia, dysgeusia, and radiological imaging. The incidence of anosmia and/or dysgeusia in Mexican COVID-19 patients is unknown, but elsewhere it has been reported of 35% [10] . This could improve the sensitivity of the second MoH definition. Only one in ten ambulatory patients are tested, and these patients could differ in important ways that we are unable to account for, such as subjective disease severity. Furthermore, false negative tests are well known and limit our definition of gold standard [11] [12] [13] . This is especially important given the high post-test probability observed throughout the study period (>10%). Accounting for false negative tests would increase the post-test probability, and thus a negative test would not rule out the disease in high prevalence areas such as this. Our analysis supports the fact that case definitions should be formally evaluated as to ensure their usefulness. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. We would like to thank the government of Mexico City for their open data policy. Also, we give our gratitude to healthcare workers. Your work makes the difference. We received no funding for this study. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 15, 2021. ; https://doi.org/10.1101/2021.06.13.21258845 doi: medRxiv preprint definition for confirmed case by epidemiological contact; Mx suspected case first: Mexico first definition for suspected case; Mx suspected case first: Mexico second definition for suspected case; WHO probable case: WHO definition for probable case; WHO suspected case: WHO definition for suspected case. Gold standard was considered to be either a positive molecular or antigen SARS-CoV-2 test. As the WHO suspected case definition had a negative predictive value of 0%, it does not appear in B). All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted June 15, 2021. ; https://doi.org/10.1101/2021.06.13.21258845 doi: medRxiv preprint COVID-19 caseness: An epidemiologic perspective Maintaining surveillance of influenza and monitoring SARS-CoV-2 -adapting Global Influenza surveillance and Response System (GISRS) and sentinel systems during the COVID-19 pandemic: Interim guidance. Geneva: World Health Organization Definición de caso COVID-19. Mexico: Comité Nacional para la Vigilancia Epidemiológica (CONAVE) Mexico: Dirección General de Epidemiología, Secretaría de Salud, Comité Nacional para la Vigilancia Epidemiológica (CONAVE) Population of Mexico by state, INEGI. 2020. Available at Effect of changing case definitions for COVID-19 on the epidemic curve and transmission parameters in mainland China: a modelling study Effectiveness of COVID-19 case definition in identifying SARS-CoV-2 infection in northern Mexico Systematic Review and Meta-analysis of Smell and Taste Disorders in COVID-19 Variation in False-Negative rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure Assessing case definitions in the absence of a diagnostic gold standard Impact of RT-PCR Test False-Negative Results for SARS-CoV-2 Surveillance in Mexico No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity