key: cord-1045487-3x5y7bvz
authors: Asahi, K.; Undurraga, E. A.; Wagner, R.
title: Benchmarking the CoVID-19 pandemic across countries and states in the U.S.A. under heterogeneous testing
date: 2020-05-06
journal: nan
DOI: 10.1101/2020.05.01.20087882
sha: 9b8c9735a213cdbea77a651bac651da0a44ed345
doc_id: 1045487
cord_uid: 3x5y7bvz

Public health officials need to make urgent decisions to reduce the potential impact of the CoVID-19 pandemic. Benchmarking based on the increase in total cases or case fatality rates is one way of comparing performance across countries or territories (such as states in the USA), and could inform policy decisions about COVID-19 mitigation strategies. But comparing cases and fatality across territories is challenging due to heterogeneity in testing and health systems. We show two complementary ways of benchmarking across countries or US states. First, we used multivariate regressions to estimate the test-elasticity-of-COVID-19-case-incidence. We found a 10% increase in testing yielded ~9% (95% CI:4.2-13.4%; p<0.001) increase in reported cases across countries, and ~2% (95%CI:0.1-3.4%; p=0.03) increase across US states during the week ending April 10th, 2020. We found comparable negative elasticities for fatality rates (across countries: beta;=-0.77, 95%CI:-1.40- -0.14; p=0.02; US states: beta;=-0.15, 95%CI:-0.30-0.01; p=0.06). Our results were robust to various model specifications. Second, we decomposed the growth in cases into test growth and positive test ratio (PTR) growth to intuitively visualize the components of case growth. We hope these results can help support evidence-based decisions by public health officials as more consistent data hopefully becomes available.

As of April 18, 2020, SARS-CoV-2 has spread globally, resulting in 2.3 million reported infections and ~160,000 deaths in 185 countries and territories [1] . Substantial evidence shows the disease burden of CoVID-19 (illness caused by SARS-CoV-2) is higher than ascertained [2] . Public health officials need to make urgent decisions about interventions to reduce the potential impact of CoVID-19 with limited available evidence [3] [4] [5] . They have to continuously assess and adapt their decisions based on available resources, disease surveillance, and emerging scientific evidence.

Benchmarking based on the increase in total cases or case fatality rates is one way to compare performance across states, regions, or countries, and could inform policy decisions about mitigation strategies. But CoVID-19 case counts depend on lab-confirmation, so the number of reported cases is a function of testing [6] . Benchmarking may thus be of limited use because countries or even states within the USA do not have the same testing policies or test availability for SARS-CoV-2 [5] . Comparing fatality rates, which may be comparatively easier than comparing CoVID-19 incidence, is also challenging. To illustrate, the case fatality rate in China, adjusted for underascertainment, demography, and censoring (deaths lag infection), was estimated at ~1.4%. Still, the crude case fatality rate adjusted only by censoring was ~3.7% [7] .

Ideally, public health officials would carry out testing based on representative random samples of the population (e.g., national, state) to estimate infection rates and case fatality rates. But countries and states have restricted testing considering its limited availability. To be sure, testing has increased globally [6] . But testing is far from random.

Diagnostic tests are often used to allocate scarce healthcare resources, so tests target patients with more severe symptoms as CoVID-19 incidence increases. A purist would argue one cannot compare countries or territories with different levels of testing without random sampling. Acknowledging these limitations, we took a pragmatic approach to suggest two relatively simple ways of comparing territories, such as countries or US states, with varying degrees of testing. First, we used multivariate regressions to estimate the test-elasticity-of-CoVID-19-incidence, that is, the proportional increase in testing of CoVID-19 and in reported cases (both in logarithms). Our results were robust to various model specifications. Second, we decomposed the growth in cases into test growth and positive test ratio (PTR) growth to intuitively visualize the components of case growth. We hope our results can help public health officials when they need to benchmark vis-á-vis other territories.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 6, 2020. 

We estimated the test-elasticity-of-CoVID-19-incidence for countries with CoVID-19 transmission globally (n=42; limited to countries that report the number of lab tests) [8] and for US states (n=51) [9] . We used a multivariate regression with the week-on-week change in the number of weekly lab-confirmed CoVID-19 cases or case fatality rates as our dependent variable. As independent variables, we used testing per capita, health expenditures, number of days since the 100 th lab-confirmed CoVID-19 case occurred, the share of the population ≥70 years of age with chronic respiratory disease. Variable definitions are shown in the Online Resource (Table S1 ).

We used a multivariate regression to estimate the test-elasticity-of-case-incidence. The regression specification is:

Where the subscript (i) stands for country or state, and (k) for covariates included. Because we used logarithms for

Yi and Testi, β captures the test elasticity of CoVID-19 case incidence, controlling for a vector of covariates Xk (Table S1 ). ε is an error term. Note that by first differencing both the dependent and the key variable, β accounts for any unobserved and time-invariant effect in a territory that may impact the level of testing and the level of the disease (e.g., income, quality of healthcare, age distribution). Still, we kept covariates as controls to test their impact on β.

A complementary way of benchmarking the evolution of cases vis-à-vis countries or US states with heterogeneous testing strategies is to use a simple algebraic decomposition. Because weekly cases can be decomposed as the multiplication of total tests and PTR ( / ), transforming to logarithms and taking the difference over weeks yields that the growth of cases can be decomposed as

This growth decomposition is useful to visualize the source of change in reported cases for comparison across countries or US states.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint

Test-elasticity Figure 1 shows a "raw" (i.e., not controlling for covariates) estimate of the test-elasticity-of-cases, that is, the association between logarithmic changes in testing per capita and reported cases. Figure 1A shows a consistent association between the increase in testing per capita and reported CoVID-19 cases by country (β=0.86; 95%CI: 0.57-1.15; p<0.001). In other words, a 10% increase in testing was associated with a ~9 % increase in reported cases. Considering that a test elasticity=1 was included in the 95% confidence interval, we cannot reject that growth in weekly cases was fully proportional to increase in testing. By contrast, Figure 1B shows that for the US states, the test-elasticity of cases was substantially lower (β=0.17; 95%CI: 0.03-0.31; p=0.02), suggesting that the evolution of CoVID-19-case-growth in the US was largely driven by changes in PTR rather than testing. We get larger elasticities, more comparable to the global sample, when excluding outlier states in the USA and focusing on the same range of testing growth (β=0.54, p<0.0001, 95%CI: 0.36-0.71; Online Resource, Figure S1 ). Table 1 are consistent with the findings in Figure 1A for the cross-country sample. With and without regression controls, we get point estimates for the elasticity around 0.85-0.88 (p-value <0.001; columns 1 through 3). A 10% increase in testing yielded about 9% (95% CI: 4.2-13.4%; p<0.001) increase in reported cases. Moreover, changes in testing explained about 40% of the variance in cases for our regressions (r 2 =0.41) and adding health expenditures, population, days since the 100 th CoVID-19 case, and share of the population ≥70 years of age with chronic respiratory disease added little explanatory strength to the model (r 2 =0.46). We obtained similar results for case fatality rate, but in the opposite direction (Table 1 , columns 4 through 6). We found a test-elasticity-of-casefatality slightly above minus one (β=-0.89; column 6). A 10% increase in testing yielded about 9% (95%CI: -12.2--5.6%; p<0.001) decrease in case fatality rates, with covariates adding limited explanatory strength (r 2 =0.45). To test robustness, we re-ran the regression with the most recent available data (April 11 th through April 17 th ); results were largely consistent. We found an elasticity for reported cases of β=0.77 (95%CI: 0.23-1.30; p=0.01; r 2 =0.43, column 3), and β=-0.77 (95%CI:-1.40--0.14; p=0.02, r 2 =0.18, column 6) for case fatality rates (Online Resource, Table   S2 ).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint Table 2 shows results across states in the USA. As in Figure 1B , the magnitude of the elasticity was smaller for reported cases (β=0.18) and case fatality rate (β=-0.16). A 10% increase in testing yielded about 2% (95%CI: 0.1-3.4%; p=0.03, r 2 =0.25, column 3) increase in reported cases, and ~2% (95%CI: -3.1--0.2; p=0.03, r 2 =0.14, column 6) decrease in case fatality rates (only significant with controls). R 2 was smaller for regressions comparing US states than countries globally. Elasticities for US states using the most recent data available (April 11 th through April 17 th ) showed some qualitatively similar results. We found a test elasticity of β=0.17 (95%CI: 0.08-0.27; p<0.001, r 2 =0.29, column 3) for reported cases, and β=-0.15 (95%CI: -0.30-0.01; p=0.06, r 2 =0.12, column 6) for case fatality rates, though the latter was not significant at α=0.05 (Online Resource, Table S3 ). shows both testing and PTR remained the same as the previous period. Figure 2 shows four quadrants where countries would fall when they increased or decreased in either testing or PTR. When both testing and PTR grow (quadrant I) or decrease (quadrant III), or countries o states move along the y-axis (change in PTR) or x-axis (change in tests), weekly change has a straightforward interpretation. But for quadrants II and IV, the net effect in cases is not obvious. To aid interpretation, we plotted a downward sloping line that represents the zero-case-growth (i.e., where Δ ln + Δ ln = 0; hence Δ ln = − Δ ln ). Countries or states above the line increased case growth in the past week; those below the line have decreased case growth. Case growth in most countries moves in complex trajectories. The distance to the zero-case growth line is a visual clue for the overall increase in cases.

In Figure 2A , we see that countries like Italy (ITA) and the United Kingdom (GBR) both increased the number of tests performed between April 10 th and April 17 th . However, because PTR increased in the UK and decreased in Italy, we can conclude, solely by looking at Figure 2A , that CoVID-19 cases increased in the UK and decreased in Italy between April 3 rd and April 10 th . On the other hand, countries like Ecuador (ECU) and Israel (ISR) appear on top of the line, meaning that growth in COVID-19 cases was approximately the same as in the previous week (Δ ln = 0). Notably, this null change in case growth is accounted for by different levels of testing. In the graph, Israel decreased reported testing but increased the positive rate in the same proportion. In contrast, Ecuador . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint massively increased testing during that week with a declining PTR, leaving case growth unchanged. This decomposition helps track case growth visually across jurisdictions with heterogeneous testing. Differences in testing need to be taken into account when benchmarking and communicating the evolution of the pandemic.

Our results show two relatively simple ways of comparing the reported evolution of COVID cases across states, regions, or countries with heterogeneous testing. We used test-elasticities or a growth decomposition between test and PTR growth. Data required for our estimates are readily available to public health officials through COVID-19 data repositories; data and code for our analyses are also available. As the pandemic unfolds and more consistent data hopefully become available [10], we hope these results can help support evidence-based decisions by public health officials. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. Notes * p < 0.05, ** p < 0.01, *** p < 0.001. Standard errors shown in parentheses. ln stands for natural logarithm. a Confirmed cases of COVID-19" during the week ending April 10th, 2020. b Case fatality rate is the ratio between deaths due to COVID-19 and cases in the same period. c Testing is the amount of negative and positive tests informed to each country's health authority in the same period of cases [6] . d The number of days since the cumulated number of cases was equal or greater than 100 [6] . e Share over age 70 with a chronic respiratory disease is the share of the population with age equal or greater than 70 years old with a chronic respiratory disease, including chronic obstructive pulmonary disease and asthma.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint Case fatality rate is the ratio between deaths due to COVID-19 and cases in the same period. c Testing is the amount of negative and positive tests informed to each country's health authority in the same period of cases [6] . d The number of days since the cumulated number of cases was equal or greater than 100 [6] . e Share over age 70 with a chronic respiratory disease is the share of the population with age equal or greater than 70 years old with a chronic respiratory disease, including chronic obstructive pulmonary disease and asthma.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 6, 2020. Notes. Points are countries (A) or US states (B). In both panels, we plot the growth of tests and PTR between the week ending April 10 th and the previous week. The functional form of the previously mentioned growth is the change in the natural logarithm of per capita rates per week. The growth decomposition of cases is the sum of the increase in testing (horizontal axis) and PTR growth (vertical axis). The line is not a regression but represents zero case growth as Δ ln = Δ ln + Δ ln = 0. While territories above the line have growing cases, territories below the line have decreasing cases. The four quadrants (I to IV) in gray show the various combinations B A . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint of increasing or decreasing testing and PTR. Importantly, quadrants (II) and (IV) include entities with growing and decreasing cases, depending on the side of the zero-case growth line. For instance, in (B), KY appears with decreasing cases but associated with fewer tests and more PTR. The opposite combination happened in (A) for Ecuador. For visual purposes, countries in A were restricted to population of over 5 million people, and US states in (B) exclude WA. Countries are represented by ISO 3 codes and US states, by two-digit codes. The Online Resource shows how the USA moved over several weeks in the cross-country plot.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 6, 2020. . https://doi.org/10.1101/2020.05.01.20087882 doi: medRxiv preprint

An interactive web-based dashboard to track COVID-19 in real time

Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (COVID-19)

Scientific and ethical basis for social-distancing interventions against COVID-19

School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review

Countries test tactics in 'war' against COVID-19

To understand the global pandemic, we need global testing -the Our World in Data COVID-19 Testing dataset

Estimates of the severity of coronavirus disease 2019: a model-based analysis