key: cord-0716829-jxwb7oqn
authors: Spoer, Ben R.; McCulley, Edwin; Lampe, Taylor M.; Hsieh, Pei Yang; Chen, Alexander; Ofrane, Rebecca; Rollins, Heather; Thorpe, Lorna E.; Bilal, Usama; Gourevitch, Marc N.
title: Validation of a neighborhood-level COVID Local Risk Index in 47 large U.S. cities()
date: 2022-05-24
journal: Health Place
DOI: 10.1016/j.healthplace.2022.102814
sha: 22a30fe72becc16f7102d6ddaf396c853297ceed
doc_id: 716829
cord_uid: jxwb7oqn

OBJECTIVES: To present the COVID Local Risk Index (CLRI), a measure of city- and neighborhood-level risk for SARS COV-2 infection and poor outcomes, and validate it using sub-city SARS COV-2 outcome data from 47 large U.S. cities. METHODS: Cross-sectional validation analysis of CLRI against SARS COV-2 incidence, percent positivity, hospitalization, and mortality. CLRI scores were validated against ZCTA-level SARS COV-2 outcome data gathered in 2020–2021 from public databases or through data use agreements using a negative binomial model. RESULTS: CLRI was associated with each SARS COV-2 outcome in pooled analysis. In city-level models, CLRI was positively associated with positivity in 11/14 cities for which data were available, hospitalization in 6/6 cities, mortality in 13/14 cities, and incidence in 33/47 cities. CONCLUSIONS: CLRI is a valid tool for assessing sub-city risk of SARS COV-2 infection and illness severity. Stronger associations with positivity, hospitalization and mortality may reflect differential testing access, greater weight on components associated with poor outcomes than transmission, omitted variable bias, or other reasons. City stakeholders can use the CLRI, publicly available on the City Health Dashboard (www.cityhealthdashboard.com), to guide SARS COV-2 resource allocation.

As the SARS COV-2 pandemic continues, local response efforts have been hampered by a lack of timely and geographically granular information. In the United States (U.S.), SARS COV-2 incidence (The New York Times, 2021), mortality (Centers for Disease Control and Prevention/National Center for Health Statistics), and vaccination (Centers For Disease Control And Prevention, 2021) data are widely available at the county level, but similar data have not been as easily accessible for smaller geographies, like cities or neighborhoods. One reason for this is that the collection, cleaning, and dissemination of SARS COV-2 surveillance data is the responsibility of local health departments, which may lack the funding and staff required to undertake such efforts, especially in small and mid-sized cities (NACCHO, 2020) .

In this situation, local health departments often rely on readily available county-level data. While county-level data are essential for public health surveillance and planning, city and sub-city data are vital to guide local pandemic response efforts, particularly because more than 80% of the U.S. population lives in urban areas (U.S. Census Bureau, 2021) . City populations often differ substantially from the populations of counties in which they are located, causing county-level metrics to be insufficient proxies for city-level measures (Spoer et al., 2020) . This is consistent with Tobler's first law of geography, which states "everything is related to everything else, but nearer things are more related than distant things". Spatially granular data is most effective at describing the health-related conditions in a specific place, and as such, can inform more effective responses to the ongoing SARS COV-2 epidemic.

To address the need for spatially granular data that can guide city-☆ Sources of support: This work was supported in part by NIH grant DP5OD26429, and Robert Wood Johnson Foundation grants 77644 and 78325. Funders did not have substantive roles in study design, data collection, analysis or interpretation, nor writing or submission of this manuscript.

level SARS COV-2 response, the City Health Dashboard (Department Of Population Health Nyu Langone Health, 2021 ) (the Dashboard), a website that provides free access to a range of health and health determinant metrics for over 750 U.S. cities, created the COVID Local Risk Index (CLRI) in June 2020. The CLRI provides a city-and neighborhood-level metric that characterizes risk of poor SARS COV-2 outcomes (high SARS COV-2 transmission and potential for severe SARS COV-2 illness) to help guide resource allocation and interventions. Given the urgent need at the time for sub-county SARS COV-2 data tools, the Dashboard released the CLRI before sufficient small-area SARS COV-2 outcome data were publicly available to validate the index.

To validate the CLRI at the smallest possible geography, the Dashboard has partnered with Drexel's Urban Health Collaborative (UHC). The UHC compiled SARS COV-2 data from several U.S. cities, leveraging publicly available data when possible, and requesting data not otherwise publicly available directly from city health departments, including SARS COV-2 positivity, incidence, hospitalizations, and mortality counts (Bilal et al., 2022b) . We selected a range of SARS COV-2 outcome metrics because risk factors, transmission, and poor outcomes tend to cluster geographically, and because the CLRI was designed to capture both transmission and severity. We chose to capture both transmission and severity as they have different policy implications. On the one hand, preventing transmission may require a broader focus on avoiding exposure to SARS-COV 2, which may be the key driver of disparities (Bilal et al., 2022a) . On the other hand, addressing severity may require longer term policies that reduce chronic disease burden. While the range of metrics available for each city varied, we identified a core set of metrics to validate the CLRI for a subset of cities displayed on the Dashboard. In this paper we describe the methods used by the Dashboard team to calculate the CLRI and then use SARS COV-2 data from the UHC to validate whether the CLRI accurately captures differences in risk for SARS COV-2 transmission and severity in select U.S cities.

The Dashboard, a Robert Wood Johnson Foundation-funded data platform, provides data for 766 U.S. cities, including all U.S. cities with population 50,000 or greater and 10 smaller New Jersey cities. The Dashboard team published the CLRI in June 2020 at the city-and neighborhood-level after reviewing other SARS COV-2-related risk indices (Surgo Ventures, 2020a; Social Progress Imperative, 2020) and emerging literature on demographic factors and health conditions related to SARS COV-2 transmission and severity. The Dashboard team then re-evaluated and updated the CLRI in March 2021 based on new high-quality research published through October 2020 (Gottlieb et al., 2020; Gupta et al., 2020; Hamidi et al., 2020a Hamidi et al., , 2020b Hirsch et al., 2020; Kim et al., 2021; Petrilli et al., 2020; Reichberg et al., 2020; Rosenberg et al., 2020; Rozenfeld et al., 2020; Van Gerwen et al., 2020; Williamson et al., 2020) , in consultation with SARS COV-2 expert researchers. Information about the index calculation is available in the Dashboard's technical documentation .

The CLRI provides a combined city-and neighborhood-level assessment of SARS COV-2 infection risk and illness severity (census tracts were used to proxy neighborhoods). The CLRI is comprised of three groups of metrics: (1) social vulnerability, which includes metrics serving as a proxy for SARS COV-2 infection risk, (2) SARS COV-2related chronic health conditions, which includes metrics contributing to potential increased severity of SARS COV-2 illness, and (3) SARS COV-2-related demographics, which captures groups that may be higher risk for SARS COV-2 infection and severity of SARS COV-2 illness (Table 1) .

The social vulnerability component group captures neighborhood social and demographic factors associated with increased risk for SARS COV-2 infection. We measured social vulnerability through the Centers for Disease Control and Prevention's (CDC) Social Vulnerability Index (SVI). The SVI is a validated, peer-reviewed index that measures a community's vulnerability to harm caused by a natural disaster, including disease outbreak (Flanagan et al., 2011 , Centers For Disease Control And Prevention, 2018 . SVI is correlated with SARS COV-2 positivity, incidence and mortality (Nayak et al., 2020; Bilal et al., 2021) . The SVI was calculated following the procedure created by the CDC using U.S. Census American Community Survey (ACS) 2014-2018 5-year estimates (Flanagan et al., 2011 , Centers For Disease Control And Prevention, 2018 Nayak et al., 2020) . A list of the variables included in 

The SARS COV-2-related chronic health conditions component group incorporates known risk factors for increased severity of SARS COV-2 illness. The pool of potential components was limited to metrics for which estimates were available from CDC's PLACES Project (2018, 1year Modeled Estimates; methods detailed elsewhere) (Places: Local Data For Better Health, 2020, 500 Cities: Local Data for Better Health, 2018). Candidate components with equivocal evidence were excluded. The following chronic health conditions were included: obesity (Williamson et al., 2020; Kim et al., 2021; Van Gerwen et al., 2020; Ebinger et al., 2020; Petrilli et al., 2020; Rozenfeld et al., 2020) , chronic kidney disease (Rozenfeld et al., 2020; Gottlieb et al., 2020; Van Gerwen et al., 2020; Petrilli et al., 2020; Kim et al., 2021) , diabetes (Ebinger et al., 2020; Azar et al., 2020; Van Gerwen et al., 2020; Hirsch et al., 2020; Kim et al., 2021; Williamson et al., 2020) , chronic obstructive pulmonary disease (COPD) (Van Gerwen et al., 2020 , Cummings et al., 2020 Kim et al., 2021; Williamson et al., 2020) , and coronary heart disease (Azar et al., 2020; Gottlieb et al., 2020; Hirsch et al., 2020; Cummings et al., 2020; Gupta et al., 2020; Williamson et al., 2020) .

The SARS COV-2-related demographics component group includes demographic factors related to both SARS COV-2 infection and severity. This group includes density of older adult and racial/ethnic minority populations (Rozenfeld et al., 2020; Azar et al., 2020; Gottlieb et al., 2020; Van Gerwen et al., 2020; Ebinger et al., 2020; Hirsch et al., 2020; Williamson et al., 2020; Petrilli et al., 2020; Kim et al., 2021; Gupta et al., 2020; Cummings et al., 2020; Gottlieb et al., 2020 Gottlieb et al., , 2020 . Though the SVI accounts for density of non-white individuals and adults aged 65+, we added weight to these specific demographic groups because older age has consistently been among the strongest predictors of poor SARS COV-2 outcomes, and racial/ethnic minority groups have experienced a higher burden of SARS COV-2 cases and mortality due to myriad factors related to structural racism (Bassett et al., 2020; Berkowitz et al., 2020; Chen and Krieger, 2021; Millett et al., 2020; Raifman and Raifman, 2020; Bailey and Moon, 2020; Bilal et al., 2021) . Our inclusion of proportion minority population is intended to proxy neighborhood-level consequences of structural racism (e.g., racial residential segregation, neighborhood disinvestment, etc.), and is not intended to suggest that minority individuals are at higher risk for SARS COV-2 transmission or negative SARS COV-2 outcomes due to biological differences between minority and majority race individuals. These demographic factors have a substantial effect on the final CLRI score, which is consistent with research to date on the importance of these factors in SARS COV-2 transmission and severity. Data are from U.S. Census ACS 2014-2018 5-year Estimates (U.S. Census Bureau, 2020).

We weighted the social vulnerability group at 30% of the overall index score in order to distribute additional weight across the demographic and health conditions component groups, for which we found robust evidence of association with poor SARS COV-2 outcomes (see Table 1 for all component weights). SVI components were weighted equally within the social vulnerability group, consistent with CDC's SVI calculation methods (Centers for Disease Control and Prevention/-Agency for Toxic Substances and Disease Registry/Geospatial Research Analysis and Services Program, 2020, Flanagan et al., 2011) . The weighting scheme was developed based on theory drawn from scientific research, before outcome data were available. Similar data projects have found success with a priori weighting schemes (Catlin et al., 2010) .

The remaining 70% of the index score weight was distributed based on a population-normalized average effect size. Specifically, we reviewed "high-quality" evidence on SARS COV-2 infection and severity (defined as research with sample sizes >100, which surveyed a population-based sample, and controlled for common confounders in regression models that estimated effect sizes for specific health conditions). We then averaged effect sizes from the included evidence for each component, and multiplied the component's average effect size by its U. S. population prevalence.

The Dashboard calculated the CLRI separately at the city-and census tract-levels. We followed the analytic strategy utilized by CDC's SVI (Flanagan et al., 2011 , Centers for Disease Control and Prevention/-Agency for Toxic Substances and Disease Registry/Geospatial Research Analysis and Services Program, 2020). We assigned percentiles relative to other Dashboard cities or census tracts for each component (Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry/Geospatial Research Analysis and Services Program, 2020), multiplied each component's percentile by its weight, summed the weighted percentiles of all components, and reported this sum in deciles as the CLRI (calculated using SAS v9.4) (Sas Institute Inc, 2015).

For validation purposes, and since SARS COV-2 data are rarely released at the census tract level, we aggregated SARS COV-2 outcomes to the zip code tabulation area (ZCTA) level using the U.S. Census Bureau's ZCTA to Census Tract Relationship File (U.S. Census Bureau, 2010) . Tract CLRI values were weighted by the proportion of the ZCTA population that resided in both the census tract and ZCTA in question, then summed.

) CLRI values were computed only for ZCTAs with ≥70% population overlap with Dashboard census tracts. CLRI values were not calculated for cities with fewer than 10 ZCTAs. We conducted a sensitivity analysis using a 90% threshold for ZCTA to census tract population overlap.

To obtain sub-city SARS COV-2 data, the Dashboard partnered with the Drexel Urban Health Collaborative (UHC). The UHC systematically accessed and collected geographically granular ZCTA-level SARS COV-2 data on count of tests conducted, count of positive tests, confirmed cases, incidence, hospitalizations, and mortality cumulatively from onset through the dates specified in Table 2 . Since most cities reported only cumulative counts at the ZCTA-level, we opted to not conduct longitudinal analyses with data on trends. These data were obtained by identifying repositories of data from U.S. cities, including the 30 cities that are members of Big Cities Health Collation for which UHC is funded to provide SARS COV-2 data (Bilal et al., 2022b) . Data from public dashboards were either accessed directly or downloaded, or, in select cases, copied into a spreadsheet. In cases where data were not publicly available, UHC requested data directly from health departments, entering into data sharing agreements as needed. All data were checked for consistency and outliers. In total, we obtained data for 47 of the 766 Dashboard cities. The dates of access, sources for each city, and outcomes available are displayed in Table 2 .

We identified four ZCTA-level SARS COV-2 outcomes to validate the CLRI, as they represent measures of either risk of infection or illness severity: positivity (number of people that tested positive for SARS COV-2/number tested for SARS COV-2), incidence (confirmed SARS COV-2 cases/total population), hospitalization (SARS COV-2 hospitalizations/ total population), and mortality (SARS COV-2 deaths/total population). Population denominators were obtained from 2015 to 2019 ACS 5-year estimates.

We also compared select metrics for cities in the validation sample to the same metrics for cities on the Dashboard but not in the validation sample in order to gauge similarity between the two groups. The demographic factors compared included racial/ethnic diversity, percent children living in poverty, percent of the population experiencing excessive housing cost burden, and CLRI rank. Metrics were analyzed as presented on the Dashboard and are defined in the Dashboard's technical documentation .

This analysis assessed the CLRI's construct/convergent validity in measuring risk of SARS COV-2 infection and illness severity. We tested the hypothesis that the CLRI was positively associated with SARS COV-2 infection (positivity and incidence) or severity (hospitalizations and mortality); each outcome was tested independently. First, we graphically depicted correlations using scatter plots. Second, we fitted a negative binomial model with the count of positive tests, cases, hospitalizations and mortality as the outcome, the number of tests (for positivity) or population counts (for the other outcomes) as the offset, and the CLRI as the only predictor. We chose a negative binomial model as the four outcomes were found to be overdispersed for a Poisson model. We extracted the rate ratios (RR) and 95% confidence intervals for the CLRI coefficient. Before introducing CLRI into this model, we standardized ZCTA-level CLRI values by centering by the city ZCTA CLRI mean and scaling by city standard deviation, both for each city separately. This standardization was conducted to make RRs comparable across cities. We also compared RRs to RRs produced using the same methods for another publicly available SARS COV-2 vulnerability index, the Surgo Ventures' COVID Community Vulnerability Index (CCVI) (Surgo Ventures, 2020a). The CCVI does not provide city-level values.

We also produced an overall pooled estimate of the association between the CLRI and the CCVI and the four outcomes using a mixedeffects negative binomial model of ZCTAs nested in cities, with a fixed and random coefficient for the CLRI or the CCVI. The exponentiated fixed coefficient of this model represents associations for the median city. To test whether there was heterogeneity in the association between the CLRI (or CCVI) and the outcomes, we compared this model with a model without a random slope for the CLRI (or CCVI) using a log likelihood ratio test. We fitted this model using a Laplace approximation with the glmmTMB package in R 4.1. To test whether the number of quadrature points of the generalized mixed model influenced our results, we tested increasing the number of quadrature points using Sta-taMP v17 (see Supplemental Fig. 4 comparing results using the Laplace approximation vs. an increased number of quadrature points). Table 3 displays a list of RRs and 95% confidence intervals per city. Scatterplots displaying the association between the CLRI and the four analyzed SARS COV-2 outcomes are available upon request. In the pooled analysis, we found that higher CLRI was associated with higher positivity (RR = 1.16, 95% CI 1.09 to 1.24 per 1-SD increase, p < 0.01), incidence (RR = 1.09, 95% CI 1.04 to 1.14, p < 0.001), hospitalization (RR = 1.43, 95% CI 1.31 to 1.56, p < 0.001), and mortality (RR = 1.22, 95% CI 1.15 to 1.30, p < 0.001). This means that, in the pooled analysis, a 1-SD higher ZCTA-level CLRI score was associated with 16% higher risk of positivity, 9% higher incidence, 43% higher risk of hospitalization, and 22% higher risk of mortality. These pooled numbers varied by city (see Appendix Table 5 for comparisons of model with and without random slopes, and Supplemental Fig. 2 for a comparison between coefficients from the mixed effects model and from the stratified model), and do not account for differences due to compositional differences between cities, differences in the course of the outbreak across cities, and differences in when data were accessed.

We found that the CLRI was a good predictor of city-level positivity, with positive associations in 11 of 14 cities with positivity data (p < 0.05 in 8/11 cities). RRs in these 11 cities ranged from 1.04 (95% CI 0.98 to 1.11, p > 0.05) in Virginia Beach (VA) to 1.35 (95% CI 1.20 to 1.53, p < 0.01) in Boston (MA). For example, in Chicago each city-specific SD increase in ZCTA CLRI was associated with a 26% higher positivity rate (RR = 1.26, 95% CI 1.14 to 1.39, p < 0.01). Results regarding incidence were mixed; in 33 cities the association between CLRI and incidence was positive, ranging from 1.01 (95% CI 0.78 to 1.31) in Shreveport (LA) to 1.85 (95% CI 1.49 to 2.28) in Oakland (CA) (p < 0.05 in 17/33 cities). We found no association in 1 city, and a negative association in 13 cities, ranging from 0.81 (95% CI 0.60 to 1.10) in San Diego (CA) to 0.98 (95% CI 0.84 to 1.13) in Jacksonville (FL) (p < 0.05 in 6/13 cities).

The CLRI performed more consistently with regards to indicators of SARS COV-2 illness severity. In all 6 cities with hospitalization data (p < 0.05 in all) and in 13 of 14 cities with mortality data (p < 0.05 in 9/14 cities) the CLRI was associated with hospitalization or mortality. Specifically, the association between CLRI and hospitalization ranged from 1.29 (95% CI 1.08 to 1.55, p < 0.01) in Madison (WI) to 2.07 (95% CI 1.69 to 2.52, p < 0.001) in Seattle (WA). In the 13 cities with a positive association between CLRI and mortality, the RR varied from 1.05 (95% CI 0.92 to 1.21, p > 0.05) in Tulsa (OK) to 1.87 (95% CI from 1.37 to 2.54, p < 0.001) in Seattle (WA). A sensitivity analysis using a more stringent threshold to aggregate from census tracts to ZCTAs produced similar results to the main analysis (Supplemental Fig. 3) .

We explored degree of similarity between this validation sample (n = 47) and other cities (n = 713) on the Dashboard. On average, cities included in the validation sample had higher total population count and higher average CLRI scores than did Dashboard cities that were not included. Validation cities were also more diverse and had higher percent of children living in poverty than other Dashboard cities. There were not statistically significant differences in housing cost (Table 4 ). Supplemental Fig. 1 depicts a histogram of CLRI values for cities in the validation sample; a histogram for the full sample is not shown as cities were, by design, equally distributed across CLRI values.

Finally, the CLRI and Surgo's CCVI performed similarly overall with respect to association with SARS COV-2 outcomes. Overall, the CCVI was similarly associated with positivity and slightly more strongly associated with incidence (RR of 1.13 vs. 1.09 for the CLRI); CCVI RRs for incidence were larger than CLRI RRs in 29/47 cities, RRs were identical in 7 cities, and CLRI RRs were larger in 13 cities. RR differences were typically less than 0.1. CLRI was more frequently and strongly positively associated with hospitalizations and mortality (Table 3) (A  table of city-level R 2 values is available upon request).

We validated the CLRI as an accurate tool to capture small area-level SARS COV-2-related risk in 47 U.S. cities, demonstrated most strongly by our pooled model results. In the majority of included cities, we found the CLRI to be strongly associated with positivity and SARS COV-2related hospitalizations and mortality. These results underscore the extent to which social determinants of health, demographic factors, and the population prevalence of specific health conditions affect neighborhood-level risk for SARS COV-2. Given large total populations in U.S. cities, even a small increase in ZCTA-level positivity, hospitalization, and mortality (i.e. statistically significant RRs >1.00) can have important implications for policy makers and public health practitioners.

Associations between the CLRI and SARS COV-2 incidence were weaker and more heterogeneous across cities. In 13 cities CLRI was negatively associated with SARS COV-2 incidence (p < 0.05 in 6 of 13). In 4 of these 13 cities positive associations were found with either positivity, hospitalization, or mortality. These weaker and more heterogeneous associations may be due to imperfect reporting of SARS COV-2 cases, especially at the beginning of the pandemic, differences in the course of the outbreak across cities, unequal access to testing (Rader et al., 2020) , or other factors that cause incidence data to be less reliable than positivity, hospitalization, and mortality data (Wu et al., 2020) . This indicates that either measurement error related to SARS COV-2 incidence is higher than measurement error related to the other SARS COV-2 outcomes analyzed here, or that the CLRI is a better predictor of SARS COV-2 illness severity than of incidence. If the latter, this may be because caused by the components included in the CLRI and how they were weighted. Old age and comorbidities are weighted heavily in the CLRI, while, for example, potential for occupational exposures and area proportion of essential workers are not included. Inclusion of these variables (for which we could not find sufficiently granular data) or a different weighting approach may have produced different results. The heterogeneity in the incidence results could also be related to omitted variable bias. The CLRI does not measure how individuals interact with their social and built environments, which is a source of potential variation in SARS COV-2 exposure. The CLRI may produce more robust incidence results were such variables included.

There was substantial variation in strength of association between the CLRI and SARS COV-2 outcomes across cities. This could be driven by a number of factors. Some of these factors are related to the course of SARS COV-2 in a given city, including when and how the disease was introduced, and the course of the outbreak; cities that were exposed early in the pandemic or had larger outbreaks will likely have more cases, and so will produce larger effect size estimates. Differences may also be caused by differential access to testing, reporting of test results, or data tampering that may artificially lower counts of positive tests, hospitalizations, or mortality. This could also be caused by differences in how city residents interact with their social and built environments, as mentioned above.

Surgo's CCVI, also developed to assess neighborhood-level SARS COV-2 risks, produced a slightly stronger association with SARS COV-2 incidence than CLRI and was positively associated with incidence in more cities. In contrast, the CLRI was more strongly associated with positivity, hospitalization, and mortality. This could be for several reasons. First, Surgo's CCVI includes variables related to risk of infection that are not included in the CLRI, for example percent of population working in high infection risk settings, and long-term care residents per 100,000 population (Surgo Ventures, 2020b). Second, CCVI includes Table 3 City-level associations (RRs) between ZCTA-Level CLRI scores, CCVI scores and four SARS COV-2 outcomes. more metrics, which may reduce the impact of health conditions included in the CCVI on the final index score (Surgo Ventures, 2020b), in turn lowering strength of association with severity outcomes. Though both indices are associated with SARS COV-2 outcomes, the CLRI may be preferable for two reasons. First, CLRI is better correlated with a range of SARS COV-2 outcomes, including severe SARS COV-2 outcomes, which put more stress on the health system and have more dire consequences than do mild cases. Second, the CCVI includes numerous county-and state-level variables, reducing neighborhood-level specificity; the CLRI exclusively incorporates neighborhood-level variables. The CLRI can be used by policy makers in several ways. Recent research has described associations between county-level SVI and vaccination coverage (Barry et al., 2021) ; the CLRI can help policy makers identify at-risk places at a more geographically granular level. For example, Waco, TX has used the CLRI to interpret neighborhood-level distributions of confirmed SARS COV-2 cases (Dashboard, 2020b) , while Manchester, NH has used the CLRI to identify high SARS COV-2 risk neighborhoods (Dashboard, 2020a) . Going forward, testing and vaccination sites could be located in neighborhoods that have high CLRI scores, and vaccine outreach and education initiatives could focus on these neighborhoods. Furthermore, given that vaccines are now widely available in the U.S., neighborhood-level Table 3 . City-Level Associations (RRs) between ZCTA-Level CLRI Scores, CCVI Scores and Four SARS COV-2 Outcomes (cont)COV-2 Outcomes ~Pooled coefficient comes from a multilevel Poisson model of ZCTAs nested in cities, with a fixed and random slope for the CLRI or the CCVI. Coefficient shown is the fixed portion of the CLRI or CCVI effect, representing the effect in the median city. ******.

Summary statistics for COVID local risk index and select metrics for cities included in and excluded from validation sample. year estimates. b Two sample t-tests comparing means for these metrics were significant at the p < 0.05 level. Note: Model without a random slope includes a fixed effect for the CLRI or CCVI and a random intercept for city; model with random slope further includes a random slope for the CLRI or CCVI. P-value tests the null hypothesis that there is no heterogeneity in the association between CLRI or CCVI with the outcome by city. disparities in vaccine access and uptake may emerge. As disparities are identified, resources should be preferentially guided to neighborhoods in which there is slow vaccine uptake that also rank highly on the CLRI. Similarly, in the event of another surge of SARS COV-2 cases, state officials could allocate more resources to cities with higher CLRI scores. Finally, in cities with neighborhood-level SARS COV-2 surveillance data, the CLRI can provide an additional point of reference, helping policy makers to bolster the case for additional state and federal resources, or to identify neighborhoods that are performing better or worse than the CLRI would suggest based on their surveillance data. This validation analysis could have been produced sooner if granular SARS COV-2 infection and illness severity data were more readily available. Furthermore, though the results of this analysis are encouraging, they are based on data from just over 6% of Dashboard cities because geographically granular data are not available for more cities, or because smaller cities do not have enough ZCTAs for validation. Whenever possible, granular SARS COV-2 data should swiftly be made publicly available.

Though the CLRI is calculated at the neighborhood level, the present validation analysis was conducted at the ZCTA level. ZCTAs are larger than census tracts and may capture different populations. We took steps to reduce bias introduced when aggregating index values from tract to ZCTA by only including ZCTAs that had high population overlap with census tracts on the Dashboard (>70%). Sensitivity analysis results using a more stringent population overlap threshold (>90%) were similar to results from the main analysis. Data from cities were accessed at different times, and the course of the pandemic varies across cities, so validation results for different cities may not be easy to compare. Incomplete and differentially reported outcome data may bias the results described here. However, it is likely that areas with higher SVI and CLRI scores experience more substantial underreporting. As such, this bias may dilute the association between CLRI and SARS COV-2 outcomes. Some of the variables included in the CLRI intended to capture the consequences of structural racismnamely proportion of minority individuals. While these measures may not capture structural racism as directly as other measures, for example the Index of Dissimilarity, which measures residential segregation in larger areas (e.g. cities), or the Index of Concentration at the Extremes, which measures residential segregation in smaller areas (e.g., census tracts) (White, 1986) , we used proportion of minority individuals to maintain consistency with published literature while calculating the index. At the point when we constructed the index we did not identify sufficient literature examining associations between other measures of structural racism and SARS COV-2 transmission and disease outcomes. Future CLRI updates may contain these variables. Similarly, though the CLRI, via the SVI, includes three variables related to the housing built environment (housing in structures with 10+ units, mobile homes, and household overcrowding), recent research suggests public built environments also affect SARS COV-2 severity (Wali and Frank, 2021) . Variables related to the public built environment, and other recently emerged SARS COV-2 risk factors, may be added to future versions of the CLRI. Finally, cities included in the validation sample were not representative of all Dashboard cities. Validation cities were larger, more diverse, had higher percent children living in poverty, and higher average CLRI ranks than did Dashboard cities not included in the validation sample. Though this limits the generalizability of these results, it also suggests the present analysis validates the CLRI for use in cities with substantial at-risk populations, where SARS COV-2 resources may be most needed.

In terms of strengths, the CLRI exclusively incorporates census tractlevel data, and has been updated as our understanding SARS COV-2 has improved. The CLRI is publicly available from the Dashboard through downloadable data and neighborhood-level maps (http://www.cityh ealthdashboard.com).

The CLRI can help guide city-and neighborhood-level SARS COV-2 resource allocation and interventions. In the absence of needed validation data, public health researchers can build potentially valid data tools while seeking validation as soon as possible.

Available: https://chronicda ta.cdc.gov/500-Cities-Places/500-Cities-Local-Data-for-Better-Health-2018-relea/rja 3-32tc

Disparities in outcomes among Covid-19 patients in A large health care system in California

Racism and the political economy of Covid-19: will we continue to resurrect the past? J. Health Polit

Patterns in Covid-19 vaccination coverage, by social vulnerability and urbanicity -United States

Variation in racial/ethnic disparities in Covid-19 mortality by age in the United States: a cross-sectional study

Structurally Vulnerable Neighbourhood Environments and Racial/ethnic Covid-19 Inequities

Spatial inequities in Covid-19 testing, positivity, confirmed cases, and mortality in 3 U.S. Cities: an ecological study

Racial/ ethnic and neighbourhood social vulnerability disparities in Covid-19 testing positivity, hospitalization, and in-hospital mortality in a large hospital system in Pennsylvania: a prospective study of electronic health records

Tracking Covid-19 inequities across Jurisdictions represented in the Big cities health Coalition (Bchc): the Covid-19 health inequities in Bchc cities dashboard

Different Perspectives for Assigning Weights to Determinants of Health

Social Vulnerability Index 2018 Database Us [Online]. Agency for Toxic Substances and Disease Registry/Geospatial Research Analysis and Services Program

Covid Data Tracker: Covid-19 Integrated County View

Centers For Disease Control And Prevention/Agency, 2020. For Toxic Substances and Disease Registry/Geospatial Research Analysis and Services Program. Cdc Svi

Provisional Death Counts for Coronavirus Disease (Covid-19): Index of Covid-19 Surveillance and Ad-hoc Data Files

Revealing the unequal burden of Covid-19 by income, race/ethnicity, and household Crowding: us county versus zip code analyses

Epidemiology, clinical course, and outcomes of critically ill adults with Covid-19 in New York City: a prospective cohort study

Raising awareness of Covid risk and testing in Manchester

Using Data to Inform Comprehensive Covid Response Strategy in Waco

Department Of Population Health Nyu Langone Health, 2021. City Health Dashboard

Pre-existing traits associated with Covid-19 illness severity

A social vulnerability index for disaster management

Clinical course and factors associated with hospitalization and critical illness among Covid-19 patients in

Longitudinal analyses of the relationship between development density and the Covid-19 morbidity and mortality rates: early evidence from 1,165 metropolitan counties in the United States. Health Place 64

Does density aggravate the Covid-19 pandemic?

Risk factors for intensive care unit admission and in-hospital mortality among hospitalized adults identified through the us coronavirus disease 2019 (Covid-19)-Associated hospitalization surveillance Network (Covid-Net)

Assessing differential impacts of Covid-19 on black communities

National Profile of Local Health Departments. National Association of City and County Health Officials

Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease

Geographic access to United States Sars-Cov-2 testing sites highlights healthcare disparities and may bias transmission estimates

Disparities in the population at risk of severe illness from Covid-19 by race/ethnicity and income

Rapid emergence of Sars-Cov-2 in the greater New York metropolitan area: Geolocation, demographics, positivity rates, and hospitalization for 46 793 persons tested by Northwell health

Cumulative incidence and diagnosis of Sars-Cov-2 infection in New York

A model of disparities: risk factors associated with Covid-19 infection

Health and health determinant metrics for cities: a comparison of county and city-level data

Zip Code Tabulation Area (Zcta) Relationship Files

Available Apis

Urban Area Facts

Risk factors and outcomes of Covid-19 in New York City; a retrospective cohort study

Surgo Ventures, 2020a. The Covid-19 Community Vulnerability Index (Ccvi)

Surgo Ventures, 2020b. Covid-19 Community Vulnerability Index (Ccvi) Methodology

Neighborhood-level Covid-19 hospitalizations and mortality relationships with built environment, active and sedentary travel

Segregation and diversity measures in population distribution

Substantial underestimation of Sars-Cov-2 infection in the United States

The authors have no disclosures to report.

Supplementary data to this article can be found online at https://doi. org/10.1016/j.healthplace.2022.102814.