key: cord-0796137-id7nhhmy
authors: Li, Y.; Li, M.; Rice, M.; Su, Y.; Yang, C.
title: Phased implementation of COVID-19 vaccination: rapid assessment of policy adoption, reach and effectiveness to protect the most vulnerable in the US
date: 2021-02-23
journal: nan
DOI: 10.1101/2021.02.19.21252118
sha: 0eb5aae614fe8b095bc559bd851846d7664a2ce0
doc_id: 796137
cord_uid: id7nhhmy

Objectives: The US and rest of the world have suffered from the COVID-19 pandemic for over a year. The high transmissibility and severity of this virus have provoked governments to adopt a variety of mitigation strategies. Some of these previous measures, such as social distancing and mask mandates, were effective in reducing the case growth rate yet became economically and administratively difficult to enforce as the pandemic continued. In late December 2020, COVID-19 vaccines were first approved in the US and states began a phased implementation of COVID-19 vaccination. However, there is limited quantitative evidence regarding the effectiveness of the phased COVID-19 vaccination. This study aims to provide a rapid assessment of the adoption, reach, and effectiveness of the phased implementation of COVID-19 vaccination. Methods: We utilize an event-study analysis to evaluate the effect of vaccination on the state-level, daily COVID-19 case growth rate. Results: Through this analysis, we assert that vaccination is effective in reducing the spread of COVID-19 shortly after the first shots were given. Specifically, the case growth rate declined by 0.124, 0.347, 0.345, 0.464, 0.490, and 0.756 percentage points corresponding to the 1-5, 6-10, 11-15, 16-20, 21-25, and 26 or more day periods after the initial shots. Conclusions: The findings could be insightful for policymakers as they work to optimize vaccine distribution in later phases, and also for the public as the COVID-19 related health risk is a contentious issue.

In late December 2019, a pneumonia-like illness was reported in Wuhan, China, which came to be known as COVID-19. In the US, the first case was documented in late January in Washington state. By mid-March, all 50 states had reported cases and by mid-April the US had the most COVID-19 deaths in the world (Kantis et al., 2020) . To combat the rapidly growing pandemic, many states implemented lockdown measures which coincided with an economic downturn (IMF, 2020) . Therefore, despite growing cases, lockdowns were eased in many states by April (Washington Post Staff, 2020) . Additionally, there has been a decline in compliance towards social distancing measures which are often voluntary and hard to enforce (Boseley, 2020) . As lockdowns are generally not economically sustainable, alternative methods to control the pandemic have received growing attention. Accordingly, many research groups have worked on developing a COVID-19 vaccine. The Centers for Disease Control and Prevention (CDC) has stated that vaccines are vital in tandem with other social distancing measures. In the US, many companies started developing COVID-19 vaccines in March (HHS, 2020) . In December, the Food and Drug Administration (FDA) authorized the emergency use of both the Pfizer-BioNTech and Moderna vaccines. Despite their notoriously rapid development, clinical trials have found both vaccines to be more than 90% effective with only a few, minor side effects (Luthi, 2020) .

Once the vaccine efficacy was thoroughly tested, the government started planning distribution efforts through a program known as Operation Warp Speed (HHS, 2020). The first round of vaccines was delivered to states, territories, and some federal agencies in late December (Silberner, 2020; Ivory et al., 2021) . Ultimately, the states are in charge of the local administration of vaccines. However, as the supply is currently limited the Advisory Committee on Immunization Practices (ACIP) has suggested taking a phased approach to vaccination (CDC, 2020) . This approach was made in an attempt to decrease deaths and serious illnesses while preserving a functioning society and reducing the burden on those who have been disproportionately harmed by the virus (CDC, 2021) . The federal government anticipates vaccinating 100 million people in the US by the end of February 2021. However, the vaccine rollout has been slow (Silberner, 2020) likely due to the limited supply and inefficient distribution (CDC, 2020; Robbins et al., 2020; Romero et al., 2021) . It is important to rapidly assess the phased vaccination approach in terms of adoption, reach, and effectiveness, despite this low coverage, in order to better plan for vaccination later on. Indeed, an evaluation of the pandemic in California near the end of January 2021 noted that cases (The COVID Tracking Project, 2021) and hospitalizations were starting to decline at the same time vaccination coverage was increasing (Los Angeles Times Staff, 2021) . Similar to social distancing policies, the COVID-19 vaccination has become a contentious issue especially as there is limited direct evidence regarding the effectiveness of vaccination. Many people who protested early pandemic measures, such as stay at home orders and mask mandates, are likewise protesting mandatory vaccines (Luthi, 2020) . This opposition may be due to firm beliefs in medical autonomy, vaccine conspiracy theories (Ball, 2020) , concerns about vaccine side effects, mistrust of the government (Dwyer, 2020) , and mistrust of healthcare (Coustasse et al., 2020) . However, COVID-19 vaccine opinions and knowledge have varied overtime. There has been a noticeable increase in the public's willingness to take the vaccine as it has moved into later stages of development (Luthi, 2020; Dwyer, 2020) . Increasing the public trust in COVID-19 vaccines is critical as many experts predict that a vaccination rate between 70% and 80% is needed for herd immunity (Maragakis, 2021) . As manufacturing and distribution become more systematic, it will become more important to eliminate other barriers to widespread vaccination efforts such as public opposition.

In order to increase the public's willingness to take the vaccine, some studies have recommended utilizing public information campaigns, financial investments, and other measures to highlight the effectiveness, benefits, and safety of the vaccine (Paltiel et al., 2020) .

Furthermore, other studies have emphasized the importance of making sure these campaigns receive high coverage and are spread to the public quickly, as the pandemic is an urgent issue (Buonomo et al., 2020) . Without sufficient knowledge, it is often hard to persuade people to accept recommendations (Wolfson, 2020) . For example, mask mandates ultimately gained more widespread acceptance once thorough research highlighted their success (Lyu et al., 2020) .

Indeed, as the vaccine supply increases, distribution will move towards phases that include a broader range of people who are eligible for vaccination (CDC, 2020) . Therefore, evaluating the . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 23, 2021. ;  effectiveness of vaccination in controlling the spread of COVID-19 is important to ensure a smooth transition between phases by increasing vaccination acceptance and coverage.

Currently, there is limited research done to quantitatively evaluate the effectiveness of the phased COVID-19 vaccination approach. Through a natural experiment, we investigate how statewide vaccination efforts affect the spread of COVID-19 in the US during the early stages of vaccination. Specifically, we use an event-study analysis to estimate the effects of state vaccinations on the state-level confirmed case growth rate over different time periods.

Vaccination may play a vital role as states are reopening and social distancing policies are becoming less stringent (Boseley, 2020; Hills, 2021) . It is necessary to do a rapid assessment of vaccination effectiveness during the initial stages of implementation as concrete evidence of success and safety is important to inform policymakers who direct administration efforts, and the public in order to achieve widespread vaccination.

Records detailing when the COVID-19 vaccine was first administered in every state were gathered in this study by searching and reviewing the public news and governmental pages. Our study focuses on state-level COVID-19 vaccination beginning with the first shot which was given in New York on December 14, 2020. Following the CDC guidelines, frontline healthcare workers and long-term care facility staff and residents were given the highest priority in receiving the vaccine in phase one (CDC, 2020) . However, the vaccination process, including the definition of groups given priority, have varied among the states. Detailed information on the adoption of vaccination in different US states can be found in the result section.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ;

In addition, we collected vaccination data from October 1, 2020 to January 26, 2021. A repository shared by the Johns Hopkins University Centers for Civic Impact COVID-19 data analysis and visualizations (https://github.com/govex/COVID-19) contains state-level reports which record the daily cumulative number of doses administered to the public. Specifically, we divided the cumulative number of doses administered by a state's population size to find the daily ratio of doses administered.

Statewide policy data was produced by the NSF spatiotemporal innovation center based on the Oxford COVID-19 Government Response Tracker project schema (Hale et al., 2020) .

and their implementation overtime. These policies include school closures, workplace closures, the cancellation of public events, restrictions on gatherings, public transport closures, stay at home orders, internal movement restrictions, and international travel controls. The extent of government action is reflected using a stringency index ranging from 0 to 100; the larger the value, the more stringent a certain policy measure is. The entire dataset can be found on Github (https://github.com/OxCGRT/covid-policy-tracker). A lot of research has found that social distancing policies and mask mandates affect the growth rate of COVID-19 cases (Courtemanche et al., 2020; Lyu et al., 2020) , thus, the stringency index was included in this study to control for the effects of policy stringency during the study period.

The NSF spatiotemporal center provided a data repository detailing daily COVID-19 data Yang et al., 2020) . Specifically, our study used the US state-level confirmed . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.19.21252118 doi: medRxiv preprint case data. This level of data was used as most vaccination plans were designed and implemented by individual states (Silberner, 2020; Ivory et al., 2021) . Only confirmed case data was used in this research since we focused on the impact of vaccines on the spread of COVID-19. The daily confirmed case data was gathered from this Github repository, (https://github.com/stccenter/COVID-19-Data/tree/master/US), and daily growth rates of confirmed cases were derived from the original dataset as introduced in the methodology section.

(1) Vaccination State-level vaccination data was encoded as binary data, with 0 indicating that the COVID-19 vaccine had not been administered in a certain state and 1 indicating that people had started to take the vaccine in that state. To assess the vaccination effects, a reference group representing the 1-5 days before the start of vaccination was selected. Additionally, we analyzed the effects of the vaccination over six post-periods (1-5 days, 6-10 days, 11-15 days, 16-20 days, 21-25 days, and 26+ days)-which represented days after the first shot of the COVID-19 vaccine was administered, and four pre-event trends (6-10 days, 11-15 days, 16-20 days, and 20+ days)-which represented days before the vaccination started. Multiple 0-1 indicators were generated to indicate these pre-and post-periods. Each day in the analytical period was assigned with one indicator by comparing t to the time range each indicator represented. Here, t is the cumulative number of days since the first shot of the vaccine was administered in a certain state.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10. 1101 (2) Confirmed case growth rate

To estimate the effect of vaccination, we calculated the daily, state-level confirmed case growth rate, which is the difference between the natural logarithms of the cumulative counts of confirmed cases on a given day and that of the prior day. This difference was then multiplied by 100 to enable the daily growth rate to be given in percentage points and so that the estimated regression coefficients could be interpreted as percentage point changes in the growth rate.

(1)

In this study, we employed a fixed effects panel regression model incorporating the event-study design to examine the effects of statewide phased COVID-19 vaccination on the spread of COVID-19 in the US. The differences between the availability of vaccines before and after the initial vaccination in the states enabled estimation in the context of a natural experiment.

We examined if the trend in the COVID-19 confirmed case growth rate was differentiated before and after the start of vaccination. The response variable in the model was the confirmed case growth rate as defined in Eq (1). The effect of periods of interest was defined in the Vaccination section (the six post periods after the first shot of the COVID-19 vaccine were coded

, the four pre-event periods were coded

, and the reference period was

). In addition to the differences in vaccination efforts, we also included the test growth rate, the policy stringency index (Hale et al., 2020) , and the dose administrative ratio as control variables.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ;

In the above regression specification, 

Vaccination policy adoption . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10. 1101 Vaccination efforts were guided at the state level. All states started the vaccination of priority groups by December 16; however, some states initiated their vaccine administration quicker than others. The collected information shows that about half of all states administered their first shot of the COVID-19 vaccine on December 14, 2020, with the overall first shot in the US being given in New York on the same day. Most other states started to vaccinate priority groups in the following days. Detailed information of the dates and news sources can be found in the appendix. The delivery of vaccinations to the public was implemented in settings such as healthcare facilities, stadiums, and pharmacies, which received doses from the state (Gold, 2021) . Since the supply of vaccines is limited, the ACIP suggests offering vaccination in a phased approach (CDC, 2020) . This approach gives vulnerable populations high priority. According to the recommendation, Phase 1a includes healthcare personnel and long-term care facility residents.

Phase 1b includes persons ≥ 75 years of age and frontline essential workers. Phase 1c includes . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.19.21252118 doi: medRxiv preprint persons 65-74 years of age, persons 16-64 years of age with high-risk medical conditions, and other essential workers. However, as distribution was left up to individual states, many phases were defined slightly differently. The CDC phase definition that most closely aligned to the state's most recently vaccinated populations (as of January 26, 2021) was used to categorize the state in Table 2 , which represents the state-level implementation of vaccination. Information regarding each state's specific vaccination phases were gathered from USAFacts (https://usafacts.org/visualizations/covid-vaccine-tracker-states/#). The states also show variation in their vaccination progress. Although all states are still in phase 1 as of January 26, some states have already entered phase 1c while others are still in phase 1a. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 Arkansas, for example, had low numbers of administered doses during the whole period.

Overtime, Texas and Florida likely became more efficient in their vaccination efforts.

The attempts of each state to reach the public can also be seen in Figure 2 , which shows the percentage of the population, in each US state who received at least 1 vaccine dose by the same four dates described above. Throughout the analysis period, Texas and Florida were able to notably increase the percentage of their populations who received at least 1 dose of the vaccine.

West Virginia also saw large increases in this population percentage and has been commended for its vaccination efforts (Anderson et al., 2021) . In terms of vaccinations, some states have been more effective in reaching the public as seen by changes in the number of doses administered and the percent of the population with at least 1 vaccine dose overtime. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.19.21252118 doi: medRxiv preprint Figure 2 : Percentage of the population who received at least 1 dose in the study period. Figure 3 displays the effects of statewide COVID-19 vaccination on the state-level daily confirmed case growth rates based on the collected data. In addition, change in the daily growth rate percentage for the six post-periods and the four pre-event periods with respect to the reference period can also be visualized in Figure 3 . The red vertical line in Figure 3 represents the reference period and the black square symbols describe the point estimates (along with their 95% confidence intervals) of the vaccination effect over different time periods. The figure shows that there were significant decreases in the daily COVID-19 case growth rate after the initial shots of the vaccine, with 0.124, 0.347, 0.345, 0.464, 0.490, and 0.756 percentage point declines in the 1-5, 6-10, 11-15, 16-20, 21-25, and 26 or more day post-periods after the start of vaccination.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101/2021.02.19.21252118 doi: medRxiv preprint Figure 3 : Event-study estimates of the effects of state vaccination on the daily state-level growth rate of COVID-19 confirmed cases.

To verify the robustness of the estimates from the regression model, robustness checks were performed using different controls and samples. The main results column in Table 1 shows the effects of vaccination on the case growth rate with controlling for the Christmas holidays and pre-vaccination trends as explained previously. Again, this is the main regression we adopted, of which the average vaccine effects were plotted in Figure 3 . Three other versions of the model were run-controlling for the Christmas holidays, controlling for the pre-vaccination trend, and . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10. 1101 without controlling for the Christmas holidays, as shown in Column 2, 3 and 4 respectively.

These results showed similar effects on the case growth rate as seen with the main result.

Columns 5-7 are robustness checks using the main regression specification but with some states excluded. Alabama was excluded in Column 5 because it has one of the lowest vaccination rates, while West Virginia was excluded in Column 6 because it has one of the highest vaccination rates. New Mexico, Wisconsin, Delaware, Tennessee, and Alaska were excluded from the test in Column 7 as these states represented a wide range in their percentage of distributed vaccines that have been administered (Adams et al., 2021) . The results are robust in regard to these exclusions as they display similar results to what was seen in the main analysis. is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10.1101 https://doi.org/10. /2021 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted February 23, 2021. ; each state reached the public. For example, West Virginia increased vaccine administration dramatically throughout this analysis period. Successful vaccination in West Virginia, was partly attributed to the use of a network of pharmacies which facilitated local distribution (Anderson et al., 2021) . Indeed, vaccination allocation may play a role in reaching different communities (Baumgartner et al., 2020) . Additionally, as vaccination is a contentious issue and many people are skeptical of the vaccine due to a lack of quantitative information highlighting its effectiveness, many experts have recommended utilizing public information campaigns and media outlets to help spread information. These sources are vital in highlighting the safety and effectiveness of the vaccine to the public (Baumgartner et al., 2020; Wakefield et al., 2010) .

Making sure mitigation measures reach the public is an essential step in the ultimate goal of widespread vaccination and the fight against COVID-19. Other states can learn from these successful implementation strategies in order to reach a larger percentage of the public in later phases.

Our study contributes direct, empirical evidence, which highlights the effectiveness of the COVID-19 vaccination on the case growth rate. Overall, vaccination has been successful in decreasing the case growth rate during the early stages of vaccination despite the initial limited supply and slow rollout. Furthermore, the decreases increased in magnitude overtime. Although the vaccine is still in the early stages of administration. This information could be used to convince the public that vaccination is worthwhile and can help policymakers optimize implementation strategies.

Despite these promising results, this study was limited by the available data. For example, the real number of COVID-19 cases is likely much higher than the totals being reported (Beusekon, 2021) . Future work may also investigate possible consequences of the vaccine using . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 23, 2021. ;  death case data which can better represent the severity of the virus. Similarly, the model used in this analysis failed to control for factors such as mask use and seasonality due to data availability. The vaccination approach and system will shift when more COVID-19 vaccines are approved by the FDA for regular distribution and supplies become more readily available (Means et al., 2020) . Future research is warranted to estimate the reach, effectiveness, adoption, implementation, and maintenance of regular COVID-19 vaccination.

This study found that the phased vaccination approach helped reduce the daily, state-level COVID-19 case growth rates in the US. Despite initially limited vaccine availability, the CDC guidelines regarding the phased vaccination approach has helped protect the most vulnerable from COVID-19. Even though the COVID-19 vaccine is still in the early stages of distribution, it has already been associated with significant decreases in the case growth rate. This is promising for the subsequent phases of vaccination. Overall, the results presented in this research have implications in regard to general public vaccine concerns. For example, our assertion that the COVID-19 vaccine is effective, even in the initial stage, may help persuade skeptical vaccine critics that this mitigation measure is worth it. This information is vital as states are reopening and social distancing measures are becoming less feasible both administratively and economically.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted February 23, 2021. ; https://doi.org/10. 1101 

States ranked by percentage of COVID-19 vaccines administered: Feb

Most successful vaccine rollouts in US: 4 state strategies

Anti-vaccine movement could undermine efforts to end coronavirus pandemic, researchers warn

How Prepared Are States to Vaccinate the Public Against COVID-19? Learning from Influenza and H1N1 Vaccination Programs

Study: US COVID cases, deaths far higher than reported

Distancing compliance in decline among young people

Effects of information-dependent vaccination behavior on coronavirus outbreak: insights from a SIRI model

Covid vaccine rollout gives US hope amid variant concerns

CDC. ACIP COVID-19 Vaccines Work Group

Frequently Asked Questions about Vaccination

Interim Considerations for Phased Implementation of COVID-19 Vaccination and Sub-Prioritization Among Recommended Populations

Who Gets Vaccinated First?

Taking the pulse of COVID-19: A spatiotemporal perspective

Strong Social Distancing Measures In The United States Reduced The COVID-19 Growth Rate: Study evaluates the impact of social distancing measures on the growth rate of confirmed COVID-19 cases across the United States

COVID-19 and Vaccine Hesitancy: A Challenge the United States Must Overcome. The Journal of Ambulatory Care Management

The COVID Tracking Project. Average daily new COVID-19 cases per 100k people

Poll: Americans Are Growing Less Reluctant To Take COVID-19 Vaccine

Pharmacies Will Start to Get a Big Infusion of Vaccines

HHS. Fact Sheet: Explaining Operation Warp Speed

Variation in government responses to COVID-19. Blavatnik school of government working paper

Chapter 2: Dissecting the Economic Effects

See How the Vaccine Rollout Is Going in Your State

UPDATED: Timeline of Coronavirus

The Impact of Policy Measures on Human Mobility, COVID-19 Cases, and Mortality in the US: A Spatiotemporal Perspective

Community Use Of Face Masks And COVID-19: Evidence From A Natural Experiment Of State Mandates In The US: Study examines impact on COVID-19 growth rates associated with state government mandates requiring face mask use in public. Health affairs

Vaccine opponents rebrand as rollout of Covid-19 shots looms

Coronavirus Second Wave? Why Cases Increase

Clinical Outcomes Of A COVID-19 Vaccine: Implementation Over Efficacy: Study examines how definitions and thresholds of vaccine efficacy, coupled with different levels of implementation effectiveness and background epidemic severity, translate into outcomes

Here's Why Distribution of Vaccine Is Taking Longer Than Expected

New Pandemic Plight: Hospitals Are Running Out of Vaccines

A Spatiotemporal Data Collection of Viral Cases for COVID-19 Rapid Response. Big Earth Data

What You Need To Know As The First COVID-19 Vaccines Head Your Way

Use of mass media campaigns to change health behaviour. The Lancet

Where states reopened and cases spiked after the U.S. shutdown

COVID Vaccines Appear Safe and Effective, but Key Questions Remain