key: cord-0940767-qi9ske7a
authors: Dave, Dhaval; Friedson, Andrew I.; Hansen, Benjamin; Sabia, Joseph J.
title: Association Between Statewide COVID-19 Lottery Announcements and Vaccinations
date: 2021-10-15
journal: JAMA Health Forum
DOI: 10.1001/jamahealthforum.2021.3117
sha: 37dd724aad7923bb128d8e77796932f1cf18f30c
doc_id: 940767
cord_uid: qi9ske7a

This case-control study assesses if announcements of cash drawings in 19 states were associated with increased vaccine uptake by comparing vaccination trends in states that announced drawings with states that did not using a difference-in-differences framework.

We draw control variables from several sources. The first is daily state-level COVID-19 reported cases. These are publicly available through the New York Times and can be accessed here: https://github.com/nytimes/covid-19-data. For each state we use the seven-day moving average of COVID-19 cases per 1,000 population from 1 week prior as a control for the severity of that state's COVID-19 epidemic (which could influence individual attitudes towards COVID-19 vaccines).

Our second set of control variables come from the SafeGraph Point of Interest (POI) data series. These data are free to academic researchers but must be applied for here: https://www.safegraph.com/academics. The POI data counts cell phone pings at points of interest -specifically businesses, which can be classified by type using six-digit North American Industry Classification System (NAICS) industry codes. For each state date we aggregate up the number of pings at bars and at restaurants. We then use the number of pings per 1,000 population from 1 week prior as a control for general sentiment about infection risk and willingness to engage with others (which could influence individual attitudes towards COVID-19 vaccines).

Our third set of controls accounts for states having state-wide non-cash/in-kind drawings. For these we again use Hassan and Kannapell (2021) and National Governor's Association (2021) as primary sources, and then used Google-based web searches to validate the nature and announcement date of each non-cash drawing.

Our final set of controls accounts for states not having state-wide cash drawings, but having a substantial portion of the population covered by a local cash drawing. We again used Googlebased web searches for each state, county and city sequentially to find cash drawings that meet our above criteria but at the local level. We then used U.S. Census Bureau (2020) estimates of state and county population to calculate the proportion of the state population that was influenced by a local drawing.

Our estimation strategy follows a difference-in-differences approach (see Dimick and Ryan 2014) , which compares states with lotteries to those which did not before and after the announcements. The main advantage of this type of estimation is that it uses a comparison group so that any observed changes in outcomes in states with cash drawings are not interpreted in a vacuum. We estimate the following equation via ordinary least squares,

where is the number of doses of COVID-19 vaccine per 1,000 population reported given in state s on date t.

is the difference-in-differences estimator, a variable which takes on the value of one if state s announced a lottery-style cash drawing on or prior to date t and takes on the value of zero otherwise.

is a set of control variables for each state on each date; they include foot traffic at restaurants and bars per 1,000 state population seven days prior, average new COVID-19 cases per 1,000 population seven days prior, and percent of state population covered by a local cash drawing, and an indicator for statewide non-cash/in-kind lotteries. is a set of state fixed effects, that is a set of dummy variables for each state, which control for date invariant state characteristics (such as demographic makeup of the state). is a set of date fixed effects, that is a set of dummy variables for each day, which control for state invariant date characteristics (such as national policy recommendations from the Centers for Disease Control and Prevention). Where i indexes days relative to the announcement date, so i=0 is the date of the announcement, i=1 is the date after the announcement, i=-1 is the date before the announcement and so forth. is again the number of doses of COVID-19 vaccine per 1,000 population reported given in state s on date t.

is a variable which takes on the value one if the state ever announces a cash drawing, and zero otherwise.

is a variable which takes on a value of one if the observation is for date i relative to announcement and zero otherwise. All other variables are defined as above. Figure 1 plots the estimates and 95% CIs for the coefficients. In the period after announcement ( > 0), these coefficients represent the estimated association between having announced a lottery and the outcome (the incident immunization rate per 1,000) at i days post announcement. Prior to the announcement ( < 0), the coefficients represent the estimated association between announcing a lottery and the outcome at i days prior to announcement. Any estimate that is statistically different from zero during this time period could be considered evidence that states with and without cash drawings had systematically different patterns in the outcome variable leading up to the announcement. This is known empirically as the existence of non-parallel trends in the pre-period. One of the underlying assumptions of valid difference-indifferences estimation is that there are parallel trends between "treated" states (in this case states that announced a cash drawing) and "control" states (in this case states that did not). Thus, the estimates where ( < 0) serve as a test for violations of the parallel trends assumption.

Panel B of Figure 1 repeats estimation of the event-time model using methods developed by Callaway and Sant'Anna (2020) that adjust for bias in difference-in-differences models that may be present when the event being studied occurs at different times in different places and there are heterogeneous dynamic treatment effects (see Goodman-Bacon 2021), as could, in theory, be the case with state cash drawing announcements. Our approach uses counterfactuals for treatment states that never announced or had not yet announced a vaccination lottery ("not yet adopters"). The findings are not sensitive to restricting the counterfactual states to only states that never announced a vaccine lottery over the April 28 th through July 1 st period.

In the Discussion Section we compare the upper bound of our 95% CI of 0.30 to a hypothetical value of 1.22 which would be the size of the estimate we would expect had the state cash drawings generated enough new vaccinations to reach the President's goal of 70% of the adult population with a first vaccine dose by July 4 th . What follows is how we arrived at this number. The calculation is not meant to be an exact calculation, but a ballpark number to demonstrate the magnitude of impact needed and that our upper 95% CI was not close.

For the states that announced, on the day prior to the lottery announcement, the cumulative onedose vaccination rate was 647.9 per 1,000 adults. That implies that 64.8% of the adult population in the treated states had at least one dose, on the day prior to the lottery announcement.

So the pre-announcement 1 dose vaccination rate was 64.8%, and with a mean adult population in these states of 6,420,582 this implies a total number of adults with at least 1 dose was 4,160,537.

The target amount is to have 70% with at least 1 dose, which with the same mean population would imply 4,494,407 people with at least 1 dose. This means that the policy would need to increase first-dose vaccinated adults on average by 333,870 for the average treated state over 33 days (from June 2nd till July 4th).

Our outcome is in terms of the daily incidence rate (per 1000 population). Thus, we divide the required increase in first-dose vaccinated adults (333,870) by the average treated state population, and convert to per 1,000.

(333,870 / 8,280,780)*1,000 = 40.32 This is the total increase in the incidence rate needed by July 4th.

Since our effect is the average effect on daily incidence, we divide 40.32 by the number of days (33 days between June 2nd and July 4th):

40.32 / 33 = 1.22

This is the effect size that would be required to close the gap.

Difference-in-differences with multiple time periods

Methods for evaluating changes in health care policy: the difference-in-differences approach

Difference-in-differences with variation in treatment timing

These are the U.S. states trying lotteries to boost COVID vaccinations. The New York Times

COVID-19 Vaccine Incentives

Population and Housing Unit Estimates Tables

We compiled our list using Hassan and Kannapell (2021) and National Governor's Association (2021) as primary sources, and then used Google-based web searches to validate the nature and announcement date of each cash drawing, eTable1 below details the dates of each announcement.