key: cord-1038835-oseu3k93
authors: Bean, Hamilton; Grevstad, Nels; Meyer, Abigail; Koutsoukos, Alex
title: Exploring whether wireless emergency alerts can help impede the spread of Covid‐19
date: 2021-09-30
journal: Journal of Contingencies and Crisis Management
DOI: 10.1111/1468-5973.12376
sha: a09455b0629bc45d25ad36f87341ee5f1ced637b
doc_id: 1038835
cord_uid: oseu3k93

Officials worldwide have sought ways to effectively use mobile technology to communicate health information to help thwart the spread of Covid‐19. This study offers a preliminary exploration of whether state‐level (N = 6) and local‐level (N = 53) wireless emergency alert (WEA) messages might contribute to impeding the spread of Covid‐19 in the United States. The study compares changes in reported rates of infections and deaths between states and localities that issued WEA messages in March and April of 2020 with states that did not. Small sample sizes and differences in the rates of Covid‐19 spread prohibit robust statistical analysis and detection of clear effect sizes, but estimated effects are generally in the right direction. Combining statistical analysis with preliminary categorization of both WEA message content and social media themes suggests that a positive effect from WEA messages cannot be ruled out.

| 1 Hospitals are nearly overwhelmed. By public health order, masks are required in high transmission areas.

Social gatherings are limited to 10 or fewer (Asmelash & Toropin, 2020, para. 2) .

'Be careful!' The message also warned, and it included an embedded reference hyperlink containing county-level portals for further information.

Even if health conditions do not call for statewide WEA issuance, communities can leverage the geolocation affordances of the WEA system. For example, also in October 2020, Massachusetts officials announced that communities at high risk for Covid-19 transmission would receive WEA messages reminding them about safety rules (Klein & Rosenberg, 2020 Evidence supporting the efficacy of mobile alerting for is now beginning to emerge. While no nationwide mobile alert was issued in the United States, this study explores preliminary evidence that state-level (N = 6) and locality-level (N = 53) Wireless Emergency Alert (WEA) messages might contribute to impeding the spread of Maryland, Michigan, New Mexico, South Carolina, and Florida. Localities refer to territory/ county/municipal areas. Although the U.S. WEA system is used primarily for issuing severe weather warnings (i.e., for tornados, flood, snow squall, etc.), messages are also issued that can help protect lives and property from various types of hazards (fire, industrial accident, drinking water contamination, etc.). As of 2020, 'pandemic' has been added to the list of hazards for which WEA messaged are issued.

Research concerning the use of WEA messages to warn at-risk publics typically focuses on correlations between message attributes (i.e., source, hazard, guidance, timeframe, location, style, and map and URL inclusion) and recipients' interpretations (i.e., comprehension, belief, and personalization) and behavioural intentions and actions (i.e., protective action decision-making and response) (Bean et al., 2016; Casteel & Downing, 2016; Doermann et al., 2020; Kim et al., 2019; Kuligowski & Doermann, 2018; Liu et al., 2017; Sutton & Kuligowski, 2019; Sutton et al., 2018; Wood et al., 2018) . By contrast, this study explores a novel set of correlations: changes in reported rates of Covid-19 infections and deaths between states and localities that issued WEA messages in March and April of 2020 with states that did not. Combining preliminary statistical analysis with an exploration of both WEA message content and social media responses to statewide Orders suggests that WEA messages could play an important role in instructing people to take protective actions that hinder Covid-19 transmission.

In what follows, we situate our exploratory study in the context of public warning and mobile health communication ('mHealth') research. We then describe the purpose and operation of the WEA system. We subsequently describe how the WEA system was used during the onset of the Covid-19 outbreak in the United States.

Following a discussion of our research questions and methods, we offer three preliminary analyses of Covid-19 WEA messages:

(a) statistical model comparisons between states and localities that issued WEA messages in March and April of 2020 and states that did not; (b) comparison of Covid-19 WEA messages with social science best practice for 'complete' messages; and (c) social media themes in response to the issuance of statewide Orders delivered over the WEA system. We conclude with several ideas for 'next steps' stemming from our analyses.

The study of public warning involves understanding why people take protective action in response to information and instruction about hazards and disasters (Drabek, 1986; Lindell & Perry, 2012; Mileti & Sorensen, 1990) . Typically, this research asks disaster survivors to account for when and how they received, understood, and acted upon an alert or warning message, or it asks what stakeholders might do in a hypothetical situation. The results of this research are then used to help officials optimize their public warning systems to increase warning message reception, comprehension, and appropriate response (Ripberger et al., 2019) .

Mobile technology offers new ways to conduct public warning (National Academies of Sciences Engineering and Medicine, 2018).

However, the rapid development and implementation of mobile public warning systems have generally outpaced research concerning their actual benefits, limitations, and efficacy (Bean, 2019) .

Researchers have outlined a theoretical and applied communication research agenda for mobile public warning messages that involve studying (a) how hazard-related information can best be communicated in short messages, (b) how a map or other location-related information might be included, (c) how messages can be configured and disseminated to minimize delay time and maximize personalization, and (d) how contextual and message receiver factors influence mobile public warning reception, comprehension, and response (Bean et al., 2015 ; National Academies of Sciences Engineering and Medicine, 2018; Wood et al., 2018) . These themes have also been taken up in the growing body of literature concerning 'terse' warning messages, a category of risk communication that includes both WEA messages and social media messages (Doermann et al., 2020; Kim et al., 2019; Sutton, Gibson, et al., 2015; .

One research arena associated with mobile public warning research is mHealth (mobile health communication), defined as 'medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices' (World Health Organization, 2011, p. 6). mHealth technology allows for 'place-shift' so that people can receive health messages when they are not able to easily access mass media (e.g., radio, television, or the Internet). mHealth messages can also reach publics when they are most amenable to behaviour change, such as when they need to take an immediate protective action to keep themselves and their loved ones safe from harm. mHealth messages delivered via the WEA system also have the benefit of extensive reach: Most members of the public own mobile devices and typically have them turned on and nearby.

The WEA system was not explicitly designed with a pandemic in mind (Bean, 2019) . Although it is still unclear exactly what type of instructional communication within a WEA message is needed to maximize protective action among diverse audiences during a pandemic, mHealth has demonstrated its effectiveness in other contexts (Gold, 2020; Gurman et al., 2012) . It is important to note, however, that mHealth campaigns typically involve personalized, repetitive, and 'opt in' SMS messages, whereas officials might view cell broadcast WEA messages as an impersonal 'bell ringer' designed to alert people located in a large geographic area (Bean, 2019) .

Adaptively tailoring WEA messages for smaller communities and frequently issuing them is possible, however (see Klein & Rosenberg, 2020) , and a few U.S. communities used the WEA system in this way during the onset of the Covid-19 crisis. To our knowledge, this study is the first to explore the efficacy of mHealth-type messages issued via the WEA system during the 2020 Covid-19 pandemic.

The WEA system is a partnership between FEMA, the Federal Communications Commission (FCC), and the nation's wireless service providers. Launched in 2012, the WEA system is designed to enhance public safety by allowing authorized federal, state, and local officials to send 90-character (recently 360-character), geotargeted, text-like messages to the public's mobile devices during an emergency.

According to the FCC (2020a), the WEA system is an essential part of U.S. emergency preparedness and has been used more than 56,000 times to warn the public about dangerous weather, missing children, and other critical situations. The WEA system is designed to enable officials to send 'imminent threat' alerts, as well as AMBER alerts for missing and abducted children. A third type of alert, 'public safety message', became available for alert originators in July 2019 (related messages include recommendations for saving lives and property). A fourth type of alert, a 'presidential alert', allows the President of the United States to send a message to the entire nation in the event of a catastrophic disaster, such as a nuclear attack. All four alert types involve a text-like message that appears on the screen of the recipient's mobile device, accompanied by an audible attention signal and vibration. WEA messages can be issued in English and Spanish, and they may also include an 'embedded reference' hyperlink for additional information. The first three types of WEA messages can also be 'opted out' of, that is, turned off or blocked on one's mobile device. A presidential alert cannot be blocked. An example of a WEA message is provided in Figure 1 .

WEA messages reach mobile devices using cell broadcast technology that is less likely to become backlogged during times of network congestion, but what makes the WEA system truly unique is its ability to 'push' broadcast alert and warning messages to all mobile devices located in a geographic area specified by an alerting authority.

Typically, an emergency manager can use a WEA system interface to draw a 'polygon' across a designated hazard area on a map rendered on a screen. The coordinates of the polygon are defined using latitude and longitude points. Once the WEA message is issued, cellular towers both within and just outside the polygon broadcast the WEA message to all enabled mobile devices in the designated alert area.

The alert area can, in theory, be a small as a city block or as large as the entire nation.

Because WEA is an opt-out system, iOS and Android devices are sold to users opted-in to all categories of WEA messages by default (some devices also allow users to opt-in to WEA system tests at the local level). It is estimated that 99% of Americans receive wireless service from a provider that voluntarily participates in the WEA system. As of 20 May 2020, scores of emergency management and public safety organizations in each state have earned FEMA's authorization to issue WEA messages, although some states only have a relative handful of authorized alerting authorities. These studies can be grouped under the topics of (a) technical aspects of the WEA system and its integration with other systems, (b) geotargeting, (c) cybersecurity, (d) public education, sentiment, and response, (e) system and message diffusion, and (f) accessibility.

The 2018 National Academies report, 'Emergency Alert and Warning Systems: Current Knowledge and Future Research', provided summaries of many of these studies, which range from broad reviews of prior research to narrow measurements of research participants' heartrates, skin conductance (sweaty palms), and other physiological responses when receiving experimental WEA messages. Most of these research projects were initiated around the same time as the WEA system began nationwide rollout in 2012. A synthesis of the research contained in the National Academies (2018) report is outside the scope of this study, but it is important to note that its authors concluded that 'fairly little is known about how to maximize the effectiveness of messages whose content is limited by technology constraints or policy decisions, or how best to make use of alerts and warnings in today's information-rich environments' (p. vii).

Importantly for this study, Doermann et al. (2020) recently proposed a short message creation tool for wildfire emergencies, which is a type of rapid WEA message generator. The authors reviewed 33 research publications regarding short alert message best practices to develop evidence-based guidance and a tool that officials can use to rapidly create informative and effective 360-character wildfire evacuation WEA messages. Information extracted from each research publication reviewed included: the topic, objectives, methods, findings, and recommendations. The authors related each information item to the Protective Action Decision Model (Lindell & Perry, 2012) to develop a tool that included five essential categories: message source, hazard identification, hazard location, timeframe, and guidance (sixth category, 'general', was also included, but not in relation to the research literature). Researchers have consistently demonstrated that these five categories are vital for public sensemaking and response irrespective of hazard type (National Academies of Sciences Engineering and Medicine, 2018). Doermann et al. (2020) claimed that issuing a 'complete' warning message makes it 'likely that even more people will take appropriate protective actions, and they will take them sooner (i.e., with less milling behaviour)' (p. 8). While wildfire and Covid-19 are extremely different hazards, they share the need for the public to take specific and timely protective action to reduce risks and save lives. Therefore, in this study, we assessed Covid-19 WEA messages for their 'completeness', that is, their inclusion of the five essential categories of information.

Finally, due to their forced reception, WEA messages can be seen as a privacy invasion, and missteps with the WEA system have generated a public backlash in some communities (Bean, 2019) . We therefore assessed public reactions to the issuance of statewide Orders via the WEA system through preliminary scrutiny of social media themes (Facebook and Twitter). Our aim was to merely determine which themes were salient, as well as how these themes might reflect (or not) prior WEA research that has indicated that short messages (90-and 280-characters) can spark confusion and anxiety among recipients (Bean et al., 2015; Wood et al., 2018) .

On 23 March 2020, Washington Post contributing reporter Dan Stillman published an article, 'My Cellphone Should Have Buzzed with a Coronavirus Emergency Alert' (Stillman, 2020 

Integrating research concerning public warning, mHealth, and the WEA system, noted above, we generated the seven research questions below (grouped by theme). To repeat, this study tends to correlate WEA message issuance with recipients' interpretations (i.e., comprehension, belief, and personalization) and behavioural intentions and actions (i.e., protective action decision-making and response) (Bean et al., 2016; Casteel & Downing, 2016; Doermann et al., 2020; Kim et al., 2019; Kuligowski & Doermann, 2018; Liu et al., 2017; Sutton & Kuligowski, 2019; Sutton et al., 2018; Wood et al., 2018) . It is important to note that the public warning, mHealth, and the WEA system literature does not anticipate correlating WEA message issuance with disease case rates and death rates, as we do herein.

Because some states issued state-wide Orders, but not statewide WEA messages, we compared states that issued statewide WEA messages with states that did not issue WEA messages but did issue statewide Orders. Also, because some states issued neither statewide Orders nor WEA messages, we compared the states that issued statewide WEA messages with states that issued neither Orders nor WEA messages. For non-WEA issuing states, we marked day zero as the date the U.S. National Emergency was declared: 13 March 2020.

A handful of U.S. localities issued WEA messages (sometimes multiple messages) in March and April 2020. Issuance occurred in states that both did and did not issue statewide Orders or WEA messages. We compared county-level per capita increases in rates of Covid-19 cases and deaths 30 days and 60 days after the first WEA message issuance with states that issued no WEA messages. We recognize that we are comparing counties with states, but we do not have a rationale for picking non-WEA message-issuing counties for comparison. We wanted to see whether Covid-19 case and death rates were lower, per capita, in counties that issued WEA messages compared with per capita cases and deaths in states that did not.

As mentioned above, recent studies have compared WEA-type messages to social science best practices for short public warning messages (Doermann et al., 2020; Sutton & Kuligowski, 2019) . These studies suggest that 'complete' warning messages may generate better protective action outcomes because recipients will not need to 'mill' for additional and confirming information. We, therefore, asked two research questions related to localities. To gain a better understanding of how WEA message recipients reacted to Covid-19 messages, we inductively analysed Facebook comments and Twitter posts to identify broad social media themes following the issuance of statewide Orders via WEA messages.

To obtain data for this study, the first author contacted FEMA on 26

April 2020, about its Covid-19 response efforts and was provided with a spreadsheet summary of Covid-19 related WEA messages to date (produced for internal tracking purposes). The first author was informed that further updates to the spreadsheet would not be available. The FEMA spreadsheet included the WEA issuance data used in this study (available from the corresponding author upon request were modelled via an exponential growth model (Bertozzi et al., 2020) ,

where I(t) is the cumulative number of cases (or deaths) at time t, I 0 is the initial number of cases (or deaths) at time t = 0 (the date of WEA issuance), e is the exponential constant, and α is the rate constant at which individuals infect others (or at which cumulative deaths grow).

A smaller value of α corresponds to a slower rate of transmission (or growth in deaths). The actual rate at which the cumulative number of cases (or deaths) increases, however, also depends on the value of I 0 .

A larger value of I 0 results in much faster growth in the number of cases (or deaths). Intuitively, the more people that are infected, the faster the number of infections grows. Thus, even if one state's transmission rate α is lower than another's, for example, due to Order or WEA issuance, it can have a substantially higher cumulative number of cases (or deaths) at t = 30 or 60 days if it's starting number I 0 was higher. The doubling time is the time it takes to double the number of cumulative infections (or deaths) and is a common measure of how fast the contagion spreads: if we start with I̅ infections, then at time

we achieve I 2 × ̅ infections. A smaller value of α corresponds to a longer doubling time. The doubling time does not depend on the value of I 0 .

Because Model (1) is parsimonious, it is well suited to developing policy-relevant insights into the pandemic (Bertozzi et al., 2020) . To investigate whether WEAs lower the transmission rate α (or slow the growth rate of deaths) and lengthen the doubling time T d , we modelled α as a function of whether or not a state (or county) issued a WEA: and counties that issued WEAs and those that did not is beyond the scope of our study. However, we note that the level of outbreak at time t = 0 (in mid-March and early April 2020) was actually slightly higher, on average, for the states and counties that issued WEAs than for those that did not (Tables 1-10) , although the reasons for this are unclear.

In addition to statistical comparisons, all 213 messages contained in the FEMA spreadsheet were analysed for their 'completeness'.

We added columns in the FEMA spreadsheet for the five essential categories of message source, hazard identification, hazard location, timeframe, and guidance (Doermann et al., 2020) , as well as a column for whether or not an 'embedded reference' hyperlink was included.

The fourth author then assessed whether each message included information related to each category. The short length of each message allowed for rapid identification of the presence or absence of the information types. The first author then repeated the fourth author's analysis to ensure accuracy, discussing and/or adjusting a handful of minor discrepancies. An example coded Covid-19 WEA message is given in Figure 2 .

To identify social media themes, in August 2020 the third and fourth authors examined public posts on the official Twitter and

Facebook accounts of five states whose governors issued stay-at- problems (the problem here appears to be message insufficiency, as we discuss below). We, therefore, did not set out to pinpoint mutually exclusive content categories derived from prior research, and we acknowledge that our categorization scheme allows for items to be interpreted differently or in ways that span multiple categories.

T A B L E 4 Estimated initial per capita cases and deaths (at date of issuance), Covid-19 transmission rate, growth rate of deaths, and doubling times in states that did and did not issue wireless emergency alerts (WEAs) (but did issue Orders) However, a significant proportion of the responses had been assigned to an 'unclear' category. We subsequently developed three additional categories to account for these responses ('financial support', 'doubting efficacy', and 'critical of late response'). The third and fourth authors repeated their analysis, assigning each response to one of the nine categories. The first author then repeated the entire analysis to assess its reasonableness, obtaining similar category counts for each state's social media responses. We did not formally establish intercoder reliability for this exploratory study and, therefore, presume that responses could be categorized differently; however, our aim was simply to identify themes within the social media responses that would be apparent to the casual observer, which our generalized approach enabled. We next present the findings of our comparative, 'completeness', and thematic analyses.

6 | COMPARING COVID-19 WEA MESSAGE ISSUANCE AMONG STATES AND LOCALITIES

The six states that issued statewide WEA messages (and Orders) during March or April 2020 had a higher mean number of cases per capita on the date of issuance than the 44 states that did not issue WEAs, but the difference in means narrowed after 30 and 60 days (Table 1 and is higher for WEA-issuing states than for those that did not issue WEAs. per capita on the date of issuance (I 0 ) is higher for WEA-issuing states than for those that did not issue WEAs.

The six states that issued statewide WEA messages (and Orders) during March or April 2020 had a higher mean number of cases per F I G U R E 5 Number of cases (left) and deaths (right) per 100,000 residents at 0, 30 and 60 days after issuance of wireless emergency alerts (WEAs) for states that did and did not issue WEAs (but did issue Orders). Red and green lines are individual states. Black lines are the observed means (from Table 3 ) [Color figure can be viewed at wileyonlinelibrary.com] capita on the date of issuance than the 36 states that issued statewide Orders but not WEA messages but had a lower mean after 30 and 60 days (Table 3 and Figure 5 ). Those six states had a lower mean number of deaths per capita on the date of issuance than the states that issued Orders but not WEAs, but they had a slightly higher mean at 30 and 60 days. 

The six states that issued statewide WEA messages (and Orders) during March or April 2020 had a substantially higher mean number of cases per capita on the date of issuance than the eight states that issued neither Orders nor WEA messages, and also had a higher mean after 30 and 60 days (Table 5 and (Table 6 and Figure 8) , and the effect is F I G U R E 6 Number of cases (left) and deaths (right) per 100,000 residents at 0, 30 and 60 days after issuance of wireless emergency alerts (WEAs) for states that did and did not issue WEAs (but did issue Orders). Red and green points are individual states. The red and green curves show the fitted exponential growth model I(t) for states that did and did not issue WEAs (but did issue Orders). The grey dashed curve shows the expected exponential growth trajectory for states that issued WEAs if their transmission rates (or growth rates for deaths) had been the same as those of states that did not issue WEAs (but did issue Orders) (i.e., if Δα had been 0) [Color figure can be viewed at wileyonlinelibrary.com] BEAN ET AL.

| 11 statistically significant (z = −4.284, p = .000). The estimated mean number of deaths per capita on the date of issuance (I 0 ) is substantially higher for states that issued both WEAs and Orders than for those that issued neither.

The 53 localities that issued WEA messages during March or April 2020 had a substantially higher mean number of cases per capita on the date of issuance than the 24 states that issued no statewide WEA messages, nor issued any county-or municipal-level WEA messages, but they had a lower mean after 30 and 60 days (Table 7 and Figure 9 ). Those 53 counties had a mean number of deaths per capita on the date of issuance that was more than double the mean for the 24 states that issued no WEA messages but had a considerably lower mean than those states at 30 and 60 days. (Table 8 and Figure 10 ), although the effect is not statistically significant (z = −1.46, p = .072). The estimated mean number of deaths per capita on the date of issuance (I 0 ) is slightly higher for WEA-issuing counties than for states that did not issue WEAs.

The four communities that issued 'complete' WEA messages during March or April 2020 had a higher mean number of cases per capita on the date of issuance than the 24 states that issued no statewide WEA messages, nor issued any county-or municipal-level WEA messages, but they had a considerably lower mean at 30 and 60 days (Table 9 and Figure 11 ). Those four communities had a lower mean number of deaths per capita on the date of issuance as well as 30 and 60 days afterward.

The estimated Covid-19 transmission rate parameters are F I G U R E 7 Number of cases (left) and deaths (right) per 100,000 residents at 0, 30 and 60 days after issuance of wireless emergency alerts (WEAs) for states that issued both WEAs and Orders and states that issued neither. Red and green lines are individual states. Black lines are the observed means (from Table 5 ) [Color figure can be viewed at wileyonlinelibrary.com]

The estimated mean number of cases per capita on the date of issuance (I 0 ) is higher for 'complete' WEA-issuing counties than for states that did not issue WEAs.

The estimated growth rate parameters for deaths are α 0 = .0460 (SE = 0.0023) and Δα = −.0053 (SE = 0.0075). The negative sign of Δα suggests a slower growth rate and longer doubling time when 'complete' WEAs are issued (Table 10 and Figure 12 ), although the effect is not statistically significant (z = −0.71, p = .239). The estimated mean number of deaths per capita on the date of issuance (I 0 ) is lower for 'complete' WEA-issuing counties than for states that did not issue WEAs.

In sum, for all five research questions Q1-Q5, there is some evidence that WEA messages might be effective in lowering

Covid-19 transmission rates and growth rates in cumulative deaths, but the observed effects are not statistically significant except in conjunction with the effect of an Order (Q3). The nonsignificance of statistical test results may be explained by relatively small sample sizes, especially for Q5.

The six states that issued statewide WEA messages during March or April 2020 saw markedly lower Covid-19 transmission rates and, to a lesser extent, slower growth rates in deaths after message issuance than the 44 states that did not issue WEA messages (Q1).

Those observed effects diminish, however, when the non-Orderissuing states are removed from the comparison group. In this case, the six states that issued WEA messages (and Orders) saw only slightly lower transmission rates and only slightly slower growth rates in deaths than the 36 states that issued statewide Orders but not WEA messages (Q2).

On the contrary, the observed effects of WEAs increase substantially, and become statistically significant, when the Order-issuing states are removed from the comparison group. In this case, the six states that issued WEA messages (and Orders) saw dramatically lower transmission rates than the eight states that issued neither statewide Orders nor WEAs, and they also saw dramatically slower growth rates in deaths (Q3).

The 53 counties that issued WEA messages during March or April 2020 saw lower Covid-19 transmission rates and, to a greater extent, slower growth rates in deaths after message issuance than the 24 states that issued neither statewide nor non-statewide WEA messages (Q4).

The effects of 'complete' countywide WEA messages were even more pronounced. The four counties that issued 'complete' WEA messages during March or April 2020 saw considerably lower transmission rates and somewhat slower growth rates in deaths after F I G U R E 8 Number of cases (left) and deaths (right) per 100,000 residents at 0, 30 and 60 days after issuance of wireless emergency alerts (WEAs) for states that issued both WEAs and Orders and states that issued neither. Red and green points are individual states. The red and green curves show the fitted exponential growth model I(t) for states that issued both WEAs and Orders and states that issued neither. The grey dashed curve shows the expected exponential growth trajectory for states that issued both WEAs and Orders if their transmission rates (or growth rates for deaths) had been the same as those of states that issued neither ( . Several messages were "healthcare professionals needed" requests, one was alerting the county that the courthouse's procedures were changing (appointment only), one was a warning about disinformation, and two were blank in the FEMA spreadsheet. (Perrin & Turner, 2019) . More public attention to mobile technology inequities is needed. As sociotechnological changes unfold, we must consider that some people will continue to neither own nor have access to mobile technology, which creates profound and unacceptable inequalities in terms of safety and security. Likewise, we must also consider how differences in the capabilities of mobile devices (e.g., display settings) and wireless networks (e.g., connectivity)

can create similar inequalities (Bennett & LaForce, 2019) . The potential usefulness of mobile technology in combatting Covid-19 transmission indicates that these issues should not be ignored.

F I G U R E 12 Number of cases (left) and deaths (right) per 100,000 residents at 0, 30 and 60 days after issuance of wireless emergency alerts (WEAs) for counties that issued 'complete' WEAs and states that did not issue WEAs. Red and green points are individual counties or states. The red and green curves show the fitted exponential growth model I(t) for counties that issued 'complete' WEAs and states that did not issue WEAs. The grey dashed curve shows the expected exponential growth trajectory for counties that issued 'complete' WEAs if their transmission rates (or growth rates for deaths) had been the same as those of states that did not issue WEAs ( Second, studies of WEA message efficacy are exceedingly rare.

As the use of the WEA system expands, health and safety outcome comparisons among groups of people who did and did not receive WEA messages should be conducted. Demonstrating the efficacy of WEA messages across hazards is an important evolution in understanding how to maximize the benefits of mobile technology (National Academies of Sciences Engineering and Medicine, 2018).

However, to repeat, WEA messages do not exist in a vacuum, so parsing out their unique influence within a given communication ecology will be extremely difficult.

Third, WEA messages for all types of hazards and disasters should strive for completeness to reduce milling behaviour among message recipients . Importantly, this study found that 'seeking clarity' was a salient social media theme in response to statewide Orders issued via the WEA system; yet, the embedded reference hyperlink capability of WEA messages was underused.

Roughly 50% of the messages analysed contained them. Embedded reference inclusion in every message would allow officials to offer (or reinforce) instructional guidance and rapidly provide communities updated information. Of course, hazard type influences the kinds of information that can be included in an embedded reference, but officials should not rule out using this affordance of the WEA system.

Finally, there are other factors not explored in this study that may help account for its findings. This study was limited by small sample sizes. The difficulty of determining effect sizes within different geographical areas experiencing different rates of spread, case reporting, and deaths is clear. We did not assess the influence of repeated WEA issuance, and repetition could have affected compliance with a statewide order or protective action guidance. We also did not have access to mobility data, which might have shown whether WEA issuance correlated with reductions in people's movements (see Fowler, 2020) . It may be impossible to delimit a causal, law-like relationship between WEA message issuance and Covid-19 outcomes. Nevertheless, the evidence presented herein suggests that a positive effect cannot be ruled out. Complete WEA messages appear to be correlated with better health outcomes in the communities that issued them, but much more evidence is needed to rule out other factors or mere coincidence. Despite this uncertainty, social media responses reveal that many users sought clarity about the meaning or implications of statewide Orders delivered via the WEA system, which bolsters prior research that has warned of the insufficiency of short WEA messages (Bean et al., 2015; Wood et al., 2018) . To help identify more factors that might explain, support, or challenge our findings, we urge researchers to conduct similar and expanded studies as more communities (both in the United States and internationally) issue WEA, WEA-like, or SMS messages in response to the Covid-19 pandemic.

All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request:

FEMA Covid-19 WEA message spreadsheet; statistical models in R, and message completeness tabulations.

Utah sent every phone in the state an emergency alert warning about rapidly rising Covid-19 cases and overwhelmed hospitals

Fitting linear mixedeffects models using lme4

Mobile technology and the transformation of public alert and warning

Disaster warnings in your pocket: How audiences interpret mobile alerts for an unfamiliar hazard

The study of mobile public warning messages: A research review and agenda

Text-to-action: Understanding the interaction between accessibility of Wireless Emergency Alerts and behavioral response, Risk Communication and Community Resilience

The challenges of modeling and forecasting the spread of COVID-19

Assessing risk following a wireless emergency alert: Are 90 characters enough

Public health messages about COVID-19 prevention in multilingual Taiwan

From social science research to engineering practice: Development of a short message creation tool for wildfire emergencies

Human system responses to disaster: An inventory of sociological findings

Public notice: Enhanced wireless emergency alerts available for coronavirus pandemic

Rhetorical criticism: Exploration and practice

Smartphone data reveal which Americans are social distancing (and not)

How governments are using text alerts to fight the coronavirus pandemic

Effectiveness of mHealth behavior change communication interventions in developing countries: A systematic review of the literature

Wireless emergency alert messages: Influences on protective action behaviour

Mass. using new tool in COVID fight: Phone alerts for high-risk communities

A review of public response to short message alerts under imminent threat

The protective action decision model: Theoretical modifications and additional evidence

Is a picture worth a thousand words? The effects of maps and warning messages on how publics respond to disaster information

COVID-19: New Zealand sends emergency alert to citizens, says 'We Are Depending On You

Coronavirus: 5 things New Zealand got right

Communication of emergency public warnings: A social science perspective and state-of-the-art assessment. Oak Ridge National Laboratory Rep

Smartphones help Blacks, Hispanics bridge some-but not all-digital gaps with whites

Measuring tornado warning reception, comprehension, and response in the United States

My cellphone should have buzzed with a coronavirus emergency alert

A cross-hazard analysis of terse message retransmission on Twitter

Alerts and warnings on short messaging channels: Guidance from an expert panel process

Terse messaging and public health in the midst of natural disasters: The case of the Boulder floods

Designing effective tsunami messages: Examining the role of short messages and fear in warning response

Qualitative research methods: Collecting evidence, crafting analysis, communicating impact

Milling and public warnings

World Health Organization. World Health Organization