key: cord-0934850-9fpchesw
authors: Djulbegovic, Benjamin; Weiss, David J.; Hozo, Iztok
title: Evaluation of the U.S. governors' decision when to issue stay‐at‐home orders
date: 2020-08-13
journal: J Eval Clin Pract
DOI: 10.1111/jep.13458
sha: 9c2e3a6e13c31b565d793a360bb5a713d50a0f97
doc_id: 934850
cord_uid: 9fpchesw

RATIONALE, AIMS AND OBJECTIVES: In the United States, the reluctance of the federal government to impose a national stay‐at‐home policy in wake of COVID‐19 pandemic has left the decision of how to achieve social distancing to individual state governors. We hypothesized that in the absence of formal guidelines, the decision to close a state reflects the classic Weber‐Fechner law of psychophysics – the amount by which a stimulus (such as number of cases or deaths) must increase in order to be noticed as a fraction of the intensity of that stimulus. METHODS: On 12 April 2020, we downloaded data from the New York Times database from all 50 states and the District of Columbia; by that time all but 7 states had issued the stay‐at‐home orders. We fitted the Weber‐Fechner logarithmic function by regressing the log(2) of cases and deaths, respectively, against the daily counts. We also conducted Cox regression analysis to determine if the probability of issuing the stay‐at‐home order increases proportionally as the number of cases or deaths increases. RESULTS: We found that the decision to issue the state‐at‐home order reflects the Weber‐Fechner law. Both the number of infections (P = <.0001; R (2) = .79) and deaths (P < .0001; R (2) = .63) were significantly associated with the decision to issue the stay‐at‐home orders. The results indicate that for each doubling of infections or deaths, an additional four to six states will issue stay‐at‐home orders. Cox regression showed that when the number of deaths reached 256 and the number of infected people were over 16 000 the probability of issuing “stay‐at‐home” order was close to 100%. We found no difference in decision‐making according to the political affiliation; the results remain unchanged on 16 July 2 020. CONCLUSIONS: when there are not clearly articulated rules to follow, decision‐makers resort to simple heuristics, in this case one consistent with the Weber‐Fechner law.

As the COVID-19 pandemic spreads, keeping people apart has been hailed as the most important preventive measure currently available.

In the United States, the reluctance of the federal government to issue a national order has placed state governors in the position of having to decide when to escalate from vague wishes for social distancing to stay-at-home directives. What information guides that decision? We suggest there are two clearly identified, readily available signals that convey crisis magnitude, namely, the numbers of cases (people reported as infected) and people pronounced dead of the disease. When those numbers increase sufficiently, action will be taken.

The two numbers are correlated, of course, but we might expect each of these numbers to affect the governor's decision to issue a stay-at-home order.

People vary in their sensitivity to stimulus differences. According to the classical Weber-Fechner law of psychophysics, 1,2 the amount by which a stimulus must increase in order to be noticed, the "just noticeable difference" or JND, is a fraction of the intensity of that stimulus. For example, Ernst Weber, a 19th century psychologist, found that people could not discriminate between 20.5 and 20.0 g weights but could usually discriminate between 21 and 20 g. 2 When a series of baseline weights was 40, 60, 80 and 100 g, the JND was 2, 3, 4 and 5 g, respectively. That is, to appreciate the differences between weights (JND), the weight (ie, stimulus) should increase by a constant percentage of the stimulus itself, that is, by at least 5% of the original weights in this example. 2 Gustav Fechner, another 19th century psychologist, proposed that the "JND" can be conceptualized as a unit of psychological intensity rather than physical intensity. [1] [2] [3] Subsequently, the relationship between the intensity of a signal and how much more intense the signal needs to increase before a person can reliably tell that the signal had changed has become known as the Weber-Fechner Law of psychophysics. As expected for physiological systems, the law is valid within certain domains of stimulus-response ratios. Therefore, if we accept Fechner's assumption that JNDs are psychologically equal in size, then there will be a logarithmic relation between stimulus intensity and perception. The Weber-Fechner law has been well documented across different physiological and psychological phenomena 4it describes responses in a variety of domains in which increasingly intense stimuli can be placed along a continuum, from the perception of loudness or brightness 1,2 to the subjective value of money 5 to the mental line for numbers 6 and risky choices. 7 In the present context, we anticipate that the perception of danger from the virus will depend upon the number of cases and/or deaths observed within a governor's jurisdiction.

Our hypothesis is that in order to move an uncommitted governor to action, the additional number of infected people or deaths required will be logarithmically larger as more cases or deaths are reported.

That is, the governors will not "notice" that conditions are sufficiently worse until an increasingly larger number of people are infected or die.

We obtained the daily numbers of cases and deaths from the New York Times daily database (https://github.com/nytimes/covid-19-data). We examined both numbers separately, cases and deaths, to see whether one signal was more clearly linked to the stay-at-home decision. The original analyses were performed on data downloaded on 12 April 2020 from all 50 states and the District of Columbia (we regarded the mayor of that city as a governor in our analyses); by that time all but 7 states had issued stay-at-home orders. Four of the governors had issued stay-at-home orders before any deaths were recorded.

The data cannot be regarded as precise. The Times consider a case confirmed when it is reported by a federal, state, territorial or local government agencies. Confirmed cases are patients who test positive for the coronavirus. However, it must be acknowledged that the performance of diagnostic tests vary, limiting diagnostic accuracy. 8 Deaths are more visible, but may be inaccurate because of incorrect attributions. Some states updated records on a daily basis, while others updated only after a certain number of cases or deaths had accumulated. In sum, the reported numbers may not be exact, but the numbers we worked with and the numbers the governors saw are the same.

Because the virus surfaced at different times across the country, in order to count the number of days between the signalling event and the stay-at-home order, we had to start a separate calendar for each state. We called that starting date t 0 . We suspected that setting t 0 to the day when the first death was reported would be appropriate, because officials are more likely to trust reports of deaths, which are concrete events, than of cases, which depend on the infected person's decision to seek medical attention. It also would be possible to set t 0 on the basis of cases. We tried that out to see if it mattered, but rather than starting the count after the first case in the state, as customarily in these types of the analyses, we started it after the first 50 cases had been reported. Because both cases and deaths accumulated exponentially, the growth rate was similar for both signals and so we expect similar results whichever way the calendar is defined.

We used two analytical approaches. First, we fitted the Weber-Fechner logarithmic function by regressing the log 2 of cases and deaths, respectively, against the daily counts. If the logarithmic notion is correct, these should appear as straight lines. The idea is that relatively few cases or deaths may be needed to inspire the first governor to issue an order, while governors who do so later will require increasingly stronger signals. In the case of the latest states, many more cases or deaths may be needed. Using the base 2 for the logarithmic analysis allows us to see how many doublings of the signal strength (which we defined as JND) are required to inspire the various governors to issue the stay-at-home orders. Because we were interested in learning how the states responded in sequence, we first ranked the states according to the order of declaring the stay-at-home order after each state had cases ≥50 or deaths ≥1. We then regressed the stay-athome state rank list on log 2 (cases) or log 2 (deaths). Thus, the coefficient of the regression represents the number of additional states introducing stay-at-home order for each doubling of cases or deaths.

For the analysis using cases, we excluded the 7 states that had not issued stay-at-home orders by April 12, the date of our analysis, thereby leaving us with data for 44 states. For the analysis using deaths, we also excluded an additional four states whose governors had issued the stay-at-home order before any deaths had occurred.

The rationale for the latter is that these governors were not using the signals as we had defined them. Because of increasing polarization of the American public related to COVID-19 risk attitudes along the political party lines, 9 we also performed the analysis according the governors' political affiliations. We repeated the original analysis from April 12 data set on the data set downloaded on 16 July 2020.

Another way to approach the data is to track the changes in the probability of making the stay-at-home decision as days pass. To do this, we used Cox regression modelling. We expected that the probability of issuing the stay-at-home order would proportionately increase as the number of cases or deaths increases. For the Cox analysis we used data for all eligible states but censored those states that had not issued the order by the time of our analysis.

Because the data were somewhat noisy, we also repeated the analyses using 3-day moving averages for cases and deaths. We performed sensitivity analyses using number of cases or deaths per million, and population size per area of state (density) as the stimuli rather than the actual numbers. The results were sufficiently similar that we elected to not present them.

All analyses were done in STATA statistical software. 10 No patients or public were involved in the design, recruitment and conduct of this study. No ethics/institutional review board (IRB) F I G U R E 1 shows that the number of both cases, A, and deaths, B, is significantly associated with the decision to issue a "stay-athome" order. For each doubling of cases beyond 50 or deaths beyond 1, it takes an additional 4 to 6 states to issue stay-at-home orders. The overall and regression analysis according to the party affiliation is shown (red: Republicans; blue: Democrats) approval was required. The analysis is based on publicly available, anonymized data. Data are available at https://github.com/nytimes/ covid-19-data. Statistical code is available from the authors upon request. Figure 1A ,B show that both the numbers of cases (P = <.0001; R 2 = .79) and deaths (P < .0001; R 2 = .63) are significantly associated with the decision to issue the stay-at-home orders order. The results indicate that for each doubling of infections or deaths within their state, an additional four to six governors will issue the stay-at-home order. The figures also show that that the regression lines were not statistically different between the governors who were affiliated with Democratic vs Republican party (except for the slopes for cases).

When the analysis was repeated on the data set downloaded on 16 July 2020, we also observed no statistical difference (both for the slopes and regressions) according to the party affiliation (data not shown). The analysis of the residuals for the linear regression analysis indicated no deviation from normality, thus supporting a customary assumption of the analysis.

Our decision regarding t 0 , when to start each state's calendar, was somewhat arbitrary. To see whether it mattered, we performed an additional set of exploratory analyses. In these, we started each state's calendar after a particular number of cases or deaths were recorded. We tried fitting the logarithmic function to n values from 1 to 100 for cases and from 1 to 20 for deaths. In general, as one would expect, better fit was observed when we started t 0 at higher numbers of cases or deaths, but at the expense of fewer data points.

For example, R 2 increased to about 80% when we started at ≥10 deaths or ≥100 cases but was based on data from fewer states -28 and 42, respectively.

The Cox regression curves in Figure 2A 

During the COVID-19 crisis, one commentator remarked 'we are all getting tired looking at the exponential graphs', wondering how to act on the information shown in those graphs. From a theoretical perspective, we found that the decision to issue the state-at-home order reflects the classical psychophysical Weber-Fechner law of psychophysics. The signalscases and deathsare strongly correlated with the decision to announce the stay-at-home order. Here, for each doubling of the number of cases beyond 50 or deaths beyond 1, additional 4 to 6 governors will issue the stay-at-home order. When the signal is truly strongmore than about 250 deaths or 16 000 cases in our analysisthe probability of issuing state-at-home order approaches near-certainty.

Given the noise introduced by differences between states in terms of population density, political persuasion and media coverage, this appears to be a remarkably simple result. The percentages of variance accounted for are quite high (60%-80%) by industry standards; the typical R 2 for a one-variable predictor in judgement research is between 5% and 30%. This appears to be another instance of a F I G U R E 2 Cox regression analysis shows that the probability of issuing a stay-at-home order proportionally increases as the number of cases, A, and deaths, B, increases (the numbers expressed as an increase in log 2 of cases or deaths are represented by different colour lines on the graphs) [To convert log 2 (cases or deaths), raise the exponent to 2. For example, the largest number of cases in our database was log 2 (14) which converts into 2 14 = 16 384; the largest number of deaths was log 2 (8) which translates into 2 8 = 256 deaths]. S@H, stay-at-home order familiar phenomenon; when there are not clearly articulated rules to follow, or people decide to ignore them (e.g., U.S. government decided not to activate prescriptive National Pandemic Strategy Documents 11 ) people rely upon simple heuristics 12 of which they need not be aware.

These powerful rule-of-thumb decision-making strategies can surprisingly be more accurate than complex statistical models, but sometimes can be disastrously wrong. 12 Here the heuristic is to wait until the day that the number of cases or deaths is striking. We suggest that The current discourse in social media seem to indicate that the party political affiliation affects the governors' attitudes toward measures of social distancing. A typical assumption is that Democrats are more pro-active than Republicans in issuing state-at-home orders.

Indeed, the governors of the seven states who never issued stay-athome orders were all Republicans. However, when we analysed data according to the political affiliation, 9 we found no difference in decision-making between the Democratic and Republican governors.

The major limitation of our approach is that it does not incorporate the kind of informationperhaps personality characteristics, perhaps availability of advice from colleagues or agenciesthat might enable us to predict how many doublings in the numbers of deaths a particular governor needs to observe before taking action. Of course, our analysis demonstrated association between stimuli (cases, deaths) and the decision to issue a stay-at-home order, but not direct causation between the two. We have identified the stimulus that underlies the decision, but we do not know why some governors acted more quickly than others. As the governors seemed to be wired to act according to the Weber-Fechner law of psychophysics, we believe that the findings are important for the public to understand how their elected officials make importantlife and deathpublic health decisions. This is ever more important as the states and countries are pondering their decisions how to re-open their economies.

Introduction to Mathematics for Life Scientists

Weber's law modeled by the mathematical description of a beam balance

Rational Choice in an Uncertain World

The Winner's Curse. Paradoxes and Anomalies of Economic Life

The neural basis of the Weber-Fechner law: a logarithmic mental number line

Risky choice and Weber's law

Report from the American Society for Microbiology COVID-19 International Summit, 23 March: value of diagnostic testing for SARS-CoV-2/COVID-19

Democrats Move Even Further Apart in Coronavirus Concerns

Center for Disease Control and Prevention. National Pandemic Strategy

Homo heuristicus: why biased minds make better inferences

The role of "Prominent Numbers" in open numerical judgment: strained decision makers choose from a limited set of accessible numbers

Evaluation of the U.S. governors' decision when to issue stayat-home orders

We want to thank Jerome Hoffman, MD, and Dr Amy Price for helpful comments on the earlier version of the paper.

The authors declare no conflict of interest.

Benjamin Djulbegovic: conceived idea, wrote the first draft, analysis; David J. Weiss: revised draft, provided further input from the perspective of psychophysics; Iztok Hozo: created a software program, analysis.

https://orcid.org/0000-0003-0671-1447