key: cord-0908428-d4rukftj
authors: Levin, Joel M.; Davis, Billie S.; Bukowski, Leigh A.; Kahn, Jeremy M.
title: Willingness to Treat with Therapies of Unknown Effectiveness in Severe COVID-19: A Survey of Intensivist Physicians
date: 2022-04-01
journal: Annals of the American Thoracic Society
DOI: 10.1513/annalsats.202105-594oc
sha: f5872ca654d7b214d69f538c6d28038a2fe79910
doc_id: 908428
cord_uid: d4rukftj

RATIONALE: Little is known about how physicians develop their beliefs about new treatments or update their beliefs in the face of new clinical evidence. These issues are particularly salient in the context of the coronavirus disease (COVID-19) pandemic, which created rapid demand for novel therapies in the absence of robust evidence. OBJECTIVES: To identify psychological traits associated with physicians’ willingness to treat with unproven therapies and willingness to update their treatment preferences in the setting of new evidence in the context of COVID-19. METHODS: We administered a longitudinal e-mail survey to United States physicians board certified in intensive care medicine in April and May 2020 (phase one) and October and November 2020 (phase two). We assessed five psychological traits potentially related to evidence uptake: need for cognition, evidence skepticism, need for closure, risk tolerance, and research engagement. We then examined the relationship between these traits and physician preferences for pharmacological treatment for a hypothetical patient with severe COVID-19 pneumonia. RESULTS: There were 592 responses to the phase one survey, conducted prior to publication of trial data. At this time physicians were most willing to treat with macrolide antibiotics (50.5%), followed by antimalaria agents (36.1%), corticosteroids (24.5%), antiretroviral agents (22.6%), and angiotensin inhibitors (4.4%). Greater evidence skepticism (relative risk [RR], 1.40; 95% confidence interval [CI], 1.30–1.52; P < 0.001), greater need for closure (RR, 1.19; 95% CI, 1.06–1.34; P = 0.003), and greater risk tolerance (RR, 1.17; 95% CI, 1.08–1.26; P < 0.001) were associated with an increased willingness to treat, whereas greater need for cognition (RR, 0.85; 95% CI, 0.75–0.96, P = 0.010) and greater research engagement (RR, 0.91; 95% CI, 0.88–0.95; P < 0.0001) were associated with decreased willingness to treat. In phase two, most physicians updated their beliefs after publication of trial data about antimalarial agents and corticosteroids. Physicians with greater evidence skepticism were more likely to persist in their beliefs. CONCLUSIONS: Psychological traits associated with clinical decisions in the setting of uncertain evidence may provide insight into strategies to better align clinical practice with published evidence.

Efficient translation of evidence into clinical practice remains a vexing problem in health care. Many evidence-based therapies are not routinely provided to patients, even many years after publication of robust clinical trial data demonstrating effectiveness (1) . At the same time, many therapies are routinely provided to patients despite absence of any evidence of benefit, or even evidence of harm (2, 3) . Central to addressing these problems is a greater understanding of how clinicians form their opinions about clinical evidence prior to the publication of data and then how they update those opinions after the publication of data (4) . Specifically, little is known about the factors that predispose clinicians to adopt, or not adopt, therapies when the evidence is uncertain (5) . Equally little is known about the factors that predispose clinicians to either update or not update their beliefs in response to new evidence (6) . Identifying these factors could provide important insight into strategies not only to speed adoption of effective treatments but also to speed deadoption of ineffective treatments.

These issues became more salient during the coronavirus disease (COVID-19) pandemic (7) . COVID-19 forced physicians to rapidly grapple with a highly morbid disease for which no effective treatments existed. During this time, anecdotal reports coupled with mechanistic hypotheses derived from past experience created early enthusiasm for several existing pharmacological agents (8) . Yet at the time, no robust clinical data existed to guide practice. Eventually clinical trials were published demonstrating the effectiveness of some treatments, like corticosteroids (9) (10) (11) , and the ineffectiveness of other treatments, like the antimalarial agent hydroxychloroquine (12) (13) (14) . We used these events to study the factors associated with the willingness to treat with unproven treatments, and willingness to update beliefs in the setting of new evidence, by means of a survey of intensivist physicians in the United States.

We developed and fielded a longitudinal survey of board-certified U.S. physicians in intensive care medicine about their treatment preferences for patients with COVID-19. We focused on intensivist physicians because of the likelihood that they were directly involved in the longitudinal care of patients with acute severe COVID-19 during the early stages of the pandemic, although we did not require direct experience with patients with COVID-19 as a condition for participation. We surveyed physicians at two points in time: once early in the pandemic prior to the publication of any trial data, and once later in the pandemic after the publication of trial data, enabling us to examine treatment preferences over time. We used the American Medical Association Masterfile to identify boardcertified U.S. physicians in intensive care medicine with a base specialty in internal medicine, emergency medicine, anesthesiology, or surgery. All aspects of the study were reviewed and approved by the University of Pittsburgh Human Subjects Protection Office.

The survey instrument was developed by the investigative team as part of a larger study on medical decision-making under uncertainty. The portion of the survey pertaining to this report contained three sections: demographic characteristics, treatment preferences, and psychological traits. To assess demographics characteristics, we asked participants about their age, sex, base clinical specialty, practice setting, and proportion of time spent performing clinical care.

To assess treatment preferences, we presented participants with a standardized description of a patient with severe COVID-19 pneumonia and asked them to indicate the likelihood that they would treat the patient with a pharmaceutical agent in any of five drug classes: a quinine-based antimalarial agent (e.g., chloroquine or hydroxychloroquine); a macrolide antibiotic (e.g., azithromycin); a corticosteroid (e.g., hydrocortisone or dexamethasone); an antiretroviral agent (e.g., lopinavir/ritonavir); or an angiotensin receptor blocker (e.g., losartan) (15) . The full text of the scenario is provided in the SUPPLEMENTARY METHODS section of the online supplement. We focused on these drug classes because, at the time of the survey, they were widely discussed as potential therapeutic options and readily available for actual use. For each drug class, participants indicated their treatment preferences along a four-point scale: "definitely would," "probably would," "probably would not," or "definitely would not" treat.

To assess psychological traits, we adapted four previously validated scales: need for cognition (16) , need for closure (17) , actively open-minded thinking (18) , and risk tolerance (19) . We also developed two de novo scales, one related to evidence skepticism (20) and one related to engagement with new research (21) . These six traits were chosen based on a review of the literature as potentially relevant to clinical decision-making under uncertainty. Full definitions for each trait and additional scale information are provided in Table 1 . For need for cognition, need for closure, actively open-minded thinking, and evidence skepticism, participants were asked to rate their level of agreement with items along a five-point Likert scale ranging from "strongly agree" to "strongly disagree." For research engagement, participants were asked to indicate the range of activities they performed to keep up with the medical literature during the last week, with more activities indicating higher engagement. For risk tolerance, participants were asked to place themselves on a five-point scale ranging from "extremely comfortable taking risks" to "not at all comfortable taking risks," with higher values indicating greater risk tolerance.

We piloted the survey among 21 practicing intensivists who did not participate in the study, with revisions made for clarity and content based on their feedback. We then entered the survey into an electronic survey tool (Qualtrics) for administration. The final survey items a summary of the theoretical rationale for their inclusion in the study are provided in the SUPPLEMENTARY METHODS.

We administered the survey in two phases using established best practices for internet surveys (22) . Phase one occurred in April and May of 2020. In this phase, physicians were sent e-mail invitations from a thirdparty contractor (Medical Marketing Services) with a link to the survey. Four invitations were sent approximately 1 week apart, beginning on April 16, 2020, and ending on May 7, 2020. Participants were offered a $50 gift card in exchange for participation. Because this e-mail list was ORIGINAL RESEARCH maintained by a third-party, we did not have access to information about the physicians who received the phase one survey. Within the survey, we collected physicians' e-mail addresses so we could directly administer follow-up surveys.

Phase two occurred in October and November of 2020, after the publication of clinical trial data. Key publications included trials demonstrating the effectiveness of corticosteroids and the ineffectiveness of quinine-based antimalarial agents (9) (10) (11) (12) (13) (14) . The phase two survey contained treatment preference questions using identical vignettes as phase one and did not reference or identify the relevant publications. It also did not contain demographics or psychological traits, since these were not expected to have changed in the interim. The exception was an expanded set of questions to measure evidence skepticism. Since the evidence-skepticism scale was developed de novo, we took the opportunity to better determine the scale's psychometric properties. The expanded list of survey items for this trait is provided in the SUPPLEMENTARY METHODS. The phase two survey was limited to physicians who responded to phase one and was sent directly from the investigators. Four invitations were sent approximately 1 week apart beginning on October 12, 2020, and ending on November 5, 2020. Participants were offered a $75 gift card in exchange for participation.

For phase one, we calculated the overall response rate as the number of unique responses received divided by the number of unique e-mail addresses targeted. E-mails were sent by the direct marketer and, due to spam filters or the inaccuracies in the marketing database, might not have been viewed by the participants. Therefore, we also estimated an effective response rate, defined as the number of responses received divided by the maximum number of unique e-mails opened across the three mailings. For phase two, we calculated the response rate as the number of unique completed surveys divided by the number of physicians contacted.

For phase one, we had no data on physicians ahead of the survey. Therefore, we could not compare characteristics between phase one respondents and nonrespondents. For phase two, we compared characteristics between respondents and nonrespondents using chi-square tests.

Prior to analyzing the survey results, we examined the internal consistency of the multi-item psychological constructs by calculating each item's Cronbach's a and interitem covariance (23) . For the multiitem de novo measure (evidence skepticism), we also examined test-retest reliability and correlation with the expanded scale (24) . We dropped constructs with poor internal consistency, defined as Cronbach's a , 0.60. For the remaining constructs, we created summary scores by averaging the individual items within each construct. We examined a correlation matrix of the summary scores to evaluate for colinearity between constructs.

To analyze the phase one survey, we first created binary versions of the willingness to treat for each drug class: either yes ("definitely would" or "probably would" treat) or no ("probably would not" or "definitely would not" treat). We then created a composite count measure of willingness to treat across all drug classes, which ranged from 0 (meaning that the physician would treat with none of the drugs) to 5 (meaning that the physician would treat with all five drugs). For the primary analysis, we used Poisson regression with robust standard errors to examine the relationship between this count measure and each individual psychological construct (25) . In secondary analyses, we examined the relationship between the psychological constructs and each individual drug.

To analyze the phase two survey, we focused on the two drug types for which large clinical trials had been published in the interval between phase one and phase two: corticosteroids and quinine-based antimalarials (9) (10) (11) (12) (13) (14) . We restricted the analysis to physicians with the potential to have updated their treatment preferences based on this evidence. For corticosteroids, this included physicians who indicated they definitely would not treat during phase one. For quinine-based antimalarials, this included physicians who indicated that they definitely would treat during phase one. Need for cognition The degree to which an individual engages in and enjoys effortful cognitive endeavors.

Evidence skepticism The degree to which an individual is skeptical of clinical evidence and places higher weight on anecdotes and experience.

The degree to which an individual desires an answer on a given topic, any answer, compared to confusion and ambiguity.

Actively open-minded thinking

The degree to which an individual is disposed toward fairness toward different conclusions even if they go against one's initially favored conclusion.

A count measure of how many ways a participant engaged with research within the previous week.

The degree to which an individual is predisposed toward risk taking. Table E2 in the online supplement.

Within these groups, we identified physicians who did not update their treatment preferences (i.e., their phase two responses were the same as their phase one responses), creating a binary variable indicating that they either did or did not update between phase one and phase two. We used Poisson regression with robust standard errors to examine the relationship between unwillingness to update and each individual psychological construct (25) . For the phase one analysis, we fit both unadjusted regression models and regression models adjusting for physician characteristics, including community practice setting (academic or community/ other), clinical time (all or almost all, not all but more than 50%, or less than 50%), and base specialty (internal medicine or other). For the phase two analysis, we only fit unadjusted regression models, since the low numbers of physicians who did not update precluded a multivariate analysis. The regression results are presented as relative risks along with confidence intervals and P values. A P value of 0.05 or lower was considered significant. All statistical analyses were performed using Stata 16.1.

A flow chart of study participants is given in Figure E1 in the online supplement. In phase one, we received 592 completed surveys in response to e-mails sent to 14,090 unique e-mail addresses, for an overall response rate of 4.2%. The maximum number of unique opened emails was 1,778, for an estimated effective response rate of 33.3% (Table E1) . In phase two, we received 371 completed surveys in response to 592 unique physicians contacted, for an overall response rate of 62.7%. Participants varied in age, base specialty, and clinical time, although most respondents practiced in an academic setting ( Table 2) . Compared with phase two nonrespondents, phase one respondents were more likely to have a base specialty in internal medicine and were more likely to practice in an academic setting ( Table 2) .

Of the four multi-item psychological constructs, need for cognition, evidence skepticism, and need for closure demonstrated acceptable psychometric properties and were retained in the analysis (Table E2) . Actively open-minded thinking demonstrated low internal consistency (Cronbach's a = 0.42) and was dropped from the analysis (Table E2) . Additional psychometric evaluation of the evidence skepticism scale showed good test-retest reliability, supporting the decision to retain this scale (Table E3) . A correlation matrix including all remaining constructs demonstrated little correlation between measures, supporting the decision to analyze them independently (Table E4) .

In phase one of the survey, respondents were most likely to treat the hypothetical patient with COVID-19 with macrolide antibiotics, followed by antimalaria agents, corticosteroids, antiretroviral agents, and angiotensin inhibitors (Figure 1 ). In the regression analysis of the composite outcome measure, greater evidence skepticism, greater need for closure, and greater risk tolerance were statistically significantly associated with increased willingness to treat, while greater 

need for cognition and greater research engagement were statistically significantly associated with decreased willingness to treat (Table 3) . Similar results were obtained when analyzing each drug class individually and when repeating the regression controlling for practice setting, clinical time, base specialty, and perceptions of evidence quality (Table 3) .

In phase two of the survey, 241 of 367 respondents (65.7%) were eligible to update their treatment preferences for quinine-based antimalarial agents, in that in phase one they did not say they "definitely would not" treat. Of these, only 16 (6.6%) did not update their treatment preferences. A total of 354 of 367 respondents (96.5%) were eligible to update their treatment preferences for corticosteroids, in that in phase one they did not say they "definitely would" treat. Of these, only 12 (3.4%) did not update their treatment preferences. Physicians with greater evidence skepticism were more likely to not update their treatment preferences for both quinine-based antimalarial agents and corticosteroids (Table 4 ).

The COVID-19 pandemic revealed a pressing need to understand how clinicians develop treatment preferences about therapies of unknown effectiveness. We found that several psychological traits were associated with an increased willingness to treat with unproven pharmaceutical treatments in severe COVID-19, including greater evidence skepticism, greater need for closure, greater risk tolerance, lower need for cognition, and lower research engagement. Reassuringly, only a small minority of clinicians failed to update their treatment preferences after publication of clinical trials related to quinine-based malarial agents (in which the trials were negative) and corticosteroids (in which the trials were positive). Only greater evidence skepticism was associated with decreased willingness to update. Use of unproven treatments was widespread during the early stages of the pandemic. For example, studies reported rates of antimalarial agent administration ranging from 34.6 to 92.1% despite any clinical data to demonstrate efficacy (26) (27) (28) . Frequent use of unproven treatments like antimalarials underscores the tensions inherent in clinical decision-making during a pandemic and highlights the need for systems to rapidly generate clinical evidence to guide decision-making during public health crises (29, 30) . More broadly, our results suggest that there may be a phenotype of physicians who rapidly adopt unproven treatments in the face of uncertainty, physicians who weight experience over evidence, are less likely to engage with the research literature, are more tolerant of risk, and are less tolerant of ambiguity.

Our results also suggest that some types of physicians are more steadfast in their beliefs about unproven treatments than others. Specifically, physicians that weighted experience over scientific evidence were less likely to update their beliefs in response to clinical trials, regardless of whether those trials were negative (in the case of antimalarial agents) or positive (in the case of corticosteroids). This finding should be interpreted with caution given that the vast majority of physicians did update their preferences. Nonetheless, this finding generates important hypotheses for future studies examining variation in physician responses to evidence and provides context to past work demonstrating the existence of physician-specific practice patterns that are distinct from behaviors learned over time (31) (32) (33) (34) (35) .

Together, these results provide insight into potential strategies to better align clinical practice with published evidence. It is unlikely that any intervention will change physicians' underlying psychological traits. However, physicians with specific psychological traits may be more susceptible to certain interventions, enabling "personalized" interventions based on individual psychological profiles. For example, interventions to reframe risk, socialize decision makers to group norms, and nudge decision makers toward desirable decisions might deemphasize the influence of direct experience and risk tolerance, in certain types of physicians (36) (37) (38) (39) . Ultimately, a better understanding of how these traits impact decision-making could lead to behavior change interventions that are specifically tailored to individual physicians' psychological profiles, rather than "one size fits all" interventions that are agnostic to the fact that physician responses to evidence vary systematically.

Our study has several limitations. First, many of the associations we observed were small, and as an observational study, these associations do not imply causation. However, the goal of our study was to not to infer causation or quantify the magnitude of these associations. Rather, the goal was to identify novel associations. Regardless of magnitude or mechanism, these correlations provide a valuable framework for 

considering interventions designed to improve clinician decision-making. Second, the phase one response rate was modest, raising the possibility that our study population differs from the population of U.S. intensivists in systematic ways. However, it is relatively unlikely that these differences led to significant response bias. Response bias is relatively uncommon in studies of psychological associations like this one, since response bias requires that differences in populations moderate the relationship between two associated variables, which is a relatively uncommon occurrence (40, 41) . Third, relatively few physicians were unwilling to update their treatment preferences after the publication of new evidence, reducing our ability to examine the factors associated with willingness to update. Fourth, we only ascertained physicians' expressed treatment preferences, not their actual behaviors. It is possible that physicians' responses about their treatment preferences differed from their actual practice or that the physicians differed in how they interpreted the vignettes. Nonetheless, clinical vignettes are a robust strategy for assessing practice patterns (15) , and there is no reason to think our use of clinical vignettes led to substantial bias, particularly since directly measuring individual physician treatment patterns would also lead to known biases (42) . Fifth, the reliability of our psychological traits was only moderate, although any measurement error would serve to make our estimates more conservative, biasing our results toward the null. Sixth, we cannot rule out the possibility that these psychological traits might change over time, perhaps in a way that was influenced by the pandemic. Future research should examine this important point.

Our study provides new insight into how intensivist physicians form their beliefs about new treatments and helps explain variation in the adoption and deadoption of new treatments as evidence evolves. A better understanding of these patterns could lead to behavioral interventions designed to better align treatment preferences with clinical evidence.

Author disclosures are available with the text of this article at www.atsjournals.org. Multivariable analysis controls for community practice setting (academic or community/other), clinical time (all or almost all, not all but more than 50%, or less than 50%), and base specialty (internal medicine or other). This analysis contains only physicians who responded to phase one and were eligible to update their references in phase two (n = 123 for quinine-based antimalarials and n = 296 for corticosteroids). A multivariable analysis was not preformed due to relatively low numbers of physicians who did not update. All estimates are risk ratios reflecting the change in willingness to update for each one unit change in the measurement scales.

Crossing the quality chasm

Towards understanding the de-adoption of low-value clinical practices: a scoping review

Effect of published scientific evidence on glycemic control in adult intensive care units

Evidence-based medicine: a unified approach

Practice variation and physician-specific effects

Evidence-based medicine as Bayesian decisionmaking

Uncertainty and COVID-19: how are we to respond?

More than 80 clinical trials launch to test coronavirus treatments

RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with Covid-19

Writing Committee for the REMAP-CAP Investigators. Effect of hydrocortisone on mortality and organ support in patients with severe COVID-19: the REMAP-CAP COVID-19 corticosteroid domain randomized clinical trial

COALITION COVID-19 Brazil III Investigators. Effect of dexamethasone on days alive and ventilator-free in patients with moderate or severe acute respiratory distress syndrome and COVID-19: the CoDEX randomized clinical trial

RECOVERY Collaborative Group. Effect of hydroxychloroquine in hospitalized patients with Covid-19

National Heart, Lung, and Blood Institute PETAL Clinical Trials Network. Effect of hydroxychloroquine on clinical status at 14 days in hospitalized patients with COVID-19: a randomized clinical trial

Coalition Covid-19 Brazil I Investigators. Hydroxychloroquine with or without azithromycin in mild-to-moderate Covid-19

Clinical vignette-based surveys: a tool for assessing physician practice variation

The efficient assessment of need for cognition

Item selection and validation of a brief, 15-item version of the need for closure scale

Actively open-minded thinking in politics

Measuring generalized risk orientation with a single survey item

On the belief that beliefs should change according to evidence: implications for conspiratorial, moral, paranormal, political, religious, and science beliefs

Describing knowledge encounters in healthcare: a mixed studies systematic review and development of a classification

Internet, mail, and mixed-mode surveys

Reliability and validity assessment

The intraclass correlation coefficient as a measure of reliability

A modified Poisson regression approach to prospective studies with binary data

Association of treatment with hydroxychloroquine or azithromycin with in-hospital mortality in patients with COVID-19 in

The minimal effect of zinc on the survival of hospitalized patients with COVID-19: an observational study

Northwell Health COVID-19 Research Consortium. Examination of patient characteristics and hydroxychloroquine use based on the US Food and Drug Administration's recommendation: a cross-sectional analysis in New York

Optimizing the trade-off between learning and doing in a pandemic

Life at the editorial "COVID frontline

Personal, organizational, and market level influences on physicians' practice patterns: results of a national survey of primary care physicians

The formation and evolution of physician treatment styles: an application to cesarean sections

Patient and healthcare professional factors influencing end-of-life decision-making during critical illness: a systematic review

Evaluation of physicians' cognitive styles

Physician-driven variation in nonrecommended services among older adults diagnosed with cancer

Failure to adopt beneficial therapies caused by bias in medical evidence evaluation

Framing effects on physicians' judgment and decision making

The development and enforcement of group norms

Nudge units to improve the delivery of health care

The impact of nonresponse rates on selfselection bias a meta-analysis

Consequences of survey nonresponse

Physician clinical performance assessment: prospects and barriers