key: cord-283175-kh2rm2ir
authors: Roma, Paolo; Monaro, Merylin; Muzi, Laura; Colasanti, Marco; Ricci, Eleonora; Biondi, Silvia; Napoli, Christian; Ferracuti, Stefano; Mazza, Cristina
title: How to Improve Compliance with Protective Health Measures during the COVID-19 Outbreak: Testing a Moderated Mediation Model and Machine Learning Algorithms
date: 2020-10-04
journal: Int J Environ Res Public Health
DOI: 10.3390/ijerph17197252
sha: 
doc_id: 283175
cord_uid: kh2rm2ir

In the wake of the sudden spread of COVID-19, a large amount of the Italian population practiced incongruous behaviors with the protective health measures. The present study aimed at examining psychological and psychosocial variables that could predict behavioral compliance. An online survey was administered from 18–22 March 2020 to 2766 participants. Paired sample t-tests were run to compare efficacy perception with behavioral compliance. Mediation and moderated mediation models were constructed to explore the association between perceived efficacy and compliance, mediated by self-efficacy and moderated by risk perception and civic attitudes. Machine learning algorithms were trained to predict which individuals would be more likely to comply with protective measures. Results indicated significantly lower scores in behavioral compliance than efficacy perception. Risk perception and civic attitudes as moderators rendered the mediating effect of self-efficacy insignificant. Perceived efficacy on the adoption of recommended behaviors varied in accordance with risk perception and civic engagement. The 14 collected variables, entered as predictors in machine learning models, produced an ROC area in the range of 0.82–0.91 classifying individuals as high versus low compliance. Overall, these findings could be helpful in guiding age-tailored information/advertising campaigns in countries affected by COVID-19 and directing further research on behavioral compliance.

Public willingness to comply with the protective health measures proposed by authorities is critical for controlling the outcomes of an infectious disease outbreak [1], given that "behavioral changes can significantly affect the epidemic spread both qualitatively [ . . . ] and quantitatively" [2] . Coronavirus disease 2019 (COVID-19, also known as 2019-nCoV), an acute respiratory illness with an unknown cause, emerged in China in December 2019 and, since then, has spread rapidly throughout most of the world. In January 2020, the World Health Organization declared it an international public health emergency and, shortly thereafter, a global pandemic [3] . On 30 January 2020, the first two cases of COVID-19 in Italy were confirmed by the Italian government. In the following weeks and months, the health emergency generated devastating consequences for both local residents and national health workers. Due to the lack of supported treatments and vaccines, Italy (similar to other countries) implemented a policy of social and physical distancing, as well as a mandatory recommendation to "stay at home". However, the virus continued to escalate at an overwhelming rate, due in part to the failure of a portion of Italian residents to observe the recommended health measures, despite the government's prevention information campaigns (issued from February onwards). To dissuade unsafe behaviors, the government was forced, on 19 March, to introduce penalties for those violating the guidance by leaving their domiciles for reasons other than stringent and absolute need. It is worth noting that, at this point in time, the contagion was already very advanced: on 18 March, the official cases reported by the government included 33, 190 infected, 4440 recovered, and 3400 deceased [4] [5] [6] [7] .

The present research stemmed from the observation that, throughout this health crisis, Italian residents (similar to those in other affected countries) have engaged in incongruous behaviors, thereby severely limiting the effective management of COVID-19. To investigate this phenomenon more deeply, we selected and explored the effect of multiple psychological and psychosocial variables that were thought to relate to individual differences in compliance with the intervention measures and safety behaviors during the first phases of the COVID-19 outbreak in Italy.

The belief that a recommended health behavior will have positive consequences and/or will reduce the public health threat (or its seriousness) is commonly defined as perceived efficacy or perceived benefits. In actuality, these terms indicate slightly different constructs, originating from different theoretical frameworks: the health belief model (HBM) [8] and protection motivation theory (PMT) [9, 10] , respectively. Although perceived benefits also include non-health-related positive outcomes (e.g., having more money after quitting smoking), the term is often used synonymously with perceived efficacy. In this study, we use the term perceived efficacy very specifically to describe people's perception of the efficacy of the recommended preventive measures in reducing the risk of contagion. Perceived efficacy has been studied in relation to a variety of general health measures, including engaging in physical activity, receiving vaccinations, and complying with medical treatment among psychiatric outpatients [11] [12] [13] ; the literature suggests that it is a key determinant for compliance with preventive health behaviors and that "only when a person feels that the recommended behavior is likely to lead to the desired outcome will adoption of the recommendations occur" ( [14] , p. 193). Perceived efficacy has also been studied in the context of past epidemics/pandemics (e.g., H1N1) and the current COVID-19 outbreak, demonstrating that it is one of the strongest predictors of compliance with preventive health behaviors [15] [16] [17] .

Self-efficacy is defined as "people's beliefs about their capabilities to produce designated levels of performance that exercise influence over events that affect their lives" ( [18] , p. 71). Of note, the concept also refers to an individual's belief about their ability to perform specific behaviors in particular situations [19] . A direct relationship between self-efficacy and behavioral change has been found in many health contexts (relating to e.g., cigarette smoking, weight control, contraceptive behavior, and alcohol abuse) [20, 21] and during past pandemics [22] , indicating a strong relationship between self-efficacy and health behavior change and maintenance. According to the health action process approach (HAPA) [23] , three specific cognitions lead to an intention to act (i.e., motivation): (a) perceiving oneself as at risk (i.e., a risk perception), (b) believing that the recommended health behavior will reduce the threat (i.e., an efficacy perception), and (c) self-efficacy. Although efficacy perception has been found to be a stronger predictor than self-efficacy in developing the intention to act [24] , "individuals need to know the contingencies between behaviors and outcomes [ . . . ], but they also need to be confident that they can really perform the behavior in question" (p. 493). Accordingly, the present study aimed at revealing whether self-efficacy plays a mediating role in this relationship. We posited that feeling able to perform the recommended health behaviors would mediate the relationship between the perceived efficacy of those measures and compliance.

Research Question 1 (RQ1): Self-efficacy would mediate the relationship between the perceived efficacy of the government health measures and compliance (simple mediational model). Higher levels of perceived efficacy would increase compliance through higher self-efficacy.

Risk perception, defined as an individual's belief about the risk of potential harm, is a key construct of many theoretical frameworks for health behaviors (e.g., HBM, PMT, HAPA). Risk perception is influenced by the perceived severity of the specific health threat and the perceived likelihood of harm. As noted by Brewer et al. [25] , previous studies have often used the terms "likelihood" and "vulnerability"/"susceptibility" interchangeably, even though the first represents "one's probability of being harmed by a hazard under certain behavior conditions" (p. 136), whereas the second can be defined as individual resistance or constitutional vulnerability.

Research on past epidemics/pandemics (e.g., H1N1) has not only shown risk perception to be a key driver of health behaviors, but it has also consistently found an association between risk perception and precautionary behaviors [26] [27] [28] , even though a meta-analysis highlighted that this association involves only small effect sizes [25] , especially with respect to perceived severity (r = 0.16). In the context of health psychology, significant interactions between risk perception and self-efficacy have been found; for example, research has identified an association between motivation to think about cardiovascular disease, use of health information, and knowledge acquisition [29] . However, some research has reported no significant direct effect of risk perception on preparedness measures. Bourque et al. [30] for instance, found that the effect of risk perception was largely mediated by knowledge, perceived efficacy, and milling behavior in household preparedness for terrorism behavior in the United States.

Research Question 2 (RQ2): Risk perception would moderate the relationship between the perceived efficacy of the government health measures and compliance (Research Question 2a), as well as the positive relationship between the perceived efficacy of the health measures and self-efficacy (Research Question 2b), and the positive relationship between self-efficacy and compliance (Research Question 2c) (moderated mediation models). Higher perceived efficacy of the health measures would be associated with increased compliance with protective health measures, especially alongside higher risk perception.

Civic engagement can be defined as "the process of believing that one can and should make a difference in enhancing his or her community" [31] . It is typically thought to have two dimensions: attitudes and behaviors. However, since the present study was focused on cognitive variables, we primarily examined civic attitudes.

Due to the lack of a strong theoretical model for civic engagement in health contexts, no previous research has examined the link between civic engagement and health behaviors (or specifically, compliance with health measures). However, studies have focused on specific aspects of civic engagement, such as perceived moral responsibility. For example, a recent study [13] investigating the vaccination rate among nurses in Hong Kong considering HBM constructs and using perceived moral responsibility as a moderator found an insignificant effect. Furthermore, previous studies have examined individuals' level of information regarding community health concerns during pandemics (including the current COVID-19 pandemic), demonstrating a positive association between knowledge about a specific threat and preventive behaviors [16, 32, 33] . These results suggest that promoting knowledge about COVID-19 might encourage the adoption of preventive behaviors.

Other facets of civic engagement-namely political efficacy and an interest in public affairs-have been found to be positively associated with self-efficacy [34] . Furthermore, research on mobile donation as a new platform for technology-mediated civic engagement has demonstrated that perceived effectiveness (i.e., the degree to which consumers believe that the company will really donate as much as promised and that this donation will actually reach the needy recipients) has a positive effect on the intention to donate via mobile phone [35] .

In this vein, the present study sought to explore the relationship between civic engagement (specifically, civic attitudes), perceived efficacy, self-efficacy, and compliance with protective health measures during the COVID-19 pandemic. The COVID-19 outbreak presents a unique opportunity to investigate the role of civic engagement-together with the aforementioned variables-since even asymptomatic individuals are able to spread the virus to vulnerable populations (e.g., older adults and the elderly) [36] . Therefore, we included this variable as a moderator because it could potentially provide insight into how to improve compliance to the recommended preventive behaviors. Indeed, in the Research Question 3 (RQ3): Civic attitudes would moderate the relationship between the perceived efficacy of the government health measures and compliance (Research Question 3a), as well as the positive relationship between the perceived efficacy of the health measures and self-efficacy (Research Question 3b), and the positive relationship between self-efficacy and compliance (Research Question 3c) (moderated mediation models). Higher perceived efficacy of the health measures would be associated with increased compliance with the protective health measures, especially alongside higher levels of civic attitudes. Figure 1 shows the proposed moderated mediation model. Other facets of civic engagement-namely political efficacy and an interest in public affairshave been found to be positively associated with self-efficacy [35] . Furthermore, research on mobile donation as a new platform for technology-mediated civic engagement has demonstrated that perceived effectiveness (i.e., the degree to which consumers believe that the company will really donate as much as promised and that this donation will actually reach the needy recipients) has a positive effect on the intention to donate via mobile phone [36] .

In this vein, the present study sought to explore the relationship between civic engagement (specifically, civic attitudes), perceived efficacy, self-efficacy, and compliance with protective health measures during the COVID-19 pandemic. The COVID-19 outbreak presents a unique opportunity to investigate the role of civic engagement-together with the aforementioned variables-since even asymptomatic individuals are able to spread the virus to vulnerable populations (e.g., older adults and the elderly) [37] . Therefore, we included this variable as a moderator because it could potentially provide insight into how to improve compliance to the recommended preventive behaviors. Indeed, in the Research Question 3 (RQ3): Civic attitudes would moderate the relationship between the perceived efficacy of the government health measures and compliance (Research Question 3a), as well as the positive relationship between the perceived efficacy of the health measures and selfefficacy (Research Question 3b), and the positive relationship between self-efficacy and compliance (Research Question 3c) (moderated mediation models). Higher perceived efficacy of the health measures would be associated with increased compliance with the protective health measures, especially alongside higher levels of civic attitudes. Figure 1 shows the proposed moderated mediation model. 

The results of a previous review on demographic and attitudinal determinants of compliance with protective measures during a pandemic showed that being older, female, more educated, and non-White were associated with a higher likelihood of adopting the recommended health behaviors [38] . More recently, several studies have investigated which factors were associated with compliance with the preventive measures during the COVID-19 pandemic. The results indicated that the 

The results of a previous review on demographic and attitudinal determinants of compliance with protective measures during a pandemic showed that being older, female, more educated, and non-White were associated with a higher likelihood of adopting the recommended health behaviors [37] . More recently, several studies have investigated which factors were associated with compliance with the preventive measures during the COVID-19 pandemic. The results indicated that the demographic variables of male gender and younger age were associated with lower levels of compliance [38] [39] [40] , while-albeit not consistently-the demographic variables of a higher level of education and being married were associated with greater compliance [38] .

Together with age and education, we included level of personality dysfunction (as assessed by the DSM-5 approach [41] ) as a control variable, as several studies have shown that people react differently to threats according to certain personality traits [42, 43] . For instance, individuals with high levels of antagonism, which refers to aggressive tendencies accompanied by assertions of dominance and grandiosity, may argue with other people when their desires are not satisfied [44] . Moreover, high levels of disinhibition are commonly related to greater impulsivity and sensation seeking, which could lead to a tendency to ignore real dangers or threats. Finally, among personality traits, only agreeableness was associated with greater compliance, whereas aspects of the dark triad (i.e., narcissism, Machiavellianism, and psychopathy) and antisocial traits were predictive of lower levels of compliance [45, 46] . Despite the paucity of studies on personality variables and compliance with government recommendations during past pandemics, it is possible that higher levels of personality dysfunction could interfere with behavioral compliance.

Recently, researchers from a range of scientific fields (including the clinical and social sciences) have begun to emphasize the increased need to focus on prediction, rather than explanation, during data analysis [47] . Machine learning (ML) is a branch of artificial intelligence that focuses on data prediction. ML algorithms automatically learn information from a set of data and make predictions on unseen data without being explicitly programmed to do so. ML techniques have been shown to be particularly useful in predicting human behavior, including high-risk behavior [48] [49] [50] [51] . Indeed, one of the main advantages of ML is that it enables inferences to be made at the individual level, whereas traditional statistical methods focus primarily on the group level [52] . Thus, ML predictive models support researchers in making predictions for individual subjects, which is particularly useful for the development of personalized and targeted prevention campaigns. In the present study, ML algorithms were applied to the 14 psychosocial variables to predict individuals' likelihood of complying with the COVID-19 protective measures.

Research Question 4 (RQ4): Which variables would predict, with maximal accuracy, high versus low compliance with the protective health measures prescribed by the Italian government?

The research team assembled an online survey inspired by the literature and collected data over five days (18) (19) (20) (21) (22) March 2020). The questionnaire was administered cross-sectionally on an online survey platform, which participants accessed via a designated link. The link was disseminated through the main means of communication and social networks, in order to reach a large number of subjects. The study was approved by the local ethics committee (Board of the Department of Human Neuroscience, Faculty of Medicine and Dentistry, Sapienza University of Rome).

Ten questions were designed to investigate compliance with the COVID-19 protective measures (e.g., "It is suggested that all persons avoid crowded places. Are you complying with this?"). These questions were assessed on a five-point Likert scale ranging from 1 (slightly) to 5 (extremely), with Cronbach's alpha of 0.84.

When COVID-19 became pervasive, the Italian government announced guidelines for preventing infection, including "Disinfect hands often" and "Stay at home". Accordingly, 10 statements were provided to measure the perceived efficacy of 10 protective guidelines (e.g., "It is suggested that all persons avoid crowded places. Do you find this useful?") using a five-point Likert scale ranging from 1 (hardly) to 5 (extremely). Cronbach's alpha was 0.93.

Participants' self-efficacy with respect to protecting themselves from COVID-19 was measured using three questions (e.g., "I am confident in my ability to protect myself from COVID-19") adapted from previously validated measures [22] , which were assessed on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Cronbach's alpha was an acceptable 0.62.

Risk perception was assessed through two variables, perceived severity and perceived likelihood, using items adapted from Cho and Lee [22] and Liao, Cowling, Lam, Ng, and Fielding [53] . Perceived severity was assessed using four items (e.g., "If I got COVID-19, it would be severe"), with a Cronbach's alpha of 0.70. Perceived likelihood was assessed using two items ("How likely is it that you will get COVID-19 in this period?"), with Cronbach's alpha of 0.80. Six questions were assessed on a five-point Likert scale ranging from 1 (not likely at all) to 5 (certain).

The Civic Engagement Scale (CES) [31] was employed. The CES is a 14-item scale based on an understanding of civic engagement as "the process of believing that one can and should make a difference in enhancing his or her community". It measures two specific aspects of civic engagement: attitudes and behaviors. The first subscale (Attitudes), composed of eight items, assesses civic attitudes in terms of "the personal beliefs and feelings that individuals have about their own involvement in their community and their perceived ability to make a difference in that community". The second subscale (Behaviors), composed of six items, assesses civic behaviors in terms of "the actions that people take to actively attempt to engage and make a difference in their community". Both the Attitudes and the Behaviors subscale obtained high reliability in the validation study, with Cronbach's alphas of 0.91 and 0.85, respectively. In the present sample, Cronbach's alpha was 0.87 for both scales, and each item was assessed on a five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).

Personality dysfunction was investigated using the Personality Inventory for DSM-5-Brief Form-Adult (PID-5-BF) [54] . The PID-5-BF is a 25-item self-rated personality traits assessment. It measures five personality trait domains: negative affect, detachment, antagonism, disinhibition, and psychoticism. Each domain is measured through five items, which are rated on a four-point Likert scale ranging from 0 (very false or often false) to 3 (very true or often true). The overall measure generates scores in the range 0-75, with higher scores indicating greater overall personality dysfunction. Each trait domain receives a score in the range 0-5, with higher scores indicating greater dysfunction in that specific personality trait domain.

A total of 2812 respondents participated in the survey. All participants were aged 18 years or older and were living in Italy. The online survey was closed on the sixth day following dissemination of the link. All participants voluntarily responded to the anonymous survey and indicated their informed consent within the survey. The procedures were clearly explained, and participants could interrupt or quit the survey at any point without explaining their reasons for doing so. Two respondents were excluded from the sample because they were younger than 18 years, and 44 participants were excluded because they lived outside of Italy during the outbreak.

The final sample consisted of 2766 participants: 1982 (71.7%) females and 784 (28.3%) males. The mean age of the sample was 32.94 (13.2; range 18-90 years), and the majority was Italian citizens (N = 2739, 99%) . Most of the sample (N = 1194, 43 .2%) held a high school degree and were unmarried (N = 1866, 67.5%), unemployed (N = 1165, 42.1%), and childless (N = 2130, 77%) . Furthermore, most participants reported to be staying at home (N = 2368, 85.6%) and going out up to once per day (N = 2559, 92.5%).

A paired sample t-test was used to compare efficacy perception with behavioral compliance. Cohen's d [55] effect size was inspected for each significant effect.

The mediation and moderated mediation models were run using PROCESS version 3.5 [56] , as developed by Preacher and Hayes [57] for SPSS, version 25 (IBM, Armonk, NY, USA). Moderated mediation test simple mediation models (i.e., determining whether a given variable or mediator accounts for some or all of the relationship between two other variables) that may differ according to a further variable (e.g., if the mediation pathway is only present for individuals with higher or lower levels of certain variables). PROCESS estimates indirect effects (i.e., mediation) and conditional indirect effects (i.e., moderated mediation) using bootstrap confidence intervals. In the present study, the bias-corrected 95% confidence interval (CI) was calculated using 5000 bootstrapping resamples. Effects were considered significant if the resulting CI did not contain 0. All measures, including compliance with protective health measures, were treated as continuous variables except for the ordinal covariate "education". Considering PROCESS model templates [58] , we first tested a simple mediation model (Model 4) to explore if the association between perceived efficacy of the recommended health measures and compliance was mediated by self-efficacy. Next, we tested Model 76 to verify the moderated effect of risk perception and civic attitudes on the direct and indirect effects of the perceived efficacy of the health measures on compliance. "High" and "low" levels of both moderators were determined at one standard deviation above and below the mean of each scale. Taking into account previous findings [38] [39] [40] 42, 43] , all analyses controlled for age, education, and level of personality dysfunction. Finally, ML models were trained and tested in WEKA 3.9 [59] . The procedure used to build the models is reported in the "Results" section.

Descriptive statistics and scale correlations are reported in Table 1 . 

A paired sample t-test was conducted to investigate differences between the perceived efficacy of the recommended safety behaviors and compliance. The result was statistically significant-t (2765) = 37.384; p < 0.001-with an effect size (= 0.711) approaching Cohen's [55] standard for a large effect (= 0.80). The results further indicated a statistically significant reduction in scores for behavioral compliance (M = 41.7; SD = 6.20) relative to perceived efficacy (M = 44.8; SD = 6.17). Subsequent paired sample t-tests were run for each safety measure prescribed by the Italian government (e.g., "Avoid hugs", "Avoid handshakes," etc.). Since the category of protective health measures was very heterogeneous, being composed of 10 behavioral types, we conducted t-tests for each measure. Our aim was to thoroughly investigate whether a difference between perceived efficacy and compliance impacted some behaviors more than others. T-test results were significant for all protective measures except for "Avoid handshakes": t (2765) = 1.253; p < 0.210; d = 0.024 (see Table 2 ). 

Research Question 1 postulated that self-efficacy would mediate the relationship between the perceived efficacy of the recommended health measures and compliance. As shown in Table 3 , the total effect of perceived efficacy on compliance was significant (B = 0.750 (SE = 0.01); p < 0.001 (CI = 0.725, 0.775)). The mediation analyses showed that the indirect effect of perceived efficacy on compliance via self-efficacy was positive (0.034) and the bootstrapped 95% CI did not include 0 (0.026, 0.044). Furthermore, the covariates age, education, and personality dysfunction were significant: the first showed a positive association with behavioral compliance, whereas the others showed a negative association with the outcome variable. The final mediation model explained 60% of the total variance in compliance with the health measures: 97% of the total effect on compliance was explained by the direct effect of perceived efficacy, whereas 3% was explained by the indirect effect of the mediator. 

The results of the moderated mediation model related to conditional indirect effects, as presented in Table 4 . Research Question 2 was partially confirmed: risk perception emerged as a significant moderator in the relationship between the perceived efficacy of the recommended health measures and compliance (Research Question 2a), whereas no moderation effects of this variable emerged in either the relationship between perceived efficacy and self-efficacy or the relationship between self-efficacy and compliance (Research Questions 2b and 2c). Similar results were found for the moderating role of civic attitudes. As shown in Table 4 , this variable moderated the direct effect of perceived efficacy on compliance (Research Question 3a). However, no significant effects were found in the relationship between perceived efficacy and self-efficacy or the relationship between self-efficacy and compliance (Research Questions 3b and 3c).

Further, older age, lower educational levels, and lower personality dysfunction emerged as significant covariates that were positively associated with higher behavioral compliance.

As shown in Figure 2 , the simple slope analysis found that the positive relationship between perceived efficacy and behavioral compliance was significant (B = 0.78, p < 0.001) under both high (+1 SD) and low levels of perceived risk (−1 SD) 

To predict individuals' compliance with the COVID-19 protective measures based on the collected psychosocial variables (RQ1), participants were split into two classes: high compliance and low compliance. The cut-off for low versus high compliance was set to a total compliance score (i.e., the sum of the scores of all items concerning application of the safety measures) of 30, representing the midpoint of the total compliance score range (i.e., . Thus, the low compliance class included participants with a total compliance score ≤ 30 (N = 171); the high compliance class included participants with a total compliance score > 30 (N = 2595).

We followed the recommended procedure to ensure model generalization and to increase the replicability of the results, by splitting the data into two sets: a training set (used to train and validate the model) and a test set (used to test model accuracy on examples that had never been seen by the ML classifier) [54, 62] . In the present study, a percentage split of 80:20 training to test data was applied, with participants randomly assigned to one or the other set while maintaining the proportion between classes. Therefore, the training set consisted of 2213 participants (2076 high compliance and 137 low compliance), and the test set consisted of 553 participants (519 high compliance and 34 low compliance).

All the 14 collected variables were entered in the ML models as predictors: gender, age, education, work position, marital status, citizenship, child(ren), home/work, going out per day, selfefficacy, risk perception, civic attitudes, perceived efficacy, and personality disfunction. Using these 14 predictors, ML algorithms were run and validated on the training sample (N = 2213) using a 10fold cross-validation procedure. Specifically, k-fold cross-validation was used; this resampling COMPLIANCE Figure 2 . Simple slope analysis of the effect of perceived risk and civic attitudes on the relationship between perceived efficacy and compliance.

To predict individuals' compliance with the COVID-19 protective measures based on the collected psychosocial variables (RQ1), participants were split into two classes: high compliance and low compliance. The cut-off for low versus high compliance was set to a total compliance score (i.e., the sum of the scores of all items concerning application of the safety measures) of 30, representing the midpoint of the total compliance score range (i.e., . Thus, the low compliance class included participants with a total compliance score ≤ 30 (N = 171); the high compliance class included participants with a total compliance score > 30 (N = 2595).

We followed the recommended procedure to ensure model generalization and to increase the replicability of the results, by splitting the data into two sets: a training set (used to train and validate the model) and a test set (used to test model accuracy on examples that had never been seen by the ML classifier) [52, 60] . In the present study, a percentage split of 80:20 training to test data was applied, with participants randomly assigned to one or the other set while maintaining the proportion between classes. Therefore, the training set consisted of 2213 participants (2076 high compliance and 137 low compliance), and the test set consisted of 553 participants (519 high compliance and 34 low compliance).

All the 14 collected variables were entered in the ML models as predictors: gender, age, education, work position, marital status, citizenship, child(ren), home/work, going out per day, self-efficacy, risk perception, civic attitudes, perceived efficacy, and personality disfunction. Using these 14 predictors, ML algorithms were run and validated on the training sample (N = 2213) using a 10-fold cross-validation procedure. Specifically, k-fold cross-validation was used; this resampling procedure seeks to reduce variance in the model performance estimation by using a single training set and a single test set. It portions the sample into k = 10 subsets (folds), using 9 of them to train the model and the remaining subset to validate its accuracy. This procedure is repeated k = 10 times, with 1-fold left out each time as a validation set [61] . The final model metrics are obtained by averaging the metrics of the k = 10 validation subsets. Finally, the models developed through the 10-fold cross-validation were tested on the test sample (N = 553). Specific ML algorithms were selected to represent different classification strategies: logistic regression [62] , support vector machine [63] , naïve Bayes [64] , and random forest [65] . This ensured that the results were stable across classifiers and did not depend on specific model assumptions.

In running the classification algorithms, the class imbalance problem was addressed. ML methods work best with balanced datasets-a condition that is rarely met within scientific research with human subjects [66] . A different number of instances across classes may lead to an overall correct classification into the majority class and a complete misclassification of instances into the minority class. In the present research, the two classes were extremely unbalanced, with a ratio of 1:15 between the high compliance and low compliance groups. One strategy to overcome class imbalance consists of altering the relative costs associated with misclassifying the minority and majority classes, in order to compensate for the imbalance [67] . Following this methodology, in the present study, the ML algorithms were designed in such a way that an error in classifying the minority class (low compliance) was weighted 15 times more than an error in classifying the majority class (high compliance). This cost-modifying strategy has been shown to provide better results in addressing class imbalance than other methods, such as random oversampling of the minority class or random undersampling of the majority class [67] . Moreover, it should be noted that, for the goal of this task, it was more beneficial to obtain a lower number of false negatives than false positives; likewise, it was more beneficial to have a model with high sensitivity rather than high specificity. In other words, it was more important to not miss people with low compliance than to misclassify people as high compliance.

The final classification results of the 10-fold cross-validation and test set are reported in Tables 5  and 6 , respectively. The models' predictive performance was quantified using the following metrics: ROC area, accuracy, precision, recall (or sensitivity), and F-measure (F1 score). Of note, the classifiers showed an ROC area in the range of 0.82-0.91 in the test set. However, random forest and naïve Bayes classifiers highlighted a lower recall for low compliance compared to other classifiers, making them weaker models for the purposes of prediction.

Finally, to investigate the weight of the 14 predictors in the models, here we report the point-biserial correlation (r pb ) between the outcome and each predictor: perceived efficacy = 0.547; self-efficacy = 0.173; age = 0.104; going out per day = 0.101; perceived risk = 0.100; civic attitudes = 0.095; work/home = 0.080; child(ren) = 0.079; education = 0.077; marital status = 0.076; personality dysfunction = 0.072; gender = 0.060; work position = 0.057; citizenship = 0.029. 

Public willingness to comply with the protective health measures proposed by authorities is critical for controlling the outcomes of an infectious disease outbreak. The situation in many countries during the current COVID-19 pandemic indicates that, despite legal penalties and mass information campaigns, not all citizens have adopted the recommended behaviors to prevent the spread of the virus. Thus, the present study sought to identify which psychological and psychosocial factors might improve compliance.

The results confirmed our first research question, showing that self-efficacy significantly mediated the relationship between perceived efficacy and compliance. Stated differently, our findings suggest that individuals who perceive themselves as able to carry out (i.e., those with self-efficacy) those behaviors judged as effective in reducing the threat (i.e., behaviors with perceived efficacy) are more likely to comply with the government measures. This finding highlights the key role of self-efficacy in the adoption and maintenance of recommended health actions [21] and suggests a relationship between self-efficacy and compliance with the preventive measures during the COVID-19 pandemic, as previously reported for different populations (e.g., healthcare workers) [68, 69] . Furthermore, some covariates were also significant: older age, lower education levels, and lower levels of personality dysfunction were all associated with increased compliance. These results parallel preliminary findings on the COVID-19 outbreak, indicating that younger persons and those with higher education levels are less likely to comply with the recommended measures, especially those related to hygiene [70] . To the best of our knowledge, this study represents the first attempt to provide data on the association between personality functioning and compliance with health measures. The findings suggest that overall personality functioning may be significant in influencing individuals to adopt the protective measures recommended by authorities. For this reason, personality functioning should be assessed more frequently, and those already known to have a personality impairment (e.g., clinical patients) should be supported and controlled more promptly than others.

Our second and third research questions were only partially confirmed: on the one hand, risk perception and civic attitudes as moderators made the mediation of self-efficacy insignificant, thereby invalidating research questions 2b, 2c, 3b, and 3c. With respect to this unexpected result, we propose that the two relevant moderators made self-efficacy less important in influencing compliance with the health measures; however, further research investigating the associations between these variables is recommended. On the other hand, both risk perception and civic attitudes were found to significantly moderate the relationship between perceived efficacy and compliance. Thus, the impact of perceived efficacy on compliance varied in accordance with risk perception and civic attitudes, thereby confirming research questions 2a and 3a. As the moderation coefficient for both constructs was positive, higher risk perception and civic attitudes were associated with a stronger effect of perceived efficacy on compliance. Regarding risk perception, this result is in line with previous studies reporting a strong association between risk perception and changes in (or maintenance of) health behaviors in a variety of contexts, including epidemic/pandemics [26] [27] [28] 71] . With respect to civic attitudes, this is a novel finding of our study, as such attitudes have not previously been studied in the context of pandemic behavioral responses. The current finding of a link between civic attitudes and preventive health actions suggests the importance, for instance, of teaching civic education in the early school grades.

Finally, as regards the ML classification models outcome (RQ1), it has been shown that the above-mentioned psychological and psychosocial variables are able to predict which individuals have high versus low compliance, with an ROC area in the range of 0.82-0.91 and high sensitivity for the target class (low compliance). Taken together, the results of the moderated mediation and ML models underline that the most important variable for compliance with the recommended health behaviors is perceived efficacy, as has been consistently indicated by previous studies on behavioral responses to epidemics [15, 16, 28] . This result suggests that, to foster compliance, communications regarding COVID-19 containment measures should focus on perceived efficacy by highlighting the utility of the recommended behaviors.

Overall, our results indicated significantly lower scores in behavioral compliance compared to efficacy perception. The introduction of risk perception and civic attitudes as moderators rendered the mediating effect of self-efficacy insignificant. The impact of perceived efficacy on the adoption of recommended behaviors varied in accordance with risk perception and civic engagement. Finally, the ML classification models' outcome showed that the psychological and psychosocial variables considered are able to predict which individuals have high versus low compliance.

The present results should be interpreted with caution, due to some limitations. First, the cross-sectional study design, implemented during the first phases of the COVID-19 outbreak in Italy, prevented us from drawing causal inferences. We were unable to assess individuals' psychological functioning before the virus spread, and we were similarly unable to report behavioral compliance with the recommended health measures in more advanced phases of the outbreak. Furthermore, the data collection via a web-based survey relied on voluntary sampling and self-reported data; thus, the data may be distorted by selection or social desirability biases. Indeed, the survey clearly indicated the health authority recommendations (i.e., "It is suggested that . . . "). Thus, by disagreeing with the written precautions, participants would have revealed their violation of the official recommendations. Given that such violations would have been seen as socially undesirable, participants may have been reluctant to disclose this.

Despite these limitations, our findings have several significant implications due to the lack of relevant research on targeted interventions to enhance public compliance with government health recommendations during the COVID-19 outbreak. Government awareness communications and campaigns regarding COVID-19 and related protective measures should be tailored to specific segments of the population, as defined by age and level of education. Furthermore, countries affected by COVID-19 should consider relevant psychological dimensions alongside their lockdown protocols [72] . In particular, government strategists may use the findings of the present study on the psychological characteristics of people who do and do not comply with the containment measures (i.e., perceived efficacy, risk perception, civic attitudes) to target their COVID-19 communications more effectively. Overall, we believe that our findings will be helpful in guiding age-tailored information/advertising campaigns in countries affected by COVID-19 and directing further research on behavioral compliance. Funding: This research received no external funding.

The authors declare no conflict of interest.

Monitoring the level of government trust, risk perception and intention of the general public to adopt protective measures during the influenza A (H1N1) pandemic in the Netherlands

The Effect of Risk Perception on the 2009 H1N1 Pandemic Influenza Dynamics

Emergency Committee Regarding the Outbreak of Novel Coronavirus (2019-nCoV)

A Nationwide Survey of Psychological Distress among Italian People during the COVID-19 Pandemic: Immediate Psychological Responses and Associated Factors

The effect of the COVID-19 lockdown on parents: A call to adopt urgent measures

How Personality Relates to Distress in Parents during the Covid-19 Lockdown: The Mediating Role of Child's Emotional and Behavioral Difficulties and the Moderating Effect of Living with Other People

Individual Differences, Economic Stability, and Fear of Contagion as Risk Factors for PTSD Symptoms in the COVID-19 Emergency

The Health Belief Model and Sick Role Behavior

A Protection Motivation Theory of Fear Appeals and Attitude Change1

Cognitive and psychological processes in fear appeals and attitude change: A revised theory of protection motivation

Utility of the health belief model in examining medication compliance among psychiatric outpatients

Measuring perceived benefits and perceived barriers for physical activity

Can the Health Belief Model and moral responsibility explain influenza vaccination uptake among nurses?

When to accentuate the negative: The effects of perceived efficacy and message framing on intentions to perform a health-related behavior

Public perceptions, anxiety, and behaviour change in relation to the swine flu outbreak: Cross sectional telephone survey

Study of H1N1 Influenza Preventive Behaviors Predictors Based on Health Belief Model in Jiroft People

Predictors of non-adherence to public health instructions during the COVID-19 pandemic

Self-efficacy

Self-efficacy models. In Behavioral Health: A Handbook of Health Enhancement and Disease Prevention

Self-efficacy and the health belief model

Stress, Self-Efficacy, and Health

The influence of self-efficacy, subjective norms, and risk perception on behavioral intentions related to the H1N1 flu pandemic: A comparison between Korea and the US

Self-efficacy in the adoption and maintenance of health behaviors: Theoretical approaches and a new model

Social-cognitive predictors of health behavior: Action self-efficacy and coping self-efficacy

Meta-analysis of the relationship between risk perception and health behavior: The example of vaccination

Perceived risk, anxiety, and behavioural responses of the general public during the early phase of the Influenza A (H1N1) pandemic in the Netherlands: Results of three consecutive online surveys

H1N1 Preventive Health Behaviors in a University Setting

Prevalence of preventive behaviors and associated factors during early phase of the H1N1 influenza epidemic

Perceived Risk and Efficacy Beliefs as Motivators of Change

An Examination of the Effect of Perceived Risk on Preparedness Behavior

Community Responses during Early Phase of COVID-19 Epidemic

Chinese public's knowledge, perceived severity, and perceived controllability of the COVID-19 and their associations with emotional and behavioural reactions, social participation, and precautionary behaviour: A national survey

User-generated content on the internet: An examination of gratifications, civic engagement and psychological empowerment

Donation via Mobile Applications: A Study of the Factors Affecting Mobile Donation Application Use

Why does COVID-19 disproportionately affect older people? Aging

Demographic and attitudinal determinants of protective behaviours during a pandemic: A review

Public Trust and Compliance with the Precautionary Measures Against COVID-19 Employed by Authorities in Saudi Arabia

Sociodemographic and Psychological Correlates of Compliance with the COVID-19 Public Health Measures in France

Prevalence and Predictors of Anxiety and Depression Symptoms during the COVID-19 Pandemic and Compliance with Precautionary Measures: Age and Sex Matter

Diagnostic and Statistical Manual of Mental Disorders

Vigilance for threat: Effects of anxiety and defensiveness

The Impact of Worry on Attention to Threat

A Review of Systems for Psychology and Psychiatry: Adaptive Systems, Personality Psychopathology Five (PSY-5), and theDSM-5

Who complies with the restrictions to reduce the spread of COVID-19?: Personality and perceptions of the COVID-19 situation

Compliance with containment measures to the COVID-19 pandemic over time: Do antisocial traits matter? Pers

Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning

Machine Learning Approaches for Clinical Psychology and Psychiatry

Indicators to distinguish symptom accentuators from symptom producers in individuals with a diagnosed adjustment disorder: A pilot study on inconsistency subtypes using SIMS and MMPI-2-RF

Introducing machine learning to detect personality faking-good in a male sample: A new model based on MMPI-2-RF scales and reaction times

Use of mouse-tracking software to detect faking-good behavior on personality questionnaires: An explorative study

Anxiety, worry and cognitive risk estimate in relation to protective behaviors during the 2009 influenza A/H1N1 pandemic in Hong Kong: Ten cross-sectional surveys

The Personality Inventory for DSM-5-Brief Form

Statistical Power Analysis for the Behavioral Sciences

Partial, conditional, and moderated moderated mediation: Quantification, inference, and interpretation

SPSS and SAS procedures for estimating indirect effects in simple mediation models

Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach

The WEKA data mining software

The reusable holdout: Preserving validity in adaptive data analysis

A study of cross-validation and bootstrap for accuracy estimation and model selection

Ridge Estimators in Logistic Regression

Improvements to Platt's SMO Algorithm for SVM Classifier Design

Estimating Continuous Distributions in Bayesian Classifiers

Random forest

The class imbalance problem: Significance and strategies

The class imbalance problem: A systematic study1

Factors associated with preventive behaviours of COVID-19 among hospital staff in Iran in 2020: An application of the Protection Motivation Theory

Knowledge, perceptions and preventive practices towards COVID-19 early in the outbreak among Jimma university medical center visitors

Non-compliance with COVID-19-related public health measures among young adults in Switzerland: Insights from a longitudinal cohort study

The dynamics of risk perceptions and precautionary behavior in response to 2009 (H1N1) pandemic influenza

Incremental conditions of isolation as a predictor of suicide in prisoners

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license