key: cord-1011648-pn908w1n authors: Insfran-Rivarola, Andrea; Tlapa, Diego; Limon-Romero, Jorge; Baez-Lopez, Yolanda; Miranda-Ackerman, Marco; Arredondo-Soto, Karina; Ontiveros, Sinue title: A Systematic Review and Meta-Analysis of the Effects of Food Safety and Hygiene Training on Food Handlers date: 2020-08-25 journal: Foods DOI: 10.3390/foods9091169 sha: 10b0311a98e9cfc3be6ca04b4305a8c7ac6ed933 doc_id: 1011648 cord_uid: pn908w1n Foodborne diseases are a significant cause of morbidity and mortality worldwide. Studies have shown that the knowledge, attitude, and practices of food handlers are important factors in preventing foodborne illness. The purpose of this research is to assess the effects of training interventions on knowledge, attitude, and practice on food safety and hygiene among food handlers at different stages of the food supply chain. To this end, we conducted a systematic review and meta-analysis with close adherence to the PRISMA guidelines. We searched for training interventions among food handlers in five databases. Randomized control trials (RCT), quasi-RCTs, controlled before–after, and nonrandomized designs, including pre–post studies, were analyzed to allow a more comprehensive assessment. The meta-analysis was conducted using the random-effects model to calculate the effect sizes (Hedges’s g) and 95% confidence interval (CI). Out of 1094 studies, 31 were included. Results showed an effect size of 1.24 (CI = 0.89–1.58) for knowledge, an attitude effect size of 0.28 (CI = 0.07–0.48), and an overall practice effect size of 0.65 (CI = 0.24–1.06). In addition, subgroups of self-reported practices and observed practices presented effect sizes of 0.80 (CI = 0.13–1.48) and 0.45 (CI = 0.15–0.76) respectively. Food safety is a global public health threat with frequent incidents of foodborne diseases. Additionally, the COVID-19 outbreak has put more pressure on global public health; particularly, organizations of producers and providers along the food supply chain are facing an ongoing challenge to improve and to extreme food safety and hygiene due to the pandemic. In this context, foodborne diseases are responsible for major economic costs for a country [1, 2] . In terms of global estimates, in 2010, 31 foodborne hazards caused 420,000 deaths and 600 million foodborne illnesses derived from disease agents, such as non-typhoidal Salmonella enterica, Salmonella typhi, Taenia solium, hepatitis after applying the inclusion criteria, only 31 papers were eligible for inclusion in the literature review (see Figure 1 ). We classified the 31 final papers into three categories based on their main outcomes: changes in knowledge, attitude, and practices toward food safety and hygiene following training interventions. Twenty-six of the 31 studies reported changes in knowledge, 12 discussed changes in attitude, and 16 reported changes in food safety practices. Regarding the publication rate, we found that food safety and hygiene training interventions seem to have increased since 2011. Regarding the country of origin, most of the studies were published in the United States (29%), followed by Malaysia (13%), and Canada, Brazil, and the United Kingdom, with equal proportions (6.5%), see Supplementary Tables S2 and S3 . As for the research settings, the studies were conducted mainly in schools or universities (5/31), food process facilities (4/31), hospitals (4/31), restaurants (3/31), street food establishments or food trucks (3/31), farms/greenhouses (2/31), and multi-settings (2/31), among others. Regarding the sample size, the studies varied from n = 10 to n = 194. There were 64 different interventions conducted among the 31 studies, with face-to-face/lectures (25/64) being the most frequent type of training intervention, followed by lectures combined with practice demonstrations (14/64) , computer-based training (6/64), videos (4/64), videos combined with either a lecture (1/64) or a lecture and a demonstration (2/64), lectures combined with an incentive (1/64) or with demonstrations and incentives (2/64), and booklets (2/64), among others. We found that no studies We conducted a comprehensive search on the following databases: PubMed, Cochrane Controlled Register of Trials (CENTRAL), Ebsco, Scopus, and Web of Science. Also, we searched for grey literature on Google Scholar and ProQuest. In relation to the search strategy, we relied on both the Peer Review of Electronic Search Strategies (PRESS) [61] and the PICOS (population, intervention, comparator, outcome, and study design) elements. The ultimate search strategy is described in the Supplementary Foods 2020, 9, 1169 4 of 24 Data S1. We searched for publications in English published between January 1997 and December 2019. Likewise, we examined the reference lists of the retrieved articles to look for further relevant literature. The last search was run in April 2020. Two authors reviewed the titles and abstracts of the work retrieved during the search. Discrepancies were resolved by discussion and consensus with a third author. All of the reviewed works were conducted among food handlers from different steps of the food supply chain, including farms, food processing facilities, and restaurants (i.e., from farm to fork). Interventions were defined as food safety and hygiene training sessions covering aspects such as personal hygiene, hand washing, cleaning and sanitization, cross-contamination, foodborne diseases, and temperature control. Training was given in the form of talks, demonstrations, self-practice, and different sources of communication, including posters, videos, booklets, slideshows, and fact sheets. We searched for randomized controlled trials (RCTs), quasi-RCTs, and controlled before-after (CBA) studies. In addition, we searched for non-randomized designs, including uncontrolled pre-post studies, to allow a more comprehensive and complete assessment of the available evidence in the area, recognizing that RCTs may not be feasible for many large-scale food safety education interventions [62] [63] [64] [65] [66] . The reported food safety training sessions were aligned with regulations, protocols, and guidelines, including, but not limited to, the United Nations' (UN) Codex Alimentarius, the HACCP, the Food and Drug Administration (FDA) Food Code (including the hand-washing guidelines and protocol), the FDA's Employee Health and Personal Hygiene Handbook, the United States Department of Agriculture (SDA) Food Safety Education campaign, the European Union General Food Law, Regulation (EC) No. 852/2004, the United Kingdom's Safety Act, the GMPs, and the Good Hygiene Practices (GHPs). In all of the studies, the comparison group included either participants (i.e., food handlers) who did not receive food safety training or those who had not yet received proper food safety training. As the main outcomes, all included studies evaluated changes in knowledge, attitude or practice among food handlers. Knowledge refers to the degree of understanding of food handlers about the food safety information given during training sessions. In contrast, attitude refers to a predisposition or tendency to respond positively or negatively to training. Finally, practices are the actions of an individual in response to the knowledge and attitude involved in the training sessions. Similarly, food safety practices can be defined as the increased use of evidence in healthcare practice and policy when both knowledge of, and attitude toward, food safety are present. Changes in levels of knowledge were measured in the studies through survey-questionnaire data gathered in Likert-type scales with sub-dimensions such as food poisoning, cross-contamination, temperature control, and personal hygiene. Changes in self-reported attitudes toward food safety and hygiene were also measured through survey-questionnaire data on Likert-type scales. Finally, changes in practices were measured, such as self-reported practices and observed practices, the former through survey-questionnaire data in a Likert-type scale and the latter through checklists. Both used different sub-dimensions, including personal hygiene, food safety, and hygiene, temperature control, cross-contamination, sanitation, storage, and food display. We discarded any case report/series and/or review studies with data missing (e.g., sample size, mean, standard deviation), as well as studies conducted among people other than food handlers (e.g., consumers and food transporters). Two independent reviewers screened each potential article to identify its abstract, title, keywords, and concepts reflecting both the article's contribution and the research context. Disagreements were overcome by discussion. Then, the relevant full-text studies were retrieved and independently assessed by two reviewers against the review's inclusion/exclusion criteria. Once more, disagreements were overcome by discussion and consensus with a third author. The data were extracted by one reviewer and checked by a second reviewer. The extracted raw data from each study included authors' names, Foods 2020, 9, 1169 5 of 24 year of publication, country of origin, title, study setting, study length, study aim, study design, study population, participant demographics, details on the training interventions and control conditions, recruitment and study completion rates, outcomes, measurement times, and information on the risk of bias. The data were arranged manually and tabulated using standardized forms including data from studies that fulfilled our requests for additional information. We stratified data into comparable subgroups for meta-analysis for each outcome: knowledge, attitude, and practice. Furthermore, we separated practice into two subgroups: self-reported practices and observed practices. As in similar cases [57, 66] , due to studies using different measurement instruments and scales, we calculated the Hedge's g standardized mean differences (SMD) to measure the effect size, as proposed by Borenstein et al. [67] . Due to variation across studies, we conducted a random effect meta-analysis using Hedges's g with a 95% confidence interval (CI) and the two-sided p-value for each outcome [67, 68] . Heterogeneity among the studies in terms of effect measures was assessed using the I 2 statistic. This index can be interpreted as the percentage of total variability in a set of effect sizes due to true heterogeneity (between-studies variability) [69] . Higgins et al. 2003 suggested the use of I 2 values of 25%, 50%, and 75% as low, moderate, and high, respectively [70] . Thus, an I 2 value greater than 50% is indicative of substantial heterogeneity. We also assessed the evidence of risk of publication bias through a funnel plot and statistical tests, including Egger's test [71] and the Begg's test [72] (with a 95% confidence interval). We ran the meta-analysis in RStudio using the metafor package [73] and the meta package [74] . To reduce the risk of bias, two independent reviewers assessed each study. Randomized studies were assessed by using Cochrane's tool RoB2 [75, 76] . Here, the judgment criteria included 3 levels (low risk of bias, some concerns, or high risk of bias) for each of the 5 bias domains. Nonrandomized studies were assessed by using the ROBINS-I tool [77] ; the judgment criteria included 5 levels (low, moderate, serious, critical, and no information) for each of the 7 bias domains [78] . The risk of bias visualization was done using robvis [79] . Finally, we summarized the findings reported in each study (Table 1) . During the initial search, we found 1094 papers. Then, after removing duplicates, our database was reduced to 321 papers. Following data screening and the application of exclusion criteria, we removed 200 more studies. One hundred twenty-one studies underwent full-text review. However, after applying the inclusion criteria, only 31 papers were eligible for inclusion in the literature review (see Figure 1) . We classified the 31 final papers into three categories based on their main outcomes: changes in knowledge, attitude, and practices toward food safety and hygiene following training interventions. Twenty-six of the 31 studies reported changes in knowledge, 12 discussed changes in attitude, and 16 reported changes in food safety practices. Regarding the publication rate, we found that food safety and hygiene training interventions seem to have increased since 2011. Regarding the country of origin, most of the studies were published in the United States (29%), followed by Malaysia (13%), and Canada, Brazil, and the United Kingdom, with equal proportions (6.5%), see Supplementary Tables S2 and S3 . As for the research settings, the studies were conducted mainly in schools or universities (5/31), food process facilities (4/31), hospitals (4/31), restaurants (3/31), street food establishments or food trucks (3/31), farms/greenhouses (2/31), and multi-settings (2/31), among others. Regarding the sample size, the studies varied from n = 10 to n = 194. There were 64 different interventions conducted among the 31 studies, with face-to-face/lectures (25/64) being the most frequent type of training intervention, followed by lectures combined with practice demonstrations (14/64), computer-based training (6/64), videos (4/64), videos combined with either a lecture (1/64) or a lecture and a demonstration (2/64), lectures combined with an incentive (1/64) or with demonstrations and incentives (2/64), and booklets (2/64), among others. We found that no studies used any kind of intervention involving social media. Regarding the type of study, eleven studies were pre-post studies, twelve relied on RCT, and eight performed a cross-sectional study with a trained group and a non-trained group. As for the measurement instruments, twelve studies administered surveys, one administered a test, two used checklists, and the rest did not report the used measurement instruments. Regarding gender, 13 papers reported that the majority of participants were female, while males represented the majority in nine studies, and one study had an equal proportion (50% of each). Eight studies did not report gender. The main outcomes, descriptions, statistics, and other relevant information of each study are summarized in Table 1 . We performed a meta-analysis of the effects of food safety training interventions on the KAP of food handlers. Overall, we found that food safety training interventions had a significant effect on knowledge changes, with an SMD of 1.24 (CI = 0.89 to 1.58; p-value = 0.0001). In relation to attitude, our analysis results indicate that food safety training has a positive effect, giving an SMD of 0.28 (CI = 0.07 to 0.48; p-value = 0.008) for the attitudes of food handlers toward food safety and hygiene. Finally, with respect to practice, the overall effect size was estimated to be SMD = 0.65 (CI = 0.24 to 1.06; p-value = 0.0018). For those interventions with self-reported practices, we found an effect size of SMD = 0.80 (CI = 0.13 to 1.48; p-value = 0.0201). In contrast, for studies reporting observed practices, the effect size was SMD = 0.45 (CI = 0.15 to 0.76; p-value = 0.0035). Figures 2-6 show the forest plot for each outcome. Overall, food safety KAP was significantly higher as a result of training interventions. This phenomenon was particularly noticeable in the knowledge component. The forest plot in Figure 2 shows that most of the individual results lay close to 1. Such results strongly suggest that training increases knowledge of food safety and improves food safety attitudes and practices among food handlers. 95.3%), attitude (I 2 = 77.7%), and for practice (I 2 = 94.9%). Regarding the risk of bias for randomized studies, six studies were evaluated with some concerns of risk of bias, four studies with low risk, and two studies with high risk. For nonrandomized studies, ten were evaluated with moderate risk of bias, nine studies with serious risk, and none as low risk. The visualization data are shown in Supplementary Tables S4 and S5. Crags-Dino (2002) This systematic review has summarized the effects of training interventions on the knowledge, attitudes, and practices of food handlers towards food safety and hygiene. Change in knowledge was assessed in 26 out of 31 studies; therefore, this was the most frequently reported outcome. This result is consistent with previous studies [58, 107] , and a significant amount of information is available, so it is probably easier to measure knowledge than attitude or practice. We found evidence that training interventions have a significant effect on increased knowledge toward food safety and hygiene across different type of settings such as fresh produce [91] , food service operators [108] , schools [80] , restaurants [82] , households [101] , and multi-settings [97] . On the other hand, one study found no difference in knowledge between a control and an intervention group except for a positive attitude, so it can be considered to be optimistically biased [90] . This phenomenon has been demonstrated in previous research [90, 109, 110] . Attitude was assessed in 12 out of 31 studies, most of them assessing one intervention while some studies evaluated two [9, 95] or three interventions [87] . Considering the summarized effect size, a SMD = 0.28 suggests a moderate effect for the positive attitude of food handlers; this is similar to previous studies [57, 58, 66] . Both studies [9, 95] reported similar improvements in attitudes, either 9.1% 9.4% 8.9% 9.6% 9.4% 9.5% 9.3% 9.3% 6.7% 9.5% 9.3% Figure 6 . Forest plot-Self-reported practice. We graphically assessed the risk of publication bias through funnel plots which, as the supplementary Figures S1-S5 depict, were symmetric. The null hypothesis for the Begg's and Egger's tests indicated an absence of bias in the selected studies. For knowledge, the Egger's test did not indicate any risk of publication bias, while the Begg's test did indicate a moderate level of risk (i.e., Begg's test: p-value = 0.044 and Egger's test: p-value = 0.054); however, the data seem symmetric in the funnel plot (see Figure S1 ). As for the effects of food safety training on attitude changes, we also found no evidence of publication bias, both in tests and in the plot (Begg's test: p-value = 0.653 and Egger's test: p-value = 0.763). Finally, we found no statistical evidence of a risk of publication bias for the practice component (Begg's test: p-value = 0.472 and Egger's test: p-value = 0.608) and the graphic shows symmetry as well. In this review, the heterogeneity was considered high for knowledge (I 2 = 95.3%), attitude (I 2 = 77.7%), and for practice (I 2 = 94.9%). Regarding the risk of bias for randomized studies, six studies were evaluated with some concerns of risk of bias, four studies with low risk, and two studies with high risk. For nonrandomized studies, ten were evaluated with moderate risk of bias, nine studies with serious risk, and none as low risk. The visualization data are shown in Supplementary Tables S4 and S5. This systematic review has summarized the effects of training interventions on the knowledge, attitudes, and practices of food handlers towards food safety and hygiene. Change in knowledge was assessed in 26 out of 31 studies; therefore, this was the most frequently reported outcome. This result is consistent with previous studies [58, 107] , and a significant amount of information is available, so it is probably easier to measure knowledge than attitude or practice. We found evidence that training interventions have a significant effect on increased knowledge toward food safety and hygiene across different type of settings such as fresh produce [91] , food service operators [108] , schools [80] , restaurants [82] , households [101] , and multi-settings [97] . On the other hand, one study found no difference in knowledge between a control and an intervention group except for a positive attitude, so it can be considered to be optimistically biased [90] . This phenomenon has been demonstrated in previous research [90, 109, 110] . Attitude was assessed in 12 out of 31 studies, most of them assessing one intervention while some studies evaluated two [9, 95] or three interventions [87] . Considering the summarized effect size, a SMD = 0.28 suggests a moderate effect for the positive attitude of food handlers; this is similar to previous studies [57, 58, 66] . Both studies [9, 95] reported similar improvements in attitudes, either with face-to-face training or computer-based (CB) instruction. This is consistent with [84] , who stated that participants learned equally well whether the instructional format was CB or instructor-led training. In addition, in studies where food handlers had attended food hygiene training previously [97, 103] , food safety attitude remained the same. According to our findings, most studies reporting an increase in knowledge also reported an increase in attitude [9, 97, 105, 106] . However, an increase in knowledge might not necessarily bring about an improvement in attitude. This was the case for four studies [80, 85, 86, 100] . The reason for this is unclear, yet some factors that could partially explain this could be length of the training [80] , lack of repetition of the training [86] , or previous hygiene enforcement program within the control group [85] . Attitude is a measure of the degree to which a person has a favorable or unfavorable evaluation of behavior [27] . In this regard, providing employees with training that does not promote a positive change with attitude [80] , subjective norms, and perceptions of control may not contribute to improving intention (and ultimately behavior) to perform the behaviors [111] . Practice and behavior were measured in 16 studies, two of them assessing two outcomes (self-reported and observed practice) and the rest just one. The summarized effect of food safety training on practices showed that the interventions increased food safety practices, both for the 11 studies with self-reported practices and the seven studies with observed practices. Previous studies reported similar improvements, either self-reported or observed practices, but with a slightly smaller effect for the self-reported practices [38, 97] ; this consistent agreement between self-reported and observed behaviors was reported previously [23] . However, this is contrary to expected, since self-reported data are usually susceptible to social desirability bias [112] , i.e., the tendency of respondents to give socially desirable responses in such a way as to be viewed favorably by others [113] . Thus, respondents tend to overestimate their food safety practices as being higher than their actual practices deserve [38, 66, 114, 115] . On the other hand, observed practices could be affected by the "Hawthorne effect" where the changes in a person's behavior may be due to the presence of an observer. In this research, inconsistencies between self-reported and observed practices were detected by [106] , with 95% being the self-reported rate of washing hands and 82.5% for keeping hair covered with a cap; however, the observations showed only 50% and 17.5% of compliance, respectively. For studies assessing practices thorough observations, evaluation was mainly done using a checklist [38, [97] [98] [99] 116] . The implementation of food safety and hygiene practices has the final objective of preventing foodborne illnesses. Food safety behaviors are often subdivided into specific behavioral constructs such as personal hygiene, adequate cooking of foods, avoiding cross-contamination, keeping foods at safe temperatures, and avoiding food from unsafe sources [117] . Behavior outcomes provide a more direct measure of intervention effectiveness compared to knowledge and attitudes [66] ; however, food safety practices were measured in only 16 out of the 31 studies. This is consistent with the proportions reported by Viator et al. [107] . Moreover, an integrative review conducted by Zanin et al., [118] stated that 50% of the selected studies reported no translation of knowledge into attitudes/practices. In this review, we found evidence of close to 25% translation into both attitudes and practices. In addition, food safety practices of food handlers are associated with the type of management, i.e., tending to be higher in corporate-managed than owner-operated [31] . Incorporating practical assessment, such as observations, could help owner-operated organizations, since in some cases observation is more important than self-reported practices in order to represent actual behaviors [99, 119] . Overall, all nine food safety training interventions that incorporated theory and practice (T&P) demonstrations were more effective in terms of knowledge gain than those that only incorporated theoretical training. This is consistent with [83] , who found that training that incorporated active participation was more effective than traditional passive instruction. Nevertheless, those studies reporting T&P presented a poor improvement in attitude [85, 86] . Finally, the seven and eleven interventions based on T&P and theory, respectively, showed similar practice improvement in 71% and 80% of the studies, respectively. Although the ultimate goal is to prevent foodborne diseases, no study reported an impact on this goal. As expected, the results were based around the change in KAP as a mean to avoid food safety risk. Thus, theoretical training based on KAP is commonly used to improve handlers' food safety performance [106] . However, some authors have reported flaws, mainly in the assumption that the received information is translated into practices and behaviors [100, 103] . Food safety and hygiene are critical in all steps in the farm-to-fork chain. In an ideal scenario of the farm-to-fork continuum, a total absence of foodborne pathogens and opportunistic bacteria is obviously desired [120] . Nevertheless, despite good knowledge, attitude, and self-reported practices, there may be poor performance in hygiene [121] and food safety practices. Bacteria might exist in nature in a range of different metabolic stages, such as dormant, active, and growing; thus, it is important to detect bacteria and ascertain whether they are potentially active [120] . Despite the central role that food workers' hands play in bacterial transfer among food and various surfaces [81] , only one study assessed the number of bacteria growing on cultures obtained from the hands [86] , while another demonstrated cross-contamination with hand hygiene sessions using GloGerm ® powder and UV light [91] . Both studies showed improved knowledge of food handlers. Similarly, it is well known that an effective way to control food poisoning is to maintain hygienic surroundings [103] . Thus, additional evaluations and inspections including surface cleanliness and hand cultures seem to be a suitable part of training [122] . Similarly, frequent practical and hands-on sessions will create a much more vivid experience for workers [83, 89, 91] . Active learning, e.g., a training session that raises awareness of the possibility that E. coli bacteria may accumulate under the fingernails should also demonstrate the correct handwashing procedure and require the learner to practice until he or she can successfully demonstrate effective performance of that procedure [85] . Also, risk perception acts as a guide for decisions about behavior and can be a barrier to following a particular activity or procedure or not [123] . In this regard, there are different approaches to food safety training. Some include cases of victims of food poisoning [91] during food safety training to connect with audiences' lifestyles, incorporate fear, and enhance the perception of risk [58] . Moreover, to be effective, training programs should be based on appropriate adult education theory [124] , the possibility of human error [125] , and make sure that the reading comprehension level of the text is suitable for most food handlers [9] . Training programs that are more closely associated with a worksite are potentially more effective, especially if supported by practical reinforcement of the message [85, 126] . The frequency [51] and length of exposure [127] for a training program are significative factors in the obtained outcome. For studies reporting the length of intervention, the majority were conducted in one day with a follow-up period between 2 and 8 weeks, with 1 year being the longest follow up period [82] . Moreover, because knowledge decreases over time [5] , food safety and hygiene training should be provided frequently [51] to prevent the information from being forgotten and also to increase the level of knowledge [86] . Some studies suggest refresher retraining after 2 years [108] and before 5 years from initial certification [5] . For food establishments, we found that the educational level and professional training have significant effects on knowledge, practice [49, 98] , and food handlers' positive attitudes [49, 103] . However, the inclusion of adult education concepts, skill-based programs with interconnected sessions [85] , and even the use of YouTube ® videos [91] can be effective for low literacy audiences. In this regard, farm employees with low educational attainment have also demonstrated significant knowledge gain [85, 91] . Commitment and motivation from supervisors and management, as well as proper support and facilities given to staff are critical for the success of food safety and hygiene intervention. Training moves people in the right direction but not far enough [88] . In this regard, food handlers' attitudes are significantly related to the management environment [31] , thus supervisory support enforcement plays a significative role [85] in demonstrating and emphasizing the importance of following proper food safety practices [88] , as well as being role models themselves [91] . Moreover, because transforming knowledge into behavior is complex, training from top management to all employees is crucial [128] , inasmuch as successful food safety intervention must be based on firm theories [99] . Furthermore, additional key factors are the supervisors' years of experience [5] , clear responsibilities of food managers, and written agreement related to practicing sanitization procedures [99] , as well as trained and certified managers helping to reduce critical food safety violations [129] . In terms of settings, most of the studies were carried out in restaurants and street food establishments, hospitals and schools, greenhouses and farms, and industrial food processing companies. This is in accordance with a previous study which found that the most frequently reported settings were restaurants and street food establishments [58] . In this context, the restaurant industry has been labeled as one of the most recurrent sources of foodborne illness outbreaks [130] . Therefore, food safety certification of kitchen managers appears to be a significant factor in outbreak prevention in restaurants [131] . A combination of inspection results with a mandatory training and certification program may mitigate food safety risks [132] . Many barriers and factors (environmental, social, cultural, belief systems, and so on) can affect whether food handlers effectively implement food safety practices in their workplaces [30, 31, 122, 133] , including a lack of adequate food safety training, time pressure, competing job tasks, lack of or inconvenient locations of equipment/resources, lack of managerial support, lack of motivation/incentive, lack of reminders, or lack of clarity in food safety messages [25, 90, 98, 122, [134] [135] [136] . As expected, studies from developing countries have experienced some fundamental barriers, including a lack of infrastructure, poor working conditions, ill-functioning equipment, a lack of water, and insufficient supervision [89, 93] . Interestingly some studies from developed countries have experienced some limitations regarding literacy [94] and a potential language barrier [83] , mainly because food handlers were not native speakers. Regarding the training interventions among the selected studies, 27% were based on international guidelines (including WHO, HACCP, GMP, and ServSafe ® ), 18% on national guidelines, 18% on previous studies, and the remaining studies did not report this information. The guidelines vary by sector (restaurants, meat industry, dairy industry, etc.), legislation, or requirements of the country or region in which a company is located, market conditions, and certifications. Despite the frequent food-related incidents attaching great importance to the certification system [137] , only 41% of the included studies awarded some national or international certification for food handlers. High costs could discourage companies from implementing certifications. In this sense, local governments should support organizations [137] , mainly those that rarely invest in training or certification. A powerful way to win the interest of politicians and policy makers is to be able to attach a monetary value to food-related illness [138] . In this regard, the overall annual estimated cost of foodborne illness has remained relatively constant since 2005 at approximately GBP 1.5 billion in England and Wales and 152 billion USD in the USA [138] . Even though regulations and voluntary certifications are commonly thought of as driving forces to improve the safety and quality of food products [137] , legislation might lead food handlers to undergo training only for certification without being motivated to acquire and use new knowledge [97] . A study found that the number of food safety violations did not differ as a function of certification [129] . Thus, certifications and legal requirements may not guarantee food safety [139] . Our study has several major limitations. Firstly, differences in data (settings and data collection/processing approaches) and the multi-component nature of food safety and hygiene training makes it difficult to generalize the results. Second, most studies used observational pre-post designs. As a result, the absence of matched comparison groups, the potential presence of confounding variables, and the lack of randomization prevented the reported outcome improvements from being causally linked to the interventions. Third, the evaluation of KAP limited our ability to make conclusions about the behavior of the food handler. Fourth, knowledge, attitude, and practice are often subdivided into specific constructs; however, our ability to investigate these concepts in detail was limited by the availability and reporting of primary research, as many studies only reported overall scores or scales. Moreover, the determination of workers' behavior using the self-reported technique before education was an important limitation in some included studies. Finally, there is a possibility that the "Hawthorne effect" led to the improvements reported in the studies. Foodborne diseases continue to be a global problem, causing substantial morbidity and mortality and significant costs. According to our results, food safety and hygiene training have positive impacts on food handlers' knowledge, attitude, and practice. Effective and frequent food safety training of food handlers continues to be an initial step in ensuring that food safety concepts are at least introduced. Despite knowledge being delivered by training, it cannot just be translated into desired changes in attitudes and practice. The inclusion of practical demonstration and continuous support might increase positive attitudes towards food safety and hygiene practices among food handlers with the ultimate goal of minimizing the incidence and prevalence of foodborne hazards. Moreover, effective food safety training should be relevant to the situation, promote active learning, increase risk perception, and consider the work environment. Because computer-based (CB) training was not found to differ from face-to-face training in terms of the outcome obtained, CB programs could be used more extensively, since they are an efficient and cost-effective way to educate staff. In this regard, we identified several barriers to attaining proper food safety and hygiene practices, which should be considered by educators with appropriate adjustments according to the stage of the food supply chain, as well as the market, regional, and cultural characteristics. Similarly, training interventions should be based on international or national guidelines and adapted to different sectors, legislations, and certifications. Furthermore, local governments should support organizations, especially those that rarely invest in training and certification like SMEs, small farms, restaurants, or street food services. Finally, certifications and legal requirements may not guarantee food safety and hygiene, but when properly supported by resources, commitment, leadership, and a receptive management culture, food safety and hygiene practices may improve. Supplementary Materials: The following are available online at http://www.mdpi.com/2304-8158/9/9/1169/s1, Table S1 : PRISMA checklist, Data S1: Search strategy, Table S2 : Geographical distribution of studies selected, Table S3 : Distribution per year of studies selected, Figure S1 : Funnel plot for knowledge, Figure S2 : Funnel plot for attitude, Figure S3 : Funnel plot for overall practice, Figure S4 : Funnel plot for self-reported practice, Figure S5 : Funnel plot for observed practice, Table S4 : Risk of bias for randomized studies, and Table S5 : Risk of bias for nonrandomized studies. Estimating global mortality from potentially foodborne diseases: An analysis using vital registration data Reported foodborne outbreaks due to fresh produce in the united states and European Union: Trends and causes Foods 2020 Disease Burden Epidemiology Reference Group; World Health Organization. WHO Estimates of the Global Burden of Foodborne Diseases; World Health Organization Food safety, Hazard Analysis and Critical Control Point and the increase in foodborne diseases: A paradox? Food Control Evaluation of food safety knowledge, attitudes and self-reported hand washing practices in FOODSAFE trained and untrained food handlers in British Columbia Foodborne and Food-Handler Norovirus Outbreaks: A Systematic Review The growing burden of foodborne outbreaks due to contaminated fresh produce: Risks and opportunities Healthline Editorial Team; Higuera, V. Worst Foodborne Illness Outbreaks in Recent U Comparison of Knowledge and Attitudes Using Computer-based and Face-to-Face Personal Hygiene Training Methods in Food Processing Facilities Epidemiology of food borne diseases: A worldwide review WHO surveillance programme for control of foodborne infections and intoxications: Preliminary results and trends across greater Europe Clinical microbiology laboratories' adoption of culture-independent diagnostic tests is a threat to foodborne-disease surveillance in the United States Quality and safety standards in the food industry, developments and challenges Application of ISO22000 and comparison to HACCP for processing of ready to eat vegetables: Part I The Role of Auditing, Food Safety, and Food Quality Standards in the Food Industry: A Review Food safety knowledge, optimistic bias and risk perception among food handlers in institutional food services The Health Belief Model: A Decade Later The Health Belief Model and Sick Role Behavior A survey of environmental health officers' views of food hygiene training Hygienic food handling behaviours. An application of the Theory of Planned Behaviour Using the Theory of Planned Behaviour to design a food hygiene intervention An application of the theory of planned behavior-A randomized controlled food safety pilot intervention for young adults Efficacy of the theory of planned behaviour model in predicting safe food handling practices Food handlers' beliefs and self-reported practices From Intentions to Actions: A Theory of Planned Behavior The theory of planned behavior A review of food safety and food hygiene training studies in the commercial sector Evaluation of a food hygiene training course in Scotland Food hygiene training: Introducing the Food Hygiene Training Model Investigating a link of two different types of food business management to the food safety knowledge, attitudes and practices of food handlers in Beirut Advocacy, Communication and Social Mobilization for TB Control A Guide to Developing Knowledge, Attitude and Practice Surveys; World Health Organization Bridging the gap between formal and informal science learning Organizational learning in practice: How do small businesses learn to operate internationally? Hum. Resour Speed of Internationalization, the Knowledge Link and Effects of Creativity Health Behaviour and Health Education: Theory Research and Practice The evaluation of food hygiene knowledge, attitudes, and practices of food handlers' in food businesses in Turkey Evaluation of the food safety training for food handlers in restaurant operations Attitude is Everything World Health Organization. Foodborne Disease: A Focus for Health Education Knowledge, attitude and practice the three pillars of excellence and wisdom: A place in the medical profession Attitudes, and Practices of Food Service Staff Regarding Food Hygiene in Shiraz Food Hygiene Knowledge, Attitudes and Practices of the Food Handlers in the Military Hospitals Food safety knowledge, attitudes and practices of food handlers in lebanese hospitals: A cross-sectional study Assessment of Knowledge, Attitudes and Practices (KAP) Among food handlers at residential colleges and canteen regarding food safety Food Safety Attitudes, Beliefs, Knowledge and Self-Reported Practices of College Students Before and After Educational Intervention Food safety attitudes in college students: A structural equation modeling analysis of a conceptual model Knowledge, attitude and practices of food safety amongst food handlers in the coastal resort of Guangdong, China Food court hygiene assessment and food safety knowledge, attitudes and practices of food handlers in Putrajaya Food Safety Knowledge, Attitude and Practice of Food Handlers and Microbiological and Chemical Food Quality Assessment of Food for Making Merit for Monks in Ratchathewi District Knowledge, attitude and practices regarding food hygiene and sanitation of food handlers in Kuala Pilah Assessment of knowledge, attitude and practices concerning food safety among restaurant workers in Putrajaya The food safety culture in a large South African food service complex: Perspectives on a case study Knowledge and Attitudes towards Handling Eggs in the Home: An Unexplored Food Safety Issue? Consumer food safety knowledge and practices in the home in Turkey Food Safety Knowledge: The Case of Domestic Food Handlers in Accra Effectiveness of food handler training and education interventions: A systematic review and meta-analysis Meta-analysis of food safety training on hand hygiene knowledge and attitudes among food handlers Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement Cochrane Handbook for Systematic Reviews of Interventions Chapter 24: Including non-randomized studies on intervention effects. In Cochrane Handbook for Systematic Reviews of Interventions; The Cochrane Collaboration Standards of evidence: Criteria for efficacy, effectiveness and dissemination Criteria for evaluating evidence on public health interventions A systematic review and meta-analysis of the effectiveness of food safety education interventions for consumers in developed countries Introductionto Meta-Analysis Research synthesis as a scientific process Assessing heterogeneity in meta-analysis: Q statistic or I 2 Index? Measuring inconsistency in knowledgebases Bias in meta-analysis detected by a simple, graphical test Operating Characteristics of a Rank Correlation Test for Publication Bias Foods 2020 Conducting meta-analyses in R with the metafor RoB 2: A revised tool for assessing risk of bias in randomised trials RoB2 Development Group Revised Cochrane Risk-of-Bias Tool for Randomized Trials ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions The Development group for ROBINS-I. Risk of Bias. In Non-Randomized Studies of Interventions (ROBINS-I): Detailed Guidance; The Cochrane Collaboration Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments The Effect of Food Safety Education on the Knowledge Attitude and Behavior of Foodservice Workers of Inner City Public Schools Evaluating the Effectiveness of Food Safety Messages Along the Agri-Food Chain from Farm to Fork Evaluation of a HACCP pilot program for the food service industry A Comparison of Traditional Handwashing Training with Active Handwashing Training in the Food Handler Industry The Usefulness of Computer-Based Instruction in Providing Educational Opportunities for Nursing Staff Development and assessment of pilot food safety educational materials and training strategies for Hispanic workers in the mushroom industry using the Health Action Model The hygiene training of food handlers at a teaching hospital Intervention improves restaurant employees' food safety compliance rates Appreciation of food safety practices based on level of experience Will capacity building training interventions given to street food vendors give us safer food? A cross-sectional study from India An evaluation of process specific information resources, aimed at hazard analysis, in Small and Medium Enterprises in food manufacturing Food safety training and evaluation of handwashing intention among fresh produce farm workers Compare the effects of two educational methods on the health principles knowledge of employees in food preparation Evaluation of compliance to food saftey standards amongst food handlers in selected Hospitals in Kenya Development and Implementation of a "Counter-Top" Training Program to Increase Retention of Food Safety Knowledge, Alter Behavior, Improve Attitude and Increase Skills of Spanish-speaking Retail Employees Effectiveness of Food Safety Managerial Training: Face-to-Face or Computer-Based Delivery Training: A vital tool for improving the knowledge and practice of food safety and hygiene among food handlers in boarding secondary schools in Plateau State The role of theoretical food safety training on Brazilian food handlers' knowledge, attitude and practice Evaluation of the effect of person-in-charge (PIC) program on knowledge and practice change of food handlers in Dubai Effect of food safety training on food handlers' knowledge and practices: A randomized controlled trial Knowledge, attitudes and practices (KAP) on good manufacturing practices (GMP) among food handlers in Terengganu hospitals Role of courtyard counselling meeting in improving household food safety knowledge and practices in Munshiganj district of Bangladesh The effect of Food Safety Education on handwashing practices in school canteens' food handlers An assessment on pre-and post-food hygiene training on food safety's Kap level among food handlers in Kuala Terengganu and Kuala Nerus Knowledge and attitudes on food hygiene among food services staff on board ships Food safety knowledge, attitudes, and behavior of street food vendors and consumers in Handan, a third tier city in China Food safety knowledge, attitudes, and practices of brazilian food truck food handlers Preventing and controlling foodborne disease in commercial and institutional food service settings: A systematic review of published intervention studies A Comparison of Food Safety Knowledge among Restaurant Managers, by Source of Training and Experience Food hygiene and hazard analysis critical control point in United Kingdom food industry: Practices, perceptions, and attitudes Understanding and Predicting Food Handlers' Implementation of Specific Food Safety Practices using Social Cognition Models Identifying Specific Beliefs to Target to Improve Restaurant Employees' Intentions for Performing Three Important Food Safety Behaviors Predictors of young consumer's green purchase behaviour Development and validation of a scale to capture social desirability in food safety culture Self-reported and observed behavior of primary meal preparers and adolescents during preparation of frozen, uncooked, breaded chicken products Comparison between self-reported and observed food handling behaviors among latinas Socio-economic profile and food safety knowledge and practice of street food vendors in the city of Guwahati Evaluation of food safety education for consumers Knowledge, attitudes and practices of food handlers in food safety: An integrative review Examining the predictive utility of an extended theory of planned behaviour model in the context of specific individual safe food-handling High-throughput 16S rRNA sequencing to assess potentially active bacteria and foodborne pathogens: A case example in ready-to-eat food Assessment of Food Safety Knowledge, Attitude, Self-Reported Practices, and Microbiological Hand Hygiene of Food Handlers Application of the WHO keys of safer food to improve food handling practices of food vendors in a poor resource community in Ghana The interrelationship between perceived knowledge, control and risk associated with a range of food-related hazards targeted at the individual, other people and society Competency-based adult learning in food safety programs. In Food Protection Technology Factors Influencing the Human Error of Workers in Manual Assembly Lines Evaluation of Food Hygiene Education Estimating the Effects of Length of Exposure to a Training Program: The Case of Job Corps; IZA Discussion Paper Food safety knowledge and practices among food handlers in Slovenia Effect of a Manager Training and Certification Program on Food safety and Hygiene in Food service Operations Restaurant Inspection Scores and Foodborne Disease Systematic environmental evaluations to identify food safety differences between outbreak and nonoutbreak restaurants Does mandatory food safety training and certification for restaurant employees improve inspection outcomes? Factores de Éxito de Lean Seis Sigma en la Industria Alimentaria Factors Impacting Food Workers' and Managers' Safe Food Preparation Practices: A Qualitative Study Restaurant Employees' Perceptions of Barriers to Three Food Safety Practices Food safety challenges and barriers in southern United States farmers markets Government regulations and voluntary certifications in food safety in China: A review Foodborne Diseases: Prevalence of Foodborne Diseases in Europe Effectiveness of an intervention package on knowledge, attitude, and practices of food handlers in a tertiary care hospital of north India: A before and after comparison study This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license Acknowledgments: This study was supported by Mexico's National Council of Science and Technology, the Programa para el Desarrollo Profesional Docente, para el Tipo Superior (PRODEP) Program and the Universidad Autónoma de Baja California. The authors declare no conflict of interest.