key: cord-0783337-1atebjvg authors: Amer, Faten; Hammoud, Sahar; Khatatbeh, Haitham; Lohner, Szimonetta; Boncz, Imre; Endrei, Dóra title: The deployment of balanced scorecard in health care organizations: is it beneficial? A systematic review date: 2022-01-13 journal: BMC Health Serv Res DOI: 10.1186/s12913-021-07452-7 sha: c4048060c682a305516f0ea9e34b11813f1c8309 doc_id: 783337 cord_uid: 1atebjvg BACKGROUND: Balanced Scorecard (BSC) has been implemented for three decades to evaluate and improve the performance of organizations. To the best of the researchers’ knowledge, no previous systematic review has performed a comprehensive and rigorous methodological approach to figure out the impact of BSC implementation in Health Care Organizations (HCO). AIMS: The current work was intended to assess the impact of implementing the BSC on Health Care Workers’ (HCW) satisfaction, patient satisfaction, and financial performance. METHODS: The authors prepared the present systematic review according to PRISMA guidelines. Further, the authors customized the search strategy for PubMed, Embase, Cochrane, Google Scholar databases, and Google’s search engine. The obtained studies were screened to isolate those measuring scores related to HCW satisfaction, patient satisfaction, and financial performance. The Risk of Bias (RoB) in the non-Randomized Intervention Studies (ROBINS-I) tool was used to assess the quality of observational and quasi-experimental studies. On the other hand, for the Randomized Controlled Trials (RCTs), the Cochrane (RoB 2) tool was used. RESULTS: Out of 4031 studies, the researchers included 20 studies that measured the impact of BSC on one or more of the three entities (HCW satisfaction, patient satisfaction, and financial performance). Throughout these 20 studies, it was found that 17 studies measured the impact of the BSC on patient satisfaction, seven studies measured the impact on HCW satisfaction, and 12 studies measured the impact on financial performance. CONCLUSION: This systematic review provides managers and policymakers with evidence to support utilizing BSC in the health care sector. BSC implementation demonstrated positive outcomes for patient satisfaction and the financial performance of HCOs. However, only a mild impact was demonstrated for effects related to HCW satisfaction. However, it is worth noting that many of the studies reflected a high RoB, which may have affected the impacts on the three primary outcomes measured. As such, this systematic review reflects the necessity for further focus on this area in the future. Moreover, future research is encouraged to measure the real and current impact of implementing BSC in HCO during the pandemic since we did not find any. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12913-021-07452-7. Since its development by Norton and Kaplan in 1992 [1] , the Balanced Scorecard (BSC) has been utilized by many health care managers for the performance evaluation (P.E.) of Health Care Organizations (HCO) worldwide. Moreover, BSC can also be used as a strategic managerial tool by linking it to the organization's strategy [2] . The first generation of the BSC was used to evaluate four organizational perspectives: the financial perspective, the customers' perspective, the internal processes' perspective, and finally, the learning and growth perspective, all of which were steered by the organizational vision and strategy [1] . See Fig. 1 . In this regard, it should be noted that a recent review re-categorized the BSC four perspectives for HCO into further 45 sub-dimensions or categories [3] . In the second generation of BSC, strategic maps were added to describe the cause-effect relationships between strategic objectives of each perspective [4] . In the third generation of BSC, destination statements, measures, and action plans were added to achieve the intended targets [5] . It is worth mentioning that Duke Children's Hospital in the United States of America (USA) was the first HCO to implement BSC in 2000. As a result, the hospital was able to convert 11 million dollars of losses into four million dollars of profits [6] . See Fig. 2 , which shows Duke University's health system strategic map [7] . The BSC strategic maps show that the process flow of the cause-and-effect relationships ends with the customer and financial perspectives [4, 8] . More recently, the pandemic of Coronavirus Disease 2019 (COVID- 19) imposed financial burdens on many countries and health care systems worldwide. In addition, the pandemic increased the psychological stress of patients and HCW [3, 9, 10] . Moreover, implementing tasks with particular standards and guidelines was vital in tackling the spread of COVID-19 [11] . However, conflicting managerial decisions and the lack of standardization capability were factors that brought dissatisfaction about HCO [12] . Consequently, it is deemed essential to evaluate previous BSC implementation's effectiveness in HCO and determine if actual benefits remain to justify their continued use. It is worth noting that numerous systematic reviews regarding the impact of BSC implementation in non-health care-related fields, such as architecture [13] , management, marketing, and accounting, already exist [14] . However, BSC reviews in health care only described the application of BSC or its perspectives [3, 15] . They lacked evaluation of the effectiveness of using BSC in HCO through systematically reviewed and evaluated literature. In this regard, only two reviews discussed the impact of BSC; one discussed the impact qualitatively [16] , while the other mentioned a few examples of the positive impact [8] . To reiterate, this indicates that, until now, no comprehensive or rigorous methodological approach to assess the impact of BSC implementation in HCO has been recorded. Based on this gap in the literature, it was deemed essential to determine whether the previous BSC implementations at HCO were beneficial. Thus, the present systematic review aims to gather all studies which have measured the impact of implementing BSC on HCW satisfaction, patient satisfaction, and financial performance at HCO; particularly, since these three attributes represent the latest affected perspectives in the strategic maps [4, 8] . Further, this review aims to assess and compare results among the included studies. Our previous systematic review analyzed the dimensions and indicators of BSC utilized at the P.E. of HCO [3] . This systematic review was carried out by finding all studies that approached BSC implementation's impact in HCO in adherence with the 27-point of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [17] . See (S1 Appendix). The inclusion and exclusion criteria were set as shown in Table 1 below. In the present systematic review, the search strategy was developed by the first, second, and fourth authors; because the first two authors are experts in health care management and BSC, and the fourth author is an expert in systematic reviews and meta-analysis. The search strategy was initially developed for the PubMed database based on the PICO (Population, Intervention, Comparison, and Outcome) tool [18] , and depending on using both MeSH (Medical Subject Headings) terms and keywords. See (Table 1 ). Next, the strategy was adapted to Cochrane Central Register of Controlled Trials (CEN-TRAL), Embase, and Google Scholar databases, as per Cochrane's recommendations [19] . The strategies developed for these databases can be found in (S2 Appendix). The grey literature, pre-prints, and unpublished studies were searched on Google Scholar and Google's search engine websites to reduce publication bias. Furthermore, the authors attempted to identify other potentially eligible studies or ancillary publications by searching the reference lists of any potentially eligible studies. The databases were searched until October 2020. Afterward, the first author conducted the search strategies on the electronic databases and removed the duplicates using the EndNote X9.2 program. The first and second authors independently performed the selection of eligible studies. A discussion after each step was made or, if necessary, the third author was consulted for arbitration in case of disagreements. Initially, the titles and abstracts of the studies were examined to eliminate irrelevant studies. In the second step, the full texts of all potentially relevant studies were carefully reviewed to make a final decision based on the criteria mentioned above. Authors of studies with no available full texts or unclear impact duration were contacted to obtain further details and clarification. The following types of data were extracted from each of the final eligible studies: 1) author/s, year of [7] publication, 2) country, 3) type of study, 4) duration of data collection 5) setting, 6) the number of health facilities, 7) number of participants, 8) data collection tool or data source, and 9) outcome (impact on patient satisfaction, HCW satisfaction, and financial performance). The data extraction was carried out independently between January and March 2021 by the first and second authors. The research design of eligible studies was extracted directly from them. In case the research design was not explicitly mentioned, it was determined based on the role of the investigator of a given study. Specifically, if the BSC exposures were naturally determined and the investigator had no part, the study was considered observational. On the other hand, when the investigator assigned the BSC intervention, the study was deemed to be experimental. The impact of BSC in eligible studies was explicitly mentioned or determined by calculating the difference between before and after implementation values. After that, the unification of the units was performed. Next, charts plotting for each outcome were performed by the first author and then reviewed by the first and second authors separately. If the impact measurement unit was not reported in the eligible studies, the authors of these studies were contacted. Lastly, all differences were compared, discussed, and judged by the two authors in the final step. The Risk of Bias (RoB) assessment was performed by the first and second authors independently between March and June 2021 to assess the quality of the included studies. As per the Cochrane collaboration's guidelines, the Cochrane (RoB 2) tool was used for the assessment of randomized controlled trials (RCTs) [20] . The Risk of Bias in non-Randomized Intervention Studies (ROB-INS-I) tool was used to assess the observational and quasi-experimental studies [21] . As per the Cochrane Handbook, authors should avoid summarizing the overall RoB [22, 23] . Therefore, the RoB was analyzed at the study level and across studies. In (RoB 2) tool, five types of bias were assessed: bias arising from the randomization processes, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of outcomes, and bias in the selection of the reported results. On the other hand, in the ROBINS-I tool, seven types of bias were assessed: bias due to confounding, bias in the selection of participants in a study, bias in measurement/ classification of interventions/ exposures, bias due to deviations from intended interventions/ exposures, bias due to missing data, bias in the measurement of the outcomes, bias in the selection of the reported results. While using the RoB 2 tool, each type of bias was assessed as low, high, or unclear. While using the ROB-INS-I tool, each type of bias was evaluated into five Comparator -Initiation of BSC implementation (at least one year of implementation) -Or: Comparing 2 measurements after BSC implementation for at least one year -Or: Gross change/ difference after at least one year of implementation -Initiation of BSC implementation was in less than one year. -Gross change/ difference after less than one year -One-time measurement with no comparability. No limitation was set in the search strategy, studies that measured BSC impact within less than one year of implementation were excluded after carefully examining the full texts. -Impact on financial indicators: profitability/loss, change in total revenues, change in total cost, ROI, ROA either in currency or in percentage. -Or: Impact on the patient satisfaction rate -Or: Impact on the HCWs' satisfaction rate -The impact should be objective and measured/ quantitative. -Impact on other indicators. [24] . Lastly, it was recommended not to advocate quality appraisal as a criterion for inclusion in reviews [25] . Therefore, the authors decided to include all studies in this systematic review regardless of their quality assessment. Initially, the search strategy resulted in a total of 4031 studies. After removing the duplicates, a total of 2985 studies remained, which were screened based on their titles and abstracts. Then, irrelevant studies were excluded; thus, 202 studies remained. A careful examination of the included studies' full texts was made; based on this, only 20 studies were finally included in the current systematic review. Details of the study selection process are shown in the PRISMA flow-chart (Fig. 3 ). The main characteristics of the included studies are shown in Table 2 . Regarding the implementation location, nine studies were implemented in North America, two in Europe, one in Africa, seven in Asia, and one did not specify the location. It should be noted that 14 studies were performed in high-income countries, two in upper-middle-income countries, one in a lower-middle country, and only two in low-income countries. Out of the 20 selected studies, 16 were performed in hospitals or hospital departments, and four in health care facilities or clinics. See Table 2 . Even though no limitation was imposed on language, all of the selected 20 studies measuring the impact of BSC implementation were written in English. Out of the 20 selected studies, only three studies reported their study designs explicitly. However, our classification showed that 11 studies were observational since the investigators were not involved in implementing BSC; instead, these investigators only observed the results of already implemented BSCs at HCO. On the other hand, the remaining nine studies were experimental. One out of the nine was RCT, while the other eight were quasiexperimental studies, which included three pretest-posttest components and five Interrupted Time Series. See Table 2 . Notably, only three studies [32, 36, 37] randomly selected HCO, participants, or both. Variances among the data collection instruments used in the 20 studies are shown in Table 2 . Notably, the employed instruments were validated only in six studies [27, 28, 32, 35, 39, 42] . Additionally, only five studies [27, 28, 39, 42, 43] assessed the instruments' feasibility. The pre-testing of the instruments was carried out only in three studies [27, 32, 35] . In addition, only five studies [27, 31, 38, 42, 43] assigned weights for the indicators or assessed their importance before implementation. Further, only one study [29] evaluated the indicators depending on more than one source for the same variable. The 20 studies chosen for this systematic review utilized different BSC generations. The first generation of BSC was employed in seven studies [28-31, 33, 39, 42] which discussed explanations, the definition of perspectives and indicators, and how to measure each indicator. Besides these seven studies, one other study [34] used the first generation BSC; however, only customer and patient satisfaction were explained in the way they were measured. Further, only five of the 20 studies [27] [28] [29] [30] 33 ] specified the source for each perspective/indicator, while one study [42] mentioned them partially. The aspects of BSC's second-generation were found in five of the 20 studies [26, 28, 29, 31, 34] , where users modified the objectives of each indicator during implementation to suit strategy, vision, mission, and goals. Additionally, two other studies [42, 43] modified these objectives partially but failed to explain them sufficiently. Further, strategic maps were only illustrated in six studies [26, 28, 31, 33, 38, 40] . Finally, it is worth noting that only three studies [28, 29, 31] displayed the cause-effect cascade between indicators and targets. Regarding the third generation's aspects, seven of the 20 studies [26, 28-31, 33, 34] approached destination statements or targets within a time horizon. Besides, one study [6] approached the length of stay indicator only. Additionally, only one study [26] approached strategic initiatives or action plans to achieve the targeted performance. The included studies assessed different outcomes for implementing BSC. Out of the final 20 eligible studies, 17 studies [6, 26-29, 31-38, 42-44] measured the impact of BSC on patient satisfaction, 7 measured HCW satisfaction [27-30, 36, 37, 45] , and 12 studies measured financial performance [6, 27-30, 33, 34, 40, 41, 43-45] . However, the measured variables varied among studies, even in terms of the same dependent variable (Figs. 4, 5, 6, 7). For example, BSC's impact on patient satisfaction varied from overall satisfaction to the satisfaction of specific categories, such as adults, children, inpatients, outpatients, patients in the emergency room, patients in rehabilitation. In addition, the measured variables varied based on the service type, such as satisfaction of home care services and departmental services. Regarding HCW satisfaction, the name assigned to the targeted population varied from staff, employees to HCW. Further, the HCW satisfaction type varied, for instance, from HCW satisfaction towards their job to HCW satisfaction towards their superiors. However, the financial variable had the greatest variation among all three primary outcomes measured. Specifically, it was found that there exists a reduction in costs, expenditures, HCW budget, expenses, catering expenses, expenses/net revenues, bad debt expenses per net revenue, and supply per net revenue. On the other hand, an increase in revenues types included; returns, profits, aggregate surplus, funds, the value of drug-related groups, and return on assets. Moreover, the unit used for financial impact assessment differed among studies. For example, all studies used currencies for assessment, where these currencies also varied between studies, except for a few studies [27, 33, 34] which used a percentage method. As an attempt to reduce bias, all currencies were converted to United States Dollar to standardize and make the comparison across studies more consistent regarding the financial outcomes in the systematic review. Further, the authors of one study [28] were contacted for clarification since they did not report the currency. As a result, Figs. 6 and 7 were designed as seen below; one for the impact in currencies and the other for the impact in percentages. Most studies used a percentage score to measure the impact on patient and HCW satisfaction, except for three studies [28, 30, 45] , which performed the measurement based on a four or five point-Likert scale. However, to make the comparison consistent, all Likert scales were converted to percentages (scores out of 100%). It should be noted that only two of the 20 studies [35, 45] discussed the statistical significance of the results. Hence, the magnitude of change (in percentage) was taken into consideration in our analysis. To make an objective comparison and avoid falling into bias, we reported the time between implementing BSC and assessing its impact in the included studies. Further analysis demonstrated that the intervals between the initiation of BSC implementations and the measurements of impacts varied among studies. In particular, one study [29] reported the results based on 18 months of implementation, two studies [38, 44] took 2 years, one study [45] took 3 years, two studies [6, 35] took 4 years, and one study [28] took 7 years of implementation. The remaining 13 studies reported results based on 1 year of implementation. Due to the previously mentioned variations in measured variables, duration of implementing BSC, and differences in data collection instruments or data sources, refer to (Table 2) , the authors decided that conducting a meta-analysis would not lead to meaningful results. Instead, a comparison of the impact was performed using the bar charts. See (Figs. 4, 5, 6, 7) . The outcomes of BSC implementation in each of the 20 included studies are shown above in (Figs. 4, 5, 6, 7) , which illustrate the impact on patient satisfaction, HCW satisfaction, and financial performance (in percentage and currency). The exact impact type and duration were specified for each measurement, and the results are discussed further in the discussion section below. Since we did not limit our search strategy to the three BSC impacts defined in our aim, eight studies of different impact types resulted in the abstract screening process. Examples of the other BSC impact types include; influencing the market share, the number of new patients, the number of consultations and visits, community satisfaction, percentage of residents receiving outreach activities, number of sampled children, availability of essential drugs, and decreasing the average length of stay [46, 47] . However, after reading the full texts, the authors decided to exclude these studies as they lacked sufficient relevance to our research aim. See (S 3 Appendix). As illustrated in S4 Appendix, each study was evaluated in terms of RoB. For that purpose, the RoB 2 tool was employed to assess the sole RCT study [32] , in which the assessment was deemed fair, except for the performance bias. On the other hand, the RoB in the quasiexperimental and observational studies was measured using the ROBINS-I tool; and it was found that there was no information about analysis methods of confounders' adjustments except in four studies [33, 36, 39, 42] . The confounding agents were apparent in three studies [27, 35, 37] . However, the three studies failed to adjust for the confounders, which may have affected the precision of the measurement. Furthermore, the selection bias across studies reflected serious RoB in five studies [27, 30, 31, 33, 40] . A possible reason the intervention and the follow-up did not coincide together and a potentially substantial amount of follow-up time was missing in the analyses. The moderate risk of bias showed that the intervention status was well defined, but some aspects of the assignments of intervention status were determined retrospectively. Further, outcome measurements bias was raised either due to the non-blinding of intervention among assessors [37] or because the outcome measure was subjective and likely to be influenced by other factors [29, 38] . See (S4 Appendix). This systematic review aimed to identify all the studies which measured the impact of BSC implementation on three variables: HCW satisfaction, patient satisfaction, and financial performance at HCO, and then proceeded to analyze the effect of these BSC implementations. The analysis of the results reflected a remarkably positive impact of BSC on patient satisfaction in most studies. The same positive impact of BSC implementation holds for financial performance in both currency and percentage indicators. Notably, the authors found that almost all studies showed a positive impact, amounting to several million dollars. However, a few studies reflected a moderately negative impact on financial performance, which form three distinct categories. The first category includes study [29] , which explained the occurrence of unintended events that may have negatively affected financial performance. The second category comprises studies [30, 40] that revealed a highly positive impact on financial performance in previous or subsequent years, which may reflect a sloth in following up. The third category includes studies [28, 39] that showed a positive impact on financial performance on one or more of the other impact types. On the other hand, the analysis of BSC impact for HCW satisfaction revealed a less remarkably positive impact. See (Figs. 4, 5, 6, 7) . The findings obtained from the present systematic review are in line with a systematic review [14] that reviewed BSC's benefits in business, management, and accounting fields. Furthermore, the present study is the first to summarize all BSC implementations and their impacts on the health care sector based on quantitative comparisons. Moreover, the current study was compared with other reviews in the health care sector. For instance, a review [15] carried out a mere description regarding the application of BSC. In contrast, a review [3] only summarized the perspectives and dimensions utilized. Lastly, a review [16] only mentioned examples about BSC impact. One probable explanation for the mild impact on HCW satisfaction can be referred to the lack of managerial engagement with the non-managerial HCW upon BSC implementation, the lack of understanding by HCW about the advantages of BSC implementation s, or the fear of potential responsibility and accountability placed upon HCW due to BSC implementation. As a result, HCW may have declined to implement BSC, contributing to a lower satisfaction score. In conclusion, future researchers should consider increasing employee participation in BSC implementations. For instance, in a study [29] , the employees did not have incentives or motives to participate in BSC since they were permanent employees. Further, the study showed that HCW above 40 years old negatively influenced creativity and productivity upon BSC implementation. Other researchers in [48] also referred to this challenge, who noted that major deficiencies arose from qualified personnel and HCW aging. However, those researchers have also suggested that the high-ranking qualifications of HCW, driving learning and a growth perspective, will eventually generate motivation for new HCW to resolve this issue. Other proposed ideas to solve this problem were creating an open environment for learning and growth and encouraging active communication with HCW to ensure the successful implementation of BSC. Other researchers [49] encouraged senior management commitments to involve non-managerial HCW, promoting clear articulation of benefits and relevancy of BSC to clinicians. This challenge mirrors the findings of another review [50] , which realized that the attitude perceived by health care professionals of accreditation was negative and skeptical because of quality concerns regarding services and their cost. Therefore, the authors in the latter study suggested that health care professionals, especially physicians, require more intensive education about the potential benefits of accreditation. Finally, the quality assessment revealed that many studies had high RoB, which may have affected the impact results. A recommendation for the researchers and managers implementing BSC in the future is to dedicate more focus to raising the quality of implementation and lowering the RoB. Moreover, a better focus on the second and third generations of BSC aspects is essential. The current systematic review contains several strengths. To our knowledge, this is the first paper that has analyzed all the studies which measured the impact of BSC on patient satisfaction, HCW satisfaction, and financial performance in HCO. The results and analysis of this systemic review support a positive impact for applying BSC in HCO, especially on patient satisfaction and financial performance. Further, a greater emphasis on the role of HCW is required when implementing BSC since HCW satisfaction showed slightly positive, almost zero, or somewhat negative scores in most studies included. Additionally, the three primary outcome measures concentrated upon in this systematic review are considered the last destination for impact in the strategic maps and the causal effects at most BSC studies. Finally, unlike other BSC reviews [8, 16] , which included definitions of biobanks, pharmacies, laboratories, radiology, and medical colleges in HCO, this review limited the definition to the primary, secondary, or tertiary health care organizations. This strategy leads to the homogeneity of the resulting studies and leads to more valid comparisons among the results. Nevertheless, this paper has some limitations. First, it focused on the impact of BSC on the three chosen indicators only, whereas impacts on other types of indicators were not considered for analysis. Due to the vast variations of indicator types, analysis of these indicators presents a challenge, requiring narrowly specified modes of analysis. Secondly, no meta-analysis could be applied to this systematic review resulting from the heterogeneity of studies regarding their data collection tools and the enormous variation in the types of indicators. However, the later variation was clarified in the charts, and the data collection tool was specified for each study. Thirdly, the current review included studies that measured the impact after at least 1 year of implementation. Fourthly, it is essential to mention that the impact comparability is roughly more rational for patient satisfaction and HCW satisfaction than financial performance. This could be referred to as the comparison ability based on a percentage score of 100 for the satisfaction variables. Additionally, the change in financial performance based on currency could be influenced by other confounding factors such as the HCO size or the number of health facilities included in the study. Therefore, future studies should consider these confounding factors. Moreover, future studies should reduce the RoB due to the lack of high-quality BSC implementations in the literature. Finally, this review searched for the BSC implementation in health care databases; consequently, future systematic reviews are recommended to include studies in management and health policy databases. In conclusion, this systemic review offers evidence to HCO and policymakers on the benefits of implementing BSC in HCO. Although the quality assessment revealed that many studies had a high RoB, BSC implementation positively influenced HCO patient satisfaction and financial performance. Based on the findings in the present review, researchers are encouraged to focus on lowering the risk of bias in BSC implementation in the future. HCO managers are also advised to consider HCW satisfaction and engagement with BSC implementations. Finally, an additional assessment of the BSC impact in HCO during the COVID-19 pandemic is required, as we could not find any. The balanced scorecard--measures that drive performance Total quality management and balanced scorecard-a comparative analysis A systematic review: the dimensions and indicators utilized in the performance evaluation of health care organizations-an implication during COVID-19 pandemic Conceptual foundations of the balanced scorecard A descriptive analysis on the implementation of balanced scorecards in German-speaking countries Saving money, saving lives Duke Children' s Hospital Case Abstract. Balanced Scorecard Collaborative Hall of Fame A review of the use of the balanced scorecard in healthcare Assessment of Countries' preparedness and lockdown effectiveness in fighting COVID-19 Evaluations of Healthcare Providers' Perceived Support From Personal, Hospital, and System Resources: Implications for Well-Being and Management in Healthcare in Montreal, Quebec, During COVID-19 Using Logic Model and Key Performance Indicators to Construct A Field Hospital for Coronavirus Pandemic Cases Occupational burnout and job satisfaction among physicians in times of COVID-19 crisis: a convergent parallel mixedmethod study The sustainability balanced scorecard: a systematic review of architectures Balanced Scorecard: Is It Beneficial Enough? A Literature Review Scoping review of balanced scorecards for use in healthcare settings: development and implementation Reviewing the application of the balanced scorecard with implications for lowincome health settings The PRISMA statement for reporting systematic reviews and Metaanalyses of studies that evaluate health care interventions: explanation and elaboration SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews Chapter 4: Searching for and selecting studies Cochrane Handbook for Systematic Reviews of Interventions version 6 Available from www. train ing. cochr ane. org/ handb ook ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions Assessing Risk of Bias in Included Studies Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent Risk-of-bias VISualization (robvis): an R package and shiny web app for visualizing risk-of-bias assessments The integrative review: updated methodology Strategic management system in a healthcare setting--moving from strategy to results A study of a nursing department using the balanced scorecard and the performance measurement system: analytic hierarchy process Balanced performance measurement in research hospitals: the participative case study of a haematology department Balanced scorecard: application in the general Panarcadian Hospital of Tripolis, Greece Balanced scorecard at summa health system Balanced scorecard implementation in the government hospital Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA Intervention : a cluster randomised trial in Zambia Using path analysis to examine causal relationships among balanced scorecard performance indicators for general hospitals Implementation of the balanced scorecard at Barberton citizens hospital The balanced scorecard framework-a case study of patient and employee satisfaction Measuring and managing progress in the establishment of basic health services: the Afghanistan health sector balanced scorecard Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan Performance management tools motivate change at the frontlines Creating a balanced scorecard for a hospital system Performance improvement after implementing the balanced scorecard: a large hospital' s experience in Taiwan A balanced scorecard for Canadian hospitals Performance enhancement using a balanced scorecard in a patient-centered medical home A three year journey to organizational and financial health using the balanced scorecard: a case study at a Yale New Haven health system hospital The balanced scorecard-solution at Peel memorial hospital The use of balanced scorecard to support achievement of business plan in digestive endoscopy center Implementation and outcomes of a balanced scorecard model in Womenʼs Services in an Academic Health Care Institution Improving health services to displaced persons in Aceh , Indonesia : a balanced scorecard Applications of the balanced scorecard for strategic management and performance measurement in the health sector Adoption of the cocept of balanced scorecard within NSW health: an exploration of staff attitudes The attitude of health care professionals towards accreditation: a systematic review of the literature Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations We would like to thank Dr. Vahideh Moghaddam, Diêgo Andrade, and Wang Zhe for their help in translating and data extraction of Persian, Spanish, and Chinese studies, respectively. Also, we would like to thank Dr. Duha Shellah, Dr. Mohanad Khatatbeh, Dr. Ibrahim A. Shilbaya, Mr. Jason Titman for revising the manuscript. The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12913-021-07452-7.Additional file 1: S1 Appendix. PRISMA checklist. 27 The research was financed by the Thematic Excellence Program 2020-Institutional Excellence Sub-programme / National Excellence Sub-program of the Ministry for Innovation and Technology in Hungary, within the framework of the 2. thematic and Biomedical Engineering programme of the University of Pécs. Code: 2020-4.1.1-TEP2020. All data generated or analyzed during this study are included in this published article [and its supplementary information files]. Ethics approval and consent to participate Not Applicable since it is a systematic review. Not Applicable. The authors have declared that no competing interests exist.