key: cord-1042812-fydsvwm3
authors: Etheridge, James C.; Moyal-Smith, Rachel; Sonnay, Yves; Brindle, Mary E.; Yong, Tze Tein; Tan, Hiang Khoon; Lim, Christine; Havens, Joaquim M.
title: Non-technical skills in surgery during the COVID-19 pandemic: An observational study
date: 2022-01-04
journal: Int J Surg
DOI: 10.1016/j.ijsu.2021.106210
sha: 90016f90132e4b4e81335097d9644e9bc5ba5ccb
doc_id: 1042812
cord_uid: fydsvwm3

BACKGROUND: Non-technical skills are critical to surgical safety. We examined the impact of the COVID-19 pandemic on non-technical skills of operating room (OR) teams in Singapore. MATERIALS AND METHODS: Observers rated live operations using the Oxford NOTECHS system. Pre- and post-COVID observations were captured from November 2019 to January 2020 and from January 2021 to February 2021, respectively. Scores were compared using Schuirmann's Two One-Sided Test procedure. Multivariable linear regression was used to adjust for case mix. A 10% margin of equivalence was set a priori. RESULTS: Observers rated 159 cases: 75 pre-COVID and 84 post-COVID. There were significant differences between groups in surgical department and surgeon-reported case complexity (both P < 0.001). Total NOTECHS scores increased post-COVID on raw analysis (36.1 vs 38.0, P < 0.001) but remained within the margin of equivalence (90% CI 1.3 to 2.6, P < 0.001). Multivariable analysis demonstrated a similar increase within the margin of equivalence (2.0, 90% CI 1.3 to 2.7). Teamwork and cooperation scores increased by 1.0 post-COVID (90% CI 0.8 to 1.3); all other subcomponent scores were equivalent. CONCLUSION: Non-technical skills before and after the peak of the COVID-19 pandemic were equivalent but not equal. A small but statistically significant improvement post-COVID was driven by an increase in teamwork and cooperation skills. These findings may reflect an improvement in team cohesion, which has been observed in teams under duress in other settings such as the military. Future work should explore the effect of the pandemic on OR culture, team cohesion, and resilience.

Non-technical skills are increasingly recognized as critical to surgical safety. Although technical skills have traditionally been emphasized, technical deficits are responsible for only a small fraction of adverse events [1] . Meanwhile, deficits in non-technical skills, such as communication, teamwork, decision-making, and situational awareness, are responsible for up to 60% of surgical errors [2, 3] .

The operating room (OR) is a complex, high-risk environment which requires careful coordination among all surgical team members. Although teams have always been subject to the demands of this dynamic high-pressure setting, the COVID-19 pandemic introduced a number of unanticipated stressors. Risks to personal health in the OR are not novel, but the widespread nature of COVID-19 represented a pervasive and previously unseen degree of exposure for surgical teams [4] . Resulting personal protective equipment requirements provided a measurable sense of protection, but may also have degraded OR communications and situational awareness while inducing fatigue and impairing decision-making processes [5] .

Resource shortages, staff redeployment, and financial concerns added to the strain on surgical teams [6, 7] . Burnout among healthcare workers is at a record high, which can further degrade OR team performance and contribute to errors [8, 9] .

Although the safety implications of COVID-19 in surgical practice have been studied, little is known about its effects on non-technical skills. We began a surgical safety initiative targeting non-technical skills in an academic tertiary referral center in Singapore in 2019. As cases mounted in early 2020, we were forced to terminate baseline observations prematurely.

When COVID-19 restrictions eased, OR observations were allowed to resume. We therefore took this opportunity to assess the impact of the COVID-19 pandemic on non-technical skills among OR teams.

J o u r n a l P r e -p r o o f

We used observations conducted before the COVID-19 pandemic (pre-COVID) and after 

All observations were collected from a large academic referral center in Singapore. were included. All elective, urgent, and emergent cases during regular business hours were eligible for inclusion unless precluded for safety reasons, such as active COVID-19 infection or inability to maintain adequate social distancing in the OR. Purposive criterion sampling was used to maintain balance between specialties as much as possible given variability in observer availability, personnel restrictions, and operative schedules. Colorectal surgery was intentionally oversampled due to their use of a specific surgical device of interest.

We utilized the Oxford NOTECHS rating system, a well-validated behavioral marker system for OR teams [11] . The NOTECHS system includes four subscales: leadership and management, teamwork and cooperation, problem solving and decision-making, and situational awareness. Each OR subteam (surgery, anesthesia, and nursing) is rated on a 1-4 scale for each of these dimensions, yielding a total score between 12 and 48. Observers underwent formal training on the NOTECHS system. Interrater reliability was excellent across all subscales, as has been previously described [12] . Standardized data collection sheets were used to record observations and NOTECHS scores. Total NOTECHS scores were the primary outcome of interest. Subscale scores were analyzed as secondary outcomes. Covariates captured by observers included surgical department and surgeon-reported case complexity, rated as high, moderate, or low. Names or other potential identifiers of individual OR team members were not collected to protect anonymity.

Pre-and post-COVID case characteristics were compared using Fisher's exact test.

NOTECHS scores were compared using Schuirmann's Two One-Sided Test (TOST) procedure [13] . This method allows testing for both equality and equivalence of two means. Equality is tested using a standard Student's t-test. Equivalence is tested within a preselected margin using two one-sided t-tests. The left-sided t-test establishes whether the difference in means is greater than the preselected lower bound; the right-sided test establishes whether it is less than the preselected upper bound. Thus, if the results of both tests are statistically significant with α < 0.05, the null hypothesis is rejected and the difference in means is said to lie within the margins of equivalence.

A secondary analysis using multivariable linear regression was used to adjust for differences in surgical specialties and case complexity between periods. By convention, the 90% confidence interval (CI) of the parameter estimate for the pre/post-COVID variable was J o u r n a l P r e -p r o o f taken and compared against the preselected margin of equivalence [14, 15] . 

Observers rated 159 cases: 75 pre-COVID and 84 post-COVID. No surgical teams refused participation during either period. There were significant differences in both surgical department and case complexity between periods (Table 1) . Of note, colorectal cases were particularly oversampled in the post-COVID period, while the proportion of cardiothoracic, hepato-pancreato-biliary, obstetrics and gynecology, and urology cases declined. The proportions of both high-and low-complexity cases also increased post-COVID, possibly reflecting case prioritization strategies.

As shown in Table 2 Results of multivariable linear regression analyses are shown in Table 3 and Figure 1 .

Total NOTECHS scores remained unequal but statistically equivalent between periods (5.1%

increase, 90% CI 3.3% -6.8%). Teamwork and cooperation was not equivalent (10.2%

increase, 90% CI 7.8% -12.6%). All other subscale scores remained equivalent.

The COVID-19 pandemic has had a profound impact on the practice of surgery worldwide. As healthcare resources were strained, a backlog of millions of elective cases developed [17, 18] . For operations that could not safely be delayed, OR teams found themselves grappling with new risks to their own health as well as challenges to health system operations and patient safety [19] . In addition, many surgical team members were redeployed to unfamiliar work environments to meet the clinical demands imposed by the pandemic [6] .

The effects of these changes on OR teams remain poorly understood.

We used observations of surgical teams pre-and post-COVID using split baseline data from a surgical safety initiative as a natural experiment to assess changes in non-technical skills. Overall, post-COVID NOTECHS scores were found to be slightly higher but statistically equivalent compared to pre-COVID scores. The observed difference of 1.9 is equivalent to a single team member moving from "basic standard" to "excellent" in one category. Given the number of unmeasured confounders with the potential to impact scores, we place greater emphasis on the equivalence of scores than on this small but statistically significant increase.

Nonetheless, there was a clear increase in teamwork and cooperation in the post-COVID period on both unadjusted and adjusted analyses. Improvements in team cohesion have been described in other teams under duress, including high-risk settings such as the military, the fire service, and space flight [20] [21] [22] [23] . These improvements can translate into improved team performance and safety, but they require strong leadership and a culture of error management rather than error aversion to ensure that stressors do not degrade performance [22] . Surgeons and surgical leaders should employ collaborative leadership to identify and mitigate stressors imposed by the COVID-19 pandemic as we enter yet another wave to ensure that improved teamwork translates into improved team performance and, ultimately, improved patient safety [24] .

From a research standpoint, our results show that studies of surgical team's social and cognitive skills may be able to proceed, albeit cautiously, even if they are interrupted by the COVID-19 pandemic. The methods described herein can serve as a model for such studies.

The TOST procedure, complemented by multiple linear regression to control for confounding, is a powerful tool to establish equivalence in an interrupted dataset. However, equivalence margins should be selected carefully, taking into account both the clinical significance of the chosen equivalence level and the statistical power afforded by the available sample size.

The present study is limited by its pre/post observational design. Although no interventions targeting OR teams' non-technical skills took place during the COVID-19

pandemic, unmeasured secular trends may have contributed to the observed differences in NOTECHS scores. Given the nature of the pandemic, it is not possible to enroll a comparator group that was unaffected by COVID-19. External validity is also limited by our single center design. The reactions of teams in an academic tertiary referral center in Singapore may differ significantly from those in other settings. Singapore was able to mount a swift response to the pandemic, with country-wide public health mandates and implementation of robust safety measures in the OR [25] . Our ability to capture NOTECHS scores immediately before the pandemic and one year thereafter is a unique opportunity that cannot be recreated. This is an opportunistic analysis J o u r n a l P r e -p r o o f 8 and is therefore restricted by the design of the parent study. It is nonetheless possible to corroborate our evidence using qualitative data such as interviews with surgical staff. Future efforts will engage OR teams to assess their perspectives on the impact of the pandemic on communication, teamwork, and other non-technical skills.

Non-technical skills before and after the peak of the COVID-19 pandemic were equivalent but 

Analysis of errors reported by surgeons at three teaching hospitals

Effects of a brief team training program on surgical teams' nontechnical skills: An interrupted time-series study

Precautions for operating room team members during the COVID-19 pandemic

Impact of personal protective equipment on surgical performance during the COVID-19 pandemic

Redeployment of health care workers in the COVID-19 pandemic: A qualitative study of health system leaders' strategies

COVID-19 and the financial health of US hospitals

Surgeon burnout: defining, identifying, and addressing the new reality

RAS-ACS COVID-19 Task Force, COVID-19 pandemic and the lived experience of surgical residents, fellows, and early-career the American College of Surgeons

Preparing for a COVID-19 pandemic: a review of operating room outbreak response measures in a large tertiary hospital in Singapore

The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre, Qual. Saf. Health Care

Virtual non-technical skills assessment training is an effective, scalable approach for novice raters

A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability

Food and Drug Administration, Bioavailability and bioequivalence requirements

Biostatistical methodology in clinical trials in applications for marketing authorizations for medicinal products, CPMP Working Party on Efficacy of Medicinal Products Note for Guidance III/3630/92-EN

STROCSS 2019 Guideline: Strengthening the Reporting of Cohort Studies in Surgery

Elective surgery cancellations due to the COVID-19 pandemic: global predictive modelling to inform surgical recovery plans

COVID-19 impact on colorectal daily practice-how long will it take to catch up?

We Asked the Experts: The WHO Surgical Safety Checklist and the COVID-19 pandemic: Recommendations for content and implementation adaptations

Factors influencing small-unit cohesion in Norwegian Navy officer cadets

Resilience under military operational stress: can leaders influence hardiness?

Team cohesion and error culture in risky work environments

Sustaining individual motivation in high-demand team environments

Awareness of human factors in the operating theatres during the COVID-19 pandemic

A call to arms: a perspective on safe general surgery in Singapore during the COVID-19 pandemic

Name of the registry: Clinical Trials.gov 2. Unique Identifying number or registration ID

Author contribution Please specify the contribution of each author to the paper, e.g. study design, data collections, data analysis, writing. Others, who have contributed in other ways should be listed as contributors. Study conception and design

The World Medical Association's Declaration of Helsinki 2013 states in article 35: 'Every research study involving human subjects must be registered in a publicly accessible database before recruitment of the first subject'. Editors of IJS require that all types of research studies involving human participants should be registered prospectively and failing that retrospectively. There are many places to register your research, and you can choose which is the most suitable for your needs:•https://www.clinicaltrials.gov/ -for all human studies -free •http://www.chictr.org.cn/index.aspx -for all human studies -free •https://www.researchregistry.com/ -for all human studies -charge •https://www.isrctn.com/ -for all human studies -charge •Prospero -for systematic reviews -free •There are many national registries approved by the UN that can be found here Elsevier does not support or endorse any registry.

The Guarantor is the one or more people who accept full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish. Please note that providing a guarantor is compulsory.

J o u r n a l P r e -p r o o f