key: cord-1042064-ds1gm5gx
authors: Bisbey, Tiffany M.; Wooten, Kevin C.; Salazar Campo, Maritza; Lant, Theresa K.; Salas, Eduardo
title: Implementing an evidence-based competency model for science team training and evaluation: TeamMAPPS
date: 2021-06-08
journal: Journal of clinical and translational science
DOI: 10.1017/cts.2021.795
sha: cbe0fa0bfa34c74622553171599e62e620ccdf04
doc_id: 1042064
cord_uid: ds1gm5gx

INTRODUCTION: In response to a call issued by the National Research Council to investigate the knowledge, skills, and attitudes of effective science teams, we designed a team training program for conducting science in collaborative contexts. METHODS: We reviewed the literature to develop an evidence-based competency model for effective science teams along with exemplary behaviors that can be used for founding team training and evaluation. We discuss the progress of teamwork and team development research that serves as a foundation for this work, as well as previous research involving team-based competencies. RESULTS: Three overarching competencies emerged from the literature as key for science team effectiveness: psychological safety, awareness and exchange, and self-correction and adaptation. These competencies are fully described, including their evidence base. CONCLUSIONS: We developed a competency model and implementation plan for a team training program specific to science teams – TeamMAPPS (Team Methods to Advance Processes and Performance in Science). This paper details steps in the implementation process, including plans for consortia dissemination, evaluation, and future development.

Recently, the National Research Council [1] issued a call to identify and translate the knowledge, skills, and attitudes (KSAs) that make science teams effective. Science teams are groups of scientists collaborating interdependently to conduct research. Team-based research allows scientists to put their expertise together toward solving complex issues more efficiently than individual efforts alone. [2, 3] To address important issues from all angles, science teams often employ a multidisciplinary approach to benefit from the insight of multiple perspectives. [4] Research suggests that multidisciplinary teams make a greater impact than individual scientists in producing academic and practical contributions, forecasting the steady increase in their prevalence to only continue. [5, 6] Unfortunately, collaboration does not organically yield effective teamwork. As the scientific world is becoming more collaborative, researchers have begun to recognize a need to understand the teamwork competencies specific to the effectiveness of science teams, [7] as well as educational and training curricula for developing these competencies. [8] Working together effectively as a team requires specific KSAs that can be improved upon with team development interventions; [9] but it is currently unclear what KSAs might make science teams effective, as well as what barriers to effective teamwork are particular to this context. Science teams need an actionable model to implement in team training to develop the competencies necessary for team effectiveness in producing and translating scientific findings. In this paper, we review existing competency models for teams in general and describe the development of an evidence-based competency model specific to science teams for use in training, development, assessment, and evaluation. In doing so, we contribute an evidence-based competency model complete with behavioral markers applicable to translational science teams at all levels of maturity. Further, we focus on the implementation process across multiple constituent groups.

We begin with a background of the field of team science, including existing models of teamwork and their applicability to science teams. Next, we discuss progress in the field of team development and evaluation to lay the groundwork for applying a new competency model for translational science teams. Finally, we detail the development of a competency model for science team effectiveness (Team Methods to Advance Processes and Performance in Science [TeamMAPPS]), and describe the implementation plan.

Research investigating team science began to emerge in the early 2000s. Problems were becoming more complex and ill-defined, requiring multiple areas of expertise and new collaborations to solve them. It was becoming clear that although the people collaborating can be experts in their own respective fields, that they may not necessarily make an expert team when collaborating together. Researchers were in need of evidence-based guidelines for collaborative science. [10] There is an important distinction to make between teamwork and taskwork, as they each require unique skill sets. Taskwork skills are required for executing the specific tasks involved in completing work, and teamwork skills are required to collaborate with others and coordinate individual taskwork. [11] For instance, a pilot may pass all of their technical flight tests with a perfect score, but a plane will never leave the ground safely without appropriate skills for collaborating with the flight crew, such as closed-loop communication and conflict management. Teamwork skills are vital for effective collaboration in all fields, including conducting science. As is clear by the rarity of single-authored publications in academic journals, team science is now the norm across fields in academia, [12] medical science, [13] and translational science research, [14] and teams increasingly dominate solo authors in the production of new knowledge, producing more frequently cited research, and producing more high-impact research than individuals alone. [6] As scientific, health, and societal problems are becoming increasingly complex in the 21st century, multiple disciplines and foundations of knowledge are needed to fully understand the issues and to develop solutions that address them from multiple angles. Accordingly, the National Science Foundation and the National Institutes of Health are increasingly requesting evidence of team science capacity as a criterion for awarding funding. Consider the recent COVID-19 pandemicthere is no single discipline able to solve such a crisis alone. Large-scale, complex issues require cross-disciplinary and cross-institutional level responses due to the sheer magnitude and threat they pose to the population. Our understanding of teams and the competencies required for effective teamwork is largely based on teams working in the same or similar discipline, for the same employer, in fields outside the realm of scientists creating new knowledge. Much of what we know about teamwork is general enough to apply to all situations, but there is a clear need to examine the challenges faced by scientific research teams specifically and to define the particular competencies that make them effective. It is both timely and necessary to advance a program for the training and development of translational science teams grounded in an evidence-based competency model for their effectiveness. [15] Status of the Team Training and Development Literature

Deficiencies and other barriers to team effectiveness are often addressed with countermeasures like team training and development. Reviews of team training [16] and meta-analyses [9] support a consistent relationship between teamwork training and team effectiveness. Research on science teams also suggests links between team effectiveness and the teamwork components targeted in team training. For instance, we know that science teams who develop shared mental models yield greater success, and that team member trust predicts the sharing of unique knowledge between science team members. [17] Investigations on team leadership in science teams find that transformational leadership plays a key role in fostering a climate for excellence and predicting innovation, [18] and it is positively related to changes in team development over time. [19] Moreover, team science leadership training can significantly influence leadership self-efficacy. [20] Historically, work uncovering the competencies for evaluating team effectiveness unfolded alongside work on team training, stemming from multidisciplinary research in aviation, and the military. [21] From a more specific focus, researchers here [22] separated teamwork competencies into interpersonal KSAs (e.g., communication, conflict resolution) and self-management KSAs (e.g., goal setting, planning). Years later, researchers wondered if there was a "Big Five" of teamwork, of which all teams could be evaluated based on five core competencies, including: (1) effective leadership, (2) mutual performance monitoring, (3) backup behavior, (4) adaptability, and (5) team orientation. [23] A detailed list of teamwork competencies can be found in seminal efforts by Salas and colleagues, [24] with a recent update. [25] Recently, a Clinical and Translational Science Award (CTSA) endorsed project developed a working list of individual and team competencies for translational science, [15] which generated the competency domains of: (1) facilitating team affect, (2) team communication, (3) managing team research, (4) collaborative problem solving, and (5) team leadership.

The models afforded by existing work provide an excellent foundation for understanding the mechanisms that drive teamwork, but they require consideration of the work context in order for them to be useful in practice. The context and goals of a team dictate which competencies might be most important for any given team, as well as how key competencies might manifest. For instance, researchers in healthcare acknowledged the importance of effective teamwork for preventing medical error, [26] setting the forces in motion to eventually develop the TeamSTEPPS® training program specifically for healthcare teams. To develop TeamSTEPPS®, researchers [27] worked to understand the challenges that impact healthcare team effectiveness, then outlined the team competencies required to address those barriers to performance. The TeamSTEPPS® training program has had massive success improving the teamwork of healthcare providers across units and specialties, amassing an impressive evidence base as it continues to be implemented across the USA. [28] When it comes to science teams, no such fully delineated competency model exists to serve as a basis for team training and evaluation. Translational science teams have a unique set of goals and challenges that create a distinct context; and while we know much about what makes teams effective in general, we have yet to understand which teamwork competencies and behaviors are critical to address in science teams in the same manner that TeamSTEPPS® addresses healthcare teams.

Clearly, there is a long and impressive history of research demonstrating the effectiveness of teamwork across work contexts, as well as the impact of team training at improving team performance. Considerable literature exists concerning critical team-based KSAs, [24] as well as interdisciplinary team health research competencies. [29] This begs the question, how are science teams any different? In this section, we discuss the unique challenges facing translational science teams by highlighting three common characteristics that pose barriers to their effectiveness: (1) diverse 2 Bisbey et al.

backgrounds and skillsets, (2) distribution of teams or team members, and (3) membership and demographic changes over time.

One of the most salient characteristics of science teams is the diversity among members. As in virtually all teams and groups of people, diversity in gender, race, ethnicity, and other surface-level demographics can present hurdles for science teams to overcome. What is particularly notable in collaborative research is the characteristic presence of deep-level diversity in the functional backgrounds of team members. Because science teams are often deployed in response to complex questions posed by sponsors or in grant proposals, it is frequently the case for researchers to collaborate across disciplines, professions, and communities in effort to provide well-rounded and practically useful solutions considered from multiple perspectives; in fact, these collaborations are a stated requirement for many funding sources. [1] It is not always easy to collaborate with people who have different backgrounds or subscribe to different values, assumptions, norms, and expectations than one is accustomed to in their own domain. Nevertheless, it is important to have a variety of perspectives and expertise on a team, as demands fluctuate throughout the phases of any given project and different KSAs may be required for effectiveness. [30] In addition, having different backgrounds means they may have different agendas outside of the science team, as well as hold different expectations for interactions within the operation of their respective silos. The cultural assumptions underlying behavioral norms and practices can vary widely across disciplines. Multiple perspectives means a lot of good ideas must be integrated, thus, more integration and translation may be required than in the typical brainstorming session. Moreover, scientists might be less accustomed to not being the only expert on their science team. Sharing leadership and responsibility might be difficult to maneuver as team members must set aside egos and learn to be flexible handing off leadership roles.

As collectives grow in size, members can become more dispersed and siloed, exacerbating their differences and challenging their ability to act as a single unit. This is apparent at multiple levels, such as with departments across a university campus, large metropolitan areas with multiple universities, a country, global organizations, and the world itself. Although not all science teams are comparatively larger in nature, research suggests that their average size is increasing along with the impact of their collaborations. It is often the case that scientists collaborate across institutions to conduct research. [5] Unfortunately, large team size and dispersion of team members provide significant challenges to teams with regard to remaining coordinated, cooperating with one another, and maintaining shared mental models.

For instance, there are often struggles encountered in coordinating large groups of researchers in multi-institution projects, which essentially form teams within a larger team, or a multi-team system. [31] Such difficulties can require team members to ultimately set aside their differences to achieve their goals and produce important knowledge and practical implications, requiring significant effort to coordinate large teams across geographical and conceptual borders. [21] 

Science teams belong to no single discipline; thus, it may be rare for an individual's core job duties to consist solely of their work on a particular research team. Rather, it may be more likely for team members to have responsibilities outside of their roles on the science team. Different priorities of team members and other common attributes of science teams (e.g., larger size, geographic distribution) can result in a considerable amount of time completing and progressing through stages of a project. During this time, the team might undergo various membership changes as project goals may change or different segments of the team become more or less important in different phases of the project. [30] Moreover, when distributed team members are highly interdependent, any changes in the team structure or other project conditions have the potential to impact the entire team and extend the timeline even further, as teams must regroup and revert back (to some degree) to a phase of socialization, trust building, and other activities that typically occur in the earlier stages of team development. Long timelines can provide significant challenges as a team-based research project develops, including greater chances for conflict, misunderstandings regarding members' roles and responsibilities throughout project phases, difficulty remaining coordinated, and resistance to making necessary changes due to how long it takes to get things done.

Before developing a standardized team training program for science teams, it is essential to identify the core principles of teamwork that are essential to the performance context and the standards by which teams can be evaluated. Toward this effort, we reviewed the extant literature to substantiate an evidence base for a team training and evaluation program and a foundation for future research to build upon.

In response to a call by the National Research Council, [1] we developed an evidence-based competency model for team scientists by identifying and translating KSAs for successful team science that can be applied to science team evaluation, training, and other educational curricula. Specifically, we developed a model that: (1) addresses known issues in team science, (2) is evidencebased, (3) can be described behaviorally, (4) can be used for development and/or evaluation, (5) can be applicable to nascent or advanced teams, and (6) can be used as an intervention and/or training model.

Our model -TeamMAPPSwas developed on the basis of established team competencies and teamwork models applied to team science and to the scientific enterprise. Here, we used general team competencies, virtual team competencies, along with the core components of teamwork, and models of team behavior. A review of the team science literature and the Team Science Toolkit, [32] as well as the use of a comprehensive literature review [16] led to the logical development of three broad competency sets (Fig. 1) .

Each competency is associated with an observable behavior, allowing for observational checklists that can inform evaluation plans and enable assessment when the behaviors are being practiced (Table 1) . It is well established that training improves team-based competencies, including participant attitudes and the ability to integrate unique knowledge from others. To this extent, the TeamMAPPS would cover a number of the attitudes, behaviors, and cognitions required for effective teamwork. [16] Overall Framework of the TeamMAPPS Model Based on our literature review of existing evidence, we have extracted three "competency sets" for the purpose of training:

(1) awareness and information exchange, (2) psychological safety, and (3) adaptation and correction. Each competency set has three exemplary behaviors associated with them that will guide the assessment of the learning program (Fig. 1) . Briefly, awareness and exchange is a set of competencies that facilitate team members' unique voice, disciplinary insight, reframing and integrating alternative cognitive views; psychological safety is a set of competencies that provide a safe environment for different perspectives, including participant acknowledgement, issue identification, conflict resolution; and adaptation will result in team members being able to respond to external challenges, monitor the team and improve its performance through self-correction, and focus on development long-term.

Because science teams are often assembled of expert team members in their own respective areas, it is essential for teammates to have an awareness of where expertise lies in the team. This awareness serves as the basis of the team's transactive memory system 4 Bisbey et al.

(TMS). A TMS is the cognitive structure within a team through which they store, retrieve, and share information. It details who knows what and where to go for specific knowledge, and thus, serves an especially crucial role in teams with diverse skillsets. Research shows that high-performing teams are more likely to have high-quality TMSs that allow them to share knowledge quickly and effectively. [33] Science teams can vary in the degree to which their knowledge is shared or unique, and their effectiveness hinges on their ability to integrate their unique knowledge into an effective strategy. [34] Part of this process may involve probing each other for relevant input so that expertise is included from all key areas. Effective science teams begin to develop an understanding of everyone's areas of expertise by inquiring until enough information is shared to develop a shared idea of the team's abilities. Once that knowledge is shared, it may need to be reframed in the context of the group's goals so that it can be integrated into an effective performance strategy. [35] Expert science teams can draw upon the knowledge of each team member to create a shared vision and purpose for the team that guides goal-setting and strategy development.

Although much of the process of developing an awareness involves exchanging information, it does not mean that communication must be constant; rather, it is important for information which is unique to be shared and incorporated into the project plan. Moreover, it highlights the importance of using promotive voice to consistently improve strategies. The concept of voice in the workplace refers to expressing ideas, suggestions, opinions, or concerns about work. Exercising promotive voice means that employees are speaking up about ways to improve work processes or performance, whereas prohibitive voice refers to proposing solutions for decreasing inefficiencies. [36] Both forms of voice serve a purpose, but promotive voice is particularly important for science teams by cultivating a norm of team members building on each other's ideas with their own knowledge to improve their strategies.

In summary, an awareness of each other's expertise on a team and effectively sharing information are key for science team effectiveness. Facilitating awareness and exchange can be seen to address teams with diverse backgrounds and skills by making explicit the values, norms, and expectations of all team members. Continued awareness and exchange well-serves science teams that change in composition over time by reducing the misunderstanding of roles and goals. For large teams, and those physically distant and institutionally separate, awareness and exchange can no doubt aid in facilitating coordination and reduction of manifest differences.

A climate of psychological safety is marked by free sharing of ideas, opinions, suggestions, perspectives, and questions in groups without fear of embarrassment, threats to one's reputation, or retribution of any kind. [37] This is similar to the concept of voice, but differs in that psychological safety is about the team having a shared feeling of freedom in exercising their voice without fear of backlash. Meta-analyses show that teams with high levels of psychological safety are more engaged and committed, as well as have greater team learning, citizenship behaviors, and performance outcomes. [38] Because many science teams are distributed and must collaborate virtually, psychological safety can be difficult to maintain. Researchers have demonstrated the detrimental impact of low psychological safety in their study of virtual teams, finding that teams who perceive a lack of psychological safety were less innovative than other teams. [39] Through examination of the literature, we identify three key factors that can help to support psychological safety: (1) perspective seeking, (2) acknowledging and including others, and (3) addressing issues and resolving conflict.

By seeking the perspectives of others, team members signify an openness to feedback and encourage each other to share their unique viewpoints with the group. One way to seek perspectives of others might be by instituting regular debriefing sessions after performance episodes or at salient points in the project timeline. Debriefs give team members a safe context in which to share ideas and a clearly defined opportunity to seek each other's perspectives. This goes hand in hand with acknowledging and including those perspectives, which Edmondson [37] names as important for supporting psychological safety. In other words, it is important not only to listen to each other's ideas, but also to actually work toward implementing them in the project plan. Finally, by addressing issues as they arise rather than waiting for them to fester, teams can limit barriers to psychological safety due to conflict. Resolving conflict, particularly interpersonal conflict, keeps the focus on the objective and performance of the science team.

Psychological safety is beneficial for diverse science teams by ensuring that their interactions and sharing of different perspectives will not be overshadowed by colleagues with more social power. When psychological safety is low, it can be difficult for those with less power or rank to speak up with new ideas, suggestions for improvement, or admissions of errors because they have more to lose when it comes to fear of judgment and threats to reputation. In science teams, junior investigators may yield to more senior members and avoid challenging the status quo, withholding potentially valuable perspectives. Psychological safety is also particularly important in large disparate groups when initial trust can be critical, which is common in multi-institution collaborations. Moreover, a team's level of psychological safety can be jeopardized by the addition or loss of team members, as levels the team must adapt to changing membership while maintaining a shared feeling that speaking up is accepted and encouraged. By maintaining psychological safety, science teams can ensure that all disciplines, backgrounds, and differing generations can contribute to the scientific enterprise and to the development of the team.

Effective science teams can adapt to changing demands and self-correct when necessary. In order to self-correct, teams should be aware of each other's performance and the progress of the team as a whole toward their goals by monitoring these systems. [40] This might involve practicing a routine of monitoring progress, reflecting on performance, then creating change where it is required. Team debriefing will support these efforts to reflect on and analyze past work and progress toward the team's goals. The debrief is the most powerful, yet underutilized tool for improving teamwork and team performance. Meta-analyses suggest that teams who debrief outperform other teams by 20%-25%. [41] Debriefing provides a mechanism to head off any confusion or misunderstandings so that team members always have role clarity and maintain aligned efforts, goals, and mental models throughout their collaboration together. They also provide a platform for safely discussing conflicting ideas and creating plans for change and development.

Science teams who effectively adapt and self-correct continually reexamine acquired skills and background of team members, as well as perspectives upon the scientific problem. This is essential Journal of Clinical and Translational Science for science teams, as its members are consistently evolving with scientific developments in their respective fields. Furthermore, adaptation and correction allow for periodic reexamination of team member goals, thus facilitating coordination and integration across the life cycle of the project. Finally, adaptation and correction allow for the onboarding and socialization for transfer of knowledge over time, as well as maintaining a team development plan. This is critical for addressing the issue of changing membership in science teams, which can occur regularly over the course of a project.

Considering the unique challenges and barriers to science team effectiveness, along with supporting evidence and theory from the general team and science team literature, we identify nine core teamwork competencies that are congruent with contemporary science team requirements and able to be targeted in training and team development interventions. [42] Specifically, we believe these nine competencies impact science team performance by enabling innovation and adaptation through the use of many cognitive, attitudinal, behavioral, and interpersonal skills and abilities in conducting collaborative team research. Given the large number of potential competencies identifiable, [15] no one training program could possibly address all needs. Thus, the question is what might be the most important and amenable to development? Table 2 showsthe relationship of the nine TeamMAPPS competencies to established general team core competencies, [23] translational team competency domains, [15] identified team effectiveness drivers, [25] and recently developed team science best practices. [43] As illustrated, each of the nine TeamMAPPS competencies can be assessed as high, moderate, or low relatedness, or not applicable. The nine TeamMAPPS competencies fair well across all criteria, particularly in relation to leadership adaptability, cooperation, team communication, creative problem solving, team leadership, coaching, culture of trust, accountability, openness, inclusivity, constant learning, and interdisciplinary conventions. Table 3 details the TeamMAPPS training modules targeting the core competencies alongside action strategies to be learned in training.

A distinguishing characteristic of the TeamMAPPS program is its versatility and ability to potentially include assessment, training, and evaluation. Although other valuable development and evaluation systems exist, [44] TeamMAPPS is unique in that it allows users to independently apply some or all components for several purposes such as evaluation, development, and/or assessment for individuals, new teams, or existing teams. To this extent, the different components depicted in Fig. 1 will allow translational team science educators and researchers considerable flexibility in its use. It should be noted, however, that the nine specific competencies for science teams represent only a fraction of the potential competencies to ultimately be identified as predictive of team effectiveness.

The TeamMAPPS program, and its subsequent dissemination, will follow the model of evidence-based interventions, and more specifically one of intervention dissemination. [45] This program is based on research suggesting its components (e.g., psychological safety) have been proven to be effective in team practice.

TeamMAPPS will implement a framework that defines variables and the relationships between them. [46] By systematizing the materials and evaluation plan, TeamMAPPS can address numerous problems in implementation such as lack of a common language, inconsistent applications, or lack of embedded evaluation plans. [46] TeamMAPPS will implement specific and behaviorally defined competencies and action plans through systematic content presentation, behavioral models, and exercises. Thus, having reproducible methods [46] can greatly enhance intervention effectiveness. In addition, providing systematic methods and their standardized presentation to trainees will address fidelity concerns, as well as the implementation process gap [47] of quality control across the consortium as a function of variations in organizational capacity.

Leading academic researchers and CTSA thought leaders on team science generated a dissemination strategy after several years of program development involving web-based modules with learning content, self-assessment exercises, behavioral vignettes, application exercises, and application plans (Fig. 2) . We will deploy a multiyear, six-stage process to CTSA dissemination. Initially, a cross section of CTS-related team science specialists will conduct a collegial review of each module to ensure fidelity and external validity. After final revisions, representatives from five funded CTSAs will attend a Train-the-Trainer workshop to experience the modules first hand and practice numerous modes of program facilitation. The TeamMAPPS program allows for delivery modalities of (1) completely online asynchronous individual delivery, (2) face-to-face and web-mediated module implementation as part of an existing for-credit class, and (3) implementation of all modules as an intervention for new or existing teams.

Following the Train-the-Trainer event, TeamMAPPS will be piloted (using different delivery modalities) by the five CTSA institutions. Student and facilitator reactions and feedback will be solicited to inform necessary revisions. Areas such as clarity, topic fidelity, application relevance, and navigability will be assessed. In addition, competency-specific knowledge acquisition and selfefficacy will be assessed through pre-and post-training surveys. After final adjustments and formatting, the program will be offered to all CTSA institutions and ultimately to the scientific community, providing each participant with a completion certificate.

Following the levels of evaluation approach, [48] the TeamMAPPS program will follow a multiyear, multistage plan to ultimately answer the question of whether the training program actually works. Using best practices [49] in the evaluation of team training, the TeamMAPPS evaluation plan will use both experimental and quasi-experimental designs, and multiple levels of assessment. As shown in Table 4 , the evaluation plan will assess reactions, learning, and behavior at the individual team level using module-specific pre-and post-training measures. Specifically, survey items regarding self-efficacy will assess the degree to which participants believe KSAs have improved and will lead to future success. Knowledge acquisition will be evaluated with items validated to assess mastery of core concepts presented in learning materials. The behavioral markers depicted clearly in Table 1 will be used as criteria for behavioral observational scales and for 360°feedback processes to evaluate performance during training exercises and post-class team meetings. A standardized return-on-investment (ROI) rubric will be developed to evaluate trained and non-trained teams on subsequent funding and new (2) Reframing & Integrating

Perspective Seeking (4) project opportunities, with control variables used to address key factors such as team characteristics and previous funding.

As part of future implementation and dissemination, modifications will be implemented both as a function of evaluative data and of addressing the known needs of team scientists (Fig. 3) . We plan to expand the number of behavioral models to maximize transfer of knowledge, [50] provide mobile and web-based applications to personalize the search of specific techniques and action plans needed, provide modules specific to engaging and fully utilizing community members, [4] and entrepreneurial/ 8 Bisbey et al.

commercialization experts with scientific teams. Ultimately, we plan to provide augmented and virtual reality exercises to fully contextualize learning opportunities.

Following a call by the National Research Council [1] to identify and translate KSAs for successful team science, along with a clear need for a team-science-specific competency model and educational curricula, we developed an evidence-based competency model for team scientists -TeamMAPPS. The TeamMAPPS model is valuable for both researchers and practitioners as a framework for designing team training, as well as an intervention diagnostic for assessing teamwork proficiency. Given that there is little evidence or established models of competencies for translational team science, TeamMAPPS is a first stage effort to establish a robust and fully developed taxonomy to use in both professional development and graduate training. Having observable behavior that can be assessed and evaluated provides researchers with a tangible framework to empirically explore and refine. While there exist many potential competencies throughout the team science domain, [15] the three proposed competency sets and their nine attendant specific competencies are useful as much-needed building blocks for the development, validation, and practical application for team science skill development. Journal of Clinical and Translational Science 9

Enhancing the Effectiveness of Team Science

The 'romance of teams': toward an understanding of its psychological underpinnings and implications

NIH roadmap/common fund at 10 years

Interdisciplinary team science and the public: steps toward a participatory team science

Multi-university research teams: shifting impact, geography, and stratification in science

The increasing dominance of teams in production of knowledge

Approaches to preparing young scholars for careers in interdisciplinary team science

Advancing the science of team science

The effects of team training on team outcomes: a meta-analysis

The science of team science: a review of the empirical evidence and research gaps on collaboration in science

A configural theory of team processes: Accounting for the structure of taskwork and teamwork

The road we must take: multidisciplinary team science

Interventions to improve team effectiveness: a systematic review

A multi-level systems perspective for the science of team science

Individual and team competencies in translational teams

Does team training improve team performance? A meta-analysis

Study on knowledge sharing behavior engineering

Transformational leadership and team innovation: integrating team climate principles

Evolution of multidisciplinary translational teams (MTTs): insights for accelerating translational innovations

Integration of leadership training for graduate and medical students engaged in translational biomedical research: Examining self-efficacy and self-insight

Teams of psychologists helping teams: the evolution of the science of team training

The knowledge, skill, and ability requirements for teamwork: implications for human resource management

Is there a "Big Five" in Teamwork?

The wisdom of collectives in organizations: an update of the teamwork competencies

Teams That Work: The Seven Drivers of Team Effectiveness

TeamSTEPPS: assuring optimal teamwork in clinical settings

TeamSTEPPS™: team strategies and tools to enhance performance and patient safety

TeamSTEPPS®: Research/Evidence Base. Agency for Healthcare Research and Quality

Training for interdisciplinary health research defining the required competencies

A four-phase model of transdisciplinary team-based research: goals, team processes, and strategies

The science of multiteam systems: a review and future research agenda

The team science toolkit: enhancing research collaboration through online knowledge sharing

Transactive memory in organizational groups: The effects of content, consensus, specialization, and accuracy on group performance

Facilitating innovation in diverse science teams through integrative capacity

Learning and performance in multidisciplinary teams: the importance of collective team identification

Psychological antecedents of promotive and prohibitive voice: a two-wave examination

Psychological safety and learning behavior in work teams

Psychological safety: A meta-analytic review and extension

Unpacking the concept of virtuality: The effects of geographic dispersion, electronic dependence, dynamic structure, and national diversity on team innovation

Team adaptation and post change performance: effects of team composition in terms of members' cognitive ability and personality

Do team and individual debriefs enhance performance? A meta-analysis

An evidence-based competency model for team science training

Enhancing reproducibility using interprofessional team best practices

Team Development and Evaluation: Developmental Evaluation

A glossary for dissemination and implementation research in health

The struggle of translating science into action: foundational concepts of implementation science

An introduction to implementation science for the non-specialist

Evaluating Training Programs: The Four Levels

Twelve best practices for team training evaluation in health care

A meta-analytic review of behavior modeling training

Acknowledging & Including (5) Acknowledgments. This work was partially supported by the Institute for Translational Sciences at the University of Texas Medical Branch (UTMB, Galveston, TX), supported in part by Clinical and Translational Science Award UL1 TR001439 from the National Center for Advancing Translational Sciences, National Institutes of Health. This work was also partially supported by the Center for Clinical and Translational Sciences at the University of Texas Health Science Center (UT Health Science Center, Houston, TX), supported in part by Clinical and Translational Science Award UL1 TR003167 from the National Center for Advancing Translational Sciences, National Institutes of Health. Rice University is a partner in the award. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.Disclosures. The authors have no conflicts of interest to declare.