key: cord-0708261-j7tl8p6w
authors: Donenberg, Geri R.; Merrill, Katherine G.; Obiezu-umeh, Chisom; Nwaozuru, Ucheoma; Blachman-Demner, Dara; Subramanian, Sujha; Fournier, Amanda; Iwelunmor, Juliet
title: Harmonizing Implementation and Outcome Data Across HIV Prevention and Care Studies in Resource-Constrained Settings
date: 2022-04-07
journal: Glob Implement Res Appl
DOI: 10.1007/s43477-022-00042-7
sha: 3412fccb6c1e058b3af122691934900474e1a0bb
doc_id: 708261
cord_uid: j7tl8p6w

Harmonizing measures across studies can facilitate comparisons and strengthen the science, but procedures for establishing common data elements are rarely documented. We detail a rigorous, 2-year process to harmonize measures across the Prevention And Treatment through a Comprehensive Care Continuum for HIV-affected Adolescents in Resource Constrained Settings (PATC(3)H) consortium, consisting of eight federally-funded studies. We created a repository of measured constructs from each study, classified and selected constructs for harmonization, and identified survey instruments. Measures were harmonized for implementation science, HIV prevention and care, demographics and sexual behavior, mental health and substance use, and economic assessment. Importantly, we present our harmonized implementation science constructs. A common set of implementation science constructs have yet to be recommended in the literature for low-to-middle-income countries despite increasing recognition of their importance to delivering and scaling up effective interventions. Drawing on RE-AIM (Reach Effectiveness Adoption Implementation Maintenance) and the Implementation Outcomes Framework, items were harmonized for staff/administrators and study participants to measure reach, adoption, implementation, maintenance, feasibility, acceptability, appropriateness, and fidelity. The process undertaken to harmonize measures and the codified set of implementation science measures developed by our consortium can inform future data harmonization efforts, critical to strengthening the replication and generalizability of findings while facilitating collaborative research—especially in resource-limited settings. We conclude with recommendations for research consortia, namely ensuring representation from all study teams and research priorities; adopting a flexible, transparent, and systematic approach; strengthening the literature on implementation science harmonization; and being responsive to life events (e.g., COVID-19). SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s43477-022-00042-7.

A hallmark of scientific discovery is the application of rigorous methods to allow replication. Confidence in scientific findings depends on the ability to repeat an experiment and produce the same results. This is especially true for health promotion interventions where clearly articulated procedures, such as recruitment, enrollment, study activities, implementation strategies, and retention efforts are central to facilitate replication (Resnik & Shamoo, 2017) and establish a sound evidence base for effective interventions. Likewise, the use of similar measurement tools and survey instruments helps build a foundation in translational science that allows comparisons across diseases, populations, and projects, enables meta-analyses, and strengthens systematic reviews (Moser et al., 2011 ). Yet, health-related research in global settings requires flexibility, responsiveness to local priorities, and close attention to the context, all of which may be at odds with maintaining internal validity. Implementation science affords an opportunity to systematically document variations in intervention delivery and use rigorous methods to evaluate the impact.

Unfortunately, the proliferation of definitions (Odeny et al., 2015) , frameworks (Nilsen, 2015) , and measurement tools (Khadjesari et al., 2020) in implementation science has led to confusion about which to employ in a specific project. In many global health studies there is a lack of measurement consistency to evaluate similar concepts, including implementation science outcomes. Many projects use "homegrown" instruments, lack psychometric information on measures (Flake & Fried, 2020) , and employ unknown or non-validated scales. Consequently, existing research impedes effective comparisons across studies and populations, thereby limiting generalizability. This problem has led investigators to recommend focused attention on measurement issues (Flake & Fried, 2020; Sakaluk, 2019; Sakaluk et al., 2019) , including using similar instruments across intervention studies and evaluating their utility in capturing change (Rabin et al., 2012) . These efforts are particularly important for resource-constrained settings where research is difficult to implement, the gold standard is perhaps infeasible, and the need for accurate measurement is especially pronounced (Rabin et al., 2012) .

The absence of consistent measurement approaches in global settings is particularly problematic when classifying evidence-based implementation strategies, broadly defined as methods to enhance the adoption, use, and sustainment of evidence-based interventions, programs, or innovations (Powell et al., 2019) . Many of these studies require adaptations to the intervention or its delivery to accommodate setting characteristics. Undocumented adaptations and inconsistent measurements create challenges understanding subjective phenomena, such as mental health and treatment fidelity, and undermine outcome evaluations of health promotion programs. Without a common language to define what constitutes health, cross-study comparisons of intervention effectiveness and implementation strategies are difficult to achieve. Standardizing data elements, or "data harmonization," can advance the science of measurement in meaningful ways and guide future investigations.

Several factors influence the choice of measurement by investigators, including concerns about applicability, cultural relevance, readability, linguistic translations, and prior evidence supporting psychometric properties for the target population. Multi-project initiatives offer a unique opportunity to address these challenges, because they usually share certain components or properties. The group of studies within a consortium, for example, may be leveraged to inform the use of different measures across populations and cultural contexts to evaluate their generalizability and utility. This study reports on the efforts to harmonize measures by a consortium of eight studies targeting HIV prevention and care for adolescents and young adults (AYA) in low-tomiddle-income countries (LMICs).

Three prominent National Institutes of Health-funded initiatives propose measurement tools for harmonization on diverse topics and provide comprehensive lists of related measures. For example, the Rapid Acceleration of Diagnostics: Underserved Populations consortium was launched in response to the COVID pandemic's disproportionately negative impact on underserved communities and recommends instruments that help communities understand COVIDrelated morbidity and mortality. They launched a web-hub that includes a codebook of specific variables (United States National Institutes of Health, 2020) and a data dictionary in three languages. Likewise, the PhenX toolkit (Hamilton et al., 2011) provides an extensive catalog of measures for COVID-19, structural determinants of health, substance abuse and addiction, and mental health. Like Rapid Acceleration of Diagnostics: Underserved Populations, the PhenX toolkit is freely available on the web. Finally, the Patient-Reported Outcomes Measurement Information System program (Broderick et al., 2013) recommends measures of patient-reported physical and mental health outcomes and social well-being. All three initiatives recommend reliable, precise measures for targeted constructs and populations.

However, two important omissions from these resources exist. First, none focus on implementation science constructs or propose measures of implementation outcomes, despite growing recognition of the importance of translational research and the successful uptake and delivery of effective interventions. Second, none of the initiatives describe the process used to establish common data elements across their projects or carefully evaluate areas of interest for harmonization. Absent is any guidance on the strategies to achieve consensus on what data should be harmonized. These omissions are particularly important in the context of LMICs, AYA, and HIV prevention and care. The benefits of proven HIV interventions have been well documented, but AYA experience complex barriers in their efforts to access these interventions in real-world settings (Denison et al., 2017) . Additionally, despite enormous efforts to implement awareness and educational campaigns targeting youth, there is a lack of congruence understanding which interventions work best for young people.

Implementation strategies hold promise for increasing uptake of proven interventions for youth populations in LMICs, but little direction exists on how to operationalize these strategies in real-world settings. Likewise, implementation outcomes, defined by the Implementation Outcomes Framework (i.e., acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability) are the effects of purposeful actions (e.g., strategies) on proven HIV prevention and care interventions (Proctor et al., 2011) . Findings on the relationship between implementation strategies and implementation outcomes are relatively nascent. In light of the growing prevalence of HIV among young people, engaging AYA in the selection of implementation strategies and/or frameworks not only as beneficiaries but as partners and leaders in the scale-up of proven HIV interventions in LMIC settings (Sturke et al., 2020) can empower youth themselves to work towards achieving the goals of an AIDS-free generation.

The Prevention And Treatment through a Comprehensive Care Continuum for HIV-affected Adolescents in Resource Constrained Settings (PATC 3 H) consortium is composed of eight federally-funded projects designed to address the HIV prevention and care continuum among AYA in seven countries-Brazil, Kenya, Mozambique, Nigeria, South Africa, Uganda, and Zambia. PATC 3 H offers an integrated approach to understand the determinants and characteristics of AYA experiences in the context of HIV and a unique opportunity for data harmonization across LMICs. For additional details about the consortium and the eight studies, see the PATC 3 H website (Prevention and Treatment through a Comprehensive Care Continuum for HIV-affected Adolescents in Resource Constrained Settings, 2019) and Tucker et al. (2021) .

This paper describes a rigorous process to identify and establish a set of topics and instruments to harmonize across the PATC 3 H consortium and codifies a set of implementation science measures that may be used for data harmonization in future research. The process involved at least one principal investigator and co-investigator from each study team in partnership with program staff from the U.S. National Institutes of Health. Although centered in LMICs, the process detailed here can be applied to all research for studies with similar overarching themes. Importantly, context-specific pooled data on adolescent-friendly HIV prevention and care programs in LMICs are rare, and this paper offers recommendations for measures of implementation science outcomes and processes in this important domain (Archary et al., 2020) .

The PATC 3 H studies are funded under the U.S. National Institutes of Health UG3/UH3 mechanism, a two-phased approach in which pre-specified milestones in phase 1 must be met in order to progress to phase 2. The purpose of the first phase is to establish feasibility, pilot test methods, and optimize measures. Studies that achieved their goals in phase 1 (UG3) transitioned to phase 2 (UH3)-a randomized controlled trial. At the start of the UG3 phase, consortium members created a Data Harmonization Working Group (DHWG) recognizing the opportunity to broadly inform the science of HIV prevention and care in LMICs. Seventeen individuals volunteered to serve on the DHWG based on interest and prior expertise; six principal investigators, three additional study investigators, four staff and scientists from the funding agencies, and four staff from Westat, an outside contract research organization engaged to document project milestones and progress. Westat carefully documented the activities of the DHWG so the process could be replicated in future multi-study initiatives interested in harmonizing data across countries and populations. DHWG members represented multiple disciplines (psychology, sociology, medicine, public health, economics, epidemiology, pediatrics) and countries, brought expertise in qualitative and quantitative research methods, and extensive experience in HIV prevention and treatment for AYA in LMICs.

The primary objective of the DHWG was to establish the equivalent of a common data element for the main research questions across projects and facilitate multi-protocol analyses of similar primary and secondary outcomes. There was at least one investigator from each research team to maximize comparability and ensure that the objectives of each study and diverse contexts were represented. At the start, the DHWG met twice monthly to discuss potential topics for harmonization. Once sub-groups were established for each category of topics, the DHWG shifted to meeting monthly. The DHWG finalized the harmonization package by the end of the UG3 phase. Studies that progressed to the UH3 trial were encouraged (but not required) to integrate the measures into their design. Figure 1 shows the steps we employed for data harmonization. Each step is described in detail below.

As a first step to develop the common data (i.e., harmonization) package, Westat staff exported the primary constructs and their associated measures for each of the eight studies into a comprehensive spreadsheet that became the source document and resource for all DHWG activities moving forward. The document provided easy and efficient review of each project's outcomes and instrumentation by the DHWG. A representative from each study team carefully checked the source document for accuracy and provided needed updates and revisions to their study design and planned measurements. Changes and additions were incorporated into the spreadsheet for the next step.

Consistent with the consortium's primary objectives, the DHWG noted three broad domains across projects: (1) AYA HIV prevention continuum, (2) AYA HIV care continuum, and (3) structural and cross-cutting topics associated with the HIV prevention and care continuum. The DHWG assigned data elements to each of the three domains (see Fig. 2 ). The data elements were composed of related constructs and generally constituted the primary and secondary outcomes and relevant moderators and mediators of treatment effects. Within Domain 3 (structural and crosscutting topics associated with the HIV prevention and care continuum), the DHWG consolidated data elements into groupings (e.g., implementation science, demographics, sexual behavior). For example, anxiety and depression were planned moderators in several studies, so these were classified as "mental health" under Domain 3.

Step 2 was lengthy and required extensive discussion to avoid redundancies and incorporate different viewpoints. To achieve consensus, DHWG members examined the relevance of each data element for a specific domain (e.g., Does the data element reflect a characteristic of the domain? Does it relate to other constructs or domains? Should a new domain be created to capture the element's meaning?). There was full agreement on the subset of topics within the domains and their data elements. The DHWG made repeated efforts to include all interested stakeholders by continuously reverting to the steering committee composed of the principal investigators and co-investigators of each study and inviting review and comment. After the list of topics were agreed upon by the DHWG, Westat distributed the list to steering committee members for a final vote (agree with list/disagree with list). A new spreadsheet was created listing the three domains and a set of potential topics for harmonization (i.e., integrating the data elements and groupings) in preparation for review and feedback by the full PATC 3 H consortium.

The newly developed spreadsheet was distributed to the PATC 3 H steering committee. Investigators were asked to review the potential topics for harmonization, and together with their full study teams, discuss and then rank them according to their priority for harmonization as follows: 1 = highest priority; 2 = high priority; and 3 = lower priority. Each project submitted a single survey representing their team's consensus. The ratings were collated across studies and a final priority score was calculated for each topic by averaging scores across the eight research teams. The DHWG agreed a priori that topics with an average ranking ≤ 2 (reflecting highest and high priority) would be considered further for harmonization. In reviewing the topic rankings, the DHWG decided that some topics (i.e., substance use) should be included in the package given their important role in HIV prevention and care despite failing to reach the ≤ 2 threshold. Through this process, the DHWG identified five categories of topics for harmonization (Online Resource 1), which were subsequently approved by the study teams.

The next step in the process required selecting instruments and/or questionnaire items to reflect the harmonization topics. The DHWG revisited the initial spreadsheet created by Westat with an eye towards the surveys and instruments originally proposed by study teams. In some cases, teams had already planned to measure the topics chosen for harmonization (e.g., mental health). In other cases, the topics

Step 1. Create a repository of measured constructs

Step 2. Classify constructs

Step 3. Select constructs for harmonization

Step 4. Identify survey instruments to reflect harmonization topics

Step 5 were not included in the original study plans, and thus, new measures were proposed (e.g., implementation science). At this point, the DHWG divided into working sub-groups representing each of the five harmonization categories. The working sub-groups included at least one established expert in the topic area who led the harmonization efforts for that category. The leader of all five sub-groups were principal investigators of the PATC 3 H studies, and thus, well positioned to guide their group. Each working sub-group comprised 8-13 investigators across all study teams.

The working sub-groups followed several steps to identify the most appropriate measures to recommend across the consortium wherever possible. First, the sub-groups revisited the source document listing the original tools proposed across studies for the same topics. These were reviewed and discussed for relevance across the consortium. For example, several projects planned to evaluate antiretroviral medication adherence, but they differed in their measurement approach (e.g., Medication Event Monitoring System caps, pharmacy refills, self-report). Next, the leader of each Fig. 2 Domains, subcategories, and data elements for harmonization sub-group completed a literature review of related research to determine whether alternative validated, widely used, and culturally appropriate measures existed on the same topic. During this process, sub-group members met regularly to discuss different measurement options. Any originally proposed instruments that were similar across studies were prioritized since these had been carefully vetted and selected by the teams. Lastly, each sub-group generated a list of survey items to represent the topic areas for inclusion in the harmonization package. The recommended items for each category were submitted to the PATC 3 H steering committee for review and feedback.

Prior to distribution to the full consortium, the DWHG discussed whether individual projects could tailor the recommended instruments to meet the needs of their unique population and project. The DWHG agreed to recommend that validated measures (e.g., Patient Health Questionnaire-9; Monahan et al., 2009 ) be delivered without adaptation. However, the group recognized that measures with less psychometric data (e.g., implementation science) might require minor wording modifications to fit the study context. In these cases, teams were cautioned to ensure the spirit or meaning of revised items remained intact. At the all-team in-person meeting held in June 2019, the DHWG presented some of the measure recommendations and concerns were discussed (e.g., what if the harmonized measure differed from the originally planned instrument?). In September 2020, with consensus from the steering committee, a draft of the finalized harmonization package was provided to each study team. All remaining revisions were incorporated into item wording based on team feedback, and the harmonization package was finalized in January 2021. Study teams indicated their intention to use the package in whole or in part.

Following the onset of the COVID-19 pandemic, consortium members decided to coordinate their efforts and evaluate the impact of COVID-19 in their studies. Given the time constraints and need for quick deployment, these measures were not formally harmonized. Still, one team developed a measure to evaluate the impact of COVID on research participants and the HIV prevention and care continuum for AYA in LMICs. This measure was submitted to the PhenX Toolkit and adopted by several teams in the consortium (Center for Dissemination and Implementation Science, 2020).

Several topics were rated as important priorities for harmonization across studies. Unsurprisingly, the AYA HIV care continuum was consistently rated as higher priority followed by the AYA HIV prevention continuum as these constituted the primary outcomes for the consortium. Many of the structural factors were rated as lower priority, whereas implementation science, mental health, and economic assessment were rated highly. The final data harmonization package therefore comprised five categories: implementation science, HIV prevention and care continuum, demographics and sexual behavior, mental health and substance use, and economic assessment.

There was widespread enthusiasm for harmonizing across the five categories. All teams indicated plans to adopt at least some of the proposed items (Fig. 3) . Specifically, all eight study teams reported intentions to use the core measures for the implementation science outcomes, HIV prevention and care continuum, demographics and sexual behavior, and mental health and substance use categories, although the latter varied by classification (e.g., depression versus anxiety). Five studies reported planned economic or cost evaluations of their evidence-based interventions. Importantly, all teams identified the need for minor modifications to the recommended measures. Modifications included tailoring to culture, population, and context; adding or removing individual items within measures; simplifying response categories to improve comprehension; and varying time-specific questions and reference time points (i.e., HIV testing over the past 6 months vs. the past 3 months).

Few study teams initially proposed a formal implementation evaluation in their grant application. However, as the UG3 studies progressed, particularly in the face of the COVID-19 pandemic, the need for implementation assessments became increasingly evident. Teams recognized the value of understanding implementation determinants, processes, and outcomes in achieving study goals. To start, the implementation science working sub-group invited nominations of possible implementation frameworks to inform harmonization efforts. Three were proposed to guide measurement selection in the consortium: a primarily determinants framework (Consolidated Framework for Implementation Research; CFIR Damschroder et al., 2009 ), a mostly evaluative framework (Reach Effectiveness Adoption Implementation Maintenance; RE-AIM Glasgow et al., 1999) , and a largely process plus determinants framework (Exploration, Preparation, Implementation, Sustainment; EPIS Aarons et al., 2011) . The working group drafted a summary document describing the objectives, constructs, and associated survey instruments for the three frameworks. Research teams were asked to prioritize the implementation science outcomes and constructs based on their study and goals. A simple majority prioritized the RE-AIM framework. The study teams also expressed a desire to evaluate the outcomes proposed by Proctor et al. (2011) . Next, the implementation science working group recommended items reflecting the components of RE-AIM and Proctor's implementation outcomes that demonstrated some evidence of validity and reliability in Sub-Saharan Africa and/or applied in settings that serve AYA (see Table 1 for a summary; for the full set of items, see Online Resource 2). Seven studies reported plans to assess at least one RE-AIM domain and other implementation outcomes (Fig. 3) as follows. Implementation was the most frequently cited topic of planned evaluation (n = 7), followed by Maintenance (n = 6), Reach (n = 5), and Adoption (n = 4). Intentions to evaluate Acceptability and Feasibility were also high (n = 7) followed by Appropriateness (n = 6) and Fidelity (n = 4).

Two studies in the consortium focus on the HIV prevention continuum. Both studies will enroll non-infected and/ or unknown HIV status AYA and have plans to evaluate HIV testing uptake and yield, and PrEP uptake, adherence, and/ or persistence. Both studies reported plans to use specific HIV-risk assessment tools to assess participant risk of HIV acquisition based on the frequency of risky sexual behaviors.

Three of the PATC 3 H consortium studies focus on the HIV care continuum. These studies will engage AYA living with HIV with harmonized measures that include linkage and retention in care, ART initiation and adherence, age at Table 1 Implementation science harmonization measures *Theoretical constructs were from the RE-AIM framework (Glasgow et al., 1999) . The effectiveness construct from RE-AIM would be assessed on a project-specific level. Outcomes were from Proctor et al. (2011) Donenberg et al., 2020 (Donenberg et al., 2020 diagnosis and viral suppression. Most of the data associated with the HIV care continuum will be collected during clinical care appointments, including HIV viral load before and after study initiation. Finally, three of the consortium studies are collecting data on both HIV prevention and care.

Several demographic characteristics were selected as topics for harmonization, including sex at birth, sexual orientation, and gender, education, housing, food security, employment status, and income. Sexual behavior topics chosen for harmonization include the occurrence of sexual behavior (i.e., "Have you ever had vaginal or anal sex?"), age at sexual initiation, age of sexual partners, and sexual HIV-transmission risk, namely condom use (e.g., "Did you use a condom at last sex?"), number of sexual partners, cross-generational sex, and transactional sex. The existence of a living child was included to characterize teenage pregnancy, parenthood, and contexts of unprotected sex.

Validated and standardized instruments of mental health distress across cultural contexts were recommended for harmonization, namely depression, anxiety, and trauma symptoms. Seven of the eight studies indicated an intention to evaluate mental health and all eight planned to assess alcohol and drug use. The DHWG offered two options of the same instrument (short or long) for depression (Monahan et al., 2009 ) and for anxiety (Barthel et al., 2014) and two instruments to assess alcohol and drug use, both with strong psychometric properties. Sample items for anxiety, depression, and trauma respectively include: "In the past 2 weeks, how often have you been bothered by feeling nervous, anxious, or on edge?" and "In the past 2 weeks, how often have you been bothered by little interest or pleasure in doing things?" "In the past month, have you had nightmares about the event(s) or thought about the event(s) when you did not want to?" The measures of alcohol (Humeniuk et al., 2008) and drug use evaluate lifetime, frequency, amount, and recency of alcohol, marijuana, inhalants, and opioids. For example, "How often do you have a drink containing alcohol?" and "In the past 30 days, have you ever used marijuana (dagga, chamba, weed; Reinert & Allen, 2002) ?". Most study teams reported a plan to adapt the language related to alcohol and drugs to reflect the local vernacular.

The economics working sub-group initially examined the source document developed by Westat and created a list of key intervention activities that could be assigned costs.

These activities were then categorized into three broad groupings to differentiate costs associated with the study intervention, research, and non-labor expenditures. Examples of intervention-related study activities include preparing intervention materials, trainings, promotion and outreach, liaising with community members, and facilitating intervention sessions. Next, the working sub-group developed a Cost Collection Tool and User's Guide with standardized definitions based on a previously validated instrument (Hoover et al., 2019) for study teams to complete. All eight studies indicated a plan to collect cost data to evaluate the costeffectiveness of their interventions. Only five of the eight studies planned to use the Cost Collection Tool. Although the level of detail collected for the cost information varies across the studies, data will allow comparisons of cost across broad categories. The economics working sub-group also developed the PATC 3 H Policy Priority Survey to seek feedback from incountry decision-makers regarding the adoption of public health interventions to improve outcomes for AYA affected by HIV. The survey consists of eight open-ended questions and ratings of the importance of specific types of economic analyses. This information will facilitate harmonization of economic assessments and help identify measures and metrics required to support informed decision-making that meet the requirements of policy makers. Six of the eight study teams report a plan to implement the survey.

This paper describes a two-year process undertaken by the DHWG within the PATC 3 H consortium to identify and propose measures for harmonization for AYA across sub-Saharan Africa and Brazil in five categories: Implementation science, HIV prevention and care; demographic and sexual behavior; mental health and substance use; and economic assessment. The process highlights the value of data harmonization by identifying strong and relevant instruments that can be applied across diverse study populations and contexts to strengthen the replication of findings and fortify the science. Data harmonization permits larger sample sizes than can be achieved by a single study, thus increasing confidence in the findings and their generalizability. Data harmonization also allows for direct comparisons across studies to answer questions about cultural differences that inform measure relevance. Finally, data harmonization can foster more efficient usage of secondary data and enhance opportunities for collaborative and multi-country research-particularly in settings under-represented in the research-while limiting the persistent problem of research "waste" (Doiron et al., 2012) . This paper contributes much needed guidance regarding the harmonization of implementation science measures and provides an example of the potential benefits to this burgeoning field of study. Although none of the studies initially planned to formally evaluate implementation outcomes, after extensive discussion of the challenges in carrying out research in LMICs and the impact of COVID-19 on our research partners and projects, all eight study teams decided to adopt some or all of the recommended implementation science measures. Global health researchers are acutely aware of the need to attend to implementation factors-i.e., barriers and facilitators-and consider how to achieve the most positive outcomes possible with regard to implementation (i.e., adoption, feasibility, acceptability, etc.; Donenberg et al., 2019; Theobald et al., 2018) . We detail the harmonization process used to identify and select implementation science measures and offer a harmonized instrument for future researchers wishing to address HIV prevention and care among AYA in LMICs. Importantly, the process described herein and the measures proposed can also apply to all multi-study protocols.

There are important efforts underway to identify implementation science measures that are applicable and relevant across studies [e.g., see U.S.-based Disparities Elimination through Coordinated Interventions to Prevent and Control Heart and Lung Disease Risk (DECIPHeR) Alliance, 2020]. We urge future investigators to ensure that the implementation science measures adopted be driven by theory (in this case, we used RE-AIM; Glasgow et al., 1999) in order to build a more robust and generalizable science. Consideration of implementation determinants and outcomes-while recognizing the potential for overlap between the twowill help ensure that the implementation science measures address a wide range of processes affecting implementation.

Of note, the process of harmonization required extensive collaboration, patience, and attention to potential challenges. As a result, there are limitations to data harmonization processes worth mentioning. First, in many cases, although the proposed harmonization topics were already included in the PATC 3 H studies, specific assessment strategies differed, making it hard to select a "one size fits all" measure. Encouraging a multi-measure approach of the same construct allowed us to overcome resistance to adopt alternatives. Likewise, some researchers planned to use measures they had already developed or employed over the years, creating reluctance to integrate new instruments-particularly if they hoped to compare results with their previous studies. Offering the choice to use a short or long version to measure the same construct provided flexibility and ultimately led to the addition of the harmonized items. We recognized that allowing study teams the option to harmonize risked incomplete harmonized data. However, appealing to the "greater good" achieved by harmonization helped underscore the benefits derived, and most study teams agreed to adopt the recommended measures. Finally, minor adaptations and modified item wording may pose challenges at the analysis stage even though emphasis was placed on ensuring that the original meaning was retained. While this will likely mitigate some concerns, research has demonstrated that even small modifications to wording can change the measure's original psychometric properties. Hence, while this is a limitation of our harmonization process, future research should consider working in collaboration with scale developers to evaluate the modified measures and to ensure the use of best practices when modifying tools. Secondly, although AYA were incorporated within each individual study as a youth advisory board or as youth ambassadors, the voices of AYA were not included in the harmonization process. The measures harmonized were limited to the predetermined goals and objectives of the eight individual studies making in it difficult to generalize our findings across all youth populations in LMICs. Studies also varied in design and intervention content for youth populations. While there are significant challenges to harmonizing multi-site AYA interventions, there are also new opportunities, such as incorporating robust youth-driven implementation outcomes that improve how we deliver and scale-up effective interventions. Future harmonization efforts will benefit from AYA voices and lived experiences not only in the design of interventions (as was done here), but also in workgroups to harmonize measures. AYA contributions may accelerate the scale-up of effective interventions.

The PATC 3 H consortium opted to prioritize mental health and implementation science as key components for harmonization. This decision reflects growing concerns about mental health in the context of other health outcomes (e.g., cardiovascular disease, diabetes, obesity) and the important role of implementation science in the field broadly (Dixon & Patel, 2020; Norton et al., 2017) . Even before the COVID-19 pandemic, consortium members recognized that mental health impacts HIV prevention and care for all AYA. Interventions designed to improve the prevention and care continua, therefore, were seen as intimately connected to addressing mental health. This insight turned out to be prescient as the rates of mental health distress during COVID have risen dramatically in the past two years (Jones et al., 2021) .

Seven recommendations for future research consortia emerged from our experience:

(1) The harmonization process must include a representative from each study team to ensure adequate attention to the cultural relevance of selected measures. The process and the outcome benefits substantially from multidisciplinary perspectives and input.

(2) Cross-cultural harmonization is possible and increasingly required by funding agencies but must be flexible and clear from the beginning. For example, several teams had already selected measures for the harmonization topics. We, therefore, delineated measures that should not be modified (e.g., well validated) and those where modification in this particular harmonization process was acceptable (i.e., implementation science). (3) The harmonization process requires transparency and a careful systematic approach to allow for replication. Planning for harmonization in advance and documenting the methods and procedures will ensure sufficient time to reach harmonization decisions. (4) Certain study populations and/or contexts may drive decisions about harmonization priorities. Those leading the harmonization process should ensure that all teams and their priorities are fairly represented in the decision-making process. (5) There is value in disseminating recommendations for measures to be harmonized by topic and setting. Several efforts currently exist that promote measures for harmonization in the social determinants of health (e.g., the PhenX Toolkit; Hamilton et al., 2011) . The current paper adds new recommendations in the area of implementation science, which has not received the same attention to harmonization-particularly in LMIC contexts. (6) The data harmonization process must remain nimble and responsive to life events-in our case, COVID-19. Although we did not initially harmonize a COVID-19 measure, team discussions highlighted the importance of assessing the effects of COVID-19 on our study populations and projects, prompting quick action. The measures consortium members chose to incorporate into their individual studies will help build the broader evidence base on COVID-19 and its influences on AYA in LMICs. (7) Allowing flexibility and measurement adaptations to improve fit within the local context and population creates tension between internal and external validity. Future research should strive to maximize fit without sacrificing scientific rigor. As with efforts to document intervention adaptations (e.g., the FRAME; Wiltsey Stirman et al., 2019), investigators should record and communicate with scale developers about modifications to measures to inform future research and applications.

In summary, this paper offers guidance on the process of decision-making around creating a data harmonization package. It contributes to the literature on the challenges and benefits of harmonization across populations and contexts focused on a particular health outcome-HIV prevention and care. The processes and the recommendations could be of use to future harmonization efforts both in LMICs and in other underserved communities. Although some recommendations for specific sets of measures already exist (e.g., the PhenX Toolkit; Norton et al., 2017) , we present a harmonized package of implementation science measures, seeking to fill the gap in the literature on recommended implementation science measures specific to AYA and HIV from LMICs and also provide guidance on the harmonization process.

Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health

Adolescents and young people at the centre: Global perspectives and approaches to transform HIV testing, treatment and care

Psychometric properties of the 7-item Generalized Anxiety Disorder scale in antepartum women from Ghana and Cote d'Ivoire

Advances in patient-reported outcomes: The NIH PROMIS(®) measures

COVID-19 experiences among South African adolescent girls and their mothers

Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science

Youth engagement in developing an implementation science research agenda on adolescent HIV testing and care linkages in Sub-Saharan Africa

Disparities Elimination Through Coordinated Interventions to Prevent and Control Heart and Lung Disease Risk (DECIPHeR) Alliance

The application of implementation science to community mental health

Facilitating collaborative research: Implementing a platform supporting data harmonization and pooling

Applying the Exploration Preparation Implementation Sustainment (EPIS) Framework to the Kigali Imbereheza Project for Rwandan adolescents living with HIV

IMARA: A mother-daughter group randomized controlled trial to reduce sexually transmitted infections in adolescent African American women

Measurement Schmeasurement: Questionable measurement practices and how to avoid them

Evaluating the public health impact of health promotion interventions: The RE-AIM framework

The PhenX Toolkit: Get the most from your measures

Measuring implementation in global mental health: Validation of a pragmatic implementation science measure in eastern Ukraine using an experimental vignette design

Developing a web-based tool for collecting resource use data from colorectal cancer screening programs for economic evaluations. Preventing Chronic Diseases

Validation of the Alcohol, Smoking And Substance Involvement Screening Test (ASSIST)

Impact of COVID-19 on mental health in adolescents: A systematic review

Implementation outcome instruments for use in physical healthcare settings: A systematic review

The Program Sustainability Assessment Tool: A new instrument for public health programs

Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/ AIDS in western Kenya

Grid-enabled measures: Using Science 2.0 to standardize measures and share data

Making sense of implementation theories, models and frameworks

An exploratory study to examine intentions to adopt an evidence-based HIV linkage-to-care intervention among state health department AIDS directors in the United States

Mapping the growing discipline of dissemination and implementation science in health

Definitions of implementation science in HIV/AIDS

Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health

Prevention And Treatment through a Comprehensive Care Continuum for HIV-affected Adolescents in Resource Constrained Settings

Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health

Advancing the application, quality and harmonization of implementation science measures

The Alcohol Use Disorders Identification Test (AUDIT): A review of recent research. Alcoholism

Reproducibility and research integrity

Diffusion of a schoolbased substance abuse prevention program: Predictors of program implementation

Expanding statistical frontiers in sexual science: Taxometric, invariance, and equivalence testing

Dyadic measurement invariance and its importance for replicability in romantic relationship research

Fostering successful and sustainable collaborations to advance implementation science: The adolescent HIV prevention and treatment implementation science alliance

Implementation research: New imperatives and opportunities in global health

Accelerating adolescent HIV research in low-income and middle-income countries: Evidence from a research consortium

The FRAME: An expanded framework for reporting adaptations and modifications to evidence-based interventions