key: cord-0887927-52h5grwf
authors: Escobar-Viera, César G.; Melcher, Eleanna M.; Miller, Rebekah S.; Whitfield, Darren L.; Jacobson-López, Daniel; Gordon, Jacob D.; Ballard, Adrian J.; Rollman, Bruce L.; Pagoto, Sherry
title: A systematic review of the engagement with social media–delivered interventions for improving health outcomes among sexual and gender minorities
date: 2021-07-07
journal: Internet Interv
DOI: 10.1016/j.invent.2021.100428
sha: 3afdaa8ad709943552f33ab94f6a0302858599c9
doc_id: 887927
cord_uid: 52h5grwf

BACKGROUND: Sexual and gender minority (SGM) persons face a number of physical and mental health disparities closely linked to discrimination, social stigma, and victimization. Despite the acceptability and increasing number of digital health interventions focused on improving health outcomes among SGM people, there is a lack of reviews summarizing whether and how researchers assess engagement with social media–delivered health interventions for this group. OBJECTIVE: The objective of this systematic review was to synthesize and critique the evidence on evaluation of engagement with social media–delivered interventions for improving health outcomes among SGM persons. METHODS: We conducted a literature search for studies published between January 2003 and June 2020 using 4 electronic databases. Articles were included if they were peer-reviewed, in English language, assessed engagement with a social media–delivered health intervention for improving health outcomes among sexual and gender minorities. A minimum of two authors independently extracted data from each study using an a priori developed abstraction form. We assessed quality of data reporting using the CONSORT extension for pilot and feasibility studies and CONSORT statement parallel group randomized trials. RESULTS: We included 18 articles in the review; 15 were feasibility studies and 3 were efficacy or effectiveness randomized trials. The quality of data reporting varied considerably. The vast majority of articles focused on improving HIV-related outcomes among men who have sex with men. Only three studies recruited cisgender women and/or transgender persons. We found heterogeneity in how engagement was defined and assessed. Intervention usage from social media data was the most frequently used engagement measure. CONCLUSION: In addition to the heterogeneity in defining and assessing engagement, we found that the focus of assessment was often on measures of intervention usage only. More purposeful recruitment is needed to learn about whether, how, and why different SGM groups engage with social media-interventions. This leaves significant room for future research to expand evaluation criteria for cognitive and emotional aspects of intervention engagement in order to develop effective and tailored social media-delivered interventions for SGM people. Our findings also support the need for developing and testing social media-delivered interventions that focus on improving mental health and outcomes related to chronic health conditions among SGM persons.

Sexual and gender minority (SGM, i.e., lesbian, gay, bisexual, transgender, queer/questioning) persons face a number of physical and mental health disparities closely linked to discrimination, social stigma, and victimization (Meyer, 2003) . While sexual minorities comprise persons whose sexual identity, attraction, and practices are different from the dominant culture (American Psychological Association, 2012; Cochat Costa Rodrigues et al., 2017) , gender minorities include individuals whose gender identity (man, woman, other) or expression (masculine, feminine, other) is different from their sex assigned at birth (male, female) (American Psychological Association, 2015) . The frequently chronic trajectory of the physical and mental health conditions SGM persons face makes them potentially amenable to behavioral intervention programs that seek to boost protective factors, reduce risk factors, manage symptoms, or improve treatment adherence. Indeed, digital technology delivered interventions (e.g., via apps, web-based, social media) are widely acceptable among SGM persons (Gilbey et al., 2020) .

Social media can be defined as "a group of Internet-based applications that allow the creation and exchange of user-generated content" (Kaplan and Haenlein, 2010) , and it includes a wide range of platforms in the form of websites mobile apps (e.g., Twitter, Snapchat), messaging apps (e.g., WhatsApp, Facebook Messenger), and location-based apps, i. e., an app that allows a person to simultaneously use GPS to broadcast their location and be located. (e.g., Grindr, Scruff). SGM individuals are heavy users of social media platforms and apps (Escobar-Viera et al., 2018; Seidenberg et al., 2017) , to which they turn to explore their identity, find events, services and health information, meet friends or potential partners, and seek social support and affirmation (DeHaan et al., 2013; Harper et al., 2016) . These reasons, along with their desire to avoid potential negative experiences (e.g., stigmatization, denial, refusal of services) at in-person health care settings (Ayhan et al., 2020) , might influence SGM people to turn to and rely on digital interventions.

While social media holds potential for delivering digital health interventions and these seem acceptable to SGM persons, high levels of attrition from these interventions have been reported among the general population (Short et al., 2018) . Given this, engagement is a proposed prerequisite for the effectiveness of social media-delivered health interventions because when engagement is low, the intervention will likely fail to change behavior (Pagoto and Waring, 2016; Yardley et al., 2016) . Interestingly, assessing the engagement with social mediainterventions beyond simply usage had not been emphasized in the research (Welch et al., 2016) . This paucity of research around the subject makes it difficult to reach to significant conclusions related to engagement with social media-interventions for the general population, let alone for SGM individuals.

In the context of digital health interventions, engagement has been defined as having two dimensions: micro level engagement, or "the extent (e.g. amount, frequency, duration) of usage" and macro level engagement, or "the depth of user involvement with the behavior change process characterized by attention, interest, and affect" (Perski et al., 2017) . Some examples of micro level engagement include number of 'likes', count of post views, and frequency and number of user posts or replies to intervention messages . Meanwhile, certain types of post content produced by intervention participants, such as autobiographical posts , might be an indicator of affective response to intervention components, which are described as macro level engagement. These two dimensions are important to evaluate in social media-delivered interventions because, while good micro level engagement might signal overall engagement with the intervention, macro level engagement seems to be a more valid indicator of engagement with a process of behavioral change (Yardley et al., 2016) . More recently, a number of both qualitative and quantitative methods have been proposed for assessing these emotional, cognitive, and behavioral aspects of engagement, including interviews, think aloud methods, focus groups, self-report questionnaires, ecologic commentary assessments, system usage and sensor data, social media data, and psychophysiological measures (Short et al., 2018) .

Despite the increasing number of digital health interventions focused on improving health outcomes among SGM people (Gilbey et al., 2020) , there is a lack of reviews summarizing whether and how researchers assess this construct. Given the ongoing conversation on building consensus in the area of engagement with digital health interventions, we propose to expand previous research (Gilbey et al., 2020) , and the need to expand access to health services to hidden populations such as SGM persons (Kates et al., 2018) , we propose to put a specific focus on summarizing the literature in the field of engagement with social media-delivered health interventions for SGM people. Therefore, the objectives of this review were to (1) identify all peer-reviewed published articles that assessed engagement with social media-delivered health intervention for SGM persons, describing study characteristics and quality of study reporting; and (2) describe whether and how engagement metrics were operationalized across studies using the proposed dimensions of micro and macro level engagement.

We report this review according to the requirements of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement (PRISMA) Moher et al., 2009 ) (Appendix 1). We registered the protocol for this review in the PROPERO database (no. CRD4201942189), and we make it available here as a supplement (Appendix 2).

We included peer-reviewed references published in journals of medicine, public health, or social sciences, in English language. On a case-by-case basis, we allowed references from conference proceedings, as long as these conferences required full manuscript submission and peer-review before acceptance. Included manuscripts had to assess the engagement with health interventions delivered through social media and focused on SGM individuals. In the context of this review, social media could include social and professional networking, content production and sharing, online communities, and location-based services (i. e. Facebook, LinkedIn, YouTube, online support groups, Tinder, etc.). Because we aimed to review the findings about engagement with these interventions, we did not set specific health outcomes or conditions as inclusion criteria. Given the heterogeneity in defining engagement, we included papers that reported to assess engagement, and examined their conceptualization of engagement and assessment of it. Finally, we excluded theses, dissertations, and opinion pieces.

Literature searches were developed and conducted by a health sciences librarian (RM) using the PubMed, Ovid APA PsycInfo, EBSCOhost SocINDEX, and ACM Guide to Computing Literature databases. Search strings were comprised of natural language and controlled vocabulary representing the concepts of sexual or gender minorities and social media, and were translated for each database. The search strings were adapted in part from searches developed by Charles Wessel (Escobar-Viera et al., 2018) . The searches were limited to the publication year 2003 (the year "MySpace," the first modern social media platform launched) to present (June 2020), and references were downloaded into the EndNote citation management software After importing them into a bibliography managing software, duplicate references were removed (Bramer et al., 2016) . We provide the entire list of terms, descriptors, and search strings as a supplement (Appendix 3). We retrieved 8477 citations. Of these, 2453 were from PubMed or MEDLINE, 2829 from PsycINFO, 2250 from SocINDEX, and 945 from ACM. After removing 1539 duplicates, 6938 citations remained for the screening process.

We conducted all screening and data extraction procedures using an online platform (Distiller SR, 2020) , within which we created structured forms that systematically guided both screening and data extraction processes. Two pairs of reviewers (EMM, DJL, JDG, and AJB) were randomly assigned an equal number of citations. Within each pair, reviewers independently screened all manuscript titles and abstracts to generate a set of references that had some possibility for inclusion. Then, each pair assessed the full text of these references to determine eligibility for inclusion. We achieved substantial interrater reliability (weighted Cohen's kappa, 0.75) (McHugh, 2012) . In order to minimize risk of reviewer bias, we held consensus meetings between the coders and the rest of the team (CGEV, DLW, BLR, SP) to resolve differences. These meetings were sufficient to resolve all conflicts, and adjudication procedures were not needed.

Our extraction forms allowed for several categories of data (1) study logistics (author, full title, year of publication, methodology, research design, setting, and funding source); (2) study population characteristics (number of participants, recruitment methods, gender identity, sexual orientation, race, ethnicity, education level, employment status, and income);

(3) quality of study reporting; (4) intervention characteristics (social media platforms used, intervention name, study length, theoretical framework, comparison group, and measures of usage/engagement); (5) health outcomes (primary and secondary outcomes, measures used); and (6) results and limitations. During the extraction process, one reviewer of each pair completed a first data extraction and the second reviewer validated or disagreed with it. This procedure helped ensure accuracy of data extracted with disagreements being resolved during team meetings.

We assessed quality of reporting of all included studies. Given that all studies fit in two categories, we used either the CONSORT extension for pilot and feasibility studies checklist Thabane et al., 2016) or the main CONSORT statement for parallel group randomized trials checklist (Schulz et al., 2010) . For each checklist, we assigned values of zero for each item marked with an 'X' and one for items marked with a checkmark. Total scores could range from zero to 40 or from zero to 37, where a score of 40 or 37 means the study fully adhered to the CONSORT guidelines for pilot and feasibility studies or parallel group randomized trials, respectively (Appendices 4 and 5).

Of 6938 citations reviewed, we excluded 6812 records after title and abstract screening (Fig. 1) . During this step, we identified three potentially relevant references with manual search, and thus included them for full-assessment. Of 129 articles fully assessed for eligibility, we excluded 15 that were not peer-reviewed manuscripts, six that did not have sexual or gender minority participants, 87 studies of interventions not delivered via social media, and three that did not use a measure of usage or engagement with the intervention. We reviewed the list of references for each included manuscript in order to identify additional studies, but no new study that met our inclusion criteria were identified in this manner. Therefore, 18 research manuscripts were included in the final sample. Records aŌer duplicates removed (n = 6,938)

Full-text arƟcles assessed for eligibility (n = 129)

Full-text arƟcles included for synthesis (n = 18)

Duplicates removed (n = 1,539)

Flowchart of studies screened and included in a 2020 systematic review of engagement with social media− delivered interventions for improving health outcomes among sexual and gender minorities.

Of 18 manuscripts included in this review, 15 reported results of feasibility studies (Alarcón Gutiérrez et al., 2018; Anand et al., 2015; Elliot et al., 2016; Horvath et al., 2013; Huang et al., 2017; Lampkin et al., 2016; Lelutiu-Weinberger et al., 2015; Patel et al., 2020; Pedrana et al., 2013; Sun et al., 2015; Tanner et al., 2018; Vogel et al., 2019a; Washington et al., 2017; Young and Jaganath, 2013; Zhu et al., 2019) . Three studies reported results of either efficacy (Bull et al., 2012) or effectiveness trials (Cao et al., 2019; Vogel et al., 2019b) . Characteristics of each study are presented in Table 1 .

Eleven studies were conducted in the USA, two in China, and one in Thailand, Australia, India, UK, and Spain, respectively. Social media sites used to deliver the interventions varied across studies. Eleven studies (Anand et al., 2015; Bull et al., 2012; Cao et al., 2019; Horvath et al., 2013; Lelutiu-Weinberger et al., 2015; Patel et al., 2020; Vogel et al., 2019a; Vogel et al., 2019b; Washington et al., 2017; Young and Jaganath, 2013; Zhu et al., 2019) delivered their interventions using a single social media site (Facebook, WeChat, Adam's Love, Thrive with Me). Four studies (Alarcón Gutiérrez et al., 2018; Huang et al., 2017; Lampkin et al., 2016; Sun et al., 2015) leveraged location-based social media apps for intervention delivery (Grindr, Planet Romeo, Wapo, A4A Radar, Jack'd, Scruff), and three other studies (Elliot et al., 2016; Pedrana et al., 2013; Tanner et al., 2018) used a combination of social media sites and location-based apps to do so (Facebook, YouTube, Badoo, Gaydar, A4A Radar, Grindr, Jack'd, Scruff, Recon). Feasibility studies had an average duration of 8.7 months (range 1-40 months). Both effectiveness trials had a duration of 12 months, and the only efficacy trial included had a duration of two months.

Participant demographic characteristics varied greatly across studies. Among studies that reported mean age of participants (Bull et al., 2012; Cao et al., 2019; Elliot et al., 2016; Horvath et al., 2013; Lelutiu-Weinberger et al., 2015; Tanner et al., 2018; Vogel et al., 2019a; Washington et al., 2017; Young and Jaganath, 2013) , mean age was 27.1 years (range 16-34). Only three studies (Anand et al., 2015; Vogel et al., 2019a; Vogel et al., 2019b) recruited participants who identified as either cisgender women or transgender. Of 15 feasibility studies, 13 recruited samples of men who have sex with men (MSM), with sexual orientation comprising mainly gay men. One study (Vogel et al., 2019a) included participants with diverse sexual identity, and another (Pedrana et al., 2013) did not report participants' sexual identity. Among the three effectiveness or efficacy trials, one (Cao et al., 2019) recruited MSM only, another (Vogel et al., 2019b) reported sexual and gender minority participants as a single group, and one study (Bull et al., 2012) did not report participants' sexual identity.

The quality of reporting was variable across the included manuscripts. Among 15 feasibility studies, CONSORT scores ranged from 14 to 26 out of 40 possible points; 8/15 studies met reporting standards on their title, 14/15 did so on the Introduction section, 0/15 met the reporting standards in the Methods and Results sections, and 4/15 did so on the Discussion section. Only 2/15 studies met reporting standards regarding funding sources, ethical approval, pilot trial protocol, and registration number. Among three effectiveness or efficacy trials, CONSORT scores (Schulz et al., 2010) were 22 (Cao et al., 2019) , 17 (Vogel et al., 2019b) , and 22 (Bull et al., 2012) out of 37 possible points; 2/3 studies met reporting standards in their title and Introduction sections, 0/3 did on the methods, results and discussion reporting, and 1/3 met standards reporting trial registration, protocol availability, and funding sources.

Characteristics of social media− delivered interventions evaluated in each included study are summarized in Table 2 . A total of 17 interventions were evaluated across 18 articles. The vast majority of papers evaluated interventions related to HIV care: sexual health promotion (Pedrana et al., 2013) , HIV testing (Alarcón Gutiérrez et al., 2018; Cao et al., 2019; Elliot et al., 2016; Huang et al., 2017; Patel et al., 2020; Sun et al., 2015; Washington et al., 2017; Young and Jaganath, 2013; Zhu et al., 2019) , condomless sex (Bull et al., 2012; Lelutiu-Weinberger et al., 2015) , linkage to services (Anand et al., 2015; Lampkin et al., 2016) , and treatment adherence (Horvath et al., 2013; Tanner et al., 2018) . Two articles (Vogel et al., 2019a; Vogel et al., 2019b) evaluated different aspects of the same smoking cessation intervention.

All but one of the examined interventions consisted of exposing participants to informational and or educational content (via message boards, posts, ads, and private or group chats) about the health-related outcome of interest. One (Lelutiu-Weinberger et al., 2015) of the interventions coupled this educational information with the delivering of motivational interviewing and cognitive behavioral skills-based sessions. Interventions allowed interactions with health-related content only (Cao et al., 2019; Elliot et al., 2016; Huang et al., 2017; Pedrana et al., 2013; Vogel et al., 2019a) , with peers only (Patel et al., 2020; Washington et al., 2017) , with health educators only (Alarcón Gutiérrez et al., 2018; Lampkin et al., 2016; Lelutiu-Weinberger et al., 2015; Sun et al., 2015; Tanner et al., 2018; Vogel et al., 2019b; Zhu et al., 2019) , or both peers and health educators (Anand et al., 2015; Bull et al., 2012; Horvath et al., 2013; Young and Jaganath, 2013) . In 9/18 articles (Cao et al., 2019; Elliot et al., 2016; Horvath et al., 2013; Jacobs and Kane, 2012; Vogel et al., 2019a; Vogel et al., 2019b; Washington et al., 2017; Young and Jaganath, 2013; Zhu et al., 2019) , study design included some form of comparator condition.

All 18 studies included in this review used at least one measure of micro level engagement. However, measures of engagement varied widely across studies. Ten studies (Anand et al., 2015; Bull et al., 2012; Elliot et al., 2016; Horvath et al., 2013; Huang et al., 2017; Lelutiu-Weinberger et al., 2015; Patel et al., 2020; Pedrana et al., 2013; Vogel et al., 2019a; Zhu et al., 2019) used social media data for frequency counts of views, reads, or logins of the intervention materials or social media site. Ten studies (Bull et al., 2012; Horvath et al., 2013; Lampkin et al., 2016; Patel et al., 2020; Pedrana et al., 2013; Sun et al., 2015; Tanner et al., 2018; Vogel et al., 2019a; Vogel et al., 2019b; Young and Jaganath, 2013) leveraged this data to assess quantity and quality of user responses to intervention prompts, emails, or other users' comments.

Moreover, three articles (Alarcón Gutiérrez et al., 2018; Bull et al., 2012; Washington et al., 2017) reported intervention attrition rates as an indicator of engagement. One study (Cao et al., 2019) assessed selfreport of view counts and user responses to intervention messages. Additionally, two articles (Lelutiu-Weinberger et al., 2015; Pedrana et al., 2013) utilized qualitative assessments to learn about usability of and user experience with the intervention. Finally, three articles (Elliot et al., 2016; Horvath et al., 2013; Huang et al., 2017) assessed number of intervention activities completed by the user.

In this systematic review, we found a low number of peer-reviewed articles examining social media-delivered interventions to improve health outcomes among SGM people. We found variation in the quality of data reporting across articles, with almost all interventions focused on improving HIV-related outcomes among cisgender MSM. Importantly, we found an important variation in how studies assessed engagement, with measures focused mostly on micro level engagement. We discuss the implications of these findings below and provide suggestions for future research. We used a comprehensive set of inclusion criteria for this review. For example, we included articles from studies conducted on social media apps more commonly thought of as dating/hook-up apps. Even so, we found few studies that evaluated social media-delivered health interventions focused on SGM persons; 15 articles reported results of feasibility studies, and 3 were either efficacy or effectiveness trials. A large majority of these studies was conducted in developed countries only within the last five years. Given the ubiquitous use of social media among SGM people (Anderson and Jiang, 2018; Nesi et al., 2018) and the high acceptability of digital health interventions among this group (Gilbey et al., 2020) , social media holds potential as a delivery modality for behavioral interventions targeting these minority groups. As the world struggles with a viral pandemic which forces us to consider social distancing measures, digital health interventions may be viable interventions to ensure individuals' safety and privacy. To do this, welldesigned and well-conducted feasibility trials, preferably with comparator conditions, will provide stronger evidence regarding engagement, and potential effectiveness of these interventions.

Quality of data reporting was variable across the included studies. Most of the variability was due to incomplete reporting of the Methods and Results sections, such as changes made to the assessments after pilot study commenced, rationale for sample size, explanation of interim analyses, method used to generate and to implement random allocation, reasons for dropouts, or harms or unintended effects in each group. The variation we found in the reporting might be explained in part by the fact of relatively recent consensus around reporting standards and how to structure feasibility trials (O'Cathain et al., 2015; Tickle-Degnen, 2013) in order to generate evidence that can be used to plan and design efficacy or effectiveness trials. It is likely that researchers, publishers, and funding agencies will continue to incorporate these relatively recent elements to the common practice of future research.

In most studies included in this review, participants were between 18 and 30 years of age. While social media use was previously most common among teens (Anderson and Jiang, 2018) , usage has been growing among adults under (Drouin et al., 2020; Perrin and Anderson, 2019) and over 65 years of age (Nimrod, 2020) , especially during the COVID-19 pandemic, to address anxiety, loneliness, and other mental health concerns. Given the current events around the world, increased social media use has been linked to enhanced quality of life among older adults (Wallinheimo and Evans, 2021) . This increased usage combined with the motivation for using social media among older adults (Jung and Sundar, 2020) highlights the opportunity to expand intervention research into this particular age group.

The vast majority of articles included focused on improving HIVrelated outcomes with samples comprised of men who have sex with N/R: not reported. a Assessed using CONSORT 2010 main statement (37 possible points) and extension to randomized pilot and feasibility trials (40 possible points). b Demographic assessment allowed participants to select multiple responses and therefore, percentages may not add up to 100. c Race/ethnicity variables were reported with different categories from the ones reported in this review, therefore, percentages may not add up to 100. d Sexual orientation was reported for only 79 study participants, as opposed to the entire sample of 2656 persons. -Roberts et al., 2017) . The lack of diversity in both sexual and gender identity among these samples makes it difficult, if not impossible, to generalize our findings about engagement with social media interventions to larger groups of SGM people. Therefore, further intensive work is needed to increase our understanding of how different groups of SGM persons engage with digital health interventions in order to meet their specific needs of health information and identity representation. Given that some of these SGM groups are more hidden than others in our communities, this research will require more purposeful incorporation of more SGM people in the entire process of developing, usability testing, feasibility evaluation, and effectiveness testing of social media-delivered other digital health interventions. However, assessments among SGM people found important disparities and unmet needs for depression (Becerra-Culqui et al., 2018; Marshal et al., 2011) , alcohol and substance use (Boyd et al., 2019) , tobacco use (Jamal et al., 2018) , obesity and eating disorders (Azagba et al., 2019) , some types of cancer (Valanis et al., 2000) , and heart disease (Caceres et al., 2020) . These disparities are even greater in SGM communities of color (Bostwick et al., 2014) and SGM communities living in rural settings (Rosenkrantz et al., 2017; Willging et al., 2006) . Moreover, only three articles reported recruitment of sexual minority cisgender women and/or transgender persons. The lack of inclusion of these groups limits the ability to determine the efficacy of these types of interventions for sexual minority cisgender women and transgender persons. Given the specific health disparities and barriers to access care for subgroups of SGM people (Hafeez et al., 2017; Safer et al., 2016) , future intervention research that leverages social media should be conducted with these groups specifically to explore how digital health interventions might address the unmet needs and health disparities for sexual minority cisgender women and transgender persons.

We also found some variation regarding which social media platform was used to deliver interventions. Although the vast majority of studies were conducted on commercially available platforms (e.g., YouTube, dating apps), two studies leveraged platforms specifically developed for the research. This finding highlights a growing preference of both participants and researchers for commercially available platforms that do not require additional learning effort for the participant, a very-well known principle in community-based interventions of "meeting people where they already are." Additionally, while most interventions were delivered via Facebook, a sizeable minority of interventions were delivered on either via alternative social media platforms (e.g., You-Tube) or via location-based social media apps (i.e., dating apps). This highlights the importance of having a human-centered approach (Moore and Arar, 2018) for the designing, developing, and testing of more engaging social media interventions, focused not only on platform popularity but also on users' goals, preferences, and motivations for use.

While the lack of consensus on what constitutes engagement for digital interventions, as well as the dimensions and measures to best assess it, makes it a difficult task to summarize our findings regarding engagement. However, a key finding of this review that complements and expands previous research on a similar topic (Gilbey et al., 2020) , is that the most frequently employed engagement metrics were number of views, number of comments on intervention posts, attrition rates, and online usage statistics provided by each social media platform. Only two studies utilized qualitative assessments to explore usability and user experience, but not macro level engagement (Short et al., 2018) . These metrics speak to the extent to which the user has observed or interacted with an intervention. In other words, these are indicators of micro level engagement (Short et al., 2018) , which provides information on behavioral engagement with the intervention but less so about macro level engagement, or the user's cognitive and emotional investment with the behavior change process. Given the importance of developing interventions that mediate positive outcomes via effective engagement (which may or may not require sustained engagement) (Yardley et al., 2016) , further consideration must be paid to evaluating both micro and macro level engagement of future social media-delivered interventions.

Advancing the science of social media-delivered interventions will require complementary approaches to assessment of engagement that include both micro and macro level factors (Pagoto et al., 2019) . Leveraging mixed methods (i.e., collecting and integrating quantitative and qualitative data) can dramatically expand the findings related to aspects of macro level engagement compared to what either type of data separately might be able to tell (Fetters et al., 2013) . For instance, researchers might want to conduct thematic analysis of user feedback collected at the same time that intervention messages are delivered on social media along with frequency count of message views. By doing this, it might be possible to suggest potential relationships between emotional responses to intervention messages with number of views, as well as potential cognitive or emotional gains from intervention messages. This might prove critically important for interventions focused on minority groups, especially SGM persons. Furthermore, it is critical to determine how the intersectional identities of SGM persons (e.g., race, ethnicity, social class, religion, etc.) might impact the types of intervention messages which may be most effective.

Our work is not without some limitations. First, the low number of studies included in this review along with the variation in definition and measurement of engagement across studies and the limited sample of categories of SGM other than MSM, limited our ability to generalize our findings to interventions targeting diverse groups of SGM. Second, by limiting our results to research published in English language, we might have missed relevant research published in other languages. Thirdly, there was a prominence of interventions addressing HIV disparities, but limited interventions in other areas. Finally, we found a noticeable heterogeneity in outcome measures, study duration, and study protocol. Therefore, a meta-analysis was not possible. Notwithstanding these limitations, we utilized a comprehensive set of inclusion criteria along with four large online databases, which largely covered the scope of the research conducted on social media-delivered interventions for improving health outcomes among SGM individuals.

There is a growing interest among researchers for developing and testing effective digital health interventions for SGM persons. Digital health interventions are acceptable to SGM individuals, hold potential for cost-effectiveness, and might be able to reduce barriers to access of healthcare for SGM people, otherwise consistently reported in the extant research. Our findings support the need for this research, with a focus on social media-delivered interventions for improving mental and physical health outcomes. The objective of this review was to identify and describe the evaluation of engagement with social media-delivered interventions for improving health outcomes among SGM persons. In addition to the heterogeneity in defining and assessing engagement, we found a focus on measures of intervention usage with room for future research to improve evaluation of cognitive and emotional aspects of engagement in order to develop effective and tailored digital health interventions for sexual and gender minorities.

Supplementary data to this article can be found online at https://doi. org/10.1016/j.invent.2021.100428.

We gratefully acknowledge funding from NIMHD (grant number R00 -MD012813) and NIMH (grant number P50 -MH115838) for the completion of this research. Funding sources played no role in the study design; collection, analysis, and interpretation of data; writing of the report; or decision to submit the article for publication.

The authors declare no conflicts of interest.

Acceptability and effectiveness of using mobile applications to promote HIV and other STI testing among men who have sex with men in Barcelona

Guidelines for psychological practice with lesbian, gay, and bisexual clients

Guidelines for psychological practice with transgender and gender nonconforming people

Innovative strategies using communications technologies to engage gay men and other men who have sex with men into early HIV testing and treatment in Thailand

A systematic review of the discrimination against sexual and gender minority in health care settings

Overweight and obesity among sexual minority adults in the United States

Addressing health care disparities among sexual minorities

Mental health of transgender and gender nonconforming youth compared with their peers

Discrimination and mental health among lesbian, gay, and bisexual adults in the United States

Severity of alcohol, tobacco, and drug use disorders among sexual minority individuals and their "not sure" counterparts

De-duplication of database search results for systematic reviews in endnote

Social mediadelivered sexual health intervention: a cluster randomized controlled trial

Assessing and addressing cardiovascular health in LGBTQ adults: a scientific statement from the American Heart Association

Recalling, sharing and participating in a social media intervention promoting HIV testing: a longitudinal analysis of HIV testing among MSM in China

Sexual minorities: the terminology

The interplay between online and offline explorations of identity, relationships, and sex: a mixedmethods study with LGBT youth

How parents and their children used social media and technology at the beginning of the COVID-19 pandemic and associations with anxiety

CONSORT 2010 statement: extension to randomised pilot and feasibility trials

Identifying undiagnosed HIV in men who have sex with men (MSM) by offering HIV home sampling via online gay social media: a service evaluation

For better or for worse? A systematic review of the evidence on social media use and depression among lesbian, gay, and bisexual minorities

Achieving integration in mixed methods designs -principles and practices

Effectiveness, acceptability, and feasibility of digital health interventions for LGBTIQ+ young people: systematic review

Health care disparities among lesbian, gay, bisexual, and transgender youth: a literature review

The Internet's multiple roles in facilitating the sexual orientation identity development of gay and bisexual male adolescents

Feasibility, acceptability and preliminary efficacy of an online peer-to-peer social support ART adherence intervention

Using Grindr, a smartphone social networking application, to increase HIV self-testing among Black and Latino men who have sex with men in Los Angeles

Correlates of loneliness in midlife and older gay and bisexual men

Current cigarette smoking among adults -United States

Older adults' activities on Facebook: can affordances predict intrinsic motivation and well-being?

Users of the world, unite! The challenges and opportunities of social media

Health and access to care and coverage for lesbian, gay, bisexual, and transgender (LGBT) individuals in the

Reaching suburban men who have sex with men for STD and HIV services through online social networking outreach: a public health approach

Feasibility, acceptability, and preliminary efficacy of a live-chat social media intervention to reduce HIV risk among young men who have sex with men

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration

Suicidality and depression disparities between sexual minority and heterosexual youth: a meta-analytic review

Interrater reliability: the kappa statistic

Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: conceptual issues and research evidence

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement

Conversational UX Design: An Introduction

Transformation of adolescent peer relations in the social media context: part 2-application to peer group processes and future directions for research

Changes in internet use when coping with stress: older adults during the COVID-19 pandemic

Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers

A call for a science of engagement: comment on Rus and Cameron

What type of engagement predicts success in a Facebook weight loss group

A call for a public health agenda for social media research

An internetbased, peer-delivered messaging intervention for HIV testing and condom use among men who have sex with men in India (CHALO!): pilot randomized comparative trial

Queer as Fk: reaching and engaging gay men in sexual health promotion through social networking sites

Social media usage in the U.S. in 2019|Pew Research Center

Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis

Health and health care of rural sexual and gender minorities: a systematic review

Barriers to healthcare for transgender individuals

CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials

A national study of social media, television, radio, and internet usage of adults by sexual orientation and smoking status: implications for campaign design

Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies

Acceptability and feasibility of using established geosocial and sexual networking mobile applications to promote HIV and STD testing among men who have sex with men

Preliminary impact of the weCare social media intervention to support health for Young men who have sex with men and transgender women with HIV

Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials

Nuts and bolts of conducting feasibility studies

Sexual orientation and health: comparisons in the Women's health initiative sample

Development and acceptability testing of a Facebook smoking cessation intervention for sexual and gender minority young adults

Smoking cessation intervention trial outcomes for sexual and gender minority young adults

More frequent Internet use during the COVID-19 pandemic associates with enhanced quality of life and lower depression scores in middle-aged and older adults

Social media and obesity in adults: a review of recent research and future directions

Using Facebook as a platform to direct young Black men who have sex with men to a video-based HIV testing intervention: a feasibility study

Interactive social media interventions to promote health equity: an overview of reviews

Unequal treatment: mental health care for sexual and gender minority groups in a rural state

Understanding and promoting effective engagement with digital behavior change interventions

Online social networking for HIV education and prevention: a mixed-methods analysis

Effects of a mobile health intervention to promote HIV self-testing with MSM in China: a randomized controlled trial