key: cord-0859949-qfuq7oek authors: Salah, Louai A; AlTalhab, Saad; Omair, Aamir; AlJasser, Mohammed title: Accuracy and Quality of YouTube Videos as a Source of Information on Vitiligo date: 2022-01-07 journal: Clin Cosmet Investig Dermatol DOI: 10.2147/ccid.s330015 sha: 31295907ebb43dd8fe6e58d563a37bd66face9f0 doc_id: 859949 cord_uid: qfuq7oek PURPOSE: Vitiligo is a skin condition characterized by patchy depigmentation in the skin, affecting not just the physical well-being of patients, but also the mental, emotional, and social aspects of their lives. Social media provides a platform where patients can interact and share experiences, and for physicians and experts to disseminate knowledge on the disease. This study aimed to evaluate the accuracy and overall quality of vitiligo-related content on YouTube. METHODS: Thirty videos were screened by relevance according to the inclusion criteria. Videos were categorized as either healthcare or non-healthcare sources, and viewer engagement ratio was calculated for each. Three independent vitiligo experts rated the content using the following assessment tools: DISCERN, Accuracy in Digital-health Instrument (ANDI), and Global Quality Scale (GQS). Lastly, Cronbach’s alpha was used to estimate the inter-rater reliability and consistency. RESULTS: Most of the screened videos came from health-care sources (57%). Non-healthcare sources achieved a slightly higher viewer engagement ratio, although the difference is not statistically significant (p = 0.23). DISCERN score was low overall in most videos. However, DISCERN score was higher for health-care sources as compared to non-healthcare sources (30.5 ± 9.4 vs 22.7 ± 4,2, p = 0.009). Scores of ANDI and GQS were higher for health-care sources but not statistically significant. There was a good internal consistency in DISCERN rating among the evaluators (0.86). CONCLUSION: Our study shows a low overall accuracy and quality of YouTube videos on vitiligo. It is recommended that health-care sources improve their information material in terms of quality and viewer engagement, as very little can be done to prevent non-healthcare sources in publishing their own videos. Efforts in educating the public on making distinction between evidence and non-evidence-based claims should also be taken. Vitiligo is a long-term skin condition characterized by patches of discoloration in the skin, affecting as much as 0.5-2% of the global population. 1 Pathophysiology of the disease involves mainly autoimmune destruction of melanocytes. 2 Moreover, a strong association between vitiligo and other autoimmune diseases has been well established including Hashimoto thyroiditis, graves disease, pernicious anemia, inflammatory bowel disease and more. 3, 4 Symptoms of vitiligo are usually considered harmless and are cosmetic in nature, which may have contributed to the underestimation of the importance of treatment. 5 Apart from the visual manifestation of the disease, vitiligo also impacts the mental health and health-related quality of life of the patients, particularly those with dark skin. 5 Studies have demonstrated that vitiligo patients with worse quality of life tend to seek belongingness through support groups for their social, protective effect. 6,7 Social media usage dramatically increased over the previous decades; as much as 86% of adults between the ages of 18-29 years old in the United States use social media. 8 This increase in usage also altered the way information is circulated; some platforms including YouTube have been used to deliver and share health-related information among groups of people sharing the same medical condition. 9, 10 There have been studies which assessed dermatology-related content on different social media platforms, citing misinformation as one of the primary challenges in these setups. 11-13 Few studies have documented vitiligo-related content in the social media platform YouTube, which highlighted the same problems of misinformation, but also provided insights on patient perspectives and psychosocial support. 9, 14 One of these studies showed that alternative/ complementary medicine was discussed in YouTube videos more than traditional medical treatment (44% vs 31%). 9 One of the many ways by which consumer health information is often evaluated is through a 5-point scale called the Global Quality Scale (GQS). The GQS is usually accompanied with criteria questions to determine the usefulness and quality of the information presented to the reviewer. 15 To further assess the quality of the content, a tool called DISCERN is employed. DISCERN is designed to estimate the quality of information and set standards with regard to the production of high-quality consumer health information. 16 Lastly, to gauge the accuracy (or lack thereof) of the videos, the validated assessment tool Accuracy in Digital-Health Instrument (ANDI) was used. 17 As of writing, there are no studies which integrated DISCERN, GQS, and ANDI in their crosssectional analysis of the quality and accuracy of YouTube videos as a source of information for vitiligo. Thus, this study aims to evaluate vitiligo-related content in the social media platform YouTube using statistical tools, GQS, ANDI and DISCERN. The social media platform YouTube was accessed last February 19, 2021, using the search parameter "Vitiligo". To minimize the probability of bias attributed to search history, the incognito feature was used. The first 30 results by relevance were selected for subsequent analysis, as typical content consumers do not usually go beyond this point. 18 Only videos in the English language were included in the study. 18 Additionally, videos longer than twenty (20) minutes were excluded as they are not representative of the preferences of an average viewer. 19 The uploading channel and the number of likes, dislikes, and comments were recorded for each video. Viewer's engagement ratio was calculated by dividing the total number of likes, dislikes, and comments by the number of views. Ethical approval was waived by the Research and Studies Department in Jeddah Health Affairs since the study involved data available in the public domain. Three independent vitiligo experts evaluated the quality of each video using the Global Quality Scale and DISCERN. Additionally, the accuracy of each video was assessed using the validated tool Accuracy in Digital-health Instrument (ANDI). Criteria questions and scales are enumerated in Appendix 1. Inter-rater reliability was measured using Cronbach's alpha. Raw data were encoded in Microsoft Excel TM and exported to SPSS v24.0 (IBM Corp, Armonk, NY) for analysis. Descriptive statistics analyses were performed for both categorical and numerical variables. Mean scores from the different tools for assessment were compared between healthcare and non-healthcare sources using an independent t-test. A p-value ≤0.05 was used as a basis for statistically significant difference for all statistical tests. Thirty videos were screened according to the inclusion criteria. Videos uploaded by a hospital, or a physician constituted 40%, and those considered as educational channels were around 17% of the total videos. Both were categorized as health-care sources while the rest were regarded as non-healthcare sources. These included Three independent vitiligo experts rated the quality and accuracy of the YouTube videos using DISCERN, GQS, and ANDI. There was a significant difference between healthcare and non-healthcare sources through the DISCERN assessment tool (Figure 1 ). The scores for ANDI and GQS were slightly higher for health-care sources but the difference was not statistically significantly. A breakdown of Discern score of all videos showed very poor score in more than half of the videos (57%), poor in 33% and fair in 10%. None of the videos had very good or excellent score ( Figure 2 ). The viewer engagement ratio was calculated to be 0.016 for health-care sources and 0.019 for non-healthcare sources, albeit the difference is not statistically significant (p = 0.23). The inter-rater reliability was estimated using the Cronbach's alpha which showed a good intra-class correlation for DISCERN (0.86), acceptable for ANDI (0.76) and unacceptable for GQS (0.41). Impairment of the quality of life of vitiligo patients has been documented as early as the 1970s. 20 It is known that a range of psychological outcomes may arise from this circumstance, ranging from depression to anxiety, as a consequence of low self-esteem and social isolation. 20 With the onset of social media platforms like YouTube, patients suffering from the same condition are able to share their perspectives and experiences, and health-care sources can disseminate knowledge about a certain disease to remedy some of these psychological outcomes. Thus, it is important to evaluate the accuracy and overall quality of the content of the information being consumed through YouTube videos. With the paucity of studies on the accuracy, reliability, and quality of vitiligo-related content in YouTube, this study aims to evaluate these contents with the aid of statistics and assessment tools such as the GQS, ANDI and DISCERN. The majority (57%) of videos came from health-care sources, while non-healthcare sources comprised only 43%. However, the viewer engagement ratio was higher in non-healthcare sources (0.019) than its counterpart (0.016), although the difference is not statistically significant. This is consistent with the findings of a recent study, citing viewer experience and visibility as the areas in need of further improvement for videos produced as health-care sources. 14 Another study hypothesized that the storytelling and anecdotal pattern of delivering information by nonhealthcare sources are more emotionally engaging than statistics, numbers, and general descriptions typical of the content from health-care sources. 17 In terms of the content-assessment tools, a statistically significant difference in quality through DISCERN was observed between healthcare and non-healthcare sources (P = 0.009) although the overall quality was still considered to be poor to fair. There was a good overall agreement on that between evaluators (0.86). Moreover, GQS and ANDI also revealed a higher overall quality and accuracy for health-care sources than non-healthcare sources, although the difference was not statistically significant. This is consistent with the findings of a recent study which showed a higher ANDI score for health-care sources compared to non-healthcare sources. 14 Higher GQS scores have also been observed in health-care sources in similar studies conducted on rosacea-related content. 17 Different assessment tools may overlap and may interpret the same properties differently; this implies that DISCERN can potentially capture information missed by GQS, or vice-versa. 21 Ultimately, this highlights the importance of cross-analyses utilizing multiple assessment tools to gain a holistic evaluation of the contents of the YouTube videos. Limitations of the study include the sample size, different search results, discordance between raters in ANDI ratings and the possible associations between the engagements and the content. Although this study only sampled 30 videos, previous studies have demonstrated that a minimum of 25-50 videos are adequate in making a distinction of clinically relevant differences. 17 Using the YouTube search tool may generate different search results per search, which is one of the reasons this study employed inclusion/exclusion criteria to accommodate this possibility. Lastly, this study did not evaluate possible associations between comments, likes, dislikes, and views, with the content of the videos, as it is not included in the main objective. However, further studies may be conducted to analyze these associations. The burden associated with vitiligo symptoms prompts patients to seek information wherever it is available, including social media platforms such as YouTube. This study showed that while most of the videos came from health-care sources, a higher viewer engagement is garnered by non-healthcare sources. Although only statistically significant for Discern, ratings from three independent vitiligo experts were generally higher for DISCERN, ANDI, and GQS scores of hospital-physician produced videos compared to non-healthcare sources. However, the accuracy and quality of these videos is generally low. This highlights the prevalence of healthrelated misinformation in social media. While very little can be done to prevent non-healthcare sources to produce engaging yet inaccurate videos, future efforts by healthcare facilities and experts should consider improving quality and viewer engagement when producing their own information materials. It is also recommended to educate the public on making distinctions between evidence and non-evidence-based health information on the internet. Pharmacokinetic and pharmacodynamic effects of YM087, a combined V1/V2 vasopressin receptor antagonist in normal subjects Long-term treatment of patients with inappropriate secretion of antidiuretic hormone by the vasopressin receptor antagonist conivaptan, urea, or furosemide Vitiligo and autoimmune thyroid disorders The analysis of genetics and associated autoimmune diseases in Chinese vitiligo patients Hyponatremia and hypernatremia Short-term clinical effects of tolvaptan, an oral vasopressin antagonist, in patients hospitalized for heart failure: the EVEREST clinical status trials A review of the worldwide prevalence of vitiligo in children/adolescents and adults Vitiligo on YouTube: a cross-sectional analysis A cross-sectional study of YouTube videos as a source of patient information about phototherapy and excimer laser for psoriasis Engaging but inaccurate: a cross-sectional analysis of acne videos on social media from non-health care sources DISCERN scores of YouTube information on eczema treatments Social media use in dermatology Vitiligo health education: a study of accuracy and engagement of online educational materials Is YouTube useful as a source of information for Sjögren's syndrome? DISCERN: an instrument for judging the quality of written consumer health information on treatment choices Rosacea videos on social media: a comparison of accuracy, quality, and viewer engagement YouTube as a source of patient information for Coronavirus Disease (COVID-19): a content-quality and audience engagement analysis YouTube: searching for answers about breast cancer Vitiligo: a review Evaluation of YouTube videos about smile design using the DISCERN tool and Journal of the American Medical Association benchmarks This work has not been published previously (except in the form of an abstract, a published lecture, or academic thesis, that is not under consideration for publication elsewhere), and that its publication is approved by all authors and tacitly or explicitly by the responsible authorities where the work was carried out. Moreover, this work will not be published elsewhere in the same form, in English or in any other language, including electronically, without the written consent of the copyright holder. The authors report no conflicts of interest in this work. Clinical, Cosmetic and Investigational Dermatology is an international, peer-reviewed, open access, online journal that focuses on the latest clinical and experimental research in all aspects of skin disease and cosmetic interventions. This journal is indexed on CAS.The manuscript management system is completely online and includes a very quick and fair peer-review system, which is all easy to use. Visit http://www.dovepress.com/testimonials.php to read real quotes from published authors.