key: cord-0908828-wxdkao69 authors: Khatri, Priyanka; Singh, Shweta R; Belani, Neeta Kesu; Yeong, Yin Leng; Lohan, Rahul; Lim, Yee Wei; Teo, Winnie ZY title: YouTube as source of information on 2019 novel coronavirus outbreak: a cross sectional study of English and Mandarin content date: 2020-03-20 journal: Travel Med Infect Dis DOI: 10.1016/j.tmaid.2020.101636 sha: 16791519c4bf0f3fccd1844ba42c57b9e720327c doc_id: 908828 cord_uid: wxdkao69 BACKGROUND: The current 2019 novel coronavirus outbreak is rapidly evolving. YouTube has been recognized as a popular source of information in previous disease outbreaks. We analyzed the content on YouTube about n-CoV in English and Mandarin languages. METHODS: YouTube was searched using the terms ‘2019 novel coronavirus’, ‘Wuhan virus’ and ‘武汉病毒’ (Mandarin for Wuhan virus) on 1st and 2nd February 2020. First 50 videos in each group were analyzed. Videos in other languages, duplicate videos, those without an audio and duration >15 min were excluded .72 videos in English and 42 in Mandarin were reviewed. 2 reviewers classified the videos as useful, misleading or news based on pre specified criterion. Inter-observer agreement was evaluated with kappa coefficient. Modified DISCERN index for reliability and medical information and content index (MICI) score were used for content analysis. RESULTS: These videos attracted cumulative 21,288,856 views. 67% of English and 50% Mandarin videos had useful information. The viewership of misleading Mandarin videos was higher than the useful ones. WHO accounted for only 4% of useful videos. Mean DISCERN score for reliability was 3.12/5 and 3.25/5 for English and Mandarin videos respectively. Mean cumulative MICI score of useful videos was low (6.71/25 for English and 6.28/25 for Mandarin). CONCLUSIONS: YouTube viewership during 2019 n-CoV outbreak is higher than previous outbreaks. The medical content of videos is suboptimal International health agencies are underrepresented. Given its popularity, YouTube should be considered as important platform for information dissemination. In December 2019, a series of pneumonia cases of unknown cause were reported in Wuhan city in Hubei Province of China [1] . A novel strain of coronavirus was isolated from the affected patients on 7th Jan 2020 and the number of confirmed cases rapidly soared to 581 within 2 weeks [2] . The cases spread beyond Wuhan city and on 30th Jan 2020, World Health Organization (WHO) declared the 2019 novel coronavirus (2019-nCoV) outbreak as "public health emergency of international concern "(PHEIC). As per WHO situation report of 6th February 2020, 28276 cases had been confirmed globally, of these 216 were outside China in 24 countries worldwide [3] . In our current digital world, online platforms are perhaps the most accessible source of health related information for the public [4, 5] . YouTube is a popular video sharing website with an estimated monthly viewership of 1 billion [6] and serves as an important source of health care related information. During the Ebola outbreak in 2014, relevant You tube videos were watched millions of times [7] . Similar viewership was reported during the 2016 Zika virus epidemic [8] . Such formidable viewership makes YouTube a double-edged sword in times of disease outbreaks. While appropriate YouTube content may benefit the government agencies and health care organizations in allaying public anxiety and enforcing measures to control the spread of disease, dissemination of wrong information can lead to public paranoia and https://doi.org/10.1016/j.tmaid.2020.101636 Received 13 February 2020; Accepted 19 March 2020 failure to contain the infection [9] . The quality of information on YouTube has been evaluated during previous epidemics and endemics. Up to 20% of videos were reported to have misleading information about various aspects of disease during these outbreaks [8, 10, 11] We are not aware of any studies evaluating the role of YouTube as a source of information on 2019 -nCoV. The objectives of our study were (i) to analyze the quality of the information available on YouTube about current nCoV outbreak (ii) to compare the English language content with the information available in Chinese Mandarin henceforth mentioned as Mandarin. It is the most widely spoken language in Asia and official language of China. The current n-CoV epidemic has affected the Asia Pacific region more than the West. We think that it is therefore important to review the available content in Mandarin. YouTube was accessed on 1st and 2 nd February 2020 from Singapore using the search terms '2019 novel coronavirus' and 'Wuhan virus '. Search term '2019 novel coronavirus' yielded 16,4000,000 results while the term 'Wuhan virus' showed 20,800,000 results. The search term used in Mandarin '武汉病毒' is a translation for 'Wuhan virus' and showed 58,100,000 results. We included first 50 videos from each search term for further analysis. This screening strategy is based on previous studies indicating that the users do not go beyond the first few pages of results from a search engine [12] [13] [14] . The default YouTube algorithm of decreasing order of relevance was used for sorting the videos. These 150 videos were saved in a playlist for further analysis because the search results in YouTube can change on a day to day basis [7] . Uniform resource locators (URLs) of these videos were saved separately as back up. Our search methodology is in alignment with previous studies on YouTube content [15] [16] [17] . Of the 150 videos screened, 72 videos in English and 42 videos in Mandarin were included in further analysis ( Fig. 1A and B ). Exclusion criterion were: videos in language other than English or Mandarin, absence of audio, videos lacking information on the 2019 novel coronavirus and duration more than 15 min. An analysis of 250 top You-Tube channels concluded that the optimal length for a video is 10-16 min [18] . Videos with longer duration do not receive much viewership. Hence we excluded those videos from our study. The videos with subtitles were included in the group of the subtitle language. All videos were reviewed and analyzed by 2 independent reviewers (WZT and YYL). Both WZT and YYL have received formal education in English and Chinese and are fluent in both. WZT is a haematologist and YYL is a medicine trainee. Any discrepancies were resolved by a third reviewer NKB who is proficient in both English and spoken Mandarin. As this study required analysis of publicly available information, Institution Review Board approval was not required. Videos were classified as useful if they contained scientifically correct information about any aspect of the disease (symptoms, treatment, prevention of disease, or measures being taken to combat the disease). The videos that contained scientifically unproven (e.g., referring to nCoV as a manmade conspiracy or government propaganda) information were classified as misleading. Videos uploaded by news agencies with information on current status of the disease in terms of mortality and extent of spread only, without any information on prevention, treatment, or measures to combat the disease were classified as news update. This classification has been used in evaluation of information during previous epidemics [10, 19, 20] . We used modified DISCERN score to assess the reliability of the videos [15] . This score allocates one point each for clarity, reliable source of information, lack of bias, reference supplementation and mention of uncertainty (Table 1) . A maximum score of 5 indicates highest reliability for a particular video. For evaluating the content, we used medical information and content index (MICI) as previously defined by Nagpal et al. during the Ebola epidemic [7] . This scale uses a 5 point Likert scale to assess five components of medical information included in the videos: prevalence, transmission, clinical symptoms, screening/testing, and treatment/outcomes of the disease. The study team devised set of criterion to be used for grading each component of MICI score on a scale of 1-5. For example a video would be scored 5 for the treatment/outcome component if it mentions all of the following: the symptoms can be self-resolving but some cases can become dangerously ill requiring hospitalization or intensive care unit, disease can lead to death, treatment is mainly supportive, vaccinations not yet available. The full list of criterion is available as Supplementary Table 1 . The kappa coefficient of agreement was used to determine degree of agreement between the two researchers. Source of videos was categorized into one of five groups: academic institutions or hospitals, news agencies, government agencies like regional health departments, WHO or independent users. The authors are aware of the uncertainties regarding this disease. For example, there is still no consensus on the source of disease outbreak or the transmissibility of the virus, exact incubation period of the virus, and the proportion of infected people who develop disease. Before grading the videos, available scientific literature [21] [22] [23] and relevant information on WHO website was reviewed. Version R 3.5.3 was used for analysis. Unpaired Student t-test was used to test the hypothesis of difference in means for all variables. A pvalue of 0.05 was considered to be significant. Given the small sample size, univariate logistic regression was used to find the odds of posting a useful video based on source. 72 videos in English and 42 videos in Mandarin were included for analysis after applying exclusion criterion (Fig. 1) . The total duration of all videos was 8.45 h. These 114 videos attracted a cumulative number of 21,288,856 views and 259108 likes. The kappa coefficient of agreement regarding the usefulness of the videos was 0.72 (P value < 0.001). 60% videos were classified as useful (n = 68), 8% misleading (n = 9) and 32% as news (n = 37). Baseline characteristics of the videos are summarised in Table 2 . The majority of videos were uploaded by news agencies (76%, n = 86) followed by independent users (11%, n = 12). WHO contributed 3 videos (Table 4 ). One of these videos was in English language with Mandarin subtitles and had 29466 views. There were no videos by CDC. No videos were shared by academic institutions in Mandarin. Independent users were 7 times less likely to upload useful information than academic institutions (Table 5) . Despite being of similar length, misleading videos in English attracted significantly lesser number of views and likes than useful videos. Conversely, misleading videos in Mandarin were of longer duration than useful videos and attracted more number of views. This difference was however not statistically significant.The statements made in misleading videos are available as Supplementary Table 2 . More than 70% of useful videos in English contained information about the prevalence, transmission and treatment/outcomes of nCoV disease. Almost half of the useful videos did not address transmission or clinical symptoms ( YouTube viewership of content related to the 2019 n-CoV epidemic appears higher than previous disease outbreaks [10, 20] . This is evident by the much higher number of mean views/day of the useful videos in our study. This can be partly explained by the fact that we performed this analysis early in the course of disease evolution. The mean number of days since upload on YouTube for useful videos was 6 in our study compared to 61 days as reported by Pathak et al. during the 2014 Ebola virus disease outbreak. It has been reported that the viewership of videos tends to decrease over time and that could lead to higher number of views/day in the initial days of video upload. Regardless, the difference is substantial and high viewership of content in our study should be considered as a reflection of growing popularity of YouTube for health related information. The higher number of views can also be explained by the global spread of this disease compared to previous outbreaks like Ebola which was mainly restricted to Africa. The current content on YouTube in English is mostly informative. There were only 2 misleading videos amongst the included 72 videos. These results are reassuring. Previous studies have reported that about 15-30% of information on YouTube about a disease can be misleading [24] [25] [26] . The lower percentage of misleading videos in our study can be explained by the difference in source of these videos compared to previous studies. Individual users accounted for only 11% of total videos while news agencies contributed 68%. This is not surprising given the global nature and rapid spread of this outbreak. This has attracted attention of news agencies worldwide. Videos uploaded by independent users are more likely to contain misleading information than government/news agency videos [8] . The modified DISCERN score of videos which were characterized as useful is of concern too. There was only 1 video with a perfect DISCERN score of 5 indicating highest reliability. 20% of the useful videos had DISCERN score of less than 3 indicating low reliability. Most of the videos did not mention areas of uncertainty or additional sources of patient information. International health agencies like WHO only had 2 videos while CDC had none in the top 100 English videos related to this disease. Under representation of international health agencies on YouTube has previously been reported [8] . Previous studies have focussed on analysis of YouTube videos only in English. YouTube is gaining popularity for its content in non-English languages too. In the first week of 2019, a total of 243,254 videos were posted on YouTube, out of these 67% were in languages other than English [27] . This prompted us to evaluate the information available on YouTube about nCoV in Mandarin and compare it to the content in English. The mean DISCERN scores were similar in 2 groups indicating comparable reliability. Similarly, comparable but very low mean total MICI scores were noted in both groups, suggesting overall paucity of good quality scientific information on n-CoV outbreak. The information pertaining to screening and testing was worst with mean MICI scores of 0.95 and 0.31 in English and Mandarin groups respectively. Under representation of government agencies was noted in Mandarin videos as well. News updates constituted most of the Mandarin videos too, however compared to English new agencies, the narrators of news clips in Mandarin often commented on how Chinese government could have acted differently to contain this epidemic early. No videos were shared by academic institutions and hospitals in Mandarin which contributed to slightly higher percentage of misleading videos in Mandarin. Contrary to the English videos, the misleading videos in Mandarin were more popular than useful videos, as evidenced by higher mean number of likes. The misleading information in native language at epicentre of outbreak might have led to undue public distress during early phase of outbreak. YouTube thus should play a more active role in Our study has many limitations. This study presents only a cross sectional snapshot of the available information. The viewership and content on YouTube change on a daily basis. We are unable to comment on how the viewership and content regarding 2019 nCoV will change as the outbreak evolves. We did not analyze other video sharing platforms such as health sharing websites or news channel websites. In addition, YouTube is not accessible from China. Instead, a platform called Youku is used for video sharing. This affects the applicability of results in China which is the epicentre of this disease outbreak. Restriction of YouTube in China also explains the relative lesser number of views of Mandarin language videos compared to those in English. Nonetheless, Mandarin remains the widest spoken language in Asia Pacific region and videos in this language serve as an important source of information to native Mandarin speakers in this region. Another limitation of our study is the possibility of inter and intra observer bias though we used the kappa coefficient to ensure that the results were not too different. Our search was limited to first 50 videos for each keyword. It has previously been explained in multiple studies that most Internet users do not look beyond the first 50 search results [16, 28, 29] For the search In conclusion, the available information on YouTube about 2019 n-CoV outbreak attracted high viewership. This clearly establishes YouTube as a popular platform for information seeking in the current n-CoV epidemic. As the current outbreak evolves, international health agencies and academic institutions should release more videos with information about screening and testing of this disease. YouTube should consider screening and removing videos with misleading information to prevent public paranoia in current state of emergency. No funding was required to conduct this study. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Table 4 Source wise distribution of different video types. Situation report 9th Situation report 17th The internet as a source of health information and services Use of the internet as a health information resource among French young adults: results from a nationally representative survey YouTube videos as a source of medical information during the Ebola hemorrhagic fever epidemic Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic Panic, paranoia, and public health -the AIDS epidemic's lessons for Ebola YouTube as a source of information on the H1N1 influenza pandemic Analysis of YouTube as a source of information for west nile virus infection Are video sharing web sites a useful source of information on hypertension? Is YouTube useful as a source of information for Sjogren's syndrome? YouTube for information on rheumatoid arthritis -a wakeup call Youtube as source of patient information on abdominal aortic aneuryms A Quality analysis of disc herniation videos on YouTube YouTube as an information source for femoroacetabular impingement: a systematic review of video content Is YouTube an accurate and reliable source of yellow fever information during outbreaks YouTube as a source of information on Ebola virus disease Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia Clinical features of patients infected with 2019 novel coronavirus in Wuhan A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster Is YouTube useful as a source of information for Sjogren's syndrome? YouTube for information on rheumatoid arthritis -a wakeup call Are video sharing web sites a useful source of information on hypertension? Popular YouTube channels produced a vast amount of content, much of it in languages other than English YouTube TM as an information resource for orthognathic surgery Lingual orthodontic treatment: a YouTube TM video analysis Travel Medicine and Infectious Disease xxx (xxxx) xxxx Supplementary data related to this article can be found at https:// doi.org/10.1016/j.tmaid.2020.101636.