key: cord-0016938-cks88vc0
authors: Cakmak, Guner; Mantoglu, Baris
title: Reliability and Quality of YouTube Contents Pertaining to Pancreatic Cancer
date: 2021-03-24
journal: nan
DOI: 10.7759/cureus.14085
sha: 03b23a9d6757863e94171c930d9b8ad744dedccb
doc_id: 16938
cord_uid: cks88vc0

Objective: In our study, we aim to evaluate in terms of patients the quality and reliability of the most relevant and most-watched videos uploaded on YouTube about pancreatic cancer. Method: Before starting the study, YouTube(TM) search terms were determined by consensus by two General Surgeons. Then, on 10/10/2020, the terms such as “pancreatic cancer”, “diagnosis of pancreatic cancer” and “treatment of pancreatic cancer” were entered separately in the search bar of YouTube, “relevance” was selected among the filtering options and the most viewed videos were listed. The videos were evaluated with the Global Quality Scale (GQS), the DISCERN scoring system (Quality Criteria for Consumer Health Information, http://www.discern.org.uk), and video power index. Results: Among the 50 videos analysed, 19 videos were uploaded by hospital channels, 17 videos by health channels, seven videos by patients, four videos by blog channels, and three videos by doctors. The mean GQS score of the first researcher was 3.24 ± 0.99 and the mean GQS score of the second researcher was 3.18 ± 0.88 with a significantly high agreement between them (r= 0.628). The mean DISCERN score of the first researcher was 3.48 ± 0.77 and the mean DISCERN score of the second researcher was 3.46 ± 1.09 with a significantly high agreement between them (r= 0.814). Conclusion: In our study, the majority of the videos were found to be of moderate quality. Healthcare professionals should be encouraged to upload more videos with useful content. However, we think that the uploaded videos should definitely go through a professional peer-review process before they are published.

With more than 50,000 new cases and over 90% mortality reported each year in the United States, pancreatic cancer (PC) is one of the deadliest malignancies [1, 2] . PC can be difficult to diagnose. Because of its nonspecific and atypical symptoms, PC is often diagnosed at advanced stages [3] [4] [5] [6] . Survival varies depending on the stage at which the PC is diagnosed. If the tumor has metastasized to distant parts of the body, survival rate is only 3% [7] .

Today, patients and their relatives are trying to obtain information from various sources on PC, which is difficult to detect and treat and has a poor prognosis. The Health Information National Trends Survey revealed that nearly half of people searching for cancer information turn to the internet first, rather than seeking medical advice [8, 9] .

Today, the internet and, in particular, platforms such as YouTube (www.youtube.com) where visual content is abundantly available, are the most important and widely used source of information for people of all ages worldwide. According to 2020 statistics, there are two billion YouTube users worldwide; one billion hours of videos are watched on YouTube every day and 400 hours of video are uploaded to YouTube every minute worldwide [10] . Some studies have found that patients report that the information they obtain over the internet is at least as reliable as that given to them by doctors, or even more so [11] .

In the past, various studies have been carried out to investigate the quality and reliability of video contents related to health issues uploaded to YouTube and whether they are beneficial for patients [12, 13] . However, we did not find any study on PC in our literature search.

Therefore, in this study, we aim to evaluate in terms of patients the quality and reliability of the most relevant and most-watched videos about PC uploaded to YouTube.

Before the study began, YouTube search terms were determined by consensus by two general surgeons. Then, on 10/10/2020, the terms "pancreatic cancer," "diagnosis of pancreatic cancer," and "treatment of pancreatic cancer" were entered separately in the search bar of YouTube. "Relevance" was selected among the filtering options and the most viewed videos were listed. After excluding news, repeated videos, videos in languages other than English, and those that were less than 120 seconds or contained advertising content, the top 50 most relevant and most-watched videos were recorded. In addition, the qualification of the uploader, video content, video length (seconds), upload date, time elapsed since upload, number of views per day, number of likes and dislikes, number of comments, and video power index (VPI) were also recorded. The quality of the videos was evaluated by VPI values calculated according to the formula: VPI = like count/(like count + dislike count) x 100 [12, 13] .

The collected data were separately evaluated and scored by two general surgeons with 22 and 12 years experience on panceratic cancer, respectively. The videos were evaluated at the same time, but in different locations, in order to prevent bias during the evaluation. We used the Global Quality Scale (GQS) [14, 15] , which has been used in many studies in the literature, in order to determine the quality of the videos. The five categories of the GQS are: 1-Poor quality, poor flow of the content, most information missing, not at all useful for patients; 2-Generally poor quality and poor flow, some information listed, but many important topics missing, of very limited use to patients; 3-Moderate quality, suboptimal flow, some important information is adequately discussed, but other information is poorly discussed, somewhat useful for patients; 4-Good quality and generally good flow, most of the relevant information is listed, but some topics not covered, useful for patients; and 5-Excellent quality and excellent flow, very useful for patients.

In the 5-point scale of GQS, 1 point is associated with poorly useful, 2 points with poorly useful (limited use), 3 points with somewhat useful, 4 points with useful, and 5 points with highly useful. Each question listed below that is answered with a "yes" corresponds to 1 point and those answered with "no" receive 0 points. A maximum of 5 points can be obtained. High scores indicate the reliability of the video content. The 5-item questionnaire of the DISCERN scale (Quality Criteria for Consumer Health Information, http://www.discern.org.uk) is: 1-Are the aims clear? 2-Are reliable information sources used (e.g. physicians, health channels etc.)? 3-Is the information balanced and unbiased? 4-Are additional sources listed for patients? 5-Are areas of uncertainty stated?

As a result of the researchers' evaluations, 1 or 2 points obtained from the GQS and DISCERN scales indicate that the video content is misleading, 3 points indicate that the video content is of moderate quality and reliability, and 4 or 5 points indicate that the video content is useful for patients.

Since our study did not include human or animal subjects and only analyzed videos on the YouTube platform, ethical approval was waived.

Data were analyzed using Statistical Package for Social Sciences (SPSS) version 23 (IBM Corp., Armonk, NY, USA). During the evaluation of the study data, frequencies (number, percentage) of categorical variables and descriptive statistics (mean, standard deviation, median, minimum, and maximum) of numerical variables were given. Normality assumptions of numerical variables were examined with the Shapiro Wilk normality test and it was found that they were not normally distributed. Therefore, nonparametric statistical methods were used in the study. The relationship between two independent numerical variables was interpreted with Spearman's Rho correlation coefficient. Differences between two independent groups were examined by Mann-Whitney U analysis. The correlation between two rates was examined by Spearman's correlation analysis and the agreement between them was evaluated using alpha coefficients. P<0.05 values were considered statistically significant.

A total of 50 of the most relevant and watched videos about PC were included in the scope of the study. The average video length, total number of views, time elapsed since the upload of the video, daily number of views, average comments, likes, dislikes, and VPI values are given in Table 1 . The distribution of the videos by the video content and the qualifications of uploaders are given in Table 3 . 

When Table 5 is examined, it is determined that only seven of the 50 videos were uploaded by patients, but the highest rate of comments and likes is recorded in these videos. The most-watched video in our study was uploaded by a patient in 2019, and this video was viewed 5,251,574 times. The least viewed video was uploaded by a hospital channel in 2020 and was viewed 3,735 times.

The mean DISCERN score given by the two researchers to the videos analyzed was 3.47 ± 0.99 (min-max: 1-5), and we found that the reliability of the videos was of moderate quality. The mean DISCERN score given by the first researcher to the videos was 3.48 ± 0.77 (min-max: 2-5), and the mean DISCERN score of the second researcher was 3.46 ± 1.09 (min-max: 1-5). There was a high inter-rater reliability in terms of the DISCERN scores (r=0.814).

The mean GQS score given by the researchers to the videos was 3.21 ± 0.94 (min-max: 1-5). The mean GQS score of the first researcher was 3.24 ± 0.99 and that of the second researcher was 3.18 ± 0.88 (min-max: 1-5).

There was a high inter-rater reliability in terms of the GQS scores (r=0.628). The distribution of DISCERN and GQS scores by the qualifications of the uploaders is given in Table 6 The mean DISCERN score of the animated videos was 3.8 ± 1.13 and the mean GQS score was 3.3 ± 1.11. The mean DISCERN score of the real image videos was 3.41 ± 1.70 and the mean GQS score was 3.25 ± 1.04. According to this, it was determined that the quality and reliability of videos with both real and animated content were moderate.

Seven (14%) of the 50 videos contain misleading information, and we found that these videos were uploaded by patients and blog channels. These videos were viewed more than 6 million times. The oldest video was uploaded in 2014 and the newest one in 2019.

Eighteen (36%) of the videos contain useful information. Two of these were uploaded by doctors, and 16 by health and hospital channels.

It was found that the remaining 25 (50%) videos were of moderate quality and almost all were uploaded by health and hospital channels.

Studies have reported that eight out of 10 internet users search the internet for their health information [16] . YouTube is the visual media platform that patients and their relatives use the most to get information about their health. YouTube content can provide educational and useful information. However, many studies have shown that health-related videos can contain misleading and harmful information when not uploaded by healthcare professionals [12, 13] . For this reason, the number of studies aiming to assess the quality and reliability of YouTube content on various health issues is increasing day by day in the literature. In a study conducted in 2018, it was reported that there were nearly 2,000 studies in which YouTube videos with health content were examined [17] . Studies have been conducted on glioblastoma, social media addiction, human papillomavirus (HPV) vaccines, premature ejaculation, retinopathy, diabetes mellitus, and schizophrenia, and in almost every field of medicine [18] . Although there are studies that have analyzed YouTube videos on many types of cancers [12, 19, 20] , no study has been found on PC.

It was determined that 12 of the 36 videos uploaded by health and hospital channels are patient experience videos. It was found that the mean DISCERN score of these videos was 3.75 and the mean GQS score was 3.5. These videos with a moderate quality score were found to have post-surgical survival content. They only provide information about short-term results, and their uploaders are partially profit-oriented organizations. In Yurdaisik's study evaluating breast cancer videos, videos uploaded by healthcare professionals were found to be of medium quality [12] . Previous studies have reported that the quality of these videos is poor or moderate [12, 13] . Accordingly, it is concluded by researchers that health-related videos should only be uploaded by healthcare professionals. However, the rate of videos uploaded directly by doctors in our study is very low (6%), as in previous studies. Although the contents of videos uploaded by health channels and hospital channels are mostly presented by doctors, they are generally promotional corporate videos and do not reflect the doctors' own views as healthcare professionals. There could be various reasons for the low rate of videos uploaded by doctors to YouTube. In recent years, the use of social media with the evolution of smartphones has surpassed video platforms such as YouTube. Patients now follow doctors on social media about their own health problems, benefit from one-to-one sharing, and can even participate interactively by watching their live broadcasts. In these broadcasts, patients can directly ask the questions they are curious about and receive their answers in a descriptive way, benefiting the other participants as well. It can be said that this way of expressing their profession is more attractive to doctors than uploading videos to YouTube in terms of both self-improvement and patient guidance. Another reason that doctors refrain from uploading videos to YouTube is the fear of infringing copyright and privacy [21] . While the responsibility for the contents uploaded by corporate health channels belongs to the relevant institution, the responsibility for the content of videos directly uploaded by doctors belongs to the doctors. This suggests that doctors hesitate to upload videos for professional reasons and, for this reason, the proportion of such videos is low. However, it is not a realistic expectation that every video uploaded by doctors is useful and not misleading. In this regard, it is suggested that videos uploaded to platforms such as YouTube are only published after they are audited by a committee of healthcare professionals to confirm their scientific validity and ensure that they are not misleading. Considering the sensitivity of cancer patients and their relatives, such videos are important. Therefore, although professionals should be encouraged to upload videos about health, we think that these videos should go through a peer review process.

Only seven of the 50 videos analyzed were uploaded by patients and all of these videos contain the experiences of the patients. These seven videos have been viewed more than 6 million times in total. The most viewed video has the most comments and the most likes, with more than 5 million views. Videos uploaded by patients have an average daily view rate of 1.201. Five of these seven videos contain misleading information, and two of them are of moderate quality. In studies conducted on similar topics in the literature, it has been reported that the videos uploaded by patients are viewed more and get more likes [13, [22] [23] [24] . Considering that the diagnosis of PC is usually made at late stages and the surgical chances of these patients are very low, we think that patients and their relatives are in search of different options.

Eight of the videos analyzed have content on surgical technique. These videos are of moderate quality and usefulness and have been viewed 1,134,410 times in total. The VPI values of these videos uploaded by healthcare professionals have been found to be lower than those uploaded by patients and blog channels.

Ten of the videos analyzed are animated videos, and these videos had a lower number of likes than the videos containing real images. Due to the fact that anatomical information is given in animated videos and medical terms are used extensively, we think that these videos are not understood and liked by the audience. However, the VPI values of the animation videos were found to be higher than the VPI values of the videos containing real images.

Fifty videos included in our study have been viewed 12,440,505 times, with an average daily view rate of 280. However, different rates were reported in previous studies where YouTube contents were assessed. In the study conducted by Kunze et al., 50 videos were analyzed and the total number of views was reported as 14,141,285 [25] . However, in the study conducted by Champan et al., 202 videos were analyzed and the total number of views was reported as 2,518,512 [26] .

In this study, where we analyzed videos about PC, we found that the videos are generally of moderate quality in terms of content. We included in the scope of the study the most viewed videos after filtering by relevance which appeared when we typed our pancreas-related keywords into the YouTube search bar. In addition, we also included in our study the first videos that appeared when we entered our keywords.

This study includes considerable limitations on the data examined. First, we could not fully compare the findings of our study with previous studies conducted in this field. However, since there is no other study similar to ours, we compared our data with YouTube studies in different fields. We evaluated all the mostwatched videos related to PC, although such videos were rare. The fact that we analyzed only 50 videos in this study can be seen as a limitation. Finally, since there is no other study in the literature investigating the quality and reliability of YouTube contents on PC, we could not compare our results regarding PC videos.

In our study, the top 50 most relevant and most viewed videos on YouTube on PC, a cancer type that is difficult to treat, has a poor prognosis, and a low survival rate, were analyzed. According to our findings, the rate of videos uploaded directly by doctors is very low. Although the DISCERN and GQS scores of these videos are higher than the scores of videos uploaded by non-healthcare professionals, the number of views, comments, and likes of videos uploaded by patients is higher. In our study, the majority of the videos were found to be of moderate quality. Healthcare professionals should be encouraged to upload more videos with useful content. However, we think that the uploaded videos should go through a professional peer-review process before they are published.

Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue.

In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Jemal A: Cancer statistics

Epidemiology of pancreatic cancer: global trends, etiology and risk factors

Novel diagnostic and predictive biomarkers in pancreatic adenocarcinoma

Pancreatic cancer

Pancreatic cancer: advances in treatment

Coordination Group of The Committee on Pancreatic Cancer: Clinical analysis of 2340 cases of pancreatic cancer

Pancreatic Cancer Survival Rates: What Do They Really Mean

Evaluation of educational content of YouTube videos relating to neurogenic bladder and intermittent catheterization

Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey

YouTube Stats Every Marketer Should Know in 2021

Chavannes NH: Patients' use of the internet to find reliable medical information about minor ailments: vignette-based experimental study

Analysis of the most viewed first 50 videos on YouTube about breast cancer

Evaluation of the quality and reliability of YouTube videos on rotator cuff tears

A qualitative analysis of methotrexate self-injection education videos on YouTube

YouTube for information on rheumatoid arthritis--a wakeup call

Systematic review of internet patient information on colorectal cancer surgery

Medical YouTube videos and methods of evaluation: literature review

Dissemination of misinformative and biased information about prostate cancer on YouTube

YouTube videos related to skin cancer: a missed opportunity for cancer prevention and control

Review on the reliability of medical contents on YouTube . iJOE. 2020

Going viral: doctors must tackle fake news in the covid-19 pandemic

YouTube as a source of information on cervical cancer

Evaluation of YouTube video content related to the management of hypoglycemia

YouTube as a source of information about the posterior cruciate ligament: a content-quality and reliability analysis

Evaluation of online videos of laparoscopic sleeve gastrectomy using the LAP-VEGaS guidelines