key: cord-0903849-4qh46yzj authors: Engelmann, J.; Fischer, C.; Nkenke, E. title: Quality assessment of patient information on orthognathic surgery on the Internet date: 2020-05-28 journal: J Craniomaxillofac Surg DOI: 10.1016/j.jcms.2020.05.004 sha: e25081046472546f501bcc4b9606bfaf8015ef08 doc_id: 903849 cord_uid: 4qh46yzj OBJECTIVE: The aim of this study was to assess the quality of online information for patients on orthognathic surgery. MATERIALS AND METHODS: A selection of search terms specific for orthognathic surgery was chosen and 150 websites were identified using the Internet search engines Google, Yahoo and Bing. Irrelevant websites were excluded. The remaining websites were assessed with a modified Ensuring Quality Information for Patients (EQIP) tool. EQIP evaluates the quality of medical patient information by measuring the three key aspects of content, structure, and identification data. RESULTS: 48 relevant websites were indentified. EQIP values ranged between 2 and 28 (median 13.65). While 37 of the 48 websites described details of the surgical procedures, only 13 mentioned possible risks and complications of the surgery. No differences were found between the websites of private practices, dentists and public hospitals, universities, or others (p =0.66). Websites found by Google had a significantly lower EQIP score compared with Yahoo and Bing (11.12 vs. 16.60 for Yahoo and 16.23 for Bing; p = 0.012). The better the rank of the website, the higher the EQIP score (r = −0.411, p = 0.004). CONCLUSIONS: The results of this study reflected a large variation of quality of information on orthognathic surgery on the Internet. Therefore, surgeons must be aware that they might be confronted with unrealistic expectations of patients, who may underestimate the potential risks and drawbacks of orthognathic surgery. With the world wide web now widely available, the Internet has become the first-choice source of information, with widespread and growing use of written online information, online videos, Internet discussion groups, and blogs. Increasingly, patients use the Internet to obtain answers to medical questions. According to a recent poll, more than 50% of people consult the Internet regularly to get information on health issues (Hambrock, 2018) . It can be assumed that during the COVID-19 pandemic, this number is now significantly higher (Zimmermann and Nkenke, 2020) . Around 60% of the people gather knowledge on the Internet before or after they visit a medical specialist. Approximately 50% of these people believe that they understand the health information they have sourced and are able to rate the quality of the data. However, patients rating the quality and reliability of online information seems to be an arbitrary process. Patients tend to stick to information that confirms their preexisting opinions, prejudices, and fears. Information is preferred that supports aspects that patients have heard or read before (Hambrock, 2018) . On average, people remain on a website for only 69 seconds. This short interval of time affects their health literacy (Eysenbach et al., 2002) . Health literacy is defined as an individual's ability to read, understand, and apply information to make rational healthcare decisions and follow treatment instructions (Lee et al., 2019) . Studies have shown that insufficient health literacy has an impact on patients' autonomy and ability to make sound decisions regarding their health. The consequence can be a poorer postoperative outcome (Menendez et al., 2017; Halleberg Nyman et al., 2018; Keim-Malpass et al., 2018) . Online health information sources are preferred especially by females, younger individuals, and people with a higher socio-economic status (Zschorlich et al., 2015; Hambrock, 2018) . People with a limited scientific background almost always carry out Internet-based searches using one of the major search engines. The dominant leader is Google, with a market share exceeding 92% (Statcounter, 1999 (Statcounter, -2017 . When patients think about undergoing orthognathic surgery, it is most likely that they will use the Internet to collect additional information and to receive support and reassurance (Bhamrah et al., 2015) . For the patients it is most important to gather highquality information before surgery, because there is a clear correlation with patient satisfaction after surgery (Bailey et al., 1999; Kufta et al., 2016) . Aside from online research and the individual's overall expectations, of course the information provided by the doctor during a personal consultation has an important influence on patient satisfaction (Al Kharafi et al., 2014) . In order to fully understand patients' motivations and expectations concerning orthognathic surgery, doctors have to be familiar with the information offered on the Internet (Lee et al., 2019) . Therefore, the aim of this study was to assess the quality of websites on the Internet that provide information on orthognathic surgery. An Internet search was performed in January 2019 using the three most commonly used search engines -Google, Yahoo, and Bing (Statcounter, 1999 (Statcounter, -2017 . The search was performed by the authors J.E. and C.F. The German terms for orthognathic surgery, operation for mandibular and maxillary advancement and set-back, surgical correction of the jaw. and correction of malocclusion were used as search terms, because these 4 were the most commonly used terms when patients filled in the medical questionnaires during consultation for orthognathic surgery. In order to guarantee identical search results, the search was performed simultaneously on two personal computers using the same local network and after deleting the cache files. The search was conducted with the latest version of Google Chrome (v. 71.0.3578), because Google Chrome is the most commonly used Internet browser (Statcounter, 1999 (Statcounter, -2017 . The first 50 results on the search engine results page for each search engine were included in this study in order to achieve adequate statistical power (Aldairy et al., 2012; Karamitros et al., 2018) . Websites not related to orthognathic surgery, duplicates, and websites in languages other than German were excluded. If a website was found by at least two of the three search engines, it was selected for further analysis. The website was attributed to the search engine where it reached the best rank on the search engine results page and was excluded from analysis for the remaining two search engines. The two investigators assessed all websites separately, with any remaining conflicts being discussed and resolved by consensus. Further analysis of the selected websites was performed with a modified Ensuring Quality Information for Patients (EQIP) tool (Charvet-Berard et al., 2008; Table 1 ). The item 'coverage of all relevant issues for the topic' was removed from the EQIP tool because none of the selected websites fulfilled this criterion. The items 'use of generic names for all medications or products' and 'printed consent form contrary to recommendations' were not relevant to the study and were also removed. Because of its high relevance for orthognathic surgery, the item 'description of the sequence of the surgical procedure' was subdivided into preoperative, intraoperative, and postoperative procedures. The answer 'partially yes' has been identified previously as being unreliable and, therefore, was removed from the scoring system (Ademiluyi et al., 2003) . All items answered with 'yes' were scored equally with 1 point and summed up at the end. A maximum of 35 points for the EQIP score was attainable. For statistical analysis IBM ® SPSS ® Statistics v.25 was used. Descriptive statistics, Student's t-test, ANOVA. and Pearson's correlation coefficient (PCC) were performed. The level for statistical significance was set at p < 0.05, with all tests two-sided. Out of the initially 150 identified websites, 55 were excluded due to duplicate retrievals and another 47 were excluded due to irrelevant content. The remaining 48 websites underwent further analysis (Figure 1 ). Indications or reasons for the need for orthognathic surgery were given on 32 out of the 48 websites. Planning and sequencing of at least one surgical procedure were included on 37 websites. Risks or side effects were mentioned on only 13 websites. Links and sources for further information were offered on 18 websites. Examples of treated patients were given only on nine websites. Out of the 48 websites, 22 were provided by private practices or dentists, 13 by public hospitals or universities, 10 by patient blogs, two by online encyclopedias, and one by an online newspaper. For further statistical analysis the latter three were pooled in one group as 'other online information sources'. 42 websites had their origin in Germany, five websites in Austria, and one in the Czech Republic. The median EQIP score was 13.65, with a standard deviation of 6.289. The lowest score in the survey was 2, and the maximum score was 28 (Figure 2 ). The website with the 6 highest EQIP score in our study was www.professor-lindorf.de. This was found by Bing at rank 31. Relation to a private practice, hospital, blog, etc. was not correlated with the quality of the website. EQIP scores for the three groups 'private practices' (12.77 ± 7.45), 'public hospitals or universities ' (14.69 ± 4.90) , and 'other online information sources' (14.08 ± 5.56) showed no statistically significant difference (p = 0.66; Figure 3) . Differences in quality at the search engine level were found. Websites ranked higher on the search engine results page for Google showed a significantly lower mean EQIP score (11.12 ± 6.19) than those identified by Yahoo (16.60 ± 5.71) and Bing (16.23 ± 5.08; p = 0.012). A significant negative correlation between the rank on a search engine results page and EQIP score was found. The higher a website was ranked on a search engine results page, the higher was the EQIP score (r = − − − −0.411, p = 0.004; Figure 4 ). This study was the first one to assess online patient information relating to orthognathic surgery using a modified EQIP tool. The EQIP tool is comparable to the British Medical Association patient information award appraisal form for some criteria, and includes the three key aspects: content, structure, and identification data. The EQIP tool has adequate interrater reliability They found Wikipedia to be the source of highest quality, with DISCERN scores of 64 (Aldairy) and 63 (Lee) . Interestingly, Wikipedia achieved an EQIP score of 13 in our study, placing it very close to the average score (13.6 ± 6.2). This result supports the assumption that DISCERN is more about readability than scientific evaluation of the content and the quality of information. As a consequence, there is the risk that the lay reader may attribute quality to a well-written web page and thus might be misled. Previously, the EQIP tool has been applied for the evaluation of online information about the treatment of cleft lip and palate malformations (Karamitros et al., 2018) . The mean EQIP score was 19.1 ± 5.4, which is higher than that found in our study for orthognathic surgery (13.6 ± 6.2). Considering the fact that a number of EQIP tool items were removed from the questionnaire because they had no relevance for our study, one might have expected us to find a higher EQIP score. It seems that the quality of online information on orthognathic surgery needs significant improvement before these websites can compete with those on the treatment of cleft lip and palate malformations. The information provided on the different websites was correct from a medical point of view. However, most of the websites provided only basic information on orthognathic procedures before focusing on fixing an appointment for a face-to-face consultation. For every candidate for orthognathic surgery an individual treatment plan has to be put together depending on the anatomical situation, esthetic demands, and the surgeon's preferences (Thiele et al., 2016) . Therefore, it is clearly difficult to set up a website that covers all aspects of orthognathic surgery comprehensively. We assume that a lack of 8 in-depth information possibly led to the low mean EQIP score. Nevertheless, it was surprising to see that only a few websites included in the study described the most common operations, such as Le Fort I osteotomy, bilateral sagittal split osteotomy, and genioplasty, in detail. Obviously, adding this information would immediately increase the quality of the websites. Patients who had undergone orthognathic surgery generally were satisfied and felt well informed (Al Kharafi et al., 2014) . However, they considered information regarding surgical risks and functional limitations following surgery insufficient. Websites of public hospitals and universities were more likely to provide relevant information about the risks and disadvantages of different procedures in cleft lip and palate surgery compared with websites of private practices (Karamitros et al., 2018) . However, our study found no significant differences between websites of public hospitals or universities and private practices. Indeed, the website of highest quality came from a private practice. Only 13 websites out of 48 provided information on possible risks or side-effects of orthognathic surgery, although according to the current literature complications occur in approximately 20% of patients (Zaroni et al., 2019) . Regarding search engines, this study can refute the hypothesis that 'the bigger, the better'. Google has a market share of 92.4%, while Yahoo (2.1%) and Bing (2.4%) are comparatively minor companies (Statcounter, 1999 (Statcounter, -2017 . Although Google was able to identify most of the relevant websites that were included in our study -n = 25 (52%) vs Yahoo with n = 13 (27%) and Bing with n = 10 (21%) -Google produced a significantly lower mean EQIP score. Unfortunately, the design of our study did not enable us to find explanations for why websites identified by Google were of reduced quality. A limitation of our study was that the number of websites that underwent statistical analysis was low. There are different opinions on how many websites people visit on a search engine results page before they start a new search. Recent studies have shown that it is very unlikely that patients will open more than the first 20 websites on a search engine results page (Eysenbach et al., 2002; Aldairy et al., 2012; Karamitros et al., 2018) . Therefore, increasing the number of websites in this study might strengthen the statistical power, but would change the design of the study to a less realistic scenario. As a consequence, increasing the included websites was not taken into account. This study showed a pronounced variation in the level of quality of the analyzed websites. Therefore, surgeons must be aware that they might be confronted with unrealistic expectations of patients, who may underestimate the potential risks and drawbacks of orthognathic surgery. Ethical approval and consent to participate: Declarations of interest: The authors have declared no conflicts of interest. Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The median EQIP score was 13.65, with a standard deviation of 6.289. The lowest score in the survey was 2 and the maximum score was 28. Figure 2 . Modified Ensuring Quality Information for Patients (EQIP) score frequency. The median EQIP score was 13.65, with a standard deviation of 6.289. The lowest score in the survey was 2 and the maximum score was 28. Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet Orthognathic surgery: is patient information on the Internet valid? Orthognathic surgery: pretreatment information and patient satisfaction Assessment of patients for orthognathic surgery Internet discussion forums, an information and support resource for orthognathic patients Measuring quality of patient information documents with an expanded EQIP scale How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and indepth interviews Carpal tunnel syndrome: analysis of online patient information with the EQIP tool Google Search Quality Rating Guidelines Association between functional health literacy and postoperative recovery, health care contacts, and healthrelated quality of life among patients undergoing day surgery: secondary analysis of a randomized clinical trial Die Suche nach Gesundheitsinformationen -Patientenperspektiven und Marktüberblick. Bertelsmann Stiftung Quality of health information on the Internet in pediatric neuro-oncology The quality of rhinoplasty health information on the Internet Clefts of the lip and palate: is the Internet a trustworthy source of information for patients? Systematic review of quality of patient information on phalloplasty in the Internet Impact of patient health literacy on surgical treatment of breast cancer Components of patient satisfaction after orthognathic surgery Online patient education materials for orthognathic surgery fail to meet readability and quality standards Evaluating written patient information for eczema in german: comparing the reliability of two instruments, DISCERN and EQIP Donor information for living donor liver transplantation: where can comprehensive information be found? Ring D: Patients with limited health literacy ask fewer questions during office visits with hand surgeons A survey of the quality and accuracy of information leaflets about skin cancer and sun-protective behaviour available from UK general practices and community pharmacies Quality of information for women seeking breast augmentation in the Internet Quality assessment of patient information on the management of gallstone disease in the Internet -a systematic analysis using the modified Ensuring Quality Information for Patients tool Moving the mandible in orthognathic surgery -a multicenter analysis Quality evaluation of JAMA Patient Pages on diabetes using the Ensuring Quality Information for Patient (EQIP) tool Complications associated with orthognathic surgery: a retrospective study of 485 cases Approaches to the management of patients in oral and maxillofacial surgery during COVID-19 pandemic Health information on the Internet: who is searching for what, when and how? 10 The authors would like to thank the Center for Medical Statistics, Informatics and Intelligent Systems (CEMSIIS) at the Medical University of Vienna for its support.