key: cord-0963469-tzt497qb
authors: Treanor, Lee; Radonjic, Aleksandar
title: Assessing the reading level of online resources on COVID-19
date: 2020-06-25
journal: Can J Public Health
DOI: 10.17269/s41997-020-00363-w
sha: c764196a1921522ef730f604bb3e1e79278728a4
doc_id: 963469
cord_uid: tzt497qb

nan

Dear Editor, The National Academy of Medicine defines health literacy as the degree to which individuals have the capacity to obtain, process, and understand basic health information needed to make appropriate health decisions. Most patients use the Internet as their initial source of health information (Swoboda et al. 2018) . Currently, there is an established body of misinformation surrounding coronavirus disease 2019 (COVID-19) (Ioannidis 2020) . Therefore, it is paramount that online health resources for COVID-19 are at an appropriate reading level; we aimed to assess their readability.

In a cache-cleared and location-disabled web browser, "coronavirus information" was entered in three popular search engines (Google, Bing, Yahoo!) on March 24, 2020, using five geographically representative virtual private network (VPN) locations in the United States and Canada (San Francisco, Miami, New York, Vancouver, and Toronto). Only websites in the first three search engine pages of each unique VPN location were included, as people are unlikely to proceed past this point (Eysenbach and Kohler 2002) . Public health, governmental (including regional and national), and foundational webpages which pertained to COVID-19 information for the general public were included. Webpages under 100 words, non-English articles, photos, videos, advertisements, news articles, and webpages not aimed at the general public (e.g., for healthcare professionals or businesses) were excluded. Each webpage was assessed by two independent reviewers. Any differences in data extraction were resolved by reevaluation until consensus.

The American Medical Association (AMA) and National Institutes of Health (NIH) have both recommended that reading levels of online health material should be at or below a sixth-grade level to be comprehensible to the general public (Fahimuddin et al. 2019) . Thus, five scores were used to estimate the grade level appropriate for reading each website: Flesch-Kincaid Grade Level (FKGL), Automated Readability Index (AMI), SMOG Index (SMOGI), Coleman-Liau Index (CLI), and Gunning Fog Score (GFS). The general grade level (GGL) was calculated by averaging these five scores. Flesch Reading Ease Score (FRES) applies a scale ranging from 0 to 100 with higher scores indicating easier readability (Abu-Heija et al. 2019) . Last, websites were categorized based on reading level: a GGL < 7 was considered easy; between 7 and 10, intermediate; and > 10, hard.

Of 428 webpages, 371 were excluded (330 were duplicates, 11 were < 100 words, 19 were not for the general public, and 11 were news articles). The mean GGL of the remaining 57 included webpages was 10.8. By stratifying readability of webpages with GGL, 70% were considered hard and 30% intermediate (Table 1) . Based on FRES, 82% were considered hard to read. None of the webpages were considered easy to read based on either GGL or FRES.

In conclusion, our results show that the readability of online resources for COVID-19 exceeds thresholds previously set by the AMA and NIH. Readability may be improved by

Quantitative readability assessment of the internal medicine online patient information on

Readability of internet-sourced patient education material related to "labour analgesia

How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews

Reading level of online patient education materials from major obstetrics and gynecology societies

Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures

Odds of talking to healthcare providers as the initial source of healthcare information: updated cross-sectional results from the health information National Trends Survey (HINTS)

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Easy (< 7) 0 Intermediate (7-10) 7Hard (> 10) 50 FKGL 10.6 ± 2.2 (6.8-16.1)Intermediate (7-10) 22Hard (> 10) 34 CLI 10.8 ± 1.6 (8.0-16.0)Intermediate (7-10) 10Hard (> 10) 47 SMOGI 9.6 ± 1.6 (6.7-13.6)

Intermediate (7-10) 33Hard (> 10) 21 ARI 10.6 ± 2.6 (5.9-17.9)

Intermediate (7-10) 18Hard (