key: cord-0946527-a3kgotjw
authors: van Ballegooie, Courtney; Hoang, Peter
title: Health Services: A Mixed Methods Assessment of Canadian Cancer Patient Education Materials Related to the 2019 Novel Coronavirus
date: 2021-02-10
journal: Cancer Control
DOI: 10.1177/1073274821989709
sha: af41059b0387734de0e3c4304fd8a3d2cfbe59b1
doc_id: 946527
cord_uid: a3kgotjw

The 2019 novel coronavirus (COVID-19) pandemic has prompted the reorganization in the scheduling and method of care for many patients, including patients diagnosed with cancer. Cancer patients, who have an immunocompromised status, may be at a higher risk of severe symptoms from infection with COVID-19. While information is rapidly evolving regarding COVID-19, Canada, both nationally and provincially, has been conveying new information to patients online. We assessed the content and readability of COVID-19-related online Canadian patient education material (PEM) for cancer patients to determine if the content of the material was written at a grade reading level that the majority of Canadians can understand. PEMs were extracted from provincial cancer agencies and the national Canadian Cancer Society, evaluated using 10 readability scales, qualitatively analyzed to identify their themes and difficult word content. Thirty-eight PEMs from both national and provincial cancers associations were, on average, written above the recommended 7th grade level. Each of the associations’ average grade levels were: BC Cancer (11.00 95% confidence interval [CI] 8.27-13.38), CancerControl Alberta (10.46 95% CI 8.29-12.62), Saskatchewan Cancer Agency (11.08 95% CI 9.37-12.80), Cancer Care Manitoba (9.55 95% CI 6.02-13.01), Cancer Care Ontario (9.35 95% CI 6.80-11.90), Cancer Care Nova Scotia (10.95 95% CI 9.86-12.04), Cancer Care Eastern Health Newfoundland and Labrador (10.14 95% CI 6.87-13.41), and the Canadian Cancer Society (10.06 95% CI 8.07-12.05). Thematic analysis identified 4 themes: public health strategy, information about COVID-19, patient instructions during COVID-19, and resources. Fifty-three percent of the complex words identified were medical jargon. This represents an opportunity to improve PEM readability, to allow for greater comprehension amongst a wider target audience.

Patient education is critical during the COVID-19 pandemic which has high rates of infection, significant mortality, and a current lack of therapeutic measures. 1 This is especially important for patients who have increased risk of infection and mortality, such as cancer patients. 2 The pandemic has impacted cancer patients globally including the reorganizations of oncology departments, changes and cancellations in treatment courses, and disruptions of clinical trials. [2] [3] [4] [5] [6] [7] These disruptions and modifications to cancer care in addition to increased risk perception has led to a decrease in quality of life (QoL) for patients, including an increase in depression and anxiety. 8, 9 One method of increasing patient QoL while conveying information on the rapidly evolving situation is through effective patient education.

Readability plays a key role in patient education and directly impacts patient's ability to access, understand, and utilize information in order to create an informed decision regarding their health. This concept, also known as health literacy, has been linked to positive health outcomes such as increased patient satisfaction and adherence to treatment, improved physician communication, and decreased anxiety and depression. [10] [11] [12] [13] [14] [15] Studies have indicated that a substantial proportion of health-care content is written at a 10th grade reading level, however, the majority of Canadian adults read between an 8th to 9th grade reading level. [16] [17] [18] [19] Reading level recommendations of patient education materials (PEMs) suggest that the documents should be at, or below, a 7th grade reading level. 20, 21 This recommendation is reflective of the observations that a person's comprehension is lower than their reading level and that health literacy levels for approximately 50% of Canadian adults falls below a high school level. In order to remain accessible for all patients, PEMs should accordingly be written 2 grade levels below the population average. 18, 22, 23 In this study, Canadian cancer PEMs relating to COVID-19 from provincial and national cancer associations were assessed for their readability using 10 readability scales. The content of the PEMs was also qualitatively evaluated to identify themes.

During July 2020, all internet-based PEMs were exhaustively downloaded from the associations' websites. These organizations are listed in Table 1 along with the number of unique PEMs obtained from each website. The downloaded PEMs included materials describing COVID-19-related disruptions, risks, and resources with intended use by cancer patients. Therefore, this excluded material such as those intended for health care providers. PEMs were manually converted to a Microsoft Word document (Microsoft Corp) for further analysis. Text sections of nonmedical information such as page numbers, disclaimers, contact details, tables, images, references, website addresses, and webpage navigation were excluded from assessment. 24 The specific online locations of each province's and national cancer association's PEMs can be found in Supplementary Table 1 .

PEMs were qualitatively coded to identify themes. Two members (CVB and PH) independently identified and coded salient themes and subthemes from the PEMs. Themes underwent constant comparative analysis to systematically interpret and define themes as described below. 25 Open coding was initially completed on PEMs of similar topics from each association (e.g. Information about COVID-19, cancer screening during COVID-19, etc.) to compare between PEMs and determine their primary concepts. These were then compared with the second author in order to validate themes. PEMs of separate topics were then analyzed, and similar codes were then grouped into a concept map to generate themes and subthemes. Triangulation was completed via several methods. PEMs from multiple Canadian associations were obtained to provide perspectives from various healthcare systems. Independently formed themes were compared between the authors, who are of different professional backgrounds. Finally, a mixed methods approach was undertaken, wherein a difficult word analysis would provide context to the themes via the types and frequency of the complex words identified, particularly medical terms. 26 Themes were compared to confirm consensus between authors (CVB and PH) and for the saturation of themes, where possible.

A readability assessment was then performed on the PEMs using the software package Readability Studio professional edition version 2019.3 (Oleander Software, Ltd). The scales selected report a "grade-level equivalent", which indicates the number of years of formal education that a reader would need in order to understand the PEM. The readability scales used to determine the grade level of the PEMs included eight numerical scales and two graphical scales. scale. The two graphical scales included the Raygor Readability Estimate Graph (RREG), and the Fry Readability Graph (FRG). While these ten scales are frequently used with medical texts and offer externally validated measures of readability, special emphasis should be given to the SMOG Index readability formula as it is the recommended readability tool for PEMs by the National Cancer Institute and National Institute of Health. 20, 27, 28 While many of the readability scales assume narrative text, PEMs often contain non-narrative text that must be modified before the analysis. This includes the addition of full stops for headings as well as the modification of bullet points to form complete sentences for analysis. In order to address this limitation, PEMs were individually edited to create high-and low-sentence number documents. For example, in high sentence number documents, each individual bullet point was treated as an independent sentence. On the other hand, llow sentence number documents had each bullet point separated with a comma with the final bullet point ending the sentence. These high-and low-sentence number documents typically result in low-and high-grade level estimates respectively when analyzed with narrative based readability scales. These low-and high-grade level estimates were then averaged to take into account the nonnarrative structure that many of the PEMs contain in order to better mitigate the inaccuracies they cause in many of the readability tests. 27 The average readability levels of the high-and low-sentence number documents using the average of the eight numerical scales can be seen in Figure 1 and the eight individual scales in Figure 2 , generated using Prism 8 software. Readability Studio was used to generate the two graphical scales, Figures 3 and 4.

Readability Studio extracted individual words from each of the PEMs. The analysis included the identification of the percentage of complex words (3þ syllable words) in each PEM, the percentage of long (6þ characters) words, as well as the percentage of unfamiliar words according to the NDC criteria. In addition to the analysis performed by the software, words were further analyzed in order to provide suggestions to increase the readability of the documents. Words were compared to the NDC word list as well as the New General Service List (NGSL) and those that appeared in either of the lists were removed and considered as non-jargon words. 29 All words with 3þ syllables were then extracted, and different tenses of the same word were combined. Hyphenated words were only included if one or more of the components contained unfamiliar words. Alternative words were then proposed for any 3-syllable word that appeared in 3 or more PEMs, either using Readability Studio, the Merriam-Webster Thesaurus, or in consultation with a medical doctor (PH), in order to identify synonyms that can decrease the difficulty of the word.

Graphical data in Figure 1 and Figure 2 were reported as the mean with the error bars representing the standard deviation. Data sets had their normality tested using a Shapiro-Wilk test. Equal variance was tested using a Brown-Forsythe test. Normally distributed data with equal variance then underwent a 1-way Analysis of Variance (ANOVA). If the data was not normally distributed, then a non-parametric, Kruskal-Wallis test, was employed. Multiple comparison's tests, such as Dunn's tests, were utilized to identify differences between sample means. Statistics were analyzed using Graph Pad Prism 8.

After the bullet points were converted to plain text, the high-and low-sentence estimate PEMs were subjected to a number of readability tests, including DRP GE, FK, SMOG, CLI, GF, NFC, NDC, and FORCAST, and averaged. Figure 2 illustrates a summary of the results for cancer associations that contain more than one PEM. The average grade reading level of the eight readability scales for the 38 PEMs were: BC Cancer (11.00 95% confidence interval [CI] 8.27-13.38), CancerControl Alberta (10.46 95% CI 8.29-12.62), Saskatchewan Cancer Agency (11.08 95% CI 9.37-12.80), Cancer Care Manitoba (9.55 95% CI 6.02-13.01), Cancer Care Ontario (9.35 95% CI 6.80-11.90), Cancer Care Nova Scotia (10.95 95% CI 9.86-12.04), Cancer Care Eastern Health Newfoundland and Labrador (10.14 95% CI 6.87-13.41), and the Canadian Cancer Society (10.06 95% CI 8.07-12.05). These averaged grade levels, as seen in Figure 1 , had no statistical difference between the different cancer associations. The overall mean was 10.65 (95% CI 9.92-11.39), with a range from 3 to 18. Individual readability scores for each of the associations can be seen in Table 2 . The FRG of the high sentence estimate, as seen in Figure 3a , ranges from a 7th grade to a 17th (university educated) reading level. The FRG of the low sentence estimate ranges from an 8th grade to 17th grade reading level with the majority of PEMs requiring an 11th grade reading level (Figure 3b ). The RREG of the high sentence estimate (Figure 4a ) ranges from a 6th grade reading level to a grade level equivalent to that in university. The RREG of the low sentence estimate ranges from a 7th grade to a university grade reading level with the majority of PEMs requiring a college reading level (Figure 4b ). 

From the difficult word analysis, it was found that the PEMs, on average, were comprised of 17.36% + 0.04 complex words which contained 3 or more syllables, 38.66% + 0.05 of words that contain 6 or more characters, and 24.13% + 0.05 of words which were unfamiliar. From this analysis, it was found that the majority of unfamiliar or difficult words, 53%, were medically related terms. The most frequent terms included Outpatient (-s), Designate (-ed; -tion), Pandemic, Medication (-s), Pharmacy (-ies) [Pharmacist] , and Physician (-s). Table 3 describes all the difficult words found on the inclusion criteria, their frequency, and recommended alternatives.

Qualitative analysis identified 4 major themes: public health strategy, information about COVID-19, patient instructions during COVID-19, and resources. Subthemes are described under each individual theme.

The subthemes for the public health strategy included: protecting patients and healthcare workers, providing up to date information, and changes to visitation and appointment policies. Each province identified actions that they were undertaking as a result of COVID-19. The primary goals of each health authority were to provide up-to-date information and protect patients and healthcare workers. Due to the pandemic, changes occurred to regular services including cancer screening programs, laboratory and diagnostic testing capacity, volunteer programs, visitation policies, and appointment settings. Provinces attempted to maintain imaging, radiation, and chemotherapy for cancer patients where possible. Visitation protocols were regularly described in five provinces, and predominantly required patients to attend appointments alone, except under limited circumstances such as a new cancer diagnosis.

Symptoms and risk were subthemes regularly described under information about COVID-19. Information about the novel coronavirus included the wide range of symptoms, symptoms in the context of cancer therapy (e.g. cough after radiation), route of transmission, and the lack of a vaccine. Patients who have chronic conditions or a diagnosis of cancer, particularly if receiving or recently completed chemotherapy, were identified to be at a particularly high risk of severe symptoms.

Given the nature of these PEMs were to partly provide instructions for patients, subthemes included instructions during illness and self-protection strategies during COVID-19. Instructions were divided into actions during illness, preventative measures, and maintaining mental health. Criteria were provided based on the severity of symptoms (e.g. calling 911 or going to the emergency department as a cancer patient who has a fever with new cough or shortness of breath). Patients were asked to regularly call their oncology team and keep them updated on changes in health status. Preventative measures include self-isolation, avoiding non-essential travel and in-person gatherings, wearing a mask, and healthy lifestyle behaviors (e.g. smoking cessation, socialization, etc.). PEMs normalized mental health concerns and informed patients of signs of stress as well as coping strategies. 

There was a large variation in the number of resources provided based on the focus of the PEMs. While some PEMs included lists of resources, others provided a link to further information. Specific resources included mental health, advanced care planning, smoking cessation, and cancer services (Canadian Cancer Society, Cancer Information Service, CancerControl). Information modalities included telephone, online, and community settings; few specifically mentioned low-cost, confidential, and multi-lingual services. Most resources were provided by the individual provincial health authority. Advanced care planning was identified as a subtheme. Questions and forms were provided for patients to reflect on their wishes, goals, and support persons they wished to be involved in their care.

Accessible patient education is critical during the COVID-19 pandemic especially to those who have an increased risk of infection and mortality, such as cancer patients. 1,2 While the reorganization of oncology departments continues, cancer patients will continue to require consistent and accessible information in order to make informed decisions regarding their health during the pandemic. [2] [3] [4] [5] [6] [7] In order to create accessible information for cancer patients, the recommended reading levels of PEMs should be at least 2 grade levels lower than the target population's average reading level which, in Canada, is a 9th grade reading level. 18, 20, 21, 23 Due to this, it is imperative that PEMs are written at or below a 7th grade reading level. 28 Analyses using quantitative and graphical scales showed that PEMs were written at 10.65 grade level (95% CI 9.92-11.39), greater than the Center of Disease Control and Prevention and National Institutes of Health recommended 7th grade reading level and below. 20, 21 In addition, the difficult word analysis identified that over half of the complex words were medical terms. Qualitative analysis showed that PEMs described the ongoing public health strategy, provided information and instructions about COVID-19, and supplied resources predominantly for mental health. The provincial health strategy and information about COVID-19 were generally associated with a higher frequency of complex words from the difficult word analysis list. For example, "designate" was typically used in the setting of an appointed visitor; "pandemic" or "chemotherapy/radiation" were used when providing information about COVID-19 and risk to cancer patients. These sections are likely more complicated due to greater frequency of medical terms. Previous research assessing information needs for patients diagnosed with cancer have shown variation in the amount and type of information that these patients may seek. 30, 31 Studies measuring cancer patients' satisfaction with health information have shown that patients are generally satisfied with information provided on diagnosis and diagnostic testing. However, patients identified that areas such as mental health, support services, actions patients can take to improve their health, and ease of access to information required improvement. 32 Globally, studies have been performed to better understand cancer patients' knowledge and perception of COVID-19 in both developed and developing countries. While many cancer patients are able to identify that COVID-19 is a transmissible disease that can be spread through both contaminated objects or close contact with an infected person, many were not able to identify that cancer patients may be immunocompromised and, therefore, at higher risk of having more serious COVID-19 complications and that no vaccine or effective cure is currently available at the time of this study. [33] [34] [35] [36] Our study shows that these PEMs included substantial information on previously identified subjects of importance by patients as well as COVID-19 specific information that the literature has identified are gaps in patients' knowledge. Provinces had similar information provided in their PEMs, although there was a large range in the number of PEMs. Nova Scotia offered the most detailed PEMs, and disclosed information such as the changes to surgical caseloads as a result of COVID-19, which cancer patients may find helpful if they were on a surgical waitlist.

There are several limitations in our study. Our study identified 38 PEMs which were unequally distributed across provinces, which limits conclusions or comparisons between provinces. Additionally, the PEMs were only extracted from the associations' online resources, which does not take into account what information patients could obtain in person or through other, smaller organizations. Furthermore, we focused primarily on PEMs for Canadian cancer patients, which may limit generalizability to other settings, such as developing countries or noncancer based medical settings. In addition, while words such as coronavirus are likely to be understood by all patients, many readability scores use a component of syllabic count in their calculation, likely overestimating the grade level. On the other hand, medical jargon which contains few letters and syllables, such as renal, may evade detection on readability tests that rely only on the number of letters and syllables. 37 The difficulty word analysis also identified words as complex based on their syllabic count, many of which included medical jargon. However, in the absence of direct patient interviews, it is difficult to draw conclusions on which terms patients may or may not understand in the context of these PEMs. Other considerations towards the organization, layout, and design will also impact the education materials' comprehensibility. However, available instruments to assess such factors (including The Suitability Assessment of Materials and PMOSE/IKIRSCH33), have not yet been validated for medical texts and should be investigated in the future. 17, 38, 39 

Mixed method evaluation of PEMs in the setting of COVID-19 for patients diagnosed with cancer have identified that their readability remains above the currently recommended 7th grade reading level, with an average grade reading level of 10.65 (95% CI 9.92-11.39). Higher readability information is more frequently presented in information about public health measures and COVID-19. Most of the complex words are medical jargon. Our study shows that there are still many opportunities for improving readability of PEMs. Future studies may include patient-directed interviews in assessment of PEMs. As the government continues to provide information about COVID-19 via the internet, it is imperative that PEMs are written at a grade level understood by all patients.

Courtney van Ballegooie contributed to the study concept and design, acquisition of data, interpretation of data, analysis, and preparation of the manuscript. Peter Hoang contributed to the acquisition of data, interpretation of data, and preparation of the manuscript. 

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Canadian Institutes of Health Research (grant number 21R04868) and the University of British Columbia Graduate Fellowships. The funding sources had no role in the conduct of the research and/or preparation of the article.

Courtney van Ballegooie, BSc https://orcid.org/0000-0003-3498-8769

Supplemental material for this article is available online.

Risk communication during COVID-19

High mortality rate in cancer patients with symptoms of COVID-19 with or without detectable SARS-COV-2 on RT-PCR

Cancer patients affected by COVID-19: experience from Milan, Lombardy

Post-lockdown management of oncological priorities and postponed radiation therapy following the COVID-19 pandemic: experience of the Institut Curie

Reduced fractionation in lung cancer patients treated with curative-intent radiotherapy during the COVID-19 pandemic

Conducting phase 1 cancer clinical trials during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-related disease pandemic

SARS-CoV-2 infection in cancer patients undergoing active treatment: analysis of clinical features and predictive factors for severe respiratory failure and death

Impact of the COVID-19 pandemic on quality of life for women with ovarian cancer

Patient reported outcomes of breast cancer patients during the COVID-19 outbreak in the epicenter of China: a cross sectional survey study

Association of perceived physician communication style with patient satisfaction, distress, cancer-related selfefficacy, and perceived control over the disease

Predictors of medication adherence in the elderly

Racial/ethnic group differences in treatment decision-making and treatment received among older breast carcinoma patients

Psychological outcomes in women with early breast cancer

Psychological well being and quality of life in ovarian cancer survivors

Health literacy and shared decision making for prostate cancer patients with low socioeconomic status

A comparative analysis of the quality of patient education materials from medical specialties

Assessment of online patient education materials from major ophthalmologic associations

Skills Outlook 2013: First Results From the Survey of Adult Skills

Adult Literacy in America: A First Look at the Findings of the National Adult Literacy Survey

Center for Disease Control and Prevention. Simply Put

UCLA Previously Published Health Literacy in Canada: A Healthy Understanding

Assessment of the readability of online patient education material from major geriatric associations

A purposeful approach to the constant comparative method in the analysis of qualitative interviews

Establishing trustworthiness and authenticity in qualitative pharmacy research

Assessment of use, specificity, and readability of written clinical informed consent forms for patients with cancer undergoing radiotherapy

Assessing readability of patient education materials: current role in orthopaedics

Communication during radiation therapy education sessions: the role of medical jargon and emotional support in clarifying patient confusion

A study of information seeking by cancer patients and their carers

Differences in information seeking among breast, prostate, and colorectal cancer patients: results from a population-based survey

What information do cancer patients want and how well are their needs being met? Ecancermedicalscience

Perspectives of cancer patients and their health during the COVID-19 pandemic. Tan W

Oncology patients' perceptions of and experiences with COVID-19

Knowledge, attitudes, and practices of cancer patients towards COVID-19: a cross-sectional study in Central Nepal

Perspective of oncology patients during COVID-19 pandemic: a prospective observational study from India

Assessment of Canadian patient education material for oncology pharmaceutics

Readability assessment of patient education materials on major otolaryngology association websites

A new measure for assessing document complexity: the PMOSE/IKIRSCH document readability formula