key: cord-1020561-jlrpasv4 authors: Campbell, Alyson; Hartling, Lisa; Plourde, Vickie; Scott, Shannon D title: Parental Knowledge, Self-confidence, and Usability Evaluation of a Web-Based Infographic for Pediatric Concussion: Multimethod Study date: 2022-05-10 journal: JMIR Pediatr Parent DOI: 10.2196/36317 sha: aa3660ccfa6d22bae12876e81d431d9a3ce294e2 doc_id: 1020561 cord_uid: jlrpasv4 BACKGROUND: Concussions, which are known as mild traumatic brain injuries, are complex injuries caused by direct or indirect blows to the head and are increasingly being recognized as a significant public health concern for children and their families. Previous research has identified few studies examining the efficacy of educational interventions on parental concussion knowledge. The aim of this research was to actively work together with children who have experienced a concussion and their parents to develop, refine, and evaluate the usability of a web-based infographic for pediatric concussion. OBJECTIVE: The objective of this study was to report on the usability of the infographic, parental knowledge, and self-confidence in pediatric concussion knowledge before and after exposure to the infographic. METHODS: A multiphase, multimethod research design using patient engagement techniques was used to develop a web-based infographic. For this phase of the research (usability, knowledge, and confidence evaluation), parents who could communicate in English were recruited via social media platforms and invited to complete web-based questionnaires. Electronic preintervention and postintervention questionnaires were administered to parents to assess changes to concussion knowledge and confidence after viewing the infographic. A usability questionnaire with 11 items was also completed. RESULTS: A web-based, infographic was developed. The infographic is intended for parents and children and incorporates information that parents and children identified as both wants and needs about concussion alongside the best available research evidence on pediatric concussion. A total of 31 surveys were completed by parents. The mean scores for each item on the usability surveys ranged from 8.03 (SD 1.70) to 9.26 (SD 1.09) on a 10-point Likert scale, indicating that the usability components of the infographic were largely positive. There was no statistically significant difference between preintervention and postintervention knowledge scores (Z=−0.593; P=.55; both preintervention and postintervention knowledge scores had a median of 9 out of 10). In contrast, there was a statistically significant difference between preintervention (mean 3.9/5, SD 0.56) and postintervention (mean 4.4/5, SD 0.44) confidence in knowledge scores (t(30)=−5.083; P<.001). CONCLUSIONS: Our results demonstrate that parents positively rated a web-based, infographic for pediatric concussion. In addition, although there was no statistically significant difference overall in parents’ knowledge scores before and after viewing the infographic, their confidence in their knowledge did significantly increase. These results suggest that using a web-based infographic as a knowledge translation intervention may be useful in increasing parents’ confidence in managing their child’s concussion. Concussions, which are known as mild traumatic brain injuries, are complex injuries caused by direct or indirect blows to the head [1] . An estimated 125,000 youths in Canada and 750,000 in the United States will sustain concussions annually [2, 3] . Typical signs and symptoms of a concussion include headaches, nausea, dizziness (or fogginess), and sensitivity to light or noise [4] . For most children, recovery occurs within 1 to 4 weeks after the injury [5] . An estimated one-third of children and youths will have persistent symptoms that occur beyond 28 days. Persistent symptoms may include headaches, sleep problems, and emotional distress [2, 3] . Rapid increases in the amount of health research, particularly in pediatric concussion, and the increased accessibility to research offered through the internet, suggest a demand for reliable, evidence-based health information that is relevant and easy to understand for patients and families [6] . Innovative media have been shown to be superior to traditional methods (ie, handouts) for transferring health information to consumers [7] . Our research team conducted an environmental scan of pediatric concussion resources found on the internet and in app stores. Despite innovative media being superior to traditional methods for transferring health information, our scan revealed that most pediatric concussion resources are PDF documents, suggesting that organizations struggle to optimize the use of innovative media (eg, infographics, videos, and narratives) when sharing health information. Our environmental scan also revealed that many resources are not developed in collaboration with health consumers. Using patient engagement approaches to involve end users (eg, parents and children) in the development of educational resources is key to effective knowledge translation (KT) [8] . A systematic review [9] evaluating the effectiveness of concussion education programs for coaches and parents of youth athletes found a limited number of studies examining the efficacy of educational interventions on parental knowledge of concussion and a lack of interventions designed specifically for parents. This suggests that additional research is necessary to investigate changes in parental knowledge following educational interventions and a need for more interventions specifically intended for parents and families. Furthermore, although studies have assessed parental knowledge of concussion, most have focused on sport-related concussions [9] [10] [11] [12] [13] [14] . Few studies have evaluated changes in concussion knowledge after exposure to an educational intervention [13] , and no known studies have assessed changes in self-confidence. Confidence is an important construct for behavior change. Higher levels of confidence increase the likelihood that an individual will change a health behavior when faced with obstacles [15] . Understanding how confident parents are in their knowledge and abilities to parent is an essential component of the quality and sustainability of parenting behaviors [16] . In the context of our research, understanding how confident parents are in their knowledge of concussion and if an increase in confidence before and after an intervention is observed may provide critical insight into their health decision-making for their child, such as when to seek emergency care and at-home recovery. To date, KT efforts have largely focused on ensuring that health care professionals use the latest research to inform their practice; however, emerging evidence suggests that efforts that target health consumers (eg, patients and families) can inform their decision-making and shape their treatment expectations (eg, what to expect in the emergency department) [17] [18] [19] [20] . Although research is beginning to demonstrate that strategies targeting parents for KT can reduce health care use and improve outcomes, more research is required to fully understand the power and impact of these efforts on both children and families [21] [22] [23] . Developing innovative KT tools that present research-based information in user-friendly languages and formats provides consumers with accurate recommendations while addressing knowledge or information needs. In addition, these tools may foster and empower consumers to make informed decisions about health care for themselves and their families. To date, limited research has explored the usefulness and effectiveness of web-based infographics as an innovative way to share health information with patients and families. The purpose of this research was to actively work together with children who have experienced a concussion and their parents to develop, refine, and evaluate the usability of a web-based infographic for pediatric concussion. We also aimed to assess parental self-confidence and knowledge of pediatric concussion before and after exposure to the infographic. This paper provides an overview of the development of the infographic prototype and reports on the results of parental usability, knowledge, and confidence testing. A multiphase, multimethod study using patient engagement techniques was used to develop, refine, and evaluate the usability of a web-based infographic for pediatric concussion to promote KT. Details of phases 1 and 2 of this work are reported by Campbell et al [24] . Changes in concussion knowledge and confidence in responses were also evaluated through preintervention and postintervention tests. Ethics approval was obtained from the University of Alberta Health Research Ethics Board (Pro0096202). An environmental scan was conducted (May 2021) by the first author (AC) to identify publicly available Canadian developed resources providing information on pediatric concussion found on the internet and in apps. Information gaps in these resources were extracted (eg, resource format, target audience, and information) and used to inform the subsequent phases of this research, including the target audience for semistructured interviews (phase 2) and elements of the infographic, including target audience, content, and format (phase 3). The full results of this environmental scan are reported by Campbell et al [24] . Phase 2 of this multiphase study was a qualitative description [25, 26] . The first author (AC; trained in qualitative methods and supervised by the senior author and principal investigator (SDS) conducted semistructured interviews with a convenience sample of children who have experienced a concussion and their parents. Children and parents were recruited on the web through our research group's (Evidence in Child Health to Enhance Outcomes [ECHO] KT) social media platforms (Twitter @echoktresearch, Facebook ECHO Research, and Instagram @echoktofficial) and website [27] . The overarching purpose of these interviews was to explore the concussion experience of children and parents to understand their information needs and preferences regarding concussion. For children, questions focused on the concussion experience from the child's perspective, and for parents, questions focused on the experience of caring for a child with concussion. Interview topics included mechanism of injury, symptom experience, experience with the health care team, recovery and follow-up, and concussion information needs and preferences. The interviews were recorded and transcribed verbatim. The findings from these interviews informed the content and format of the web-based infographic. The sample demographics and full results of this qualitative study are reported by Campbell et al [28] . The development of the infographic prototype involved first creating a composite infographic skeleton, which integrated the information needs and preferences of parents and children (as reported in the qualitative interviews) with the best available research on pediatric concussion management [28] [29] [30] . This integration determined the content for the infographic. The author AC led the development of the infographic with ongoing input and feedback from all authors. We then worked with a graphic design team that assisted in developing the accompanying artwork that would coincide with the content for the infographic. Upon completion of the infographic prototype, it was thoroughly reviewed and vetted for content accuracy by the author VP (content expert). The author VP is ideally suited for this assessment as she is well abreast of the best available research evidence in this specialized field. All authors provided extensive input and feedback on each iteration of the prototype until a finalized version was agreed upon to be disseminated for user feedback and evaluation. The prototype was then shared with a group of parents from the Pediatric Parent Advisory Group. The Pediatric Parent Advisory Group is a group of parents, legal guardians, and grandparents who serve as advisers to the ECHO Research and Alberta Research Centre for Health Evidence programs (the authors SDS and LH's research programs) by providing advice and feedback on research aimed at improving child health outcomes [31] . Finally, the prototype was informally shared with a diverse group of youth for further advice and feedback. On the basis of the recommendations and feedback from content experts, parents, and youth, changes were made to the infographic (eg, changes to colors, images, word choice, and order of information). We collected a convenience sample of participants to complete our web-based surveys. Collaborating clinicians from across Canada emailed potential participants with links to the infographic, usability test, and preintervention and postintervention knowledge and confidence tests. Potential participants included any parent, legal guardian, or grandparent. Participants were required to read and understand English. We asked for assistance from clinicians with known connections to concussion clinics, as potential participants from these clinics would be best suited to assess the relevance of the infographic and are more likely to seek out similar resources. Parent participants who previously participated in the qualitative interview portion of this research were also contacted via email with links to the infographic and surveys. In addition, we recruited participants via advertisements on our research group's social media platforms (Twitter, Facebook, and Instagram) with a link to the surveys. A link to the infographic, usability test, and preintervention and postintervention knowledge and confidence tests was made available to all potential participants. The study description, including potential risks and benefits, was made available before beginning the survey. Consent was implied if the web-based survey was completed and submitted. Completion of the survey was completely voluntary. Participants were asked to complete a web-based survey that assessed their perceptions of the infographic using an adapted version of the validated User Experience Questionnaire (UEQ) [32, 33] . We adapted the UEQ by reducing the number of items from 26 to 11 based on relevance to our infographic. The adapted UEQ contained 11 items, rated on a 10-point Likert scale ranging from 1 (least favorable answer) to 10 (most favorable answer). Participants had the opportunity to provide free-text feedback on areas that required revisions or more information. To evaluate the knowledge of pediatric concussion, participants were asked to answer 10 true or false questions reflecting the most common misunderstandings about concussion. The true or false questions were adopted from a questionnaire developed by McKinlay et al [34] . Participants rated their level of confidence in their response to each question using a 5-point Likert scale (very sure to very unsure). After completing the baseline knowledge test, participants were to read the infographic, and knowledge questions were answered again to assess short-term knowledge changes. Participants were again asked to rate their confidence. The UEQ was also completed following postintervention knowledge and confidence testing. Survey data were collected from May 6, 2021, to June 14, 2021, through the Canadian web-based electronic platform SimpleSurvey. SimpleSurvey is secure, protected by several firewalls and layers of security, in alignment with Canadian privacy laws. The data collected through SimpleSurvey are completely anonymous and cannot be traced back to any individual. We followed the Checklist for Reporting the Results of Internet E-Surveys. Data were entered into SPSS (version 26; IBM Corporation) [35] , and descriptive statistics (eg, frequencies), measures of central tendency, and tests of statistical significance were completed. We conducted sample size calculations based on change in overall knowledge score to achieve a power of 80% and a level of significance of 5% (2-sided) with a 1-point difference (on a 10-point scale) between preintervention and postintervention knowledge scores (correlation estimate 0.6). A total sample size of 28 participants was required. To assess knowledge change, we statistically compared preintervention and postintervention knowledge scores overall, as well as pretest and posttest scores for each topic. To assess knowledge change in each topic, items on the knowledge questionnaire were scored as 1 for correct and 0 for incorrect. The percentage of correct answers for each item and overall was then calculated. Owing to the nonnormality of knowledge scores, the median knowledge scores for both preinterventionand postintervention questionnaires were compared using Wilcoxon signed-rank tests. To assess change in confidence, we statistically compared preintervention and postintervention confidence scores overall, as well as pretest and posttest scores for each topic. Likert scale scores were averaged, and differences in pre-and posttest scores were compared using paired 2-tailed t tests. Free-text data were analyzed using content analysis [25] . In collaboration with a creative design team, we developed a web-based infographic based on 14 interviews with children and parents who have had experiences with concussion. The infographic targets parents and children and was designed to incorporate information parents and children identified as both wants and needs about concussion, including symptoms, when to go to the emergency department, return to play and learn guidance, and recovery tips. Interactive elements of the infographic include an animated GIF depicting what happens to the brain inside the skull after a direct or indirect blow to the head, drop-down lists, horizontal scrolling, audio clips of children telling their story and experience with concussion, and downloadable PDF information sheets targeting teachers, coaches, and family or friends that can be shared (Figures 1-4) . Modifications to the infographic were also made in careful consideration of those experiencing concussions. For example, we opted to make the font size larger than average and ensured the colors used throughout the infographic were less bright (softer or dull) to accommodate the visual disturbances and sensitivity to light often experienced with concussions [36, 37] . We ensured that the characters used throughout the infographic were representative of a diverse population. On average, the infographic takes approximately 5 to 10 minutes to read and review from beginning to end. In total, 43 surveys were started, of which 31 (72%) were completed. Table 1 shows the demographic characteristics of the study population. In general, parental reaction to the infographic was positive. The mean scores for each item of the usability scale ranged from 8.03 (SD 1.70) to 9.26 (SD 1.09) out of 10 ( Figure 5 ). Only 1 parent indicated that the infographic did not meet their expectations and 1 parent commented that they would not recommend this tool to other families managing a child's concussion. Common comments on the usability survey indicated parents felt the infographic was "simple and easy to follow," "user friendly," and "concise." Other comments on the most positive aspects of the infographic included the following: Good layout and flow to answer questions, easy to navigate, and flow chart easy to follow on next steps. Some parents felt that the infographic colors were "dull" and "sometimes hard to view" (ie, Participants 26 and 29). For instance, a parent said: The median preintervention knowledge score across the 10 knowledge topics was 9 (IQR 9-10), and the median postintervention knowledge score was 9 (IQR 9-10). A Wilcoxon signed-rank test showed no statistically significant difference in median knowledge scores before and after viewing the infographic (Z=−0.593; P=.53). Individually, an overall knowledge gain (although minor) was observed for topics 2, 7, and 10, and an overall knowledge loss was observed for topics 1, 6, 8, and 9, but Wilcoxon signed-rank tests showed no statistically significant changes before and after the intervention ( Table 2 ). In the remaining 3 topics (topics 3, 4, and 5), 100% (31/31) of the parents successfully identified the correct response before viewing the infographic, and this was retained after. The mean preintervention confidence score across the 10 knowledge topics was 3.9 (SD 0.56; 3.9/5, 78%) and the mean postintervention confidence score was 4.4 (SD 0.44; 4.4/5, 88%). A paired 2-tailed t test showed a statistically significant difference in average confidence scores before and after viewing the infographic (t 30 =−5.083; P<.001). Across each knowledge topic, paired 2-tailed t tests showed a statistically significant difference in confidence in answering the true or false questions for 70% (7/10) of the topics (topics 2, 3, 5, 6, 7, 9, and 10; Table 3 ). This study evaluated the usability of a novel, web-based infographic for parents who have experiences managing a child with a concussion, in addition to parental knowledge about pediatric concussion and confidence in their responses. Previous research has explored student and parent knowledge and perceived confidence about brain injury and concussion but has not assessed changes in knowledge or confidence before and after exposure to an educational intervention [38] . Our study is the first to examine how web-based and arts-based media impact parental knowledge and confidence about pediatric concussions. Parental concussion education is critically important, as many concussion signs and symptoms may not appear until hours or even days after the incident. As such, the onus falls on parents to identify signs and symptoms to ensure prompt and proper diagnosis, treatment, management, and safe return to play or school [9] . Previous studies on parental concussion knowledge have found that parents are generally knowledgeable about concussion signs, symptoms, and recovery time even before exposure to an educational intervention, with small improvements in knowledge following exposure [9, 13] . Parents in our study were also found to be generally knowledgeable about concussion before exposure to our infographic. Overall, the results of this study showed no significant overall increase in concussion knowledge. This may be because most parents in our study had previous experience with concussion and possessed high levels of concussion knowledge at baseline. A more diverse sample (particularly those without previous concussion experience) and more pronounced knowledge deficits at baseline may have yielded more significant knowledge changes. Unique to our study was the evaluation of not only parental knowledge changes but also changes in confidence in their responses. After exposure to the infographic, parents' confidence in their responses significantly increased in 70% (7/10) of the content areas. This suggests that our infographic was effective in helping parents feel more confident about their knowledge of pediatric concussions. Increasing parental confidence in knowledge of their child's illness or injury may positively influence their child health care decision-making (eg, when seeking medical attention, home care, and recovery), ultimately improving child health outcomes [15] . This phenomenon was observed in a previous study conducted by our research team in the area of pediatric chronic pain [22] . Overall, the results on the usability of the infographic were positive, with most parents rating each aspect of the tool as very favorable or extremely favorable. Our study and others have demonstrated that innovative media, using narrative and artistic elements, is a promising approach for communicating complex health information to parents and families [18, 19, 22] . Although time and resource strain often determine how resources are developed, more innovative media may be beneficial in evoking relevance, timely accessibility, and engagement. In fact, more innovative media such as cartoons and videos have been found to be superior to standard medical sheets in transferring information to consumers [7] . Our chronic pain study yielded results similar to those in this study, suggesting that innovative, web-based, and arts-based interventions (eg, e-books and infographics) are viewed positively by parents and may increase parental confidence in their knowledge of pediatric conditions [22] . The systematic review by Feiss et al [9] found only 3 concussion education programs to be evaluated in the literature. Furthermore, although these programs include written content directed toward parents, there are no programs specifically designed for parents. Our infographic is unique in that it was designed specifically for parents based on their information needs and preferences, and parents were involved throughout the tool development process. Furthermore, our infographic was designed to be safe or comfortable for youth with concussions to view, as we carefully considered elements such as font size and colors that would not cause eye strain and potentially exacerbate concussion symptoms. Although the rapidly developing evidence base in pediatric concussion places increased demands on updating information that is included in educational interventions to ensure they are timely and relevant, researchers have an opportunity to expand on the evidence being shared by taking a more participatory approach, involving end users in the development of these interventions. The finalized version of this infographic (finalized February 2022) is now being disseminated through established social media platforms, including ECHO Research's Instagram, Twitter, Facebook, and website [27] and Translating Emergency Knowledge For Kids [39] , which is a national network of health professionals and parents whose goals are to improve emergency care for children. We relied on parent self-report data. The parents in our study consistently possessed high levels of education, and our findings cannot be extrapolated to all parents, including those with more pronounced information deficits or poor health literacy. Recruitment and data collection for this study occurred during the COVID-19 pandemic; thus, in adherence to government guidelines, recruitment occurred on the web through convenience sampling, which may not have provided the most information-rich sources. We acknowledge the approximately 25% (31/43) dropout attrition in our surveys. The reasons for dropout attrition in our study are unknown, although they are common in web-based surveys. However, careful review of the survey is warranted to potentially diminish dropout attrition in the future. Our study evaluated short-term changes in knowledge and confidence. Future research should examine changes in knowledge and confidence over time, as well as whether this change affects decisions made regarding the care and outcomes of the child. In addition, approximately half of the parents in our study had previous experience with concussions and may have received information about concussions via other sources. This may confound our results in terms of the effectiveness of the infographic on knowledge and confidence. Furthermore, the baseline knowledge score was very high, leaving little room to see an effect in terms of knowledge change. Our study did not ask about previous education sources, training, or exposure to pediatric concussions. A better understanding of the impact of various forms of education on knowledge and confidence, including participants with lower socioeconomic backgrounds, will help improve the design and format of future educational interventions. Our results demonstrated that parents positively rated a web-based infographic about pediatric concussion, and this infographic increased their confidence in knowledge of pediatric concussion. These findings hold promise for future development, application, and effectiveness testing of web-based, arts-based KT interventions for transferring complex health information to parents. Future research using innovative digital media for knowledge transfer with different clinical conditions and participant demographics (ie, children and different parents) should be explored as well as the effectiveness of different formats (eg, videos, e-books, and standard information sheets). Consensus statement on concussion in sport-the 5th international conference on concussion in sport Reliable change in postconcussive symptoms and its functional consequences among children with mild traumatic brain injury Pediatric Emergency Research Canada (PERC) Concussion Team. Clinical risk score for persistent postconcussion symptoms among children with acute concussion in the ED Diagnosis and management of paediatric concussion What is the difference in concussion management in children as compared with adults? A systematic review Protocol for a systematic review of the use of narrative storytelling and visual-arts-based approaches as knowledge translation tools in healthcare. Syst Rev The power of stories: using narratives to communicate evidence to consumers Knowledge translation for effective consumers A systematic review of the effectiveness of concussion education programs for coaches and parents of youth athletes Association between concussion education and concussion knowledge, beliefs and behaviours among youth ice hockey parents and coaches: a cross-sectional study Exploration of awareness, knowledge, and perceptions of traumatic brain injury among American youth athletes and their parents Parent knowledge and perceptions of concussion related to youth football Parental knowledge of concussion: evaluation of the CDC's "Heads up to parents" educational initiative Assessment of parental knowledge and attitudes toward pediatric sports-related concussions Parental confidence in making overweight-related behavior changes Delineating among parenting confidence, parenting self-efficacy, and competence Barriers and supports to implementation of MDI/spacer use in nine Canadian pediatric emergency JMIR Pediatr Parent 2022 | vol. 5 | iss. 2 | e36317 | p (page number not for citation purposes) departments: a qualitative study Developing and testing two arts-based knowledge translation tools for parents about pediatric acute gastroenteritis. medRxiv (forthcoming) Developing and testing an arts-based, digital knowledge translation tool for parents about childhood croup. medRxiv (forthcoming) A test of conversational and testimonial messages versus didactic presentations of nutrition information Pediatric Emergency Research Canada Network. A randomized trial of a single dose of oral dexamethasone for mild croup Development and usability evaluation of an art and narrative-based knowledge translation tool for parents with a child with pediatric chronic pain: multi-method study How to translate health research knowledge into effective healthcare action The development and evaluation of an innovative knowledge translation tool about pediatric concussion. University of Alberta The use of quotes in qualitative research echo: translating evidence in child health to enhance outcomes You Can't Fix Your Brain": exploring concussion experiences of children and parents Bottom line recommendations: concussion. Trekk: Translating Emergency Knowledge for Kids Living Guideline for Diagnosing and Managing Pediatric Concussion. PedsConcussion Development and evaluation of a parent advisory group to inform a research program for knowledge translation in child health Construction of a benchmark for the user experience questionnaire (UEQ) Adaption of user experience questionnaires for different user groups Public knowledge of 'concussion' and the different terminology used to communicate about mild traumatic brain injury (MTBI) SPSS Statistics for Windows. IBM Corp User experience (re)design and evaluation of a self-guided, mobile health app for adolescents with mild Traumatic Brain Injury Website design and usability assessment implications from a usability study with visually impaired users A comparison of student and parent knowledge and perceived confidence about brain injury and concussion 22; peer-reviewed by R Lystad, A Almeida, B Nievas Soriano Self-confidence, and Usability Evaluation of a Web-Based Infographic for Pediatric Concussion: Multimethod Study JMIR Pediatr Parent 2022 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Pediatrics and Parenting, is properly cited. The complete bibliographic information | iss. 2 | e36317 | p LH was supported by a Canada Research Chair in Knowledge Synthesis and Translation and is a distinguished researcher with the Stollery Science Lab supported by the Stollery Children's Hospital Foundation. SDS was supported by a Canada Research Chair in Knowledge Translation for Children's Health and is a distinguished researcher with the Stollery Science Lab supported by the Stollery Children's Hospital Foundation. VP was supported by the Interdisciplinary Research Chair in Mental Health of Children and Young People, Université de Moncton. This project was supported by a Canadian Institutes of Health Research Foundation grant (RES0030460) awarded to SDS and LH. The authors would like to acknowledge Joud Eddin for her assistance with the development process of the knowledge translation tool and Hannah Brooks for her assistance with the revisions to this manuscript. None declared.Abbreviations ECHO: Evidence in Child Health to Enhance Outcomes KT: knowledge translation