key: cord-0687739-sjmimcft
authors: Sullivan-Baca, Erin; Babicz, Michelle A; Choudhury, Tabina K; Miller, Brian I
title: The Relationship between Health Literacy and Comfort with Teleneuropsychology in a Veteran Sample
date: 2021-10-01
journal: Arch Clin Neuropsychol
DOI: 10.1093/arclin/acab079
sha: adc1d81185efd0f9655a40ddc87337596deec63b
doc_id: 687739
cord_uid: sjmimcft

OBJECTIVE: Health literacy is a strong psychosocial determinant of health disparities and has been found to relate to various aspects of health-related technology use. With the increased implementation of neuropsychological services performed via telehealth during the coronavirus disease 2019 pandemic, the relationship between health literacy and comfort with teleneuropsychology warrants further investigation. METHOD: The present study examined 77 Veterans participating in neuropsychological evaluations as a part of standard clinical care. The sample was diverse in terms of age ethnicity, and psychiatric and neurocognitive diagnoses. In addition to a fixed-flexible neuropsychological battery, the Brief Health Literacy Screening Tool (BRIEF) was administered to evaluate health literacy. Self-reported comfort with the teleneuropsychological evaluation was assessed using an informal 10-point scale, and qualitative comfort responses were also recorded. RESULTS: Independent samples t-tests revealed older adults were more likely to participate in the evaluation via telephone than with VA Video Connect. Although health literacy was not related to telehealth modality, it was correlated with comfort with the teleneuropsychological evaluation (r = .34, p < .01), although it is notable that average comfort levels were high across modalities (M = 8.16, SD = 2.50). CONCLUSIONS: Findings support the notion that teleneuropsychological services may feasibly be implemented with a diverse group of patients, although flexibility with modality may be necessary. Those performing these services should also be aware that patients with lower health literacy may feel less comfortable with teleneuropsychology as they seek to build rapport and optimize evaluation engagement.

In February 2020, the Centers for Disease Control and Prevention (CDC) issued guidance recommending that healthcare facilities offer clinical services through virtual means when possible in order to minimize risk of coronavirus disease 2019 (COVID-19) exposure for patients and healthcare providers (Koonin et al., 2020) . As over 77% of clinical neuropsychologists report spending professional time with older adults (i.e., >65 years old; Rabin, Paolillo, & Barr, 2016) at increased risk for complications from COVID-19 (CDC, 2020), many neuropsychology clinics have taken up this call to action and adopted telehealth practices in order to continue providing services to their patients during the pandemic (Guidotti et al., 2020; Marra, Hoelzle, Davis, & Schwartz, 2020a ). These initiatives have been supported and bolstered by structural and legislative changes, including the relaxation of temporary psychology licensure and service provision across state borders (American Psychological Association, 2020), as well as expansion of financial coverage for teleneuropsychology services by federal (U.S. Congress, 2020; Coronavirus Preparedness and Response Supplemental Appropriations Act) and private (Inter Organizational Practice Committee, 2020) insurance.

Teleneuropsychology in practice can vary widely in types of services provided (e.g., only intake interview and/or feedbacks vs. comprehensive neuropsychological assessments; Marra et al., 2020a) as well as delivery format (e.g., trained technician, assistant proctored, in-clinic hybrid, and direct-to-home models; . Research largely supports the utility of teleneuropsychology in assessing and treating older adults (see Marra, Hamlet, Bauer, & Bowers, 2020b for review); however, the majority of studies have had participants travel to local satellite clinics to complete the assessment where a technician configures the equipment for the patient and provides assistance during their televisit. Although prior neuropsychology protocols in practice primarily utilized this telehealth delivery format (e.g., Cullum, Hynan, Grosch, Parikh, & Weiner, 2014; Turner, Dorner, VanKirk, Myrick, & Tuerk, 2012) , COVID-19 social distancing guidelines and stayat-home orders have required neuropsychologists to rely more heavily on direct-to-home models of delivery (i.e., patients are assessed in their homes remotely by the clinician via telephone or televideo).

The COVID-19 pandemic required adaptability not only from providers, but also their patients, many of whom had little to no prior experience with receiving healthcare services via telehealth. The volume of reported telehealth service claims to private insurance companies during April 2020 increased by over 8000% compared with April 2019 (FAIR Health, 2020). Among Veterans treated through The Department of Veterans Affairs (VA), the largest healthcare system in the United States, there was 556% growth in telemental health encounters via video-conference in the first 6 weeks of the pandemic (March 11-April 22, 2020), and 77.5% of patients reported being first time users of the telemental health service platform (Connolly et al., in press) .

Although there is limited data available on the utilization of teleneuropsychology by patients, a recent survey of clinical neuropsychologists conducted in April 2020 revealed a stepwise increase in percentage of respondents who indicated that they (1) used teleneuropsychology prior to the pandemic for clinical interviews (24%), testing (11%), feedback (26%), and intervention (25%); (2) used teleneuropsychology currently in their clinic (52%, 15%, 41%, and 36%); and (3) intended to continue to use teleneuropsychology after COVID-19 (90%, 59%, 88%, and 70%; Hammers et al., 2020) . Taken together, these data suggest that the recent growth of teleneuropsychology services likely mirrors that of other telehealth services and will remain as a viable delivery format for neuropsychological services after the COVID-19 pandemic.

It is important to understand health literacy's potential role in successful utilization of teleneuropsychology services by patients as it is apparent that virtually provided healthcare will continue post-pandemic. Health literacy has emerged over the past 30 years as one of the strongest psychosocial determinants of health disparities by age, race/ethnicity, and socioeconomic status (see Mantwill, Monestel-Umaña, & Schulz, 2015 for review) . Health literacy is a multidimensional, dynamic construct, broadly defined as the degree to which individuals can obtain, process, understand, and communicate information about healthrelated information to make informed medical decisions (Berkman, David, & McCormick, 2010) . Adult health literacy in the United States is below the international average for literacy, numeracy, and problem solving in technology-rich environments (Goodman, Finnegan, Mohadjer, Krenzke, & Hogan, 2013) . Suboptimal health literacy is associated with poorer health outcomes (see Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011 for review), including reduced utilization of preventative medication (Cho, Lee, Arozullah, & Crittenden, 2008; White, Chen, & Atchison, 2008) , more emergency room visits (Baker, Gazmararian, & Williams, 2004; Cho, Lee, Arozullah, & Crittenden, 2008) , hospitalizations (Baker, Parker, Williams, & Clark, 1998; Baker et al., 2002) , and nonadherence to prescribed medications (Kripalani, Gatti,& Jacobsen, 2010; Murray et al., 2004) .

Older age is well established as a risk factor for lower health literacy both in general populations (Gazmararian et al., 1999; Kutner, Greenburg, Jin, & Paulsen, 2006; Paasche-Orlow, Parker, Gazmararian, Nielsen-Bohlman, & Rudd, 2005) and among older individuals (see Zamora & Clingerman, 2011 for review) . Lower neurocognitive functioning has also been associated with inadequate health literacy, both in healthy older adult populations (Federman, Sano, Wolf, Siu, & Halm, 2009 ) and among persons with HIV (Morgan et al., 2015) . As patients who present to neuropsychology clinics are commonly older adults and/or have neurocognitive complaints with medical co-morbidities, health literacy will be an important consideration in understanding patients' experience with teleneuropsychology.

To date, there have been a handful of studies examining the role of health literacy in internet access and health-related online navigation skills. In a study of 131 adults with low income, persons with lower numeracy skills were less likely to have access to internet technology at small-to-medium effect sizes (Jensen, King, Davis, & Guntzviller, 2010) . Another study of 14,102 participants with diabetes reported that persons with limited health literacy, as assessed and categorized by responses to the BRIEF Health Literacy Screening Tool, were less likely than those with adequate health literacy to register for access to an online health portal, log-on successfully to the portal, and carry out health activities on the portal (e.g., schedule an appointment, order refills, view labs, and send emails to providers; Sarkar et al., 2010) . Studies have also suggested that health literacy is associated with health-related online navigation skills in both the setting of examiner-created health platforms (e.g., a mock online pharmacy and a health records navigation tool; Woods et al., 2016) and during naturalistic searches on the internet (e.g., Previous work has demonstrated the role of health literacy in internet access and navigation of health-related online tools; however, little is known about how health literacy may contribute to one's ability to successfully utilize telehealth services (Gray, Joseph, & Olayiwola, 2020) . Similarly, although low health literacy has been discussed qualitatively as a factor to consider when conducting teleneuropsychology (Sherwood & MacDonald, 2020) , there have been no studies to date qualitatively examining the role of health literacy in patient experiences accessing and using teleneuropsychology services. Therefore, the aims of this study were threefold: (1) assess trends in the use of video versus telephone modalities for teleneuropsychology services;

(2) evaluate the comfort level of patients using video or telephone modalities for teleneuropsychology services; and (3) gain greater understanding of the role of health literacy in patients' access to and comfort levels with using teleneuropsychology services. We hypothesized that participants with lower levels of health literacy would report greater discomfort with using teleneuropsychology services, and that factors that have been established as relating to health literacy, including age, education level, and neurocognitive diagnosis, would influence this relationship.

Data from this Institutional Review Board (IRB)-approved, cross-sectional study were gathered from 86 Veterans seen for standard outpatient neuropsychological telehealth evaluation at a large VA Medical Center between April and November 2020. Of those 86 Veterans, nine were excluded from the study because they did not complete all primary outcome measures, resulting in a final sample of 77 participants. A total of 40 participants (52%) were assessed using VA Video Connect (VVC), and 37 participants (48%) were assessed over the telephone. English was the dominant language for all patients, and each assessment was performed in English.

Demographically, the sample skewed male (n = 68, 88%) and White (n = 36, 47%), generally consistent with the broader demographic makeup of patients at the VA medical center where data was collected. Other races represented included African American (n = 27, 35%), Hispanic/Latinx (n = 10, 13%), and Native American (n = 1, 1%). One participant identified as "other" (1%) and two participants did not identify their race (3%). Participants' ages ranged from 27 to 91 years old, with a mean age of 57.14 years (SD = 16.33). Education level ranged from 7 to 20 years, with a mean education level of 13.66 years (SD = 2.26). For other demographic details, including relationship status, living arrangement, and employment status, see Table 1a . Psychiatric and neurocognitive diagnoses can be found in Table 1b .

Neuropsychological assessment battery. All participants were administered a 60-90 min neuropsychological assessment battery as part of standard care. Although a fixed-flexible approach was used depending on the individual clinic where the Veteran (Randolph, 1998) , Independent Living Scale (Loeb, 1996) select subtests (Money Management, Health and Safety), Wide Range Achievement Test-Fourth Edition (Wilkinson & Robertson, 2006) select subtests (Reading), Multilingual Aphasia Examination-Third Edition (Benton, Hamsher, & Sivan, 1989 ) select subtests (Sentence Repetition), Boston Diagnostic Aphasia Examination (Goodglass & Kaplan, 1983 ) select subtests (Complex Ideational Material), Rey Complex Figure Test (Osterrieth, 1944; Rey, 1941) In addition to cognitive testing, participants also completed at least one self-report screening measure of emotional functioning from the following: Geriatric Depression Scale (Yesavage et al., 1982) , Geriatric Anxiety Inventory (Pachana et al., 2007) , Generalized Anxiety Disorder-7 (Spitzer, Kroenke, Williams, & Löwe, 2006) , Patient Health Questionnaire-9 (Kroenke, Spitzer, & Williams, 2001) , Posttraumatic Stress Disorder (PTSD) Checklist for Diagnostic and Statistical Manual of Mental Disorders -Fifth Edition (DSM-5) (Weathers et al., 2013) , and the Neurobehavioral Symptom Inventory (Cicerone & Kalmar, 1995) .

Brief Health Literacy Screening Tool (BRIEF). To broadly assess participants' baseline health literacy, the Brief Health Literacy Screening Tool (BRIEF; Haun et al., 2009 ) was administered. The BRIEF was verbally administered and includes four questions:

(1) "How often do you have someone help you read hospital materials"? (2) "How confident are you filling out medical forms by yourself"? (3) "How often do you have problems learning about your medical condition because of difficulty understanding written information"?, and (4) "How often do you have a problem understanding what is told to you about your medical condition"? For questions 1, 3, and 4, participants respond using the following 5-point Likert scale: 1 = always, 2 = often, 3 = sometimes, 4 = occasionally, and 5 = never. For question 2, the following 5-point Likert scale is used: 1 = not at all, 2 = a little bit, 3 = somewhat, 4 = quite a bit, and 5 = extremely.

BRIEF total scores range from 4 to 20. Scores of 4-12 are considered reflective of "inadequate" health literacy. Scores of 13-16 are considered "marginal," and scores of 17-20 are considered "adequate." In the BRIEF pilot study, the measure demonstrated 0.79 sensitivity (95% confidence interval [CI], 0.70-0.87) for detecting inadequate health literacy skills and 0.69 sensitivity (95% CI, 0.64-0.75) for detecting inadequate/marginal skills (Haun et al., 2009) . Internal consistency in the present study was considered good (α = 0.82). In the current study, health literacy was examined as both a linear (i.e., BRIEF total score) and categorical variable (i.e., inadequate vs. marginal vs. adequate health literacy, meeting of statement).

Telehealth comfort questions. After completion of cognitive testing and the BRIEF, participants were also asked questions about their overall sense of comfort with engaging in telehealth services. Participants were asked, "On a scale from 1-10, how comfortable were you doing your appointment via phone/video"?, where 1 = not at all comfortable and 10 = completely comfortable. Notably, this rating scale was created for the present study as a simple tool to assess comfort level. It has not been formally validated, and thus cut scores are not available to divide responses into categories. Participants were also asked the following free response question: "Was there anything that could have made your phone/video experience better"?, and qualitative responses were recorded.

All analyses were conducted using IBM SPSS Statistics (Version 23.0, released 2015) . To address the first study aim, independent samples t-tests were conducted to compare (1) telehealth comfort ratings and (2) total BRIEF score in the VVC versus telephone assessment groups. To evaluate the second and third study aims, Pearson product-moment correlation coefficients were calculated to assess the relationships between participants' age, education, BRIEF total score, and telehealth comfort ratings. Additional independent samples t-tests were conducted to assess whether comfort level differed between groups with and without examiner-recorded factors affecting the evaluation (i.e., hearing difficulty, poor internet connection, etc.), whether comfort level differed between those providing negative or neutral/positive qualitative feedback, and whether those with or without neurocognitive diagnoses evidenced differences in health literacy score or assessment modality. Finally, a hierarchical linear regression was performed to evaluate whether health literacy accounted for differences in comfort level above and beyond other variables. In this regression, comfort level was the independent variable, whereas the variables age, education, neurocognitive diagnosis, and assessment interference represented Model 1 predictors. Health literacy total score was added as Model 2.

Because it has been proposed that those with the lowest level of health literacy, commonly described as having "limited" or "inadequate" literacy, are often most negatively affected in their interactions with the healthcare system compared with individuals with marginal or adequate literacy levels (Sarkar et al., 2010) , the data was then approached categorically, with total BRIEF scores divided into the three categories described previously (i.e., inadequate, marginal, and adequate). A "Welch's" F test was performed to assess differences in comfort between these health literacy levels. A "Welch's" F test was chosen as preferable to a traditional one-way analysis of variance based on a significant "Levene's" F test result, F(2,74) = 5.86, p < .01, which indicated the homogeneity of variance assumption was not met. Effect size is reported as ω 2 with the following descriptive levels: small = 0.01 ≤ ω 2 < 0.06, medium = 0.06 ≤ ω 2 < 0.14, and large = 0.14 ≤ ω 2 . A post hoc Games Howell procedure was then used to determine differences between the individual health literacy levels.

External factors that may have affected the neuropsychological evaluation were recorded by each examiner. These factors included the patient having hearing difficulty (n = 8), reduced sound quality (n = 4), unstable internet connection (n = 3), the call being disconnected (n = 3), and difficulty understanding the patient (n = 5). As would be expected by some underlying commonalities between these factors, there was within-session overlap between these factors, such that 16 total patient assessments were affected by one or more factor. Those who did and did not have examiner-reported interference with the evaluation did not differ in terms of self-reported comfort level (t(75) = 0.64, p = .52, d = 0.17).

Responses to a comfort item spanned the full scale from 1 to 10, but were on average moderately high (M = 8.16, SD = 2.50). Qualitative responses were also recorded. Although only 17 of 77 patients (22%) provided negatively valenced qualitative feedback, concerns about the evaluation included a poor signal/connection (n = 5), difficulty hearing (n = 4), preference for in-person appointment versus telephone or video appointment (n = 4), preference for video over phone appointment (n = 1), and concerns about the assessment length (n = 3). As would be expected, those who provided negative qualitative feedback about their assessment experience had lower comfort scores than those who provided no feedback or positive feedback (Comfort M = 6.88 and M = 8.47; t(75) = 2.35, p < .05, d = 0.63).

A total of 40 patients completed the neuropsychological evaluation via the VVC video platform, whereas 37 participated using a telephone. Demographically, those participating in the neuropsychological evaluation via telephone (Age M = 66.73, 

BRIEF total score was correlated with the teleneuropsychology comfort item response (r = .34, p < .01). Age, which was hypothesized to be a potential moderating factor, was neither significantly correlated with BRIEF score (r = .02, p = .89), nor was it correlated with the comfort item response (r = .04, p = .76). Similarly, education level was neither correlated with BRIEF score (r = .11, p = .34) nor comfort response (r = .04, p = .75). Health literacy score (t(73) = 1.91, p = .06, d = 0.50) and comfort rating (t(73) = 0.13, p = .21, d = 0.31) did not differ between those with and without a neurocognitive diagnosis.

To further explore the relationship between health literacy and comfort level, total scores were divided into established qualitative categories (i.e., inadequate, marginal, and adequate). In total, 22 participants (28.6%) endorsed inadequate health literacy, 16 (20.8%) indicated marginal health literacy, and 39 (50.6%) evidenced adequate health literacy. Comfort level response for each group is reported in Table 2 . "Welch's" F test revealed there was a significant effect of health literacy level on comfort response (F(2, 30.84) = 4.18, p < .05, ω 2 = 0.07).

Post hoc analysis indicated significant differences in comfort level between those with inadequate and marginal health literacy and those with inadequate and adequate health literacy, all at the p < .05 level (d = 0.85 and d = 0.79, respectively). Comfort level did not differ between marginal and adequate health literacy groups (p = .86, d = 0.17). Hierarchical linear regression predicting comfort level included age, education level, presence of factors affecting the evaluation, the presence of a neurocognitive disorder in Model 1 and the addition of health literacy (BRIEF Total Score) in Model 2. Model 1 did not contribute to a significant amount of variance in comfort level (R 2 = .24); in addition, none of the individual factors were significantly contributory. The addition of health literacy in Model 2 contributed to an additional 10.4% of the variation, representing a significant change in R 2 (F(1,74) = 8.19, p < .01).

Although neuropsychological services performed via telehealth have increased over the past decade, significant variability has existed in which services are performed and how they are implemented (Marra et al., 2020a) . Beginning in 2020, the COVID-19 pandemic has caused a sudden and vast increase in the need for neuropsychological evaluations to be implemented remotely, often directly into patients' homes via telephone or video telehealth platforms (Guidotti et al., 2020; Hammers et al., 2020) . Health literacy, which is a strong predictor of health disparities, has been associated with reduced access to internet technology and increased difficulty navigating online health portals (Jensen et al., 2010; Sarkar et al., 2010) ; however, it has not yet been examined in relation to teleneuropsychological evaluations. Given that neuropsychologists commonly work with patients at increased risk for inadequate health literacy (e.g., older adults and those with neurocognitive concerns or diagnoses), this area warrants further consideration.

The primary findings from the present study reveal that those who rate their health literacy as lower also tend to report lower comfort level with teleneuropsychology, irrespective of modality (e.g., phone or video) and that health literacy predicted user comfort above and beyond demographic or assessment-related factors. Although this linear relationship suggests that anyone with lower health literacy ratings may be at risk for lower evaluation comfort, further categorical analyses indicated that those with marginal and adequate health literacy were similarly comfortable with the evaluation, but these groups differed from those with inadequate health literacy at a large and medium effect size, respectively. This echoes the relationship seen between limited/inadequate health literacy and other negative telehealthcare outcomes, such as difficulty navigating online patient portals (Sarkar et al., 2010) .

When considered in the context of the specific health literacy questions asked on the BRIEF health literacy screen, this suggests that those who have the most difficulty navigating areas of healthcare, such as reading hospital materials, filling out medical forms, understanding written information, and understanding verbally presented information, may also experience discomfort when participating in a teleneuropsychological evaluation. Because comfort may play a role in such crucial aspects of the evaluation as rapport building during the clinical interview, engagement with standardized testing and reception to feedback (AACN Board of Directors, 2007; Kapur & Kemp, 2016) , understanding this relationship, along with ways to potentially intervene with those who have reduced health literacy to best promote comfort in the evaluation, may be important steps forward in the emerging field of teleneuropsychology.

Encouragingly, the present sample of Veterans evaluated by a neuropsychology outpatient clinic via telehealth was diverse in terms of ethnicity, age, and both neurocognitive and psychiatric diagnoses, suggesting that teleneuropsychological evaluations can be implemented across a range of individuals with unique presenting problems and needs. In fact, teleneuropsychology, like other forms of telehealth, may increase accessibility to services for individuals who may otherwise struggle to attend inperson evaluations due to physical/mobility issues, transportation issues, or other socioeconomic barriers (Elnitsky et al., 2013; Zulman et al., 2019) . Along with usability, self-reported comfort level was also rated as generally high, which is an encouraging finding clinically. However, statistically it led to reduced variety in the sample, which somewhat limits the ability to understand individuals who have low levels of comfort. Qualitative responses revealed that lower comfort scores were associated with concerns about modality, assessment length, and technology issues. Although future studies evaluating individuals with greater levels of hesitation or discomfort in neuropsychological evaluations may be helpful to broaden the implication of results, these areas provide an early roadmap for possible evaluation pitfalls and areas for future intervention.

Although overall ability to participate in the evaluation and comfort in doing so was generally high, differences existed in the technology used for assessments, with older individuals more often assessed via telephone than video at a large effect size. This finding was consistent with literature on older adults' access to telehealth technology (Lam, Lu, Shi, & Covinsky, 2020) , which suggests that older adults have lower access to telehealth technology and more difficulty navigating telehealth, and has notable implications for implementation of teleneuropsychological services. Contrary to expectations, there were no differences in health literacy for those who preferred telephone versus video teleneuropsychology delivery format. Although there are several verbally mediated tasks of orientation (i.e., Mini Mental State Examination (MMSE) Orientation Items, Montreal Cognitive Assessment (MoCA) Blind), attention (i.e., WAIS-IV Digit Span, Arithmetic), language (i.e., COWAT, Animals, WAIS-IV Verbal Comprehension Index (VCI) Subtests), and memory (i.e., CVLT-II, RAVLT, WMS-IV Logical Memory) that can be performed via telephone, assessments of other domains, particularly visuospatial functioning or visual memory, are precluded from this form of assessment. Because deficits in these areas represent hallmark features of various syndromes and neurodegenerative disorders (e.g., vascular dementia, Lewy Body dementia), telephone assessments that omit these measures may not yield an accurate diagnostic impression of the patient. This is especially concerning in the context of a greater number of individuals with neurocognitive diagnoses in the present sample participating via telephone versus video. Given these limitations, future research may wish to further explore factors that contribute to older adults' utilization of the telephone rather than video for teleneuropsychology services to improve access to video evaluations for older adults who are at the greatest risk of neurocognitive disorder and/or decline.

It is notable that, contrary to predictions, age, education level, and neurocognitive diagnosis were not associated with health literacy report or comfort using teleneuropsychology. Although this finding promotes the idea that those presenting across a range of demographic factors may all be able to participate effectively in a teleneuropsychological appointment and feel comfortable doing so, it does leave open questions as to what demographic or modifiable factors may play a role in the perhaps complex relationship between health literacy and teleneuropsychology comfort. In particular, future studies may wish to explore socioeconomic status, which was not assessed in the present study, but has been shown to be a strong predictor of health literacy (Stormacq, Van den Broucke, & Wosinski, 2019) .

Although the present study offers important insight into the relationship between health literacy, technology usage, and comfort with teleneuropsychological assessment, there are factors that may somewhat limit generalizability of results. Due to variability in the modality of assessment between telephone and video evaluations, a self-report measure of health literacy was the most feasible option to evaluate this construct. Although this measure has been established as a valid assessment of health literacy (Haun et al., 2009) , primary analyses ultimately compared two self-report responses (i.e., BRIEF and comfort questions), Thus, it is possible that a positive or negative response bias for some patients may have influenced results. Had a visual medium been present for all participants, an alternative measure of health literacy, such as the Test of Functional Health Literacy in Adults (Parker, Baker, Williams, and Nurss, 1995) or the Rapid Estimate of Adult Literacy in Medicine (Davis et al., 1991) , may have been preferred to eliminate this potential bias, and thus future studies may consider implementing such a design.

An additional future direction involves the collection of collateral informant report. Collateral informants can provide important information about an individual's level of cognitive and functional ability, especially in cases where patients are at risk for reduced insight, including in those individuals with neurocognitive disorders. It is notable that in the present study individuals with neurocognitive disorders were more likely to be evaluated via telephone. It is possible that these individuals also had lower insight into their health literacy levels, which could have tempered differences between telephone and video telehealth groups. To better account for patients with potentially reduced insight, future studies may wish to fortify self-reported health literacy measures with informant report.

Finally, data could only be collected from patients who were able to participate in teleneuropsychological evaluations, whether via phone or video. Thus, individuals with significant access limitations that precluded assessment altogether could not be characterized. Future studies may wish to explore what percentage of our patients may be affected in such a way, and how these severe access limitations may relate to constructs like health literacy.

In conclusion, findings from a diverse Veteran sample suggest individuals spanning different ages, educational backgrounds, and neurocognitive abilities may be able to effectively participate in neuropsychological evaluations performed via telephone or video, with a moderately high level of comfort. Older individuals may choose or only have access to telephone rather than video-based assessments, a finding worthy of further investigation and intervention, as visually based information can be a crucial component of neuropsychological batteries. Those with lower self-reported health literacy may feel less comfortable with video-and telephone-based neuropsychological assessments, which has implications for rapport and engagement, and also warrants further consideration as the field of teleneuropsychology continues to develop throughout and beyond the present COVID-19 pandemic.

None declared.

American Academy of Clinical Neuropsychology (AACN) practice guidelines for neuropsychological assessment and consultation

It's got to be on this page": Age and cognitive style in a study of online health information seeking

Neuropsychological and psychological testing during COVID-19

Functional health literacy and the risk of hospital admission among Medicare managed care enrollees

Health literacy and use of outpatient physician services by Medicare managed care enrollees

Health literacy and the risk of hospital admission

Multilingual aphasia examination

Health literacy: What is it

Low health literacy and health outcomes: An updated systematic review

Hopkins Verbal Learning Test-Revised. Professional manual. Lutz, FL: Psychological Assessment Resources

Effects of health literacy on health status and health service utilization amongst the elderly

Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury

Rapid increase in telemental health within the Department of Veterans Affairs during the COVID-19 pandemic

Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment

Rapid assessment of literacy levels of adult primary care patients

Delis-Kaplan executive function system

California Verbal Learning Test

Access to the US Department of Veterans Affairs health system: Self-reported barriers to care among returnees of Operations Enduring Freedom and Iraqi Freedom

Monthly Telehealth Regional Tracker

Health literacy and cognitive performance in older adults

Health literacy among Medicare enrollees in a managed care organization

Self-ratings of spoken language dominance: A multi-lingual naming test (MINT) and preliminary norms for young and aging Spanish-English bilinguals

The assessment of aphasia and related disorders

Literacy, numeracy, and problem solving in technology-rich environments among US adults: Results from the Program for the International Assessment of Adult Competencies 2012. First Look. NCES 2014-008

Strategies for digital care of vulnerable patients in a COVID-19 world-Keeping in touch

2020 COVID-19 American Academy of clinical neuropsychology (AACN) student affairs committee survey of neuropsychology trainees

A survey of international clinical teleneuropsychology service provision prior to and in the context of COVID 19

Testing the BRIEF health literacy screening tool

State by state tele-neuropsychology resources and reimbursement

Utilization of internet technology by low-income adults: The role of health literacy, health numeracy, and computer assistance

Neuropsychological assessment: Principles, pearls and perils

Trends in the use of telehealth during the emergence of the COVID-19 pandemic -United States

Association of age, health literacy, and medication management strategies with cardiovascular medication adherence

The PHQ-9: Validity of a brief depression severity measure

The health literacy of America's adults: Results from the 2003 National Assessment of Adult Literacy. NCES 2006-483

Assessing telemedicine unreadiness among older adults in the United States during the COVID-19 pandemic

Independent Living Scales (ILS)

The relationship between health literacy and health disparities: A systematic review

Initial changes in neuropsychologists clinical practice during the COVID-19 pandemic: A survey study

Validity of teleneuropsychology for older adults in response to COVID-19: A systematic and critical review

Neurocognitive impairment is associated with lower health literacy among persons living with HIV infection

A conceptual framework to study medication adherence in older adults

Le test de copie d'une figure complexe

The prevalence of limited health literacy

Development and validation of the Geriatric Anxiety Inventory

The test of functional health literacy in adults: A new instrument for measuring patients' literacy skills

Stability in test-usage practices of clinical neuropsychologists in the United States and Canada over a 10-year period: A follow-up survey of INS and NAN members

Repeatable battery for the assessment of neuropsychological status (RBANS)

L'examen psychologique dans les cas d'encéphalopathie traumatique. (Les problems.) [The psychological examination in cases of traumatic encephalopathy

L'examen Clinique En Psychologie

Analysis of an oral paradigm for the Trail Making Test

The literacy divide: Health literacy and the use of an internet-based patient portal in an integrated health system-Results from the Diabetes Study of Northern California (DISTANCE)

A teleneuropsychology consultation service model for children with neurodevelopmental and acquired disorders residing in rural state regions

A brief measure for assessing generalized anxiety disorder: The GAD-7

Teleneuropsychology (TeleNP) in response to COVID-19: Practical guidelines to balancing validity concerns with clinical need

Does health literacy mediate the relationship between socioeconomic status and health disparities? Integrative review

A pilot trial of neuropsychological evaluations conducted via telemedicine in the Veterans Health Administration

The PTSD Checklist for DSM-5 (PCL-5)

Wechsler Adult Intelligence Scale

Wechsler Memory Scale

Relationship of preventive health practices and health literacy: A national study

Wide Range Achievement Test

Health-related everyday functioning in the internet age: HIV associated neurocognitive disorders disrupt online pharmacy and health chart navigation skills

Development and validation of a geriatric depression screening scale: A preliminary report

Health literacy among older adults: A systematic literature review

Making connections: Nationwide implementation of video telehealth tablets to address access barriers in veterans

None declared.