key: cord-1041845-rrl7rymj authors: Thompson, Louisa I.; Harrington, Karra D.; Roque, Nelson; Strenger, Jennifer; Correia, Stephen; Jones, Richard N.; Salloway, Stephen; Sliwinski, Martin J. title: A highly feasible, reliable, and fully remote protocol for mobile app‐based cognitive assessment in cognitively healthy older adults date: 2022-04-05 journal: Alzheimers Dement (Amst) DOI: 10.1002/dad2.12283 sha: 6d55f53bb1b3caa2d6213ba56d2ab0fa67b09a47 doc_id: 1041845 cord_uid: rrl7rymj INTRODUCTION: The early detection of cognitive impairment is one of the most important challenges in Alzheimer's disease (AD) research. The use of brief, short‐term repeated test sessions via mobile app has demonstrated similar or better reliability and validity compared to standard in‐clinic assessments in adult samples. The present study examined adherence, acceptability, and reliability for a remote, app‐based cognitive screening protocol in healthy older adults. METHODS: Cognitively unimpaired older adults (N = 52, ages 60–80) completed three brief cognitive testing sessions per day within morning, afternoon, and evening time windows, for 8 consecutive days using a mobile app‐based cognitive testing platform. Cognitive tasks assessed visual working memory, processing speed, and episodic memory. RESULTS: Participants completed an average of 93% (M = 22.3 sessions, standard deviation = 10.2) of the 24 assigned sessions within 8 to 9 days. Average daily adherence ranged from 95% of sessions completed on day 2 to 88% of sessions completed on day 8. There was a statistically significant effect of session time on adherence between the morning and afternoon sessions only F (1, 51) = 9.15, P = .004, η (p ) (2 )= 0.152, with fewer afternoon sessions completed on average. The within‐person reliabilities of average scores, aggregated across all 24 sessions, were exceptionally high, ranging from 0.89 to 0.97. Performance on the episodic memory task was positively and significantly associated with total score and word list recall score on the Telephone Interview for Cognitive Status. In an exit survey, 65% of participants reported that they “definitely” would complete the sessions again. DISCUSSION: These findings suggests that remote, mobile app–based cognitive testing in short bursts is both highly feasible and reliable in a motivated sample of cognitively normal older adults. Limitations include the limited diversity and generalizability of the sample; this was a largely White, highly educated, and motivated sample self‐selected for AD research. sample; this was a largely White, highly educated, and motivated sample self-selected for AD research. Cognitive testing, digital tests, early detection, memory, monitoring, pre-clinical Alzheimer's disease, remote assessment, screening, smartphone app INTRODUCTION The early detection of cognitive impairment is one of the most important challenges in Alzheimer's disease (AD) research. 1 The SARS-CoV-2 (COVID-19) pandemic has highlighted potential advantages of remote methods of cognitive assessment. 2 App-based cognitive testing on mobile devices is rapidly becoming more feasible. The use of brief, short-term repeated test sessions via mobile app has demonstrated similar or better reliability and validity compared to standard in-clinic assessments in adult samples. 3, 4 Conducted during the 2020-2021 COVID-19 pandemic, the present study examined adherence, acceptability, and reliability for a remote, app-based cognitive screening protocol in healthy older adults. Our approach involved three brief assessments administered multiple times per day across several days. Participants were the initial 52 cognitively unimpaired adults ages 61 to 80 (71% female, 86% White) recruited for an ongoing study assessing novel cognitive screening methods. We recruited from the But- We selected M2C2 cognitive measures of visual working memory (WM), processing speed (PS), and episodic memory (EM) with prior theoretical and empirical support, including evidence of sensitivity to age and/or age-related neuropathology. Each task took approximately 60 seconds to complete. The Shopping-List task (EM) is a delayed forcedchoice recognition task, in which participants incidentally encode grocery item price pairs for later recall while judging whether the item's price is "good." 7, 8 Performance is summarized as the proportion of correct responses on recall trials. The Color Shapes task (WM) is a visual array change detection task, measuring intra-item feature binding, in which participants determine whether shapes change color across two sequential presentations in which the shape locations change. 9 Performance is summarized with the hit rate (proportion of correct identifications) and signal detection calculations. Specifically, the discriminability index (d') is the difference in the inverse cumulative standard normal distribution value for the hit rate (proportion of correct identifications) and false-alarm rate (proportion of misidentified stimuli). 10 The Symbol Match (PS) task is a speeded continuous performance task of conjunctive feature search, in which participants are asked to identify matching symbol pairs. 3, 11 Performance is summarized with reaction time to complete the task (milliseconds). Participants completed an average of 93% (M = 22.3 sessions, standard deviation = 10.2) of the 24 assigned sessions within 8 to 9 days. A minority (23/52) completed all 24 sessions, while a majority (46/52) completed at least 20. We detected no differences in adherence based on age, education, sex, or race/ethnicity. Average daily adherence ranged from 95% of sessions completed on day 2 to 88% of sessions completed on day 8 ( Figure 1A ). Twenty-one participants completed at least one optional additional session on day 9. On average, participants completed fewer afternoon than morning sessions (F[1, 51] = 9.15, P = .004, η p 2 = 0.152). In an exit survey, 65% of participants reported that they "definitely" would complete the sessions again, while only 4% said that they "definitely would not." Interestingly, 80% reported that they would at least "somewhat" consider using this app as part of an annual primary care cognitive screen. About 80% reported wanting to know their performance results. About 17% reported having to effortfully motivate themselves to complete the sessions. Only 4% said that it was difficult to complete the sessions during the time windows. Most participants (89%) reported that it was "very easy" to use the app and navigate the tasks. On average, participants ranked the Shopping List task as the most challenging and least favorite task. To evaluate test stability, we examined between-and within-person To assess convergent validity with an existing cognitive screening tool, List task showed minimal variability, with 70% to 80% mean accuracy across all days (Figures 1B and 1C) . On the Symbol Match task, reac- 1. Systematic Review: Prior work suggests that using a mobile app-based approach to collect repeated data on cognitive functioning in real-world settings is both reliable and valid compared to traditional cognitive screening approaches. However, the feasibility and reliability of this approach in older adults, including those with cerebral amyloidosis, is not yet widely studied. tion times improved successively over the first 4 to 5 days before leveling off ( Figure 1D ). Adherence to this 8-to 9-day, fully remote, mobile app-based cognitive testing protocol was very good, with 93% of assigned sessions completed. Overall, adherence was lower for afternoon sessions relative to morning and evening sessions. The reason for this is unclear, but participants' afternoon schedules and routines may have been more variable, increasing the likelihood of a time conflict or a forgotten session. The protocol was largely well tolerated by participants, with a majority reporting that they would consider completing the tasks again for research or as part of their own clinical care. The Shopping List task was rated as the least enjoyable measure, and on average, performance on this task did not improve over time. These results support the feasibility of remote app-based cognitive testing. It is notable that this study showed considerably higher assessment adherence than rates typically reported in the extant digital health assessment literature. 12 The reasons for our high adherence rate may be multifactorial: (1) our BAPR sample was self-selected and motivated to contribute to AD research; (2) many had previously and adherently participated in research at our site; (3) we provided detailed verbal, written, and visual aids for the M2C2 app, and (4) session time windows were tailored to participants' schedules as much as possible. The limited diversity or our largely White and highly educated sample might limit generalizability of the findings. Additionally, it is unclear how the contextual/historical considerations related to the COVID-19 pandemic enhanced or hampered the generalizability of our method and results. This research was supported by Alzheimer's Association grant AACSF-20-685786 to Brown University and National Institute on Aging Grant T32 AG049676 to The Pennsylvania State University. Stephen Salloway has been study site investigator for the aducanumab trials; received consulting fees from Bolden Therapeutics, Ono, Genentech, Biogen, Prothena, Alnylam, ATRI, Roche, and Mayo; received payment/honoraria for Grand Rounds lecture at VA Providence, CME Talk for GME, CME Talk for PlatformQ, CME Talk for WebMD Medscape, CME Talk for PER, Biogen Master Class Series, from Biogen for RSNA Symposium, AD Comm Adv Board, Webinar for SNMMI, Genentech -Adv Board, ATRI-IMPACT AD seminar, and CTAD. He received travel payment from Acumen -Sci Board, AMGEN, AAIC, GEMVAX, and AVID. All other authors no conflicts of interest. National plan to address Alzheimer's disease: 2019 update, ASPE Implementing remote memory clinics to enhance clinical care during and after COVID-19. Front Psychiatry Reliability and validity of ambulatory cognitive assessments Remote cognitive assessment approaches in the Dominantly Inherited Alzheimer Network (DIAN) The telephone interview for cognitive status The use of the modified telephone interview for cognitive status (TICS-M) in the detection of amnestic mild cognitive impairment Overdependence on degraded gist memory in Alzheimer's disease Adult age differences in memory performance: tests of an associative deficit hypothesis Visual short-term memory binding deficits in familial Alzheimer's disease Calculation of signal detection theory measures Are processing speed tasks biomarkers of cognitive aging? Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants A highly feasible, reliable, and fully remote protocol for mobile app-based cognitive assessment in cognitively healthy older adults