key: cord-0964838-74z6xoa9 authors: Heyn, Patricia C; Goldberg, Aleaza; McGrew, Greg; Bodine, Cathy title: The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study date: 2021-05-04 journal: J Rehabil Assist Technol Eng DOI: 10.1177/20556683211009731 sha: 21dbd990b556a0d71b0279fefc4a640ba03c2a42 doc_id: 964838 cord_uid: 74z6xoa9 We designed a feasibility study to evaluate a mobile-based vocational skill building coaching technology (aka Mobile Coach) intervention by using an ecological design approach. We compared the Mobile Coach to a standard job coach (no Mobile Coach technology) assistance in a facility that employs adults with significant cognitive disabilities (CDs). Twenty working-age adults with CDs were enrolled in this feasibility study and were asked to use the Vocational Mobile Coach Technology (on an iPad) to assist with their daily job functions. Project-specific usability and self-satisfaction survey was used to evaluate the user experience in performing the selected work assembly tasks with the Mobile Coach and without it. This report has the goal to describe our feasibility study design, methods, and results. The World Health Organization (WHO) estimates that 15 percent of the world's population lives with some form of disability and almost 4 percent of them are 15-year-old and older requiring specialized services and care to function in our society. 1 The number of people with disabilities who experience significant difficulties in functioning is increasing, in part due to ageing populations as well as an increase in chronic health conditions worldwide. 1, 2 Often, people with disability experience poorer health outcomes, have less access to education and work opportunities, and are more likely to live in poverty than those without a disability (WHO). 1 Therefore, the ability to work and succeed in maintaining employment status during working age is extremely important for people with disabilities. In 2021, it will be the 31st Anniversary of the Americans with Disabilities Act (ADA) that prohibits employment discrimination against people with disabilities. 3 Despite the signing of the ADA, people with disabilities in the United States continue to experience disproportionately lower rates of employment and significantly lower wages than people without disabilities and only one in three individuals with disabilities are employed. 2 Although the trends in the employmentpopulation ratio and the labour force participation rate have overall improved in the United States for people with disabilities, these indicators continue to fall behind those for people without disabilities. 4 In 2019, the employment-population ratio of Americans with disabilities was 19.3 percent, compared to 66.3 percent for people without disabilities. 5 Moreover, when people with disability were employed, 82 percent were either under-employed or worked part-time. 5 In 2020, due the COVID-19 health crisis, there was a significant increase in the unemployment rate for people with disabilities. 6 These employment disparities are even worse for people with Cognitive Disabilities (CDs) who usually experience cognitive function challenges with tasks that required orientation, attention, learning, reading, memorizing, speaking, and social skills. Unfortunately, these tasks are critical for good workplace performance. 7 For occupational activities that require planning, memorizing tasks, and locations, such as custodial duties, mail delivery, grounds maintenance, and assembly warehouse errands, the inability to memorize task order and places are the primary barrier for individuals with CDS to attain to a productive, safe, and enjoyable work environment. 8, 9 Technology trends and demographics support the development of mobile technology applications for the working needs of adults with CDs. [8] [9] [10] Mobile technology can be described as any device with internet capability that is accessible from anywhere the user is, including smartphones and tablets. 11 In 2019 in U.S.A, over 90% of Americans own a mobile technology of some kind (i.e., smartphones) for emailing, instant-messaging, or information-seeking. 12 The benefits, ease of access, and ubiquity of online and mobile internet solutions make these mobile platforms attractive for the use and assistive technology needs of individuals with CDs. 13 A Mobile Internet Device (MID) is described as a handheld device that is portable, can access the internet, and has down-graded computing capability compared to other portable computing devices like laptops. 11 MIDs have extended multimedia capabilities for handheld devices and are generally the size and shape of a typical tablet. MIDs are not used with the intention to replace mobile or smart phones and are designed to be used as companion devices. 12 In addition, handheld devices can be especially helpful for individuals with physical and mental frailty. 13, 14 Mobile technology can be designed to be easily accessible, providing an opportunity to assist in occupational and training settings for individuals with CDs. 15 Therefore, we designed a novel Mobile Skills Vocational Coach or Mobile Coach Technology (MCT) device aimed at assisting in job training for individuals with CDs. The purpose of this pilot study was to evaluate the feasibility of the MCT in an ecological environment (warehouse) that employs adults with significant cognitive impairments. More specifically, we want to assess whether working-age adults with CDs who used the MCT (on an iPad) will show higher satisfaction in performing the selected work assembly tasks (i.e., easier and better to do the task) as compared to a non-MCT group. This study is part of a series of development and research projects from the Rehabilitation Engineering Research Center for the Advancement of Cognitive Technologies (RERC-ACT). 16 The engineering of the MCT device was based on a series of iterative design research processes prior to this feasibility pilot study. 16 The iterative design was used to inform the protocol and consisted of several visits and observations in the real-world environment. Job coaches for people with CDs participated in the development of the protocol. Therefore, we are now reporting the results of the feasibility pilot study that used an ecological approach to evaluate the MCT for working age adults with CDs. The study protocol was approved by the Colorado Multiple Institutional Review Board Ethics Committee and followed the principles embodied in the Declaration of Helsinki. Working-age individuals with CDs from the Goodwill Industries Workday Program, ages 18-64, who met the criteria for Disability Benefits accordingly to the United States of America Social Security Administration 17,18 and by their state of residence, 18, 19 were invited to participate in a single testing session of a technology-based mobile context-aware prompting system technology. The technology was designed to coach and aid individuals with Intellectual and Developmental Disabilities (IDD) in performing assembly tasks. 8, 10 All participants, or their legalized authorized representative (LAR), signed an informed consent form and completed a demographics survey. The survey collected information about education level, disability type and living arrangements. For the purpose of this study, participants were only included in the study if they could express some sort of communication, either orally or by using a communication device and able to move objects (cases/boxes) from one room to another. The MCT is a prompting tool run on an iPad intended to assist individuals with cognitive impairments to work more effectively within jobs that require them to move from one location to another (mobile job). Mobile jobs are defined as jobs where the user is required to walk a distance to perform the job (i.e., warehouse workers; custodians; grounds keeper; etc.). 9, 15 The MCT is a cloud-based application that prompts the users through a series of tasks required to complete a specific job, like stock a box on a shelf. The tasks for the mobile job required navigation, object recognition, object transportation, and task sequencing. Figure 1 represents the architecture for the MCT system. The MCT is a heuristic model that could assimilate a current activity, past work performed, and determine the next activity that should be performed. A Task Server received an incoming event and determined the next, best action to perform. The results were delivered to a Data Presentation Layer, the mobile worker's prompting device (iPad) via the cloud. For this study, the Task Server was configured to recognize time events and gesture events. Navigational events were considered during the design; however, the performance and precision of the chosen technology was not sufficient to support this interface. The MCT has a touch-based screen (iPad) that has a virtual agent (job coach) that introduces instructions related to tasks. If the participant did not understand the instruction or forgot the step, the system would re-introduce the information when participants touch the screen. Ecological environment. The Mobile Coach Project team worked in conjunction with Goodwill Industries of Denver Job Day Program. Goodwill Industries has a contract with Century Link, where they supplement the packaging of residential telecommunication equipment (e.g., DSL modem). Goodwill Industries receives the equipment on a pallet. Depending on the type of residential telecommunication equipment, each pallet contains 12-24 cases, and each case contains 10-14 residential devices. Once delivered, these pallets are temporarily stored in the Goodwill Industries warehouse. The warehouse and the assembly area are adjoining rooms. The approximate distance from the pallets to the assembly pods range between 50 to 100 feet. Intervention. Enrolled participants from the Goodwill Denver Vocational Training Day Program had the opportunity to volunteer or decline participation in the study. Consenting procedures were performed for each participant, and their understanding about the study was evaluated by asking specific questions about the study after consenting explanation, such as "what is this study about?" or "if you agree to participate what do you think that you will do in this study?" Demographic information was obtained from each participant. Enrolled participants' level of comprehension and understanding was acquired by using the Mini Mental State Exam. 20 The intervention was based on one four-hour session. Participants were assigned to one of two groups, one that used the MCT or one that used a standard vocational coach. The MCT is a technology-based mobile context-aware prompting system (CAP) designed to job coach, train, and aid work performance in adults with significant cognitive impairments ( Figure 1 ). The standard vocational intervention was job coaching performed by a human with no technology. Both groups were exposed to similar procedures while performing the assembly task job despite differences in the delivery of job coaching. MCT device tasks. The job performed by the mobile worker included six high level tasks: 1. Unwrap new pallet; 2. Get new case/box; 3. Deliver new case/box to assembler (mobile task); 4. Retrieve completed case/box from assembler; 5. Place completed case/box on a pallet in the storage room (mobile task); 6. Secure completed pallet. Completing each of these tasks required one or more prompts. The overall sets of prompting were a collection of sequential steps and dual path decision points. Sequential steps were incremented either via a timer event expiring or a gesture performed by the user on the MCT. All decision points required the user to perform the correct gesture. User usability data. A project-specific survey was designed to evaluate usability and participants' satisfaction with the intervention. The survey had questions related to their experience with the coaching intervention with a Likert scale response level of 1 to 5, 1 meaning "Not at all" to 5 meaning "Very Much". A sample of how this scale was designed for adults with CDs can be seen in Figure 2 . A total of 20 participants (5 females and 15 males) with CDs was consented and enrolled in the study. Participants' disabilities ranged from a variety of cognitive impairments, including Down syndrome, Autism, and brain injury. The sample was comprised of 65% white, 20% Hispanic and 15% African American ( Table 1) . The MCT group (N ¼ 10) reported higher satisfaction with the intervention as compared to the non-MCT group. Approximately 45-100% of the MCT responses collapsed into the "Very Much" reporting level category on questions related to work enjoyment, ease of work, use of the technology, and if the MCT was helped in doing the job better. There were also questions related to adherence, such as "Would you like to use the MCT again?" ( Table 2) . The standard human job coach group (N ¼ 10; non-MCT) also expressed positive satisfaction with their standard job and in having the facility's job coach assistance while performing their tasks. However, their response levels were lower when compared to the MCT group satisfaction at 35-75% responded "somewhat" to "very much" satisfaction with the human job coach. The results of the survey indicate that there were differences in the responses between groups, and the integration of an MCT device was well received by the workers with CDs. Most of the participants were extremely excited and eager to participate in the study, as most of them wanted to use the technology. To accommodate the high motivation to use the technology, we provided time to "play" with the technology for the group that was not assigned to the MCT after they completed the experimental procedures. There were no differences in answers between the participants who were already competent with computers or iPads versus those with no previous experience with computer or tablet technology. In addition, 25% of the sample were illiterate, and literacy levels did not influence performance when using the MCT. MID technologies have improved over the years and are more affordable and easily available than years ago. 21, 22 Smart technologies, particularly smartphones, are leading the market for mobile technologies and have the potential to be a critical assistive technology device to assist the employment needs of individuals with a variety of cognitive impairments. 8, 23 This technology is designed to be easy to use, as it can be manipulated by touch or voice command. 13 The ease of access and use of the mobile solution technologies make MIDs an attractive platform for helping individuals with cognitive impairments to perform tasks that require prompting assistance, such as an assembly job assignment. 9, 10, 21 The results of this pilot study support the notion that MIDs and related technologies can be easily engineered and adapted to suit the user needs. They can present an effective, accessible, and integrated system to meet the assistive occupational needs of individuals with CDs. These technologies should also be affordable and tailored accordingly to the person's physical and cognitive abilities. 21 Mobile technologies are recognized as flexible and powerful tools that can enhance human performance if the technology features are designed with the user in mind. 22 For individuals with disabilities the habitual involvement with these devices may have a positive and lasting impact on users' ability to think, remember, pay attention, regulate emotion, and communicate. 23 These technologies, if properly engineered, can have a powerful impact in the quality of life of a person with a disability by decreasing disability stigma and increasing socialization, societal engagement, and self-esteem by empowering the person's abilities. 24 Although, the study results support the feasibility of the MCT as an innovative employment solution for people with CDs the study also had several limitations such as sporadic MCT malfunction during the experiment. However, this occurred only during one trial and we only had 2 episodes of network malfunctioning during the trial day. This technology malfunction was carefully recorded and after the system was restored and functioning, the timer and data collection system were re-started, and the trial was resumed. Another limitation was the short time that each participant was exposed and used the MCT (4 hours). Future studies could provide more detailed information related to the technology adoption and adherence if workers are exposed to the intervention longer. Another limitation of this study was the ecological environment. Since the intention of the study was to test the technology in a real-world setting, we had to deal with non-controllable factors that a real working warehouse presents, such as constant loud noise and other unexpected distractions. On two occasions, we had to postpone a planned visit to the facility due the absence. In addition, we had to deal with continuous stops during the experiment due to breaks and lunch times as part of the worker schedule. Although this study was designed as a feasibility study, the small sample size limits the generalizability of our results. After the study implementation phase, we also learned that the vocational coaches could have been important participants in the study. They are another population that can contribute with important design and such as the practical adaptation of the MCT system into their job coach program and work routine. The findings of this feasibility study indicate that our MCT device has translational and pragmatic implications, particularly in settings that employ and provide vocational training for individuals with CDs. The results support that the MCT is feasible and that working-age adults with CDs can benefits of mobile technologies to improve work performance and enjoyment quality. Therefore, this technology system has the potential to assist workers with CDS in vocational training and job tasks that require memorizing steps that involve well-ordered action in conjunction with good spatial recognition. It is important to note that the program has great potential for commercialization and knowledge transfer. Despite the several limitations impacted the study's generalizability, we were able to recruit a well-balanced and heterogeneous sample for comparisons between groups. In addition, due to the ecological element of the study, we used mix-methods appraisals (qualitative and quantitative) to evaluate the MCT feasibility. Further research is recommended to evaluate a longer duration experimental design in a larger sample for the successful adoption and adherence of the MCT for workers with CDs. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by funding provided by the National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR) Grant #H133E090003 and the Coleman Institute for Cognitive Disabilities. Patricia C. Heyn, PhD, FGSA, FACRM. PCH and CB conceived the study. PCH developed the protocol and gained ethical approval. PCH and AG recruited and collected the data. PCH and GMG analyzed the data. PCH wrote the first draft of the manuscript. All authors edited and reviewed the manuscript. Patricia C Heyn https://orcid.org/0000-0002-9015-362X World Health Organization. WHO global disability action plan 2014-2021: better health for all people with disability Annual report on people with disabilities in America The Americans with disabilities act of 1990: an overview The relationship between disability prejudice and disability employment rates AI for people with disabilities: a research roadmap. arXiv 2019 Politicizing disability in political science, COVID-19, and police violence Intellectual disability: definition, classification, and systems of supports Assistive technology effects on the employment outcomes for people with cognitive disabilities: a systematic review Determinants of on-the-job-barriers in employed persons with multiple sclerosis: the role of disability severity and cognitive indices Comparing mobile technologies for teaching vocational skills to individuals with autism spectrum disorders and/ or intellectual disabilities using universally-designed prompting systems Mobile internet device with detachable wireless module. United States patent US 7 Smartphones as digital companions: Characterizing the relationship between users and their phones Disability citizenship and independence through mobile technology? A study exploring adoption and use of a mobile technology platform A mobile app for social anxiety disorder: a three-arm randomized controlled trial comparing mobile and PC-based guided self-help interventions A review of mobile applications to enhance coaching in occupational therapy The rehabilitation engineering research center for the advancement of cognitive technologies Effects of SNAPassociated work requirements on disability claiming The mini-mental state examination: a comprehensive review A service-based approach to developing android mobile internet device (MID) applications Technology for adaptive aging Workplace accommodations for adults with autism spectrum disorder: a scoping review Using assistive technologies in the curriculum of children with specific learning disabilities served in inclusion settings: teachers' beliefs and professionalism We would also like to acknowledge Goodwill Industries of Denver Job Day Program for allowing us to use their facility and recruit their employees for this study.