key: cord-0058243-z798z5h5 authors: Vella, Frédéric; Dubus, Nathalie; Malet, Cécile; Gallard, Christine; Ades, Véronique; Preel, William; Vigouroux, Nadine title: Multidisciplinary Experience Feedback on the Use of the HandiMathKey Keyboard in a Middle School date: 2020-08-10 journal: Computers Helping People with Special Needs DOI: 10.1007/978-3-030-58796-3_46 sha: f93f9e9873d8fa33704b09c45dc22ef438c6559f doc_id: 58243 cord_uid: z798z5h5 There is a poorly addressed input area in the accessibility field that deals with the input of scientific elements including mathematical formulas. Few studies have addressed this issue although Word and Open Office editors offer input interfaces consisting of button bars associated with mathematical symbols and an “input sheet”. The analysis of input activity with these tools with disabled children has revealed that the use of these bars is complex and tiring. HandyMathKey is virtual keyboard co-designed by specialized teacher and human-interaction researchers to address the difficulties of numerical mathematical input tools. The purpose of this paper is to describe the observation method implemented in a 4th grade class at the Centre Jean Lagarde in Toulouse and to report few results on usability of HMK. Assistive technologies are often inappropriate or even abandoned due to the lack of inclusion of end-users or their ecosystem [1] , especially in the field of inclusive education [2] . A numerical keyboard for mathematical input, HandiMathKey (HMK), was designed by implementing a co-design method in which teachers and occupational therapists expressed the demands for such an innovation based on their expertise, starting from the observation of the difficulties of young adolescents in school for the production of mathematical writings [3] . The purpose of this paper is to describe the observation method implemented in a 4th grade class at the Centre Jean Lagarde in Toulouse and to report few results on usability of HMK. Information and communication technologies on computers, or tablets, can become an assistive technology that makes the learning process more accessible. Indeed, handwriting is a difficult and tiring task for students with grapho-motor deficits. Benoit & Sagot [4] have analyzed and identified the difficulties encountered by a student with neurodevelopmental disorders in order to determine special educational needs. This is why keystroking on the computer keyboard, combined with word processing software, is recommended for text entry. Rogers & Case-Smith [5] studied the relationships between handwriting and keyboarding performance of sixth-grade students and reported "some children with difficulty in handwriting may nonetheless become proficient in using a keyboard to wordprocess".While text input has been the subject of numerous studies [6, 7] for disabled people, there has been little research for mathematical formulas input. At the elementary and middle school levels, it is difficult to be autonomous in mathematics for students who cannot write by hand. Indeed, learning in this subject requires inserting symbols not available on a conventional keyboard and writing in a non-linear way. There is a poorly addressed input area in the accessibility field that deals with the input of scientific elements including mathematical formulas. Few studies have addressed this issue although Word and Open Office editors offer input interfaces consisting of button bars associated with mathematical symbols and an "input sheet". The analysis of input activity with these tools with disabled children has revealed that the use of these bars is complex and tiring. Dmaths 1 , MathType 2 , MathMagic Lite 3 , MathCast 4 , …are some interactive applications that lets you create mathematical notation. Windsteiger [10] has designed a graphical user interface based on the possibility to have dynamic objects (sliders, menus, checkboxes, radio buttons, and more) but within the specific framework of the Mathematica programming environment. Their goal was to facilitate the use of the Mathematica programming environment. Elliott and Bilmes [11] proposed the CamMath application that allows the creation and manipulation of mathematical formulas using a speech recognition system. They reported that this input modality is useful for students or professionals with motor disabilities. In addition, the use of this modality results in fewer errors and faster input of mathematical formulas than when using a keyboard and pointing device [12] . Indeed, these authors have explored a multimodal input method combining handwriting and speech. Their hypothesis is that the multimodal input may enhance computer recognition and aid user cognition. They reported that novice users were indeed faster, more efficient and enjoyed the handwriting modality more than a standard keyboard and mouse mathematics interface, especially as equation length and complexity increased. However, although speech recognition is a useful modality for people with motor disabilities, it could be on the one hand intrusive in crowded environments (schools, etc.) and on the other hand, it would have degraded performance in noisy environments. The ASEI Jean Lagarde Centre in Toulouse and the IRIT laboratory have implemented the User Centred Design Method [8] and the ISO 9241-210 [9] standard for the codesign of the HandiMathKey (HMK) mathematical input keyboard. This HMK keyboard aims to address the limitations of use that users with disabilities encountered with dedicated mathematical editors or integrated in text editors (see [3] ). The HMK keyboard is a multi-block integrated keyboard composed of sub keyboards (See Fig. 1 ) to meet the mathematical needs of middle and high school students. The background color of the sub keyboards (See Fig. 2 ) and the font size can be adjusted to meet accessibility needs. The HMK keyboard is currently available for each of the editors (Microsoft Office and Libre Office) under the Windows operating system. The participatory action research utilized a mixed method: an interdisciplinary observation workshop and a questionnaire on the utility and usability of HMK. The objectives of this workshop are: • To support the learning of the HMK keyboard by putting it into practice in mathematics class. • To assist the student in the appropriation of the tool and autonomy in the face of writing mathematics on digital media, to encourage his or her practice during personal work in secretarial work and to bring up needs and difficulties to the multidisciplinary team. This team consists of one mathematics teacher, one occupational therapy and one specialized education assistant. They leads observe how the students accept and use HMK. This field study has lasted one school year at the Jean Lagarde rehabilitation and education Center in Toulouse. The HMK was proposed to 9 students in fourth year in middle school. Both versions (Microsoft Word and Libre Office) were offered to the students but the children learned HMK, Libre Office version, the wish of the mathematics teacher. Students were invited to use HMK in all their mathematics class to write mathematical formulas. Interdisciplinary workshops were being conducted every three weeks (i.e. five workshops, outside mathematics class). The teacher introduces the HMK features corresponding to the mathematical concepts being taught; then the students are invited to do mathematical exercises and the occupational therapist may assist them in getting to grips with the HMK. During the workshop, two types of data are recorded: 1) notes of teacher and occupational therapist in the case report form; 2) activity log of the use of HMK. After five workshop sessions the students are invited to reply to answer a questionnaire. This paper reports on the baseline data from the questionnaire and some verbatims of the disabled students as well as notes from the case report. The study population consists of 9 students (see Table 1 ), with motor disability: two of whom have associated tremors, one has a visual impairment, one an emotional disability and another a learning impairment. The modes of writing mathematical formulas were distributed as follows: 5 of them performed handwritten input (HW); 7 used personal computers (PC); 5 used personal assistant (PA). 2 of them (P2 and P9) had never used a computer before the study. On the contrary, the children (P6 and P7), who have the most difficulty with handwritten input (level 4 much difficulty), were already using the PC for input (Microsoft editor). 4 have levels of difficulty (level 3 with difficulty), one has little difficulty (level 2) and 2 have no difficulty (level 1). We have chosen to report the preliminary results of the following two questions: Q1: Is HMK easier to use than Libre Office mathematical functions? Q2: Do I feel less tired using HMK than Libre Office functions? These two questions have been evaluated on a Likert scale (1: disagree at all, 2: disagree rather than agree; 3: agree rather than disagree; 4: agree completely). Figure 3 shows a mean score of 3.25 (±1.03) for ease of use and 3.5 (±0.75) for a less tiring use situation with HMK. The ease of use score could be improved due to residual bugs in the version used during this experiment. We also wanted to find out their ability and level of satisfaction in using the HMK keyboard by means of the following two questions: Q1: Do I feel able to use HMK on my own? Q2: Overall, I am happy using HMK. These two questions are evaluated with a score ranging from 1 to 10 (Q1: 1: I am not at all capable, 10: I am fully capable; Q2: 1: I am not at all happy, 10: I am very happy). Figure 4 shows that the mean ability to use is 7.25 ± 1.16 and the mean degree of satisfaction is 6.37 ± 2.13. These results show that HMK is an easy input tool thanks to its attractive button representation and layout. The degree of satisfaction is acceptable although we observe a large variation between subjects. As observed by the multi-disciplinary workshop team, the students have increasingly taken ownership of the tool over the course of the school year and we hope that the degree of satisfaction will increase. The teacher and the occupational therapist have reported dialog and observations. First the behaviour and the enjoyment were different according the children. They expressed different assessments depending on their typing habits and acceptability of HMK. P1 said she did not like the HMK too much. She was generally reluctant to use the computer tool. However, when she was asked to use HMK, she did quite well. P2 told us she was not interested in the HMK because she preferred to write mathematics by hand. During the HMK sessions, she has been able to demonstrate some skills but she was not yet really taking the initiative to use this tool. P3 had a very good mastery of the HMK. He recognized the interest of the HMK in terms of time saving. P4 had a good mastery of the HMK. She used it in class without being requested and rarely asked for help from the teacher or the occupational therapist. P5 was able to invest himself to input mathematical formulas with HMK thanks to the support of an adult. In autonomy, he does not wish to use it. P6 reported that he found that the HMK allowed him to write mathematics formulas faster, but he was accustomed to the physical keyboard. P7 found that the HMK allowed her to be faster but no less tired. Indeed, she was a fairly good mastery of HMK. P8 told us he was faster with HMK. Since the HMK experiment, he uses the computer much more easily and makes homework perfectly readable. P9 found that it was faster with HMK and that writing was less tiring. Motivation on the subject was real for him because of his difficulties in producing fine gestures. In the inclusive school, the mathematics teacher reports that students are dependent on the AESH (Persons accompanying students with disabilities) for math class. The AESH writes in the student's place and the teacher finds this to be an obstacle to the student's progress. Indeed, the learning of mathematics is more efficient by practicing the written word as regularly as possible, even if mental arithmetic is important. The HMK perfectly meets this need. Thanks to HMK, this teacher is now able to offer a fully digital teaching for writing mathematics. This study confirms the interest of having a multidisciplinary team for observation and assistance in the appropriation phase of HMK. The students were all volunteers but during these workshops they did not seen the immediate interest of HMK. This interdisciplinary workshop showed that the use of HMK in mathematics class undeniably favors the acceptance of the tool and its appropriation. HMK becomes an assistive technology used by everyone in the classroom and does not point out a difficulty related to disability. It thus brings to each student the help corresponding to his needs, and according to the children it differs according to their difficulties and the levels of competence in mathematics. The students appropriated quickly HMK due to the interface affordance (the symbols are quickly understood by the students) even if they took time to perceive its interest (see Fig. 4 ). However, they necessarily need to be accompanied in the learning of HMK but also in the learning how to use the Libre Office text editor. The typing with HMK and Libre Office is similar to reading the mathematical formula, which makes it more affordable for students with planning and visual-spatial difficulties. Students reported that HMK is much less tiring (see Fig. 3 ). It would be interesting to highlight the benefits of this decrease of fatigue for young people in terms of the learning of the subject being taught (speed of grasping, duration of concentration, etc.). The paper presents a first field study of HandiMathKey (a software keyboard for typing mathematical formulas) used by 9 motor disabled students. Questionnaire and observations were performed by a mathematics teacher and an occupational therapist during multidisciplinary workshop. The first results show that the classroom facilitates acceptability of HandiMathKey by disabled students. In general, young people adapt quickly to HMK, thanks to its intuitive layout. However, they necessarily need to be accompanied in the learning of the mathematical editor. As the sessions progressed there is a greater participation of students. This first observation over one school year made it possible to assess the relevance of the establishment of HMK as an assistive technology. These data (observations & questionnaire) must be completed by quantitative data (speed & error rate) and with more students to confirm the benefit for college. From human-centered design to disabled user & ecosystem centered design in case of assistive interactive systems Providing assistive technology in Italy: the perceived delivery process quality as affecting abandonment HandiMathKey: mathematical keyboard for disabled person L'apport des aides techniques à la scolarisation des élèves handicapés. La nouvelle revue de l'adaptation et de la scolarisation 43 Relationships between handwriting and keyboarding performance of sixth-grade students Effect of a dynamic keyboard and word prediction systems on text input speed in patients with functional tetraplegia Text input for motor-impaired people The design of everyday things: Revised and expanded edition 9241: Ergonomic requirements for office work with visual display terminals-Part 11: Guidance on usability Theorema 2.0: A graphical user interface for a mathematical assistant system Computer based mathematics using continuous speech recognition. Striking a C[h]ord: Vocal Interaction in Assistive Technologies Evaluation of multimodal input for entering mathematical equations on the computer We are thankful to the Jean Largarde Center and national education, for giving us access to students. The authors also thanks the students in the experiment.