key: cord-1010936-5rdgna8g authors: Mingorance-Estrada, Ángel Custodio; Granda-Vera, Juan; Rojas-Ruiz, Gloria; Alemany-Arrebola, Inmaculada title: Validation of a questionnaire on the use of Interactive Response System in Higher Education date: 2021-06-28 journal: Revista latino-americana de enfermagem DOI: 10.1590/1518-8345.3374.3418 sha: c7fcc0482cebf6433b04db72aa52ca8ea6e7c48f doc_id: 1010936 cord_uid: 5rdgna8g OBJETIVO: this study aims to design and validate a questionnaire to measure the students’ perception of the use of IRS as a technopedagogical resource in the classroom. METHOD: a 24 items questionnaire (Interactive Response System for the Improvement of the Teaching-Learning Process) was designed ad hoc for this research and applied to 142 university students. RESULTS: both the exploratory and confirmatory factorial analysis yielded 3 dimensions: classroom environment, teaching-learning processes and learning assessment. The results obtained both in reliability (Cronbach’s alpha= 0.955) and in the Confirmatory Factor Analysis (χ2/df=1.944, CFI=0.97; GFI=0.78; RMR=0.077; RMSEA=0.08) reveal highly satisfactory indices. CONCLUSION: statistical analyses confirm that this instrument is a valid, reliable, and easy-to-apply tool for professors to evaluate the student perception of student-centred learning. Higher Education must establish a space for the exchange of practical experiences that promotes knowledge and research within the common framework of the European and Ibero-American Higher Education Area (1) (2) . The implementation of student-centered learning is oriented to establish a model that effectively integrates technology with knowledge of didactic mediation, evolving towards the Technological Pedagogical and Content Knowledge Model (TPACK). This system combines disciplinary, pedagogical and technological knowledge, but always considering the context in which it intervenes (3) (4) , and increasing the interaction between professors and students within a critical dialogical approach (2) . Some authors consider necessary to establish didactic knowledge through the relationship between different types of knowledge (coming from the own discipline, general pedagogy and students) and the professor's biography (5) . Thus, during the nursing initial training, both the pedagogical aspects, including the implementation of Interactive Response System (hereinafter IRS) with remote answering devices to monitor students' progress, and the involvement of expert professors lead to highquality teaching, among other issues (6) . In this regard, encouraging professors to integrate technology into the classroom is crucial, as highlighted in the Horizon Report (7) , since it will significantly impact on education in the coming years. To do this, university professors must use the technological tools they are familiar with, as well as access new technological resources to improve teaching processes (8) . Similarly, technological changes in university professors follow a tendency and are not radical; they introduce those that are consistent with their teaching practices into the learning activities they normally carry out (9) . In order to face these technological challenges, an inversion of learning process is required; students should be provided with materials in various formats, so that they can carry out preliminary work before arriving at the classroom, incorporating IRS to verify the improvements in the student centered-learning process (10) (11) (12) . IRS has been already integrates in some university classes (13) . Thus, the available literature about Higher Education on the use of this technology in recent years focuses on the fields of Science, Technology, Engineering and Mathematics, Sociology, Humanities, Health (Medicine and Nursing), Business Administration and English language (13) (14) . The reviewed studies indicate that integrating IRS in university classrooms improves three main areas: the classroom environment, the teaching-learning processes and learning assessment (15) (16) (17) (18) (19) (20) . Thus, defining the possibilities and limitations of this tool is increasingly important for the improvement of the quality of Higher Education (6) . Concerning the classroom environment factor, IRS increases attendance (21) and student participation (22) (23) (24) (25) , resulting in a higher level of involvement during classes in comparison to the traditional methodology (23, (26) (27) . Within the learning factor, some studies establish that frequent and positive interaction makes classes more dynamic when using IRS (13, (28) (29) , promoting active learning (30) (31) (32) . In addition, attention (33) (34) , concentration (35) and memory (36) (37) are encouraged during the learning process. Extensive research suggests that a better performance is the result of the use of IRS, as some studies indicate (31) (32) (38) (39) (40) (41) , although other studies do not find such an effect (42) . In relation to the assessment factor, the findings of the literature support the capacity of IRS as a tool for assessment and feedback (43) (44) . It is considered that both students and professors benefit from the feedback they receive with the use of this educational technology (20, 27) . All of this leads to a key learning process for the interaction of knowledge and know-how (45) . The aim of this study is to design and validate this study aims to design and validate a questionnaire to measure the students' perception of the use of IRS as a technopedagogical resource in the classroom. The design was transectional and descriptive, as the data were collected in a single time in order to describe the phenomenon and analyze it at a certain time. The research was carried out at Melilla Campus of the University of Granada (Spain), located in North Africa, whose students attend to the Health Sciences, Social and Legal Sciences, and Education and Sports Sciences schools. For this purpose, an intentional non-probabilistic sample was carried out. The selection criteria have been: firstly, professors who use technology in their classrooms, specifically interactive response devices. And secondly, the willingness of the students to participate in this study. Therefore, the sample comprises 142 students: To carry out this study, an ad hoc questionnaire was designed for this research, "Interactive Response System for the Improvement of the Teaching-Learning Process (IRS-ITLP)". In relation to the items of the IRS-ITLP, they were written after an extensive bibliographic review on the three factors highlighted above (21, (46) (47) . Despite of the absence of experts on this field, this process provides validity to the questionnaire items. www.eerp.usp.br/rlae 3 Mingorance-Estrada AC, Granda-Vera J, Rojas-Ruiz G, Alemany-Arrebola I. We started with 65 items grouped into categories (48) . After the analysis of the items, we selected 35 items that were grouped into the three dimensions: learning environment, process and assessment. Regarding the questionnaire response format, a Likert type scale was used, with 5 response alternatives, ranging from 1, totally disagree, to 5, totally agree. The statistical software SPSS version 20.0 has been used for the statistical processing of the data. To know the reliability of each group of items, Cronbach's alpha was used, and for the validity of the questionnaire, an Exploratory Factor Analysis was carried. For the Confirmatory Factor Analysis, the program LISREL 8.8 was used. Firstly, the reliability of the IRS-ITLP questionnaire consisting of 35 elements was analyzed using Cronbach's alpha internal consistency coefficient, which was 0.965. Although this index was high, we proceeded to eliminate those items whose item-total correlation was inferior to 0.20. Finally, the questionnaire was made up of 24 items with a α=0.955, showing homogeneity indexes ranging from 0.42 to 0.85. Subsequently, the means, standard deviations, asymmetry, and item-total correlations of each of the items were obtained. As can be seen in Table 1 , the asymmetry is negative in all items, which shows a greater concentration of responses corresponding to the high scores in those items. Table 2 shows the factors, items, factor loadings and reliability of each dimension, as well as the interpretation of these factors. To determine the dimensions, the factorial loadings criterion has been followed, being the cutoff value 0.30 (49) . Error of Approximation (RMSEA) is 0.080, being both indexes considered acceptable since they are between 0.5 and .08 (49) . The which are considered acceptable as they are between 0.5 and 0.08 (49) . In addition, the GFI representing the joint adjustment is 0.78 and CFI is 0.97, so both values are within the tolerance limits. These results confirm that the data fit to the 3-factor model, supporting the proposed dimensional structure. For all these reasons, these results allow us to be confident in the reliability and validity of this instrument. Therefore, the construction and validation of this questionnaire make possible the application of this instrument to measure the student's perception in the use of IRS in the learning process. Regarding the factor "Learning environment", reliability was measured using the Cronbach's alpha coefficient, which scores 0.926 and suggests a high This factor is decisive since university students with an adequate class environment increase class attendance (22) and improves their participation (13, 17, 44) . It also comes on interaction between professors and students (13) , and has a positive influence on attention (33) (34) and concentration (35) , as the studies analyzed show. The second dimension, which corresponds to the "teaching-learning process" factor, shows an internal These elements coincide with studies supporting that this working methodology improves performance (31) (32) (38) (39) (40) (41) thanks to the paradigm shift that implies active learning through the connection between knowledge and knowhow (45) and the frequent and positive interaction, which resulted into more dynamic classes when IRS is used (13, 28) . This method promotes material comprehension by acquiring a deeper knowledge, helping to review and understand the contents and improving long-term retention (36) (37) ; the final result is an improvement in the learning process. In relation to the third dimension, "Learning El Espacio Iberoamericano de Educación Superior. Diagnóstico y propuestas institucionales Flipped Classroom to Improve University Student Centered Learning and Academic Performance Teachers' technological pedagogical content knowledge and learning activity types: curriculum-based technology integration refrained Efficient Use of Clickers: A Mixed-Method Inquiry with University Teachers Proceso de construcción del conocimiento pedagógico del docente universitario de enfermería1. Rev. Latino-Am. Enfermagem Efficient Use of Clickers: A Mixed-Method Inquiry with University Teachers Higher Education Edition The New Media Consortium Evidence of Innovative Assessment: Literature Review and Case Studies, EUR 29882. Luxembourg: European Commission Enseñar con tecnologías digitales en la Universidad Incidence of the Flipped Classroom in the Physical Education Students' Academic Performance in University Contexts. Sustainability An Integrative Review of Flipped Classroom Teaching Models in Nursing Education Turning a crisis into an interactive drama: the introduction of a 'clickers theatre' in nurse education Carrino SS. Digital connection in a physical classroom: Clickers and the student-teacher relationship Zainuddin Z, Halili SH. Flipped classroom research and trends from different fields of study Barcelo JM. Medical laboratory science and nursing students' perception of the academic learning environment at a Philippine university using the Dundee Ready Education Environment Measure Clickers" and metacognition: a quasi-experimental comparative study about metacognitive self-regulation and use of electronic feedback devices Effects of different methods of reflection on nurses' gaze and judgement in a task using a touch panel Millennial's perspective of clicker technology in a nursing classroom: A mixed methods research study Alemany-Arrebola I. chemistry Let's kahoot! Anatomy Encouraging college student active engagement in learning: The influence of response methods Students' perception of Kahoot!'s in uence on teaching and learning Interactive audience response systems in oral and maxillofacial radiology undergraduate lectures Enhancing student engagement and immediate feedback with clickers and response cards Reflections on the use of ARS with small groups Nurse Perceptions of Interactivity during Their Onboarding Orientation: Effect of an Audience Response System [thesis Making learning fun to increase nursing students' success: Formative feedback in communication learning Impact of educational games on academic outcomes of students in the Degree in Nursing Effects of competitive learning tools on medi-cal students: A case study Improving classroom clicker practices: Effects of incentives and feedback on retention Bojinova ED, Oigara JN. Teaching and learning with Clickers: Are Clickers good for students? Impact of Student Response Systems on Initial Learning and Retention of Course Content in Health Sciences Students An Integrative Review of Flipped Classroom Teaching Models in Nursing Education Can clicking promote learning? measuring student learning performance using clickers in the undergraduate information systems class Analyzing the efficacy of the testing effect using kahoottm on student performance The use of a game-based learning platform to engage nursing students: A descriptive, qualitative study Effectiveness of various innovative learning methods in health science classrooms: a meta-analysis Are Audience Response Systems Worth the Cost ? Comparing Question-Driven Teaching Strategies for Emergency Medical Technician Education Kahoot: A promising tool for formative assessment in medical education Mobile-based assessment: A literature review of publications in major referred journals from Acceptance of clickers in a large multimodal biochemistry class as determined by student evaluations of teaching: Are they just an annoying distraction for distance students? How benefits and challenges of personal response system impact students' continuance intention? A Taiwanese context Implementing clickers to assist learning in science lectures: The clicker-assisted conceptual change model Principles and Practice of Structural Equation Modeling