key: cord-0740618-i5edhol4
authors: Santos, Simone C. dos; Bispo, Esdras L.; Santos Filho, Osmário de Carvalho; Oliveira, Rebeca Paula Alves de
title: Problem-Based Learning Diagnosis in Computing Higher Education: An Overview from Brazilian Public Institutions
date: 2022-04-20
journal: SN Comput Sci
DOI: 10.1007/s42979-022-01106-z
sha: db44dcf4b00978591479aed34960b3f9383d6747
doc_id: 740618
cord_uid: i5edhol4

In Computing Higher Education (CHE), the desired transformation of traditional teaching and learning methods, almost always based on the transmission of information and content-based curricula, has been the objective of several educational institutions that wish to combat students’ demotivation and dropout. Among successful approaches, Problem-Based Learning (PBL) stands out as one of the most effective and radical methods regarding pedagogical innovations. While the PBL implementation means a great opportunity to achieve better educational performance, it also represents many challenges that can only be managed if they are first known and understood. In this context, the motivation for this study comes from the following research question: “How to know if an institution at CHE is ready to implement the PBL?”. As a response, an institutional diagnostic model regarding the adoption of PBL is proposed. It conducted an opinion survey in two kinds of educational institutions: technical and academic ones. Thirty-eight technical educational institutions in computing answered this survey, involving 302 participants, and fifteen academic institutions, involving 20 participants. The results showed that the model reached its objective, allowing the identification of favorable, warning, and critical points regarding the adoption of PBL in these institutions. This study is an evolution of the results focusing only on technical institutions published at the CSEDU 2021 conference and conducted by the NEXT Research Group.

In 2020, the Association for Computing Machinery (ACM) and the Computing Society of the Institute of Electrical and Electronics Engineers (IEEE-CS) launched Computing Curricula 2020 (CC2020). This set of curricula deals with the paradigms of global education in Computing in Higher Education (CHE). According to the CC2020, the application of classical education 1 in most subjects means that professionals who have recently graduated in computing do not have the skills that the professional market requires. Even if they have gone through an academic curriculum with the required basic knowledge, this training is not enough; it is up to the industry to train professionals to meet these needs [12] , pp. [40] [41] . This lack of alignment between academia and the labor market is related to the traditional model of education. This model has a curriculum that extensively explores computer knowledge but has This article is part of the topical collection "Computer Supported Education" guest edited by James Uhomoibhi and Beno Csapó. little focus on developing students' interpersonal skills (ability to apply knowledge to perform a task actively) and behavioral (temperaments, predilections, and attitudes).

In Brazil, this behavior is no different. The disparity between undergraduate learning and the labor market demands implies a high shortage of professionals in the sector [43] , p. 19 ). According to a Brasscom 2 [10] survey, the demand for IT professionals in Brazil will reach 420 thousand people by 2024, but 46 thousand professionals with a technological profile graduate each year. According to [40] , considering the years 2012 to 2019, the difficulty of understanding the content, students' lack of motivation, low application of theory in real problems, lack of interpersonal skills, and student dropout during the course are among the most reported problems in computing education in Brazil. Another Brasscom study [9] about educational training and employability in ICT reports that almost 69% of students drop out of the course.

Studies in Brazilian public universities [11, 22] , responsible for major educational sector in the country, point out the main reasons of students to justify evasion: (i) lack of engagement, demotivation, or not affinity with the course, (ii) attitude and methodology used by the teacher; and (iii) incompatibility between the academic environment and the professional market. It also observes that, in Brazil, the technical training courses have a smaller curriculum and are focused on the labor market. These courses are increasingly training professionals concerning other types of higher education in computing and may soon become the preference of computing undergraduates [52] , p. 65). Thus, there are many challenges to be faced in search of a balance between these two categories of institutions and their preparation to form computing professionals capable of meeting the current and future demands of society.

Toward solving the educational challenges in CHE, successful approaches use authentic problems, attractive technologies, and a learning environment that reflects the labor market to promote the engagement and motivation of the students [3, 27, 33, 45] . In this context, it is essential to be attentive to teaching and learning methods/methodologies, with concrete proposals to help to transform the traditional classroom into a practical and stimulating environment [8] . Thus, teaching methodologies such as Problem-Based Learning (PBL) [25, 26, 32, 39, 51, 56, 57] , and its variants like Project-Based Learning [7] , Case-Based Learning [54] , Challenge-Based Learning [36] , have become popular in computing education. These bring important benefits such as increased engagement, motivation, and development of technical and non-technical skills so important to the student's professional life.

Born in medical education, PBL places students in a constructivist pedagogical context in a practical, collaborative, and interactive environment. PBL promotes selffulfillment, satisfaction, and greater participation in the teaching and learning process. It positively influences the motivation and engagement of these students, increasing academic involvement and performance, and decreasing dropout rates in computing courses [50] .

In the last decade, experiences with the application of PBL have demonstrated the use of more sophisticated learning environments, more relevant academic projects, and partnerships with professionals and companies [50] , p. 4). The effectiveness of the approach has shown many positive results, especially when implemented not only as a pedagogical practice or in isolation (in specific courses into a curriculum), but as an educational innovation strategy defended and driven by the respective changing educational institution [19] . Considered one of the most radical methods regarding pedagogical innovations, PBL advocates profound changes that involve the institution, transforming its learning environment, teachers' and students' attitudes, curricula, operational and managerial resources, infrastructure, assessment processes, relationships with the labor market, and, consequently, budget [18] . While the implementation of the PBL means a great opportunity to achieve better educational performance, it also represents many challenges that can only be managed if they are first known and understood. First, as an educational strategy, the implementation of PBL can be seen as an institutional project that needs to be planned, executed, monitored, and improved over time, following the steps of a classic PDCA (Plan-Do-Check-Act) cycle [44] . As the first step in planning, it is necessary to understand the challenges faced by all stakeholders in the educational process based on PBL and raise the requirements to be adopted this approach effectively.

With this objective, the NEXT (iNnovative Educational eXperience in Technology) research group has been investigated the use of the PBL approach in computing education for more than a decade [38, 50] . In recent consultancies in CHE institutions that wish to transform their curricula and traditional pedagogical proposals through the PBL, an imminent difficulty became evident: the institutions could not adopt the PBL, and, even more critical, they did not know how to start. With this motivation, Santos Filho and Santos [47] proposed a PBL institutional diagnosis, here called PBL-Gauge, considering two target-public: (i) teachers/tutors, giving their opinion on the essential elements for the method adoption under the pedagogical aspect, also reporting their perceptions related to students in general; (ii) course coordinators/managers, who have a more systemic view of the educational institution and, therefore, can contribute with their vision under structural and organizational aspects, complementary to the teachers' view. The PBL-Gauge aims to help higher education institutions to identify their ability to implement the PBL, showing its favorable, critical aspects and warning signs. After being created and validated by specialists, this diagnosis was applied in 38 technical institutes, showing its effectiveness in the intended objective. The results showed important evidence, indicating strengths, improvements, and warnings in the context of technical education.

Continuing this research, this article describes the results of a new investigation at CHE, this time considering ten federal public universities in Brazil in the following courses: Computer Science (CS), Computer Engineering (CE), Software Engineering (SE), Information Systems (IS), and Information and Communication Technology (ICT). As a research question, we continue to want to know: "How to know if an institution at CHE is ready to adopt the PBL?". Although the two scenarios (technical and academic ones) cannot be compared, as they deal with different contexts, this research presents an overview of the diagnoses made, highlighting the main points of attention in each one. Therefore, this article is an evolution of the study of Santos Filho and Santos [47] from a broader view of CHE in Brazilian public institutions.

To report the results of this research, this article is divided into six sections. After this introduction, "Background information" presents an overview of computing higher education, besides describing concepts, challenges, and requirements of the PBL. "Research method" describes the research methodology and its steps. The institutional diagnosis model is summarized in "PBL-gauge institutional diagnosis". To assess the practical applicability of this model, "Applying PBL-gauge: results and discussion" describes the results and discussions of two opinion surveys with technical and academic institutions in computing, which answered the proposed diagnosis involving their teachers and course coordinators' team. Finally, "General discussion" discusses the conclusions and future work.

This section is divided as follows. "Computing higher education" presents in more detail an overview of Computing Higher Education. "PBL approach" describes the Problem-Based Learning approach. At last, it points out the challenges in PBL ("Challenges in PBL") and the requirements to apply it ("Requirements to apply PBL").

Defining Computing Education (CEd) is a very challenging effort. The current CEd classifications (e.g., [17, 28] make use of various dimensions to cover all specters of this broad area. However, it is possible to define CEd research, in a general and more straightforward way, like improving how computing is taught and learned [13, 23] . In this direction, Computing Higher Education (CHE) is CEd under the more specific perspective of the higher education level.

One of the CHE specificities is the formation of students to a professional career in computing. A computing professional needs to be prepared for an environment that can change continuously. Thus, it is necessary to focus more on concepts and ideas than on specific technologies, for instance (seen these constructs are more "volatile" than those). Despite this focus, the materialization of these concepts and ideas is also crucial. Hence, an important skill for this professional is to deal with the interchanging of technologies and frameworks during your career. Lifelong learning is expected to be developed during CHE undergraduate studies [12] , p. 32).

Another formation's need of a computing professional is the preparation for an interdisciplinary environment. The computing area is pervasive and ubiquitous, requiring dialog with diverse actors in a diversified professional space. Thus, it is necessary to cultivate interaction with other professionals from different areas (including non-technical ones). It also expects interdisciplinary studies to become part of CHE undergraduate studies [12] , p. 32).

In response to these needs, higher institutions adopt the PBL approach in computing courses. PBL is looking to promote an environment closer to real scenarios of constant changing as much as the dialog among students and different actors involved in the candidate solution to the problem [15] , p. 42). Many CHE institutions around the world adopt PBL. Its use embraces ranging from a single course to the whole curriculum program [6] , p. 8).

However, there is a lack of research focusing exclusively on teachers' or organizations' views from PBL in the CHE context [37] , p. 10:6). Although works are reporting PBL curriculum assessment in this context [14] , it is necessary a more systematic approach to diagnose the condition of an institution to implement the PBL approach successfully.

According to Ribeiro [42] , the PBL is a teaching and learning method that makes use of real problems to motivate students to learn concepts, procedures, and attitudes that will be important for their future performance as citizens and professionals. As emphasized by Melo [34] , the potential of the PBL in the teaching and learning process of students refers to developing important skills such as self-confidence, problem-solving, and autonomy. Thus, the focus of the teaching process is the student, who is stimulated to learn Page 4 of 21 SN Computer Science more autonomously and cooperatively with his colleagues. Figure 1 presents the classical PBL cycle.

The PBL method still requires a more active posture from students, choosing the best way to learn, conducting research, and using educational resources beyond the classroom. It also encourages students to reflect critically on what is proposed to solve the problem, promoting important skills for future computer professionals in constant learning due to technological advances. In this context, the teacher role is to monitor and provide feedback on learning, in addition to identifying the difficulties encountered by students, facilitating and guiding the progress of their learning [16] .

PBL follows some principles such as (i) an authentic learning environment, simulating the situation found in the professional environment, (ii) the use of real problems as a learning object, and (iii) the monitoring of evaluation by continuous feedback [42] . Santos and colleagues [48] defined ten principles for the teaching of computing that founded a methodology called xPBL [49] , as shown in Fig. 2 .

The xPBL methodology defines five manageable elements for PBL planning: (i) Problem; (ii) Learning Environment; (iii) Human Capital that includes students, pedagogical team, and market partners; (iv) Content, as a guide and support to solve problems; and (v) Processes, concerning educational objectives and assessment processes. These elements are aligned with the ten PBL principles (as shown in Fig. 2 ) that are fundamental for the implementation of an authentic PBL in computing education.

The adoption of the PBL approach, as it is not trivial, requires a series of changes, both in the attitude of teachers and students [1, 35, 45] . According to Moesby [35] , it is common to evidence, during the application of the PBL method, the resistance of teachers who often do not have knowledge and experience in the teaching methodology, and for this reason, need to be trained to face the obstacles of the method.

Moreover, teachers should always be aligned with the PBL teaching process, considering that this approach requires a learning environment with a flexible curriculum geared to the demands of the professional market, in which the relevant issues are raised by the students and no longer by the teacher [16] . As for the students, according to Santos and Silva [45] , some difficulties were observed Fig. 1 The four steps from the classical PBL cycle [6] , p. 4)

Ten PBL Principles and the xPBL elements ( Source: [47] in the application of the PBL method: (i) a certain initial discomfort with the changes, (ii) lack of bibliographical research during the activities; (iii) lack of ability with technological resources; (iv) little involvement of the students in groups, and (v) student priorities. However, there are also external factors to the learning environment that make it difficult to implement the PBL, as mentioned by the students, such as adequate time management, which causes a drop in student performance and project quality [24] . In the PBL application, the students' dedication and their active participation in the learning process, inside and outside the classroom, are fundamental.

It is also necessary to face many obstacles in the planning for the implementation of the PBL methodology, such as the development of problems or projects, the planning of classes with a new approach, the lack of technological tools and trained professionals to practice this method in the institution [20] . The initial lack of security in the process of change to the PBL method needs to be overcome, as well as the difficulties of aligning the time between theory and practice, the curriculum adequacy, the availability of financial resources, the evaluation, and development of skills of the tutor/teacher [53] . Aldabbus [1] reinforces these challenges emphasizing the lack of technology, flexible schedules, and absence of curriculum policy as difficulties in adopting the method. Krusche and colleagues [31] report that there are institutional factors that hinder the adoption of the PBL method such as infrastructure, class planning, and didactic resources, in addition to an adequate environment for the development of team activities, technical-pedagogical support, and hiring of monitors/tutors.

To overcome these challenges of adopting the PBL, it is necessary to plan all the processes and resources before implementation, assisting in the correct use of the method, in the alignment of theory and practice, and respecting the principles of this approach [48] . When planning to change a teaching and learning process, it is important to carry out a diagnosis to characterize the respective educational institution that will face the changes. Thus, it is possible to identify the main aspects required for the effective implementation of the approach.

Among the aspects to be analyzed, it is important to point out: (i) collaboration between courses from a curriculum that allows the integration of knowledge acquired by students; (ii) a central committee for curriculum planning, to manage the content and topics to be addressed in the course; (iii) training of teachers to improve their didactics and learn new teaching strategies; planning of teaching in small groups and; (iv) availability of a period for study [2] . The implementation of PBL also requires the understanding and participation of various actors (teachers, students, researchers, managers, real clients) who will be active in the processes, therefore the need to focus on team development and curriculum development of PBL [18, 30] . The PBL curriculum should have a flexible format, be student-centered, be interdisciplinary, have real problems, and focus on research and investigation, promoting critical thinking and development of technical and non-technical skills [21] . Content planning should focus on practical classes, with the learning process focused on solving problems that stimulate discussion, challenge students, and stimulate their creativity [4, 31] .

Other prerequisites in the process of adopting the PBL are the provision of free space for self-learning, a review of the role of teachers and departmental autonomy, availability of financial resources to invest in infrastructure, and aligning student selection criteria to the profile expected by the PBL approach. Santos et al. [50] reinforces that the implementation of the PBL method in an environment with traditional teaching triggers the need for a series of changes, such as the adaptation of the curriculum, the formatting and organization of the learning environment, and the use of technologies.

PBL promotes many benefits for the student learning from the alignment of academic training with the requirements of the professional market [20] . At this point, the PBL requires that the institution has an approximation with companies in the labor market to provide teachers with new ideas and relevant problems to be addressed in the classroom.

There are organizations that perform PBL method consulting and analyze the institution at the organizational, pedagogical, and educational level, thus supporting the design of a PBL-based curriculum, evaluation processes and implementation of organizational and pedagogical aspects. Moesby [35] reports that many educational institutions analyze the needs of students today, and this brings about various changes at various levels: personal, organizational, and cultural. These changes require the development of an action plan, involving not only principals, but all those involved in the educational process, such as teachers, tutors, coordinators, and managers.

This study used a qualitative research methodology and a descriptive approach to its results. According to [41] , qualitative research aims to investigate what people do, know, think, and feel through data collection techniques such as observation, interviews, questionnaires, document analysis, among others. In descriptive research, the study, analysis, recording, and interpretation of facts in the physical world are carried out without the interference of the researcher. Page 6 of 21

Therefore, market and opinion research are examples of descriptive research [5] .

This research was motivated by direct observations in consulting activities of the NEXT Research Group to support educational institutions in the ample implementation of PBL, generally carried out during educational innovation and curriculum changes. In this context, institution analysis is essential to identify the requirements for PBL adoption and planning initial training for teachers and coordinators. Figure 3 shows the research steps and the main instruments used, having Cycle 3 as a new step not considered in the previous study published in [47] .

After an ad hoc literature review seeking solutions to help this investigation, the central research question was defined: "How to know if an institution at CHE is ready to adopt the PBL?" (RQ). To find objective answers, this research question was divided into three secondary questions:

(Q1) What are the favorable points for adopting the PBL? (Q2) What are the critical points for PBL adoption that can negatively impact its implementation?

(Q3) What are the warning points that the institution needs to understand better to make new decisions?

From these research questions, two objectives were defined: (1) the conception of a model to assess the ability to apply PBL and; (2) the application of this model in real educational institutions to verify its effectiveness. To achieve these objectives, two research cycles were designed, as shown in Fig. 3 . As follows, it will describe these research cycles ("Description of research cycles"), and the limitations and threats to validity for this research ("Limitations and threats to validity").

The first cycle had a focus on the "conception of the institutional diagnosis", which included searching the relevant literature on the adoption of PBL in teaching computing [38, 50] , understanding the main challenges encountered by the institutions and the essential requirements for the successful implementation of PBL, as discussed in "Challenges in PBL" and "Requirements to apply PBL", respectively. From these references, we identified three dimensions of the model and its nine aspects:

• Pedagogical, with the five aspects Problem, Environment, Human Capital, Content and Process, based on the methodology cited in "Background information"; • Structural, which includes Infrastructure and Curriculum aspects; and. • Organizational, with the Political and Evaluation aspects.

Considering the main stakeholders involved in an educational project, two target audiences were mapped: the teacher/tutor 3 and the course coordinator/manager, both key actors in transformation of the pedagogical approach. The student inquiry was also considered, but we decided to capture students' perspectives from their teachers/tutors. As a result, a set of 85 assertions was identified, with their respective justifications and literature references. Then, two PBL specialists analyzed the assertions, discarding those out of context and grouping others and, finally, defining 60 assertions distributed in two questionnaires of 30 questions each, addressed to the two target audiences. Each assertion is justified by a rationale and associated with its respective reference in the literature. The link to the list of rationales and references for the two questionnaires is indicated in the Appendix. Thus, it was created the first version of the PBL-Gauge diagnosis.

The second and third cycles were guided by the objective of applying and validating the PBL-Gauge. For this, we used the opinion research method proposed by Kitchenham and Pfleeger [29] , defined in six steps:

1. Definition of objectives; 2. Research design; 3. Develop the survey instrument (i.e., the questionnaire); 4. Evaluating the research instrument; 5. Obtaining valid data; and. 6. Analyzing the data.

The initial research objective was to evaluate and identify the necessary aspects for adopting PBL in CHE institutions, both at undergraduate and graduate levels, checking the current situation with professors and course coordinators. The research was designed in two cycles to conduct this investigation, selecting two research samples: 38 technical institutions (Cycle 2), and 20 academic institutions (Cycle 3).

To create the research instrument, the questionnaires and assertions defined in Cycle 1 were analyzed by seven specialists (6 Ph.D. and 1 Ph.D. student, all with more than five years of experience in PBL) under the aspects of syntax, semantics, level of relevance of the questions and questionnaire completeness. From this evaluation, some updates, groupings, and suggestions of new assertions were recommended: an assertion to verify if the self-regulation and metacognition of the students can be evidenced in the respective institution; an assertion on feedback in the evaluations of the teachers and other actors.

After the experts' evaluation, we created two forms (Google) directed to a participant profile (teacher or coordinator). Each form is organized into sections, starting with a section for collecting respondent identification data, followed by sections on assessment aspects. Thus, the teacher form has six sections, while the coordinator form has five sections. The answers were based on Likert's ordinal scale: (i) totally disagree, (ii) partially disagree, (iii) neither agree nor disagree; (iv) partially agree, and (v) totally agree. Figure 4 shows what the survey forms look like.

To conduct Cycle 3, we carried out a new round of validation of the research instrument to make improvements. Four teachers (3 Ph.D. and 1 M.Sc.) with different levels of knowledge and experience in PBL (none, fair, good, and high) evaluated the teacher's questionnaire. Three PBL experts (all Ph.D.) with a high degree of knowledge and All of them considered the criteria of clarity, completeness, and purpose of the research. This time, the recommendations focused on clarity, with proposals for improvements in some questions. In addition to these suggestions, we have included an optional open field for respondents' comments at each end of the form's section regarding evaluation aspects. After this round, the Google forms were updated, generating a new version of the questionnaires.

For data analysis, consolidated graphs were generated for each question, with the appropriate proportions of each chosen answer. For a qualitative discussion of the results, three statuses were defined: "favorable" (agree), "alert" (neutral), and "critical" (disagree), facilitating the analysis of the answers to the secondary research questions and future decision-making.

After Cycle 2, it was noticed that there were ties between the defined statuses in some moments, when the value of two or more statuses is equal. In this situation, we define the situation as "undetermined", signaling the need for further investigation with research participants for an effective analysis. It is important to emphasize that, in practice, the PBL-Gauge is an indicator that can help in planning the PBL implementation, showing clear and obscure points for a more informed discussion with those involved in planning educational transformations. A discussion of the results of applying the PBL-Gauge will be discussed in more detail in "Applying PBL-gauge: results and discussion".

It is important to highlight some limitations of this research. According to the "Promoting Institutional & Organizational Development" guideline, 4 a diagnosis must be a thorough task, based on a careful selection of interested parties. However, preliminary or partial analysis in the first contact with the investigated institution can serve as a reference base to have a more comprehensive and in-depth diagnosis later.

Another important point is that an institutional diagnosis is focused on a particular organization, requiring time and effort to apply, discuss, and carry out future interventions. This research focused solely on the diagnosis application stage, using the opinion survey method for large-scale application, to understand its usefulness regarding the PBL implementation, becoming evident favorable, critical, and warning points of institutions.

Finally, to facilitate the processing of the data collected in the diagnosis, a Likert scale with five values from "totally disagree" to "totally agree" was used. However, the evaluation of the results adopted a qualitative interpretation of the data, identifying favorable, critical, and warning points, in response to the secondary questions Q1 to Q3. We also chose to use an analysis based on absolute numbers for a few reasons: simpler to count the answers; it considers each answer individually, even the outliers; it gives us an overview of the three mentioned statuses (favorable, critical, and warning). However, we recognize the limitations when this simplified analysis fails to accurately inform us of a result, as the "undetermined" situation. Thus, no statistical method was applied in this study, which does not prevent it from being adopted in future works, if the planners assess that it makes sense.

The PBL-Gauge institutional diagnosis is represented by Fig. 5 , which shows the main aspects required to implement a PBL methodology and its classification in three dimensions: Pedagogical, Structural, and Organizational. Vitorino and Piantola (55, p. 102) define dimensions as parts of a whole, which come together for a certain purpose. In this sense, dimensions were defined by grouping aspects with complementary characteristics that compose the fundamental aspects for carrying out the diagnosis. It is important to emphasize that these aspects are based on theoretical references, and practical experiences carried out by several authors of scientific studies of PBL in computing education, as discussed in "Background information".

The pedagogical dimension is composed of aspects related to educational processes, focusing on the elements that contribute to the teaching and learning of students. Based on the xPBL methodology, defined by Santos and The structural dimension integrates aspects of Infrastructure and Curriculum, which support the development of pedagogical activities. Finally, the organizational dimension has the Policy and institutional Evaluation aspects of guiding the institution's educational procedures related to requirements of PBL culture. The definition of the nine aspects and its grouping into dimensions helped to elaborate a model for the PBL-Gauge, facilitating the creation of its structure based on references that respect the PBL principles and fundamentals.

Regarding the application of the PBL-Gauge, the diagnosis scope is focused on computing courses, considering the PBL references used for its definition and the CHE context. As commented in "Research method", we created two Table 2 Coordinator/manager questionnaire questionnaires addressing collaborators of the educational institution in the role of teacher/tutor and coordinator/manager, as shown in Tables 1 and 2 , respectively. To identify the respondents, the following fields were also included in the first section: name, institution, position/function, e-mail, course modalities, course, duration (in hours), teaching methodology, PBL knowledge, and experience in PBL. The "course" and "duration" fields are only included in the teacher's form, and the "teaching methodology" field asks what type of methodology the teacher uses within three alternatives (traditional, active, and hybrid), while in the coordinator's form asks if the institution recommends any type of teaching methodology.

It is important to emphasize that the results of this diagnosis provide transparency about the situation of the institution from the perception of its employees. With this information, discussions, and referrals can be made, for example, regarding the training of teachers, approximation with market companies, or even acquisition of specific technologies. Another important point to highlight is the care taken with the survey participants, considering that the PBL-Gauge is an institutional diagnosis. In practice, the selection of the respondents can be carried out in groups of teachers/tutors and coordinators/managers of a given course or group of similar courses so that we can identify strengths and improvements based on the perceptions of people who experience situations within the same context and that can generate feedback on similar or complementary needs. Considering the computing area, infrastructure needs perceiving by professors and coordinators of computer engineering courses, generally dependent on sophisticated hardware laboratories, will hardly be the same as perceived by the pedagogical staff of information systems courses, whose ideal infrastructure may not even be in the academic environment, but in a partner company, for example. In this scenario, the diagnosis applied to the information systems course may highlight the need to involve real clients with much more emphasis. In contrast, the results of the computer engineering course may emphasize the necessary technological resources.

To validate the diagnosis and stabilize the model for use, the PBL-Gauge has been applied in different real contexts, amply investigating institutions in CHE. Cycles 2 and 3, outlined in this article, focused on research in Brazilian public education institutions due to their high representation in the educational sector and the challenges commented in "Introduction". "PBL diagnosis in Brazilian technical institutions" describes the results found from Cycle 1, considering a sample involving respondents from at least one campus of 38 Brazilian technical institutes, totalizing 302 participants. The relationship between one author of this study and a graduate class formed by members of most Brazilian technical institutes was one of the reasons for this high participation of respondents. Thus, we consider the first analysis to be a scale survey. "Teacher survey application" presents the research results in Brazilian public universities, inviting unknown teachers and coordinators by emails collected on websites. Although the number of respondents was significantly reduced, when compared to the mapped institutions (20 participants), we collected subjective statements that clarify many of the identified challenges by inserting open fields in the diagnosis. Thus, we consider the second analysis a scoping survey. "Coordinator survey application" discusses the survey's highlights and the utility of the diagnosis.

Initially, the PBL-Gauge was applied in Federal Institutes of Education, Science, and Technology (FITs), institutions created by federal law 11.892/08 to promote basic, professional, and higher education, but also to offer education in different teaching modalities [46] . Each institute is led by a hierarchy of directors made up of a rector, pro-rectors, directors, coordinators, teachers, and administrators. With a multi-campus structure of more than 661 units, there are 38 FITs distributed in various Brazilian regions based on professional and technical education. Although not all campuses offer courses in the computing area, there are several courses in this area such as Computer Technician, Information Technology Management, Information System Analysis and Development, Computer Science, Databases, among others [46] . All 38 FITs responded to the survey, totaling 302 respondents (222 teachers and 80 coordinators). Considering that teachers can teach at more than one academic level, most of them work in technical (92%) and higher (75%) courses, while a small portion of them work in graduate (14%) and extension (22%) courses. This scenario was similar to the coordinators with technical (60%), higher (49%), graduate (3%), and extension (8%) ones. About 88% of the teachers stated that they apply active methodologies in their classes, while only 12% use traditional methodology. On the other hand, 62.5% of the coordinators answered that the institution where they work does not recommend a teaching methodology, while 22.5% answered that they recommend the active methodology and 15% indicate the traditional methodology. As for knowledge and experience in PBL, 57% of the teachers and 50% of the coordinators stated that they have median knowledge, with the level of experience also median or even lower (42% of the teachers and only 30% of the coordinators). Page 12 of 21

Teacher Survey Application Figure 6 shows an overall result from the 222 FIT teachers. On the "Problem" aspect, most stated using real, stimulating, and sufficiently complex problems as an element of study (favorable), but were uncertain about the appropriation of the problem-solving process by the student (warning). It was also evident the lack of interaction of students with real clients and users, compromising the authenticity of the learning environment (critical points). In PBL, real clients make it possible to build effective solutions through interactions that help students in the resolution process, providing feedback, and evaluating partial results [50] .

Regarding the "Environment" aspect, teachers partially agreed that the environment stimulates social skills, solving problems required by the professional market and fostering important concepts (favorable). However, the authenticity of the environment was again questioned, considering that problems do not always reflect real market demands, compromising the student's professional posture (critical). According to Bell [7] , the PBL promotes the construction of knowledge and skills considered relevant to their professional practice. Therefore, an inappropriate environment can impact this construction.

Concerning the "Content" aspect, the main favorable points are the alignment between theory and practice, integration between related courses, content approaches using practical projects, and appropriate subjects. Being institutions focused on professional education, it is understood that the course curricula have already been built for this purpose. Two points indicated a warning sign: flexibility of content in supporting the problem-solving process and content that promotes dynamic learning inside and outside the classroom. In PBL, the content needs to be flexible and focused on the problem-solving process [31] , with teaching that is much more "learning to learn" than "knowing a concept" that, far from practice, can easily be forgotten.

"Human Capital" was the most critical aspect. Although teachers follow the resolution of problems and stimulate group work with the formation of small teams of students (favorable), many showed uncertainty about the active posture and autonomy of students. Critical points are the lack of participation of the real client, a multidirectional relationship where everyone learns from everyone, and stimulation of collaborative projects by the course coordinator.

Finally, the "Process" aspect showed as favorable points the content planning, evaluation, and continuous monitoring by teachers and institutions. However, warning signs for learning with characteristics of metacognition and selfregulation and a course planning that meets the expectations of the students. The characteristics of metacognition and self-regulation are present in the PBL approach, enabling the student to perform self-reflection and perform monitoring and evaluation of their learning [45] . Figure 7 presents the overall result of the FIT coordinators' responses. In the "Infrastructure" aspect, the following are highlighted as favorable points: (i) the environment and technological tools that facilitate interaction and collaboration; (ii) a free space and sufficient time for self-learning of the student; (iii) the existence of resources; and (iv) the teachers with skills for tutoring activities and teachers in the role of learning facilitators. The high bias in the agreement of these points and the rate of uncertainty in the first five assertions indicate a warning sign for many institutions. Critical points were the absence of tutors to support the teacher, collaboration with real clients, and training in tutoring activities.

Concerning the "Politics" aspect, favorable points are the training realization; ease in planning, implementing and evaluating; and the culture of "learning by doing" existing in the institution. However, once again, the lack of interaction with the labor market became evident, besides the absence of a student selection process more appropriate to the pedagogical approach and the collaborative planning of courses. It is also worth highlighting a warning regarding the resistance of teachers and students to changes in the curriculum or pedagogical methodology, and the need for a planned budget for educational practices.

Regarding the "Curriculum" aspect, the following points stand out as favorable: the adequacy of the curriculum, an interdisciplinary and multidisciplinary curriculum, and a flexible curriculum. However, the result also shows a warning about the curriculum's ability to stimulate problem-solving and promote self-directed learning. Also, the lack of flexible schedule and alignment of time to content was highlighted as negative points.

Finally, for the "Evaluation" aspect, the following points stand out as favorable: monitoring and feedback, effectiveness and evaluations of the disciplines and the institution. Critical points were the absence of feedback from the evaluation system on the work of teachers and student learning, in addition to the lack of evaluation of teaching methodology and teacher performance.

Twenty Brazilian public universities were selected that offer at least one of the courses in the field of computing recognized both nationally, by the Brazilian Computer Society [58] , and internationally [12] : Computer Science (CS), Computer Engineering (CE), Software Engineering (SE), Information Systems (IS) and Information and Communication Technology (ICT).

We obtained the e-mail address of each selected individual through institutional websites. To collect these emails, The survey obtained a total of 20 respondents from 10 different institutions. Among these participants, 12 are professors (corresponding to 9.1% of the initial sample), and 8 are coordinators (corresponding to 18.18%), most from undergraduate courses (91.7% of professors and 100% of coordinators), totaling 20 respondents from 15 different institutions. Teachers of CS courses correspond to most respondents, while CE is more representative among coordinators. The SE course had the lowest number of respondents, considering few higher education courses like this in Brazil. The hybrid approach is the most used among teachers, corresponding to 46.2%, while 30.8% apply active methodologies and 23.1% use the traditional approach. We carried out this survey within the context of the Covid-19 pandemic, so when asked about other methodologies adopted, 16% of teachers mentioned that they use flipped classroom methodology and the video-based methodology in the context of remote learning. As for the coordinators, 62.5% declared that there is no recommendation of teaching methodology by the institution, 12.5% of the institutions recommend one of these methodologies: active, hybrid, or traditional.

Teachers have a good theoretical knowledge of the PBL methodology (76.9% with a level of knowledge between medium and excellent). Still, they do not have the same level of practical experience (only 46.2% declare having the same level). As for the coordinators, 75% declare they have no or little experience, against 25% who have a level of experience between reasonable and good.

Teacher Survey Application Figure 8 shows an overall result from the teachers of academic institutions. Regarding the "Problem" aspect, the results show, as favorable points, that teachers are concerned with using problems that stimulate students and reflect real challenges, just as students tend to appropriate these problems in class. However, these problems do not necessarily have the same complexity as the labor market, especially when there is no relationship with real customers. Some reports highlight this aspect in different scenarios: "In the Software Engineering course, PBL is used in 6 Problem Solving components. In three of these Fig. 8 Answers percentage of professors evaluation of academic institutions from problem, environment, content, human capital, and process aspects components, there is interaction with the user through articulation with extension projects or programs. (...)". "Some problems are closer to reality and have real customers. Others are related to reality, but can be simplified as an academic exercise". "I see that in my Unit, most of the course is disconnected from reality, but it is very difficult to change".

The institutions' learning environment introduces students to problems and concepts that reflect challenges in the professional context and develop the social skills and problem-solving capacity required by the labor market. Still, they do not have contact with clients and professionals in the market. Besides, several obscure points were also evident: students' attitude as computing professionals, if the learning environment allows experiences aligned to the professional environment, and if the infrastructure available in the learning environment is ideal for teamwork and collaborative activities. In this point, the reports of some respondents give us some highlights related to the difficulty to create a learning environment that reflects the reality of the market:

"It is difficult to reproduce the professional environment in terms of requirements and players". "The lack of historical funding from the public university has created an environment in which teaching laboratories are not valued, as most professors also did not have access to good teaching laboratories in order to understand their needs. The infrastructure is only good in the specific case of some research laboratories, which few scientific initiation students have access to".

Concerning the "Content" aspect, curricula use practical projects and activities to address content. The plan courses look for dynamics that encourage students to learn in and out of classrooms, allowing flexibility to modify the teaching plan and methodology. However, there was no evidence about the integration between related courses and whether courses are designed with content that supports problemsolving. As critical points, respondents criticize the content courses and difficulties in making changes:

"The course subjects are traditional, which is not always good. However, it is difficult to make very different things better. Outside the research environment, there is no incentive for change within the public university. Furthermore, having to support research, teaching, extension, and administration is very difficult." "The content of the course is very abstract and outdated. I always update and adapt the content of my courses with updated material and a direct relationship with the market." According to Fig. 8 , Human Capital is the most critical aspect, highlighting the following challenges: (i) the learning process is not multidirectional; (ii) there is not experience that encourages the exchange of knowledge between teachers, students, tutors, and real clients; (iii) pedagogical and coordination teams do not work together to develop a more collaborative teaching project for students. In addition, the teachers highlighted the lack of participation of real customers who collaborate in the requirements of the problems used in the classroom as commented on the other two aspects discussed. There was no evidence that the students behaved actively and autonomously and if the teachers followed up on problem-solving processes. One of the respondents reported a possible consequence of these points:

"Students have presented many questions about the 'Process' of the course, but reformulating the course is difficult, it generates a lot of controversies because each teacher has their point of view and most teachers have no experience in the 'job market,' the which results in a very theoretical course disconnected from the 'market.'"

Although we have evidence that professors apply assessment methods that match the planned content for the course, the data obtained are not sufficient to define the current scenario of assessment processes in public universities. Anyway, several warning points were signaled related to the continuous monitoring of the learning process, the encouragement of students to discuss and reflect on the content, and the metacognition and self-regulation of students. Other warning points refer to the role of teachers in monitoring students' learning along the problem-solving process and whether the educational planning meets the students' learning expectations. As for the course planning and learning goals definition, the lack of collaboration between members of the pedagogical team is a mentioned challenge by professors, who explained that the course planning is still an individual process and depends on each professor, also stating that: "I would like to have more collaboration among teachers in this knowledge area. I seek to monitor and obtain feedback from students throughout classes and at the end of the course".

These results clearly point to the need for a deeper investigation of the Figure 9 presents the overall assessment models and processes adopted by these institutions. According to the found results, the infrastructure of institutions provides learning environments and tools that promote collaboration and self-learning of students and provide adequate technological resources and systems for teachers.

Teachers help in the learning process and encourage students' autonomy. But, there is a shortage of trained tutors to accompany students, and market professionals in the academic environment.

Regarding the "Politics" aspect, these institutions invest in the formation of pedagogical teams to evolve the teaching process but do not consider the resources necessary for practical approaches to institutional budgeting. Students are adaptable and are not resistant to changes in methodology and pedagogical curriculum, but there was no evidence of the same behavior on teachers. Statuses of FIT teachers' answers from Problem, Environment, Content, Human Capital, and Process aspects ( Source: [47] It is possible to adapt the course curricula to a new methodology, but there is no collaboration between the members of the pedagogical team in teaching planning. Furthermore, evidence has shown that courses do not have flexible schedules such that teachers can tailor them to their needs. We found some indeterminate points in this aspect. There is not enough evidence to conclude whether the adopted curricula encourage problem-solving or whether there is an alignment between the duration of classes and the content that teachers should teach.

According to the coordinators, the courses have evaluative criteria that guarantee the effectiveness of the evaluation process used. Still, there is no evidence of continuous feedback provided by teachers to students. The disciplines use several aspects in their evaluation process, and the institution evaluates the results generated by the courses. However, the institution has no evaluations regarding the teaching and learning methodologies, nor are there any feedback systems for the evaluation results to improve the teaching and learning process. Figure 10 presents an overview of the results from teachers' questionnaires for the FITs. The results show a predominance of favorable points on the Problem, Content, and Process aspects, with respect to the panorama of all the institutions involved in the survey. Considering the education purpose focused on professional performance, in general, these institutions have worked with real and relevant problems, with appropriate content for problem-solving, and have a student evaluation process.

On the other hand, there was a greater predominance of critical points in the Human Capital aspect, highlighting how impactful the PBL culture can be in its adoption. Based on this understanding, PBL training recommendations can be made regarding each actor's responsibilities and roles in the learning environment and the inclusion of new actors who can make a difference, such as real clients and tutors. It is worth noting the number of warning signs on the Environment aspect, indicating that there are still many uncertainties regarding preparing the teaching and learning environment for PBL in these institutions. Figure 11 presents an overview of the results from coordinators' questionnaires for the FITs. Under coordinators' perspectives, it can be clearly seen how much the structural and organizational aspects can negatively impact the PBL implementation, having as main axes of concern the infrastructure and the politics. Again, it became evident that it is necessary to work on the institution's PBL culture, involving educational managers and investing in teacher training to prepare institutions for the radical transformations that PBL brings. Other critical points also highlight the need for flexibility of the content, maintaining the alignment of theory with practice, and a special concern with the continuous Fig. 11 Statuses of FIT coordinators' answers from Infrastructure, Politics, Curriculum, and Evaluation aspects ( Source: [47] Fig. 12 Statuses of teachers' answers from problem, environment, content, human capital, and process aspects in academic institutions evaluation process involving not only students, teachers, and coordinators. Figure 12 shows an overview of the results from teachers' questionnaires for the academic institutions. The results show a predominance of good points on the Problem and Content aspect. In these institutions, teachers have more autonomy to define their teaching methodology and make the content of their courses more flexible. However, according to participants' statements, this autonomy lacks collaboration between teachers and their coordinators as a counterpoint. Those invited to answer this survey voluntarily were also concerned with solving real problems. However, as in technical institutions, there is a lack of interaction with professionals from the labor market, bringing their realities to the academic environment.

Again, the Human Capital aspect was critical in this context, pointing out the need for teachers' training and an institutional PBL culture. Also, the difficulty in evaluating several issues was quite evident, showing many obscure points, such as the Environment and Process aspects. We believe that the main reason is the characteristic of the sample, in most of the cases, with unique participants from diverse institutions (in terms of capabilities, structures, resources). This characteristic reinforces how difficult it is to conclude assertions in the application of the diagnosis when we do not involve a consistent sample of participants from the same course. Figure 13 shows an overview of the results from coordinators' questionnaires for the academic institutions. According to coordinators, academic institutions are more prepared concerning infrastructure. However, the aspects related to a policy aimed at the PBL approach left something to be desired. The coordinators' opinion also highlighted the need to work the PBL culture in the institution, involving educational managers and investing in teacher training. Other critical points also highlight the lack of flexible curricula, aligning theory and practice. Again undetermined points were evident in the evaluation aspect. The results show that evaluations are applied and based on criteria, but coordinators do not use their results to improve the teaching and learning methodology. In short, there is planning, but there is no management of pedagogical aspects in these institutions.

We can do some considerations for the best use of the diagnosis, reflecting on two opinion surveys. First, it is necessary to pay attention to the choice of the sample of participants, involving the entire faculty of the same undergraduate course and the team that supports the course coordination, rather than just the coordinator. This strategy will allow collecting opinions from different perspectives, providing richer data for educational planning discussions. Second, it is necessary to explain the diagnosis and its objectives before its application. At this time, we can clarify specific doubts about the issues, and all participants will be aware of the ongoing educational project. Third, we recommend encouraging participants to insert feedback on open questions. Personal feedback can be of great help in understanding and clarifying the diagnostic results.

Finally, from diagnosis results, a discussion with those involved in the diagnosis is recommended with each institution to raise the points of improvements and uncertainties, identifying and managing the risks for implementing the PBL.

Publications on PBL experiences applied to computer education have grown yearly, stimulated by international conferences and journals in Computing Education Research focused on educational innovations. As PBL is a model that we can implement in different ways, we found some studies proposing methodologies based on PBL in two main categories: first, more focused on instructional design, observing the environment inside the classroom and its dynamics; second, focused on planning and managing the approach, identifying the elements inside and outside the classroom that need to be considered and managed. This study contributes to the second category, bringing a proposal for an institutional diagnosis for use in the initial stages of planning the approach and making the strengths and improvements for the adoption of PBL transparent. With the application of this diagnosis in real-world educational institutions, both in the private and public sectors, it was possible to define a more stable version of this diagnosis, which we can apply in other institutions and contexts of PBL projects in Computing Higher Education (CHE).

Regarding the CHE as an application domain, the entire investigation, design, and construction of the PBL-Gauge questionnaires were based on references to the adoption of PBL in Computing. Considering that PBL was born in the Fig. 13 Statuses of coordinators' answers from Infrastructure, Politics, Curriculum, and Evaluation aspects in academic institutions medical area but has been adopted in other knowledge areas, it is crucial to emphasize that we are in a different context from medicine, with characteristics, pedagogical or not, specific to the training of computing professionals. Therefore, we chose to be careful about the recommendations for using this diagnosis in the CHE context.

As a limitation of the literature, we highlight the absence of studies that describe assessment instruments, aiming to support academic managers and pedagogical staff in planning PBL projects in their educational institution, based on data and opinions of the stakeholders involved. Many authors comment on the challenges and requirements of the adoption of PBL, such as [35] , Hsu et al. 2016 ; [1, 20, 45] , propose models for implementing this approach [16] , Santos, Furtado and Lins 2014), review and analyze the main benefits and challenges of the approach in Computing (Santos 2020). Still, it does not instrumentalize the stage of understanding the problem of implementing the PBL from the context of the target institution. On the one hand, this scenario hampered a direct comparative analysis of PBL-Gauge with other approaches, leading us to discuss PBL challenges and requirements rather than artifacts to support PBL projects in their initial phase. On the other hand, the proposal of an institutional diagnosis with this objective came to fill this gap, providing an instrument that institutions can use or even adopt in the face of educational innovation projects, promoting a reflection as deep as desired on what needs and can be done.

Based on the motivation of how to assess whether an educational institution in the context of CHE is prepared to adopt the PBL, this study proposed an institutional diagnosis based on the pedagogical, structural, and organizational dimensions, evaluating nine aspects with two groups of stakeholders: teachers/tutors and coordinators/managers. To evaluate the model, this diagnosis was applied in 38 institutions of technical education in computing through an opinion survey, obtaining an overview of these institutions' situation with 302 respondents (222 teachers and 80-course coordinators in computing). Due to these institutions' professional characteristics, several favorable points were identified, such as the use of real problems, curriculum, and student evaluation. On the other hand, critical points such as the academy's lack of interaction with the labor market, teacher training, and a more effective and ample assessment process were also highlighted. In particular, some warning signs have also been identified regarding the suitability of the learning environment for PBL and the institutionalization of PBL culture.

The second application of this model was carried out in 15 Brazilian public universities, with 20 responses and personal comments from some participants. According to the results, the Brazilian public universities investigated already have a large part of the requirements for implementing the PBL methodology. Improvements were pointed out on the aspects of human capital, learning environment, processes, and policy. Among the main difficulties, the following stand out: (i) lack of funding and incentives to make changes; (ii) course disconnected from reality; (iii) teachers with no knowledge of the labor market; (iv) lack of collaboration between professors; (v) rigid, outdated, and difficult to change curriculum; and (vi) good infrastructure in some research laboratories that allow access to few students.

The next steps are intended to make improvements in questionnaires and apply them to individual institutions, based on a careful analysis with their stakeholders. It is also intended to apply statistical methods for a more rigorous analysis.

Project-based learning: Implementation & challenges

The conceptual framework of online problem-based learning towards problem-solving ability and programming skills

Using problem based learning environment to enhance algorithmic problem solving skill

Transitioning to the real world through problem-based learning: a collaborative approach to teacher preparation

Fundamentos de Metodologia Científica: guia para a iniciação científica (Fundamentals of Scientific Methodology: guide for scientific initiation)

Identifying good practice in the use of PBL to teach computing

Project-based learning for the 21st century: Skills for the future

Peer instruction in. Revista Brasileira De Informática Na Educação

Formação Educacional e Empregabilidade em TIC Achados e Recomendações. Brazilian Association of Information Technology and Communication Companies (Brasscom)

Procura por profissionais de TI será de 420 mil pessoas até 2024 e o Brasil forma apenas 46 mil por ano. Brazilian Association of Information Technology and Communication Companies (Brasscom)

Fatores explicativos para a evasão no Ensino Superior através da análise de sobrevivência: o caso da UFPE

Computing Curricula 2020: Paradigms for global computing education

Research this! Questions that computing educators most want computing education researchers to answer

Assessing problem-based learning in a software engineering curriculum using Bloom's taxonomy and the IEEE software engineering body of knowledge

Resources, tools, and techniques for problem based learning in computing

Promoting and supporting PBL interests world wide: the profile of the UICEE Centre for Problem Based Learning

Routledge Falmer: Taylor & Francis

UICEE centre for problem-based learning (UCPBL) at Aalborg University. In 6th Baltic Region Seminar on Engineering Education

PBL in engineering education: international perspectives on curriculum change

Problem-Based Learning Use in Higher Education: A state university faculty study of PBL. (Doctorate dissertation in Education

Problem-based learning: What and how do students learn?

O que leva os alunos dos cursos superiores de computação a evadirem? Um estudo de caso feito na Universidade de Brasília

Research agenda for Computer Science Education

Impact of applying WebGL technology to develop a web digital game-based learning system for computer programming course in flipped classroom

Generic framework design of Project-Oriented Problem-Based Learning (POPBL) for Software Engineering courses

Framework of a reflective e-portfolio supported by outcome based education and problem based learning

Integrated e-project collaborative management system: Empirical study for problem-based learning project

Have we missed something? Identifying missing types of research in computing education

Personal opinion surveys. In guide to advanced empirical software engineering

The effectiveness of problem-based learning supported with computer simulations on reasoning ability. Procedia

Experiences from an experiential learning course on games development

Assessing IOT projects in university education-A framework for problem-based learning

Problem based learning associated to the development of games for programming teaching

Estratégias de ensino e aprendizagem baseadas em problemas (PBL) no ensino tecnológico

Reflections on making a change towards project oriented and problem-based learning (POPBL)

Challenge Based Learning. White Paper

Practical problem-based learning in computing education

PBL in teaching computing: An overview of the last 15 years

Problem-based learning framework for junior software developer: empirical study for computer programming students

Um Estudo Terciário Sobre o Ensino de Computação no Brasil

Aprendizagem baseada em problemas (PBL) na educação em engenharia

Um framework conceitual para implementação e gestão da abordagem PBL no ensino de Computação

A framework for applying problem-based learning to Computing Education

Problem-based learning in a computer engineering program: quantitative evaluation of the students' perspective

Diagnóstico Institucional para Adoção de PBL em Cursos de Computação: Uma Pesquisa de Opinião em Institutos Federais de Educação, Ciência e Tecnologia do Brasil. (Institutional Diagnosis for PBL Adoption in Computer Courses: An Opinion Survey in Federal Institutes of Education

Are we ready for problembased learning? A proposal of institutional diagnosis in computing higher education

PBL-Test: A model to evaluate the maturity of teaching processes in a PBL approach

xPBL: A methodology for managing PBL when teaching computing

Two decades of PBL in teaching computing: a systematic mapping study

Problem based learning: An instructional model and its constructivist framework

Graduação em Computação no Brasil: perspectiva a partir do Exame Nacional de Avaliação de Desempenho

Aprendizagem baseada em problemas (ABP): um método de aprendizagem inovador para o ensino educativo [Problem-based learning (PBL): An innovative learning method for educational teaching

Comparing problem-based learning with case-based learning: effects of a major curricular shift at two institutions

Dimensões da competência informacional

Work in progress-a mobile computing collaborative framework for problem-based learning environment

A new theoretical PBL model for MIS course design and research

Referenciais de Formação para os Cursos de Graduação em Computação

Acknowledgements The NEXT group thanks all the institutions that participate in the survey and their respective respondents, in addition to the group of experts who evaluated and contributed to the production of the survey questionnaires. Many thanks to all involved in this study. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior-Brasil (CAPES) Finance Code 001.

A rationale and literature references for Teachers' and Coordinators' questionnaires are available in: https:// docs. google. com/ sprea dshee ts/d/ 1E0fKh-NimU-OAhCed-geJ8b KpH8Q zgmtq KD1VQ zUupo/ edit? usp= shari ng