education sciences Article Machine-Assisted Learning in Highly-Interdisciplinary Media Fields: A Multimedia Guide on Modern Art Elena Chatzara, Rigas Kotsakis, Nikolaos Tsipas, Lazaros Vrysis and Charalampos Dimoulas * Multidisciplinary Media and Mediated Communication (M3C) Research Group, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece * Correspondence: babis@eng.auth.gr Received: 9 June 2019; Accepted: 22 July 2019; Published: 25 July 2019 ���������� ������� Abstract: Art and technology have always been very tightly intertwined, presenting strong influences on each other. On the other hand, technological evolution led to today’s digital media landscape, elaborating mediated communication tools, thus providing new creative means of expression (i.e., new-media art). Rich-media interaction can expedite the whole process into an augmented schooling experience though art cannot be easily enclosed in classical teaching procedures. The current work focuses on the deployment of a modern-art web-guide, aiming at enhancing traditional approaches with machine-assisted blended-learning. In this perspective, “machine” has a two-folded goal: to offer highly-interdisciplinary multimedia services for both in-class demonstration and self-training support, and to crowdsource users’ feedback, as to train artificial intelligence systems on painting movements semantics. The paper presents the implementation of the “Istoriart” website through the main phases of Analysis, Design, Development, and Evaluation, while also answering typical questions regarding its impact on the targeted audience. Hence, elaborating on this constructive case study, initial hypotheses on the multidisciplinary usefulness, and contribution of the new digital services are put into test and verified. Keywords: modern-art; multimedia guide; painting-movements recognition; blended learning; technology-enhanced learning; machine learning 1. Introduction There is no doubt that technological evolution has always affected society and everyday human activities at multiple levels, i.e., communicational, professional, educational, etc. Following the rapid advancement of Information and Communication Technologies (ICTs) that have been conducted during the last two decades, a new trend, which is referred to as Industry 4.0, aims at further revolutionizing digital processes through end-to-end automation mechanisms. Amongst others, the media landscape has been already transformed into a new digital experience (and continues to elaborate constantly), bringing modern tools forward that accelerate mediated communication, i.e., thru easy multimedia production and sharing, sophisticated content documentation and management, semantic interpretation and conceptualization of data (and meta-data), information recognition, and retrieval. While these fresh services offer innumerous capabilities in both ends of media production and consumption, they also raise multiple concerns, uncertainties, and hesitations regarding how to exploit the released capacities without being exposed to ethical, privacy, and security risks or dangerous pathways [1–5]. Clearly, the frequently immature and unstable character of the lately launched assets/facilities, along with the lack of related operational knowledge and experience, are listed among the foremost causes that deteriorate fast adoption and utilization of pioneering machinery [4–8]. Nonetheless, contemporary Educ. Sci. 2019, 9, 198; doi:10.3390/educsci9030198 www.mdpi.com/journal/education http://www.mdpi.com/journal/education http://www.mdpi.com https://orcid.org/0000-0003-2900-4657 https://orcid.org/0000-0001-7923-9361 http://dx.doi.org/10.3390/educsci9030198 http://www.mdpi.com/journal/education https://www.mdpi.com/2227-7102/9/3/198?type=check_update&version=2 Educ. Sci. 2019, 9, 198 2 of 22 forms of human creative and communicative expressions (artistic, cultural, social, etc.) require a highly multidisciplinary approach, in which STEM skillsets (Science, Technology, Engineering, Mathematics) need to be combined with humanities and sociological disciplines [9–13]. Hence, the usefulness of ICT toolsets in the media environment and, especially for the training demands, has two-folded merit: firstly, to expedite the learning process by deploying digital assets for better content demonstration and enhancement purposes and, secondly, to organize dedicated coaching sessions, as to support the proper use of tools and overall technology. In all cases, it is vital to prepare fertile ground for embracing and exploiting cutting-edge progress and brand-new developments. 2. Background—Review or Related Work Extending the above, it is apparent that we are living in a world of data, where ICT devices and computing terminals are literally everywhere. New terms (and trends), like ambient-media, have been invented and unveiled to describe the pervasive nature of information exchange and communication that have become commonplace in today’s digital and ubiquitous society [14,15]. Novel genres, i.e., Mobile, Multimedia and Data Journalism, have also launched to support the required non-linear dataflows, seeking audience engagement through enhanced storytelling with multiple augmentation layers [1–4,10–18]. However, the development of such software/multimedia services requires highly interdisciplinary production teams, which could serve the associated aesthetic, operational, communicational, and computational specifications, without disregarding the desired content and its accessing/browsing functionalities. The latter is usually dependent on the nature of the specific topic and the selected presentation, which is also known as the “education” approach, being deployed by the collaboration of the instructional designer with the subject matter expert(s). These specialties must cooperate with the graphic designers, the software engineers and programmers, the content creation teams, the media/communication advisors, and many others, who all need to have some common background and use the corresponding terminology properly [19–25]. Undoubtedly, the procedures of web/multimedia authoring, and generally the development of Human-Computer Interaction (HCI) software, are considered to be extremely multidisciplinary, especially when it comes to rich-media presentation of informatory data. Such conditions are encountered in the processes of creative production and education, in which suitable material should be digitized, properly formed, and finally assembled to user-friendly integrated environments for entertainment, infotainment, or learning purposes. Aiming at serving such demands at a schooling level, a related Interdisciplinary, Interdepartmental Post-graduate Program has been formed and it has operated for nearly fifteen years in the Aristotle University of Thessaloniki, offering master-degree studies on “Advanced Computing and Communication Systems”. Among the available pathways, the specialization of “Audio-visual Technologies in Production and Education” combines disciplines from engineering and media departments, both in terms of academic staff and preferable graduation of applicants (e.g., School of Electrical and Computer Engineering, School of Journalism and Mass Communication, School of Musical Studies, etc.). The utmost target is to bring the different sectors closer, to combine the various talents and backgrounds, and to cultivate some mutual skills and interests in all of the aspects of technology-assisted (audiovisual) creation and learning (technological, artistic, aesthetic, ethical, functional, etc.). The paper describes Technology-Enhanced Learning (TEL) strategies for one of the highly interdisciplinary courses, which deals with the History of Audiovisual Artistic Expressions to end up in the thorough presentation of important art movements. The primary motive starts on the basis that it is rather tough to enclose art in classical rule-based teaching guides, especially for students presenting a STEM or media background, who are seeking to obtain expertise on Advanced Computing and Communication Systems. Hence, the idea was to construct a multimedia/web guide on modern-art painting styles, for both in-class demonstration and self-training support. This project started as a diploma thesis undertaking [13], which investigated the impact of different e-learning methods, to indicate, shape, and assemble the applicable HCI interfaces into an integrated browsing Educ. Sci. 2019, 9, 198 3 of 22 experience. The present paper extends development and evaluation perspectives, assessing the educational challenges of content/coaching augmentation and audience engagement following the codes and values of blended learning [26]. Multimedia elements, interaction mechanisms, and gaming components are combined and tested in the current multidisciplinary schooling scenario, following the so-called LUCID principles (Logical User-Centered Interactive Design) [19–23]. Apart from the design itself (and its evaluation), the deployed interactive implementation offers valuable insights for the impact of technology-assisted media tutoring. Specifically, the initial hypotheses can be examined by answering specific questions on User eXperience (UX) achievements, thus retrieving vital feedback. This latest feature stands as the outmost target, not only for improving the fabricated services and their utilities, but also for adopting the best practices, arriving at useful conclusions that could be used as representative use-case scenarios and featured studies in other classes (i.e., in projects of the laboratory course Multimedia Production that is also offered in the same program, supporting most of the discussed viewpoints). The above-listed exploration perspectives extend the scopes of the present work to broader research considerations on professional media training and digital literacy support. 3. Research Aims and Project Motivation The current work demonstrates the implementation of a web/multimedia guide on modern art, following the structure and the appropriate development models that are usually adopted in related projects [27]. In the same context, principal values of blended learning [26] are considered for the in-class utilization of the targeted tools, incorporating modalities that would expedite interactivity with rich user experience, intensifying audience engagement (i.e., gaming elements, simulation examples, etc. [26,28]). Furthermore, algorithmic recognition of those painting styles offers another technological domain when considering that classification of significant art movements is included in the syllabus and goals of the selected/demanding course, whereas smart systems could be used as assistive training machines. Artificial intelligence does represent a cutting-edge topic that is essential for many higher education programs [29,30], ranging from standard STEM subjects to broader multidisciplinary studies, which is also the current use-case scenario. Hence, Machine Learning (ML) modules can be tested for their contribution as schooling material, i.e., to make the teaching procedure more appealing for students having loose relation to artistic/social sciences. At the same time, such a type of exercising could force digital literacy support for less digitally oriented audience, providing applied tutoring on core functional/algorithmic features, for many of the presently elaborating/upcoming trends in our ubiquitous information society (i.e., Semantic Web, Internet of Things, Big Data, etc.). Extending the above, these processes are also suitable for obtaining useful feedback and crowdsourcing-driven indirect annotation that could further propel artificial art recognition projects [12,13]. Even if these perspectives are considered somewhat beyond the main scope of TEL, the associated efforts are thought as being amongst the most challenging and trendy examples of inter-sectoral research [31–39], fueling cross-domain collaboration and knowledge exchange. Overall, the paper attempts to investigate and thoroughly analyze the potential advantages of TEL services in the media business, detecting the difficulties and needs that are associated with the interdisciplinary nature of this area, while stressing out convincing answers and prosperous solutions. Specifically, apart from the presentation material used in class, the idea was to develop an online guide, holding all of the necessary information that someone would request concerning this course and broadly this topic. The effort was directed in the examination of related examples on other languages since there was not available such an integrated site for the history of art in Greek, following the principles of exploratory development and throwaway prototyping (i.e., selecting/implementing the useful parts/utilities, while removing/replacing the ones seem to be problematic) [19–23,40–43]. As the next section explains, the name Istoriart was chosen after the Greek word history (Iστoρἰα = Istoria) and the global term art (i.e., history of art in Greek). Thus, adopting the main phases of software engineering encountered in web-/multimedia-authoring, the current work pursues best-practices on the design, application, and evaluation of TEL resources, emphasizing the interdisciplinary nature Educ. Sci. 2019, 9, 198 4 of 22 of contemporary media services. The deployed interactive prototyping model put targeted users on the center of the design (LUCID framework), aiming at obtaining valuable feedback concerning the wanted tools and their utilization. Usability assessment is employed at almost all the phases of project evolution, while a focused investigation on small-grouped training sessions is proposed to test the accomplishment of the set goals. In this context, the paper deals with the aspects of technology-enhanced (and on-demand) media learning, having a two-folded intention. Firstly, to review the Istoriart agile development case-study, aiming at concluding the adoption of best-practices, which are suitable for the current scenario and broader usage. Secondly, to validate specific hypotheses, which are related to the wanted educatory character and impact, by answering the corresponding research questions. Based on the above analysis and given as granted that contemporary media paradigms feature many teaching difficulties for learners with different scientific background [1,2,4,5], the hypotheses besides the conducted research are stated, as follows: RH1: A properly organized multimedia guide could facilitate the process of media learning, both for enhancing the in-class teaching experience and for offering self-training support. RH2: If specific modalities are selected and appropriately assembled, online training services can propel audience engagement (even if the presented topics are different from their core disciplines, thus posing comprehension difficulties). Moreover, targeted audience (i.e., students, trainees) can provide valuable feedback, helping to improve the teaching experience (in multiple levels), while also promoting digital literacy support, for both the involved topics and broader media processes during the implementation of TEL services. In this context, risen research questions accommodated to the listed hypotheses are as follows: RQ1: Are the targeted users (media students/trainees) interested in using interactive multimedia applications in media learning topics? RQ2: What is the impact of technology-enhanced learning resources in supporting a specific highly multidisciplinary media subject (i.e., modern art), prioritizing audience engagement? Extending the above, representative users of the targeted audience can actively participate in the implementation of such TEL services, providing valuable feedback to indicate the different problems, issues, and challenges that media training is facing nowadays, caused, amongst others, by its rapidly evolving multidisciplinary nature. Specifically, the current work aims at enlightening various diverse perspectives, which are related to the rapid prototyping procedures of project implementation. Close collaboration between experts of different backgrounds and skills has been forced (including the authors of this paper) to indicate the advantageous TEL values for confronting the demanding nature of the discussed cross-domain subject. Broader conclusions and the adoption of best-practices are targeted to promote digital literacy support, for both the involved topics and wider media processes. 4. Materials and Methods As already stated, the implementation model follows the principles of human-centered design. Starting with the initial idea, the phases of analysis, design, development, and evaluation are successively repeated in a spiral model. Hence, functional requirements and implementation details are continually refined and elaborated until the targeted criteria are finally met [19–23]. In this context, risk assessment and reduction procedures are engaged at every iteration/cycle, which ensures the desired communication and collaboration with the targeted audience (that is very important in the current case). From a different perspective, the Component-Based Software Engineering (CBSE) model is also involved, which allows for existing material to be part of the process (e.g., user interfaces, multimodal content/assets, applicable/already available code, etc.). The main concept is gradually transformed to crisper functional requirements and specifications, in which the analysis of related web applications Educ. Sci. 2019, 9, 198 5 of 22 and users’ preferences guide the structure and the aesthetics of the entailed prototypes. Gaming components, interactive multimedia presentations, and simulations are then imposed, which measure the offered functionalities against the computation and communication load demands. Usability evaluation and UX analysis are also repeatedly deployed, taking valuable feedback for the accurate direction of the next design steps. Project completion takes place with the integration of all the produced or refactored components, which triggers the final evaluation within and outside the production team (alpha/beta testing). The associated stress-tests are executed in typical/simulated learning sessions, which are aimed at validating the initially stated hypotheses by answering the risen questions. 4.1. Analysis The goal of Analysis is to elaborate the initial idea, to examine the demands and preferences of the targeted audience, and to formulate the first list of functional (and non- functional) specifications. There are two main procedures that are deployed in this phase, namely, to review the related applications and to investigate the needs of selected/representative users. The first process was executed in parallel by the members of the production team (including authors of this article), leading to a brainstorming discussion group, where all of the findings were inspected in the viewpoint of exploratory development and throwaway prototyping (mentioned above). Therefore, a list with related services and their qualitative evaluation was formed to highlight the interesting ideas, usable features, and possible best practices to adopt. Table 1 contains the results of the top-ranked applications/webpages on modern art, synopsizing favorite elements and characteristics that are to be considered during the subsequent design steps. Table 1. Review/qualitative evaluation of related applications (mostly websites on modern art) [13]. Name URL Address Comments Wikipaintings www.wikiart.org clear navigation, good aesthetics, precise art-movements classification Wikipedia www.en.wikipedia.org/wiki/ Modern_art poor aesthetics, crisp texts Artcyclopedia www.artcyclopedia.com no aesthetic pleasure, disorienting navigation structure GoogleArtProject www.artsandculture.google.com good aesthetics, rich visual content, options to create personal collections Tate-modern www.tate.org.uk/visit/tate-modern good aesthetics, rich visual content, inspiring confidence, browsing flaws (missing return/back options) Greek Arts Museum www.greekstatemuseum.com/kmst/ index.html good aesthetics, clear navigation, rich visual content Theartstory www.theartstory.org clear and pleasant navigation, concise summaries with options for further/extensive information lookup, satisfactory visual content, crisp/salient texts, inspiring confidence MoMA www.moma.org bright aesthetics, rich content, pleasant navigation, crisp texts National Gallery of Art (NGA) www.nga.gov user-friendly, rich in content, good aesthetics, inspiring confidence Guggenheim Museum www.guggenheim.org rich in visual content, accurate descriptions, clear and pleasant navigation, concise summaries with options for further/extensive information lookup, bright aesthetics The second important analysis task, monitoring of users’ preferences, was conducted through an empirical survey, with the help of a corresponding questionnaire. The answers of forty-seven (47) www.wikiart.org www.en.wikipedia.org/wiki/Modern_art www.en.wikipedia.org/wiki/Modern_art www.artcyclopedia.com www.artsandculture.google.com www.tate.org.uk/visit/tate-modern www.greekstatemuseum.com/kmst/index.html www.greekstatemuseum.com/kmst/index.html www.theartstory.org www.moma.org www.nga.gov www.guggenheim.org Educ. Sci. 2019, 9, 198 6 of 22 individuals were finally selected (out of 103) to balance the diversity of the sample (therefore, the statistical reliability of the approach), which featured age ranging between thirty and fifty years, with interests and professional occupations in the domains of art and education (that was evaluated as positive). In addition, the majority of the participants attended a university program and/or they were interested in post-graduate studies (>80%), with only a small percentage posing a post-graduate degree (~11%), while most of them had frequent access to the Internet and, in many cases, via multiple devices (desktop/laptop PC, smartphone, tablet, etc.). Finally, although the majority was interested in modern art, there were significant variations regarding specific preferences (i.e., period or species) and pre-existed knowledge. More detailed information regarding this survey is provided, along with the assessment outcomes in the associated results section. At this moment, it is important to maintain that the selected group matches the set goal, while considering that the guide is primarily intended to support a post-graduate interdisciplinary course (and broader related master pathways) that deals with media, artistic expressions, and overall creative communication. The initial analysis resulted in the description of some basic requirements and guidelines to be used in the remaining design iterations. Specifically, it validated the informative and educational character of the targeted guide, positing a modular structure with articles on painting movements, artists, timelines, and game components. The enhancement of the visual element and the incorporation of featured photo-galleries was also decided, prejudging a “chic & simple” aesthetical approach. While the main purpose was to have the material available online, the necessity to be able to work without Internet connection was also pointed out (using a copy of “downloaded site”), so as to adapt in the various teaching conditions. This last remark rather excludes some easy-to-use live web authoring tools and associated generic-purpose CMS tools (Content Management Systems), hence, custom-made solutions had to be deployed, without disregarding the needs for central managing and updating mechanisms. Finally, attention was paid to maintain low computation and communication load, thus to expedite fast browsing experience, without unnecessary interruptions and delays. Overall, the Analysis outcomes include the findings of the survey (discussed in the related results section), as well as the extracted specifications that were evaluated and verified with the users that were involved in the LUCID process (especially for the parts dealing with functional and content-wise attributes). 4.2. Design Taking the deliverables of the Analysis cycles as the inputs, low-fidelity prototyping iterations were deployed during this phase (mock-up design), which refined the structure of the web-guide and the overall navigation experience (Figure 1a). Specifically, many features of “theartstory.org” were indicated as the most appropriate to be adopted in the current approach, aspiring to be the respective Greek point-of-reference for modern art and principally painting movements. The main screens of the corresponding thematic categories were sketched, while the Istoriart logo ideas were brought into the table and refined during the successive spiral design (Figure 1b,c). Color pallet, chromatic code, and page composition were decided for the entire site, providing some initial high-fidelity prototype illustrations (Figure 1d). Apart from the “painting movements” and the “artists” themes, in which popular styles and painters would be listed and linked to featured pages, further elements (i.e., timelines, games) were implanted to augment interaction, which targeted rich-media experience and audience engagement. Hence, the first two items of the navigation bar point to the primary learning resources, which might also form the syllabus of a related class (like the post-graduate course of the working scenario), while the rest of them offer enhanced ways of accessing and interacting. Timelines were again inspired by “theartstory.org” to draw infographic visualizations, which highlighted key-events and generally temporal style elaboration. In this view, alternative schemes for information organization and presentation are offered, facilitating various filtering, searching, or indexing mechanisms, which can be adapted to the (self-) instructing preferences of both in-class students and distant trainees. Such exploratory capabilities are additionally supported with the “links” section, which directs users to selected external sites. Multimedia quizzes, photo-galleries, and puzzles were lastly considered as the Educ. Sci. 2019, 9, 198 7 of 22 entertaining components, which aimed to stimulate engaging schooling activities through educational games. All of these processes and their outcomes were systematically discussed and evaluated with the participating users, to come to the final decisions. Figure 1. Design of the proposed Istoriart web guide [13]: (a) organization and navigation structure (proposed sitemap); (b) low-fidelity prototype (mock-up design); (c) alternative logo suggestions; and, (d) illustration of a high-fidelity prototype. Recalling the analysis and design conclusions, the goal for Istoriart was to develop an interactive environment that would be accessible through any computing device (smartphones, tablets, laptop, and desktop computers). A static site was advocated based on the stateless web development example to achieve this goal, allowing for the consumption of interactive contents by clients with and without internet connectivity. A significant advantage of the chosen stateless development model is the fact that there is no shared state between the servers (e.g., a database), which simplifies the scaling of the application, thus allowing it to serve high volumes of traffic [13,44,45]. A computational architecture was also designed, which incorporated all of the stated specifications and requirements into the blueprinted diagram of Figure 2. The Docker-based containerization approach allows for achieving isolation between different services running on the server, also simplifying the authoring/programming and deployment workflow. Specifically, a docker image is generated and pushed to a repository, containing the Apache web-server and Istoriart content. A virtualization layer on top of hardware resources, as supplied by Okeanos cloud platform, is employed for the hosts running the containerized application. Along with these technological suggestions, it was also decided that Istoriart would be delivered as an additional microservice through the “arutv.ee.auth.gr” domain, Educ. Sci. 2019, 9, 198 8 of 22 in the URL address http://arutv.ee.auth.gr/istoriart/. The reason behind this choice is two-folded: (a) ARUTV is an interactive-TV effort that started within the same post-graduate program, which often served as a repository of other audiovisual and multimedia products, and (b) this repository is engaged in the educational activities of some other classes, including the mentioned courses of “Modern Art” and “Multimedia Production” [13,44,45]. Figure 2. The blueprinted back-end/computing architecture of Istoriart web guide: The various layers comprising the full tech stack are presented. 4.3. Development In many aspects, the design and development processes co-exist within the iterative spirals, so the main distinction is the heavier production and programming, deployed during content/media assets construction and authoring. Hence, the project continues to actually implement the adopted aesthetic and functional plan that has been already presented in the previous section. However, some strategic decisions are clarified in this section, which are mostly related to the aims of providing additional interactions, so as to engage the audience from all of the different/targeted disciplines. As already implied, photo-galleries and puzzles (jigsaw, slide) have been incorporated as interactive/entertaining components (Figure 3a). These elements allow for users to browse through various collections of pictures, which are categorized in different movements, thus to become more familiarized with the visual features of the similar styles. Moreover, students (or trainees) and professionals working on related projects can organize their own digital painting collections that would be later utilized in the processes of gaming. Coming to the perspective of educators, various experiential teaching activities could be organized and directed within the physical or electronic classes (i.e., classify the image to the proper style, group alike examples, recognize a specific artwork to retrieve further information or historical details, etc.). These elements were created in Adobe Flash (continued as Adobe Animate) and then extracted as SWF files, taking advantage of previous experience, the offered authoring capabilities, and the availability of related/original code, which could be elaborated to adapt to any specific needs of the project, according to CBSE model [13. While it was already known that this option would exclude the possibility of reproducing these services on mobile devices, it was a conscious decision to proceed with the choice that was made for two main reasons. First, the combination of the offered interactivity/scripting and the associated design tools were advantageous when compared to the alternatives, especially if the working environment and the back-end components had to be presented and explained in a multidisciplinary audience (i.e., within the teaching activities of the mentioned post-graduate program, for instance, in the “Multimedia Production” class). Second, the specific limitation of not playing these utilities in mobile devices was actually a desired feature, since high size and resolution screens or projections are preferred (for obvious reasons). http://arutv.ee.auth.gr/istoriart/ Educ. Sci. 2019, 9, 198 9 of 22 Figure 3. Examples of the gaming components in the Istoriart web guide [13]: (a) interactive galleries and puzzles (organize pictures in related movement folders); and, (b) knowledge and image-recognition quizzes (following the movement-recognition exercise, a subsequent question is shown, requesting to justify the initial classification decision, i.e., based on colors, shapes, thematic content, intuition). Another kind of educational gaming was attempted with the implementation of quizzes, asking general knowledge questions and/or focusing on the recognition of specific visual artworks (Figure 3b). Such tools could be exploited as self-testing material and for triggering group discussion in a classroom. The utilization of SWF format (and Adobe Flash) was again decided, based on the justification that was provided in the previous paragraph, as well as to serve seamless engineering and maintenance (i.e., not to search and cope with another technology, only for questions that do not contain images). While modern art paintings cannot easily be attached with strict style recognition rules, an effort was put in that direction to help, challenge, and engage the audience (i.e., especially the technologically-oriented students coming from STEM disciplines). Particularly, a two-step questioning dialogue was decided, which asked participants/players to indicate the reason of their movement classification response (through a list of choices, as shown in Figure 3b), regardless of whether their answer was right or wrong (a “checking the right answer possibility” is available in down-left green button of Figure 3b). Therefore, erroneous and correct replies could be massively selected and associated with the different categories in order to extract empirical rules and knowledge for types or specific examples posing recognition complexity/difficultness. Anonymous submission of the game results was set, with a randomization engine generating different questions, so that users would not face the exact same content each time they would play the game. All of these features could be easily handled within Adobe Flash, without deviating from the initial requirement for web content portability and offline reproduction. Returning to the educational perspectives and the offered capabilities, the introduced “learning by example” concept can be inducted in the goals of blended learning, thus giving the opportunity to schedule physical- and/or distant-communication teaching (synchronous or asynchronous). In addition, this new approach is closer to the training of artificial intelligence systems, i.e., through the paradigms of Machine and Deep Learning (ML/DL) [12,13,31–39]. Inspired by this, a painting movement recognition modality was examined (already implied), which offered an alternative/opposite viewpoint, where humans guide the machines. Figure 4 presents the envisioned TEL framework behind the conducted work, which included already running services and future perspectives, while also demonstrating the two-folded orientation of the proposed “machine-assisted learning”. When considering the intensiveness of present-day research projects that were related to art recognition and attendance [31–33,46,47], such labors require interdisciplinary workforce and associative media training that cannot be left out of the scope of this work. Specifically, while mobile devices are not compatible with the offered gaming utilities (as already explained), there was an effort to engage smartphones and tablets in educative exercises Educ. Sci. 2019, 9, 198 10 of 22 on artificial art recognition. The utmost target is to be able to combine such intelligent ML systems with Augmented Reality (AR) modules [30], i.e., to recognize an artwork through the mobile camera, retrieving and displaying further information on the screen. Therefore, while taking advantage of previous successful implementations for crowdsourcing audio semantics [48,49], the idea was to extend such strategies in the discussed example, i.e., enabling users to validate the movement recognition results by providing semi-automated annotation feedback. A proof of concept attempt was made in algorithmic level even if such demanding/generic purpose solutions are not technically feasible yet, investigating the potential visual parameters to be used in various modern-art classification schemes [12,13]. For instance, visual features that were extracted and ranked for their descriptive saliency in the discrimination of the different painting classes could be more easily understood and comprehended to some teaching groups. Clearly, this approach can be thought of as more suitable and challenging to technologists, post-graduate students, or academics, who are (/willing to be) involved in such types of multidisciplinary research. In this case, Istoriart can also provide the necessary creative or artistic background to fill in the gap in media literacy from an art perspective. Finally, the games and the targeted AR services could both propel the gradual construction of large-volume ground-truth repositories, appropriate to advance future works on DL, not only for media and art, but also for broader collaborations on demanding multimedia semantics. Figure 4. The envisioned bundle of technology-enhanced services within the Istoriart framework. Advanced movement recognition modalities attempt to further propel machine-driven training. When considering the diversification of modern-art styles and even inner-class dissimilarities of artists under the same trend, it would be beneficial to provide an educational map for the implicated painting movements, where visual content withholds large amounts of information (colors, textures, etc.). Initial experiments were conducted for the potential discrimination of patterns through machine learning. The whole classification task is quite demanding due to the heterogeneity of the art styles, as well as the personalized characteristics of representative artists. Imaging properties had to be imprinted/reflected into mathematical expressions, transforming empirical observations or intuitions into relevant visual parameters. In the current case, the feature extraction and ranking procedures were conducted with the close collaboration of different groups of expertise within the discussed post-graduate program, i.e., in the classes of related courses and as a part of master/doctoral theses [12,13]. Figure 4 shows the implicated processes, along with the triggered TEL activities and the associated services. As it is further discussed in the Results and Discussion sections, the overall cooperation experience was found exciting, enjoyable, and engaging by the different participants, while Educ. Sci. 2019, 9, 198 11 of 22 the preliminary performance outcomes of the trained models per se, seem to be very encouraging, making a strong proof of concept. 4.4. Evaluation Assessment is considered to be an essential procedure in multimedia production and broadly in system engineering projects, especially if rapid prototyping and human-centered design are involved (as in the current work). Overall, different types of evaluation can be conducted in qualitative and/or quantitative terms throughout all of the development phases. The main scope is to indicate likely flaws, imperfect behavior, or unjustified deviations from the blueprinted plan when such analyses are deployed in parallel with the project execution, providing useful feedback for course rectification (formative evaluation). A respective small panel of multi-domain experts was formed for this purpose, including representative users of the targeted Istoriart audience along with authors of this paper. Specifically, individuals coming from the sectors of education (3), audiovisual industries, artistic, or creative media (3), were assembled with technologists, web programmers, and software engineers (4) to cover all of the necessary appraising perspectives. Most of the participants had an associated degree and related experience in the field, while the students of the relevant post-graduate program were listed among the contributors. Experimental/inspection sessions and associated discussion groups were organized to uncover and shape the required curative actions. Testing and validation mechanisms were emanated from the favorite rules and principles, i.e., the Nielsen metrics and the so-called “five Es of usability” (Effectiveness, Efficiency, Engagement, Error tolerance, Ease of learning), which highlighted the desired application attributes [13,19–23,40–44]. These codes were adapted to the characteristics of the intended Istoriart functionalities and the aimed end-users. Formative evaluation was decided to mainly proceed in qualitative terms, even if bigger teams were sometimes involved in the process (i.e., focused post-graduate classes). Apart from the above formative procedure, a final assessment session was employed at project completion with the help of a different board of evaluators. This analysis mainly incorporates subjective/empirical observations and suggestions that, except for their qualitative nature, were supplemented with a related quantification approach. The central aim was to estimate the succeeded achievements, to indicate proper use of the offered TEL services in broader learning/training scenarios in the highly multidisciplinary media sector, and to unveil the potential future deployment and elaboration directions. A different kind of evaluation was employed for the case of the artificial movement recognition utilities, i.e., to stress the performance and the expected generalization accuracy of the trained modalities. More details regarding all of these processes are provided in the corresponding results section. 5. Results Elaborating on the previous paragraph, the qualitative and quantitative results, as presented in this section, can be divided into three major categories: (a) the implemented guide and the offered services; (b) the analysis and usability evaluation outcomes; and, (c) the deployed pattern classification schemes with the associated performance scores. 5.1. Implemented Guide and Offered Services The implemented modern art guide with its featured multimedia elements and the associated TEL perspectives are listed among the undeniably positive results of the work. The delivered services managed to fill the gap in the availability of related tools, especially for the Greek language. Figure 5 presents views of the Istoriart site. The starting screen contains a tag-cloud animation with direct access to the most popular movements. The horizontal menu that follows is present in all browsing instances, offering site-map guidance, thus allowing for quick navigation to each thematic page. A short welcome text completes the composition of the introduction message, providing primary project identity information, while overall maintaining a “chic and simple” design. All of the internal Educ. Sci. 2019, 9, 198 12 of 22 nodes open in the main window (_iframe), while external sites open to new tabs (_blank). In this context, users and especially students/trainees can safely explore the informatory data needed during an educational exercise or activity, without worrying about possible disorientation or misplacing (i.e., to get lost). Figure 5. The implemented Istoriart web-guide: (a) Start page, containing a tag-cloud animation with links to the most popular modern-art movements, followed by the horizontal navigation bar and the welcome text; (b) An artist page (Degas) with his identity data and timeline, representative/favorite artworks, abstract section (in bluish color) and lookup mechanisms (links/pop-up windows) for finding further information on related art movements. In both screens, project-identity links are given in the right-top corner, while the Istoriart logo is placed an instance below, in the opposite side (left -top). The horizontal menu is always present (just below the logo) to facilitate easy access to all the browsing nodes/pages. 5.2. Analysis and Usability Evaluation Results Empirical surveys were conducted during the rapid prototyping processes to serve the analysis and evaluation tasks. Qualitative observations and feedback were received by various experts through interview and lab inspection approaches, while a broader audience anonymously selected the quantitative responses. A properly formed questionnaire was used during analysis to collect the answers of potential end-users (target group) and to reveal key characteristics for the subsequent development. The questions were related to the users’ knowledge and interests, either in art generally or exclusively in modern-contemporary art, their preferences in different artistic kinds, their needs/willingness to utilize online material (and how often) for retrieving interesting information (artistic or not), and their language preferences (English, Greek). The elaborated queries were posed to assess the availability and importance of educational platforms/web-guides on modern art, especially for the Greek language. The participants were asked to state their opinions about the appropriateness and effectiveness of the advocated guide modalities (image, audio, video) and the preferred accessing modes (online/offline with downloadable material provisions). Several demographic variables (gender, age, profession) were also recorded, along with technology familiarity metrics to extract the correlation outcomes and meaningful conclusions (i.e., estimated level of competency/engagement in digital literacy and web technologies, device preferences during navigation, etc.). Table 2 depicts the analytic structure of the questionnaire. Educ. Sci. 2019, 9, 198 13 of 22 Table 2. The analysis questionnaire. # Question (Indicative Answers—Range) A Gender (Male, Female) B Age (intervals: <30, 30–50, >50) C Education (Secondary, University, Master) D Profession related to (multimedia, art, education, other) E Technology familiarity (low, medium, high) F Place for computer utilization (home, work, both, other) G Use of Internet frequency (1–5) H Device preference (desktop computer, laptop, mobile, tablet) I Knowledge of differences between modern and contemporary art (Yes, No, Not Sure) J Do you want to know more about modern art? (1–5) K Do you search on the Web for art-related information? (1–5) L Do you search on the Web for modern art related information (1–5) M Main Interest in Art (Painting, Music, Literature, Theater, Dancing) N Do you search on the Web for general information? (1–5) O Would you read an English text for Information Acquirement? (1–5) P Do you know any featured multimedia guide for modern art in Greek language? (1–5) Q Would you be interested in a modern art guide in Greek language? (1–5) R Should a modern art guide involve multimodal content (audio, video etc.)? (1–5) S Importance of downloadable content of the guide, i.e., offline mode (1–5) T Main Interest in modern art (historical data, movements, artists, creations, all, none) U A multimedia modern-art guide should involve (extended textual descriptions, basic information, concise summaries with options for further/extensive information lookup) Questions that were related to modern art and Istoriart aspects (initial knowledge, suggested modalities, anticipated usefulness, etc.) were structured in a categorical form of potential answers, with five-point Likert scales (1–5, from “Totally Disagree” to “Totally Agree” or from “Not at all” to “Very Often”). Binary values (i.e., Gender) and higher-dimension lists were also involved. A reliability test was conducted in the formulated questionnaire via Cronbach’s alpha estimation before proceeding into an extensive correlation analysis between the various factors, which revealed a value of 0.71, thus ensuring the reliability of the test. A primal statistical analysis was made with the exploitation of Chi-Squared tests, taking into consideration the categorical nature of the involved answers, based on the calculation of Pearson Correlation Coefficient with 95% confidence level (as compared to 0.05 threshold value). The items were divided in two subsets, with the former involving basic characteristics/demographics of the users (questions A-H) and the latter containing their art-related answers (questions I-U). The correlation matrix from the Chi-Square tests presents the p-values for the combinations of row/column elements, which were compared to the 0.05 threshold of statistically significant correlations (marked as green for values below 0.05, as depicted in Figure 6). Statistical differentiations in the answers of male/female users for their interests in a on modern-art guide (like Istoriart) and the preferred multimedia elements were found. The diversified professional orientation revealed statistical correlation to the anticipated services and the Internet utilization. Educ. Sci. 2019, 9, 198 14 of 22 Table 3. Evaluation questions adapted to the Nielsen metrics and the five Es of usability [13,19–23]. # Usability Criteria 1 Crisp and simple navigation 2 Use of correct Language—Terminology 3 Visibility of System Status 4 Effective Presentation 5 Flexibility and efficiency of (multimedia) use 6 Information retrieval (recognition rather than recall) 7 Aesthetic and minimalistic design 8 Multimedia interaction/user control 9 Alternative navigation routes/user freedom 10 Help and documentation support Figure 6. P-values for the questioned items (the green-highlighted cells indicated statistical difference between the associated factors, as presented in Table 3. Regarding usability evaluation, apart from the formative feedback that was received in all the corresponding phases of the iterative design (low-/high fidelity prototypes, midway deliverables, etc.), the final assessment derived from five (5) experts, specialized in the following domains: (a) education and technology, (b) creative expressions, graphics and arts, (c) game development, (d) software engineering, web authoring and management, and (e) audiovisual signal processing and multimedia. The goal (in all phases) was to receive timely feedback regarding the inspection of functional errors, misleading content, ineffective design, etc., targeting to drive the subsequent optimization process. For this reason, a short list of questions was adapted to the associated Nielsen rules and the five Es of usability [22,40–44], inquiring discussion regarding ten (10) different multimedia perspectives of Istoriart to match the specific analysis needs in the current scenario (i.e., to form the variables presented in Table 3). The interviewed experts’ team had to respond with a Likert-like score from 1 to 5, to each of the 10 properties, which are exhibited in Table 3. Figure 7 presents the statistical results of the respective scores. The mean values were in all cases around 4.5, which point to the high-quality and functional design of the application, while the tiny deviations declare the homogeneity in the answers of the implicated experts. Despite that this outcome relies on a small group, it is essential to note the agreement from all the different angles of expertise, without any diversifications between educational, creative, and technological disciplines. Educ. Sci. 2019, 9, 198 15 of 22 Figure 7. Statistics (mean, standard deviation) of the final Istoriart evaluation, using the modified usability assessment criteria, as presented in Table 3. Regarding the formed metrics and the absence of some popular/key parameters (i.e., error rate/presence—prevention and recovery; formal documentation), formative feedback pointed out that the deployed architecture and the simplicity of the back-end design resulted in the abstain of related navigation errors. For this reason, it was decided to exclude “error queries”, while also considering that, at some point, these aspects were somewhat projected in the “Navigation, Visibility, and Control” views. In the same context, the missing of formal documentation was noticed, though, it was admitted that helping notes and supporting material adequately covered this necessity (also reflected in the variables of Table 3). In qualitative terms, small concerns were further expressed regarding the use of *.swf files and popup windows, justified on the expected difficulties and incompatibilities when executed on mobile devices. Thus, the incorporation of alternative future implementations was prompted. The possibility of a search bar was finally mentioned, realizing the subsequent/unwanted increase of complexity, with the additional risk to distract or disorient users (students/trainees) in their educational tasks. Despite the above optional points, all of the reviews were intensely positive, thinking of the stated minor flaws as insignificant, especially for the current phase and the set aims, thus agreeing that the project accomplished the vast majority of its targets. The positive remarks were also validated and verified in qualitative terms by different groups of evaluators, i.e., students that were enrolled in the referred post-graduate media courses, multi-domain researchers participating in the training of the machine-driven movement recognition system (presented next), etc. It has to be noted that this last assessment was deployed to test the initial/stated hypotheses of the TEL-contributions and not for evaluative purposes per se. Nevertheless, in-class triggered discussions and feedback is collected for a potential future release of a next Istoriart version (optimized with augmented features), whereas the additional statistics and solid results could be extracted (considered out of the scope of the current work). 5.3. Machine-Driven Movement Recognition: Classification Schemes and Performance Evaluation As already pointed out, this experimentation was decided to meet the highly multi-domain nature of the media education scenarios, as well as to trigger audience engagement and experiential learning activities. Initial classification schemes were set, which involved the following eight styles: Impressionism, Meta-Impressionism, Fauvism, Expressionism, Abstract Expressionism, Cubism, Futurism, and Surrealism. The most famous/representative artists and artworks of each movement were included for training and testing purposes (the associated numbers of painting samples per class Educ. Sci. 2019, 9, 198 16 of 22 are given in Table 4). Lossless compression image files (i.e., *.png format) were retrieved from web sources in different dimensions/resolutions; hence, a preprocessing step was necessary for smoothening this heterogeneity. Specifically, each picture was rescaled to the same horizontal dimension (400 pixels), which analogously accommodated the vertical size, for the avoidance of distortion/disproportion problems. Thereafter, a set of features was extracted as related to color components, texture properties, and spatial shape detection, representing the inputs to feed system training. Table 4. Painting collections of the different genres used for the machine-driven art recognition. # Movement Number of Paintings 1 Impressionism 20 2 Meta-Impressionism 15 3 Fauvism 15 4 Expressionism 18 5 Abstract Expressionism 18 6 Cubism 15 7 Futurism 16 8 Surrealism 16 Sum 133 Specifically, basic statistical variables, i.e., mean value, standard deviation, skewness, kurtosis, and entropy, were computed for the grayscale images (with 256 gray-levels, i.e., 8-bit quantization). These parameters were combined to form structural descriptors that could aid in mining and assigning rules for each category. Moreover, color transformations into different models were deployed since color information is essential in modern-art style discrimination (i.e., RGB, HSV, YCbCr, LIQ) and the above statistics were again calculated to contribute tp a combined color-structure identification perspective. Similarly, typical filters (i.e., edge detectors) were utilized to reveal shape characteristics, i.e., through the number of edges, corners and lines. In purely technical terms, contrast/dynamic range processing were applied, along with corner/node detection techniques (i.e., Eigenvalues, Harris methods) and their elaborated edge estimation counterparts in the extracted binary versions (i.e., Canny, Sobel, Prewitt, Roberts, Laplace-Gaussian, Zero-crossing). Furthermore, continuous lines of pixels were located and quantified via the Hough-Transform, which configured the minimum line-length threshold values through trial and error testing (a number of 7 consecutive active pixels turned to be a good compromise). An alternatively texture discovery approach was attempted with a more sophisticated property set (i.e., via the computation of the Haralick coefficients, deriving from the respective co-occurrence matrices), while Empirical Mode Decomposition (EMD) transforms were also included to indicate salient spectral/scaling attributes. Overall, a feature vector of 147 visual parameters was formed for each image, using the Matlab Computer Vision toolbox [12,13,50]. The last two paragraphs might seem entirely unfitting to the main concept of the paper, especially for average readers without an essential technological and algorithmic background. However, this is the exact point when it comes to highly interdisciplinary studies, as presented in this paper. On the one hand, creative media and communication disciplines are needed to provide the human experience in classifying modern-art painting movements, as to suggest the accompanying discrimination rules. Clearly, the interested users of this domain would be fascinated with the idea of using a machine-driven recognition guide (and the respective TEL scenarios that are listed in Figure 4). On the other hand, scientists and researchers from the computer vision perspective bring the skillsets and assets to train such smart systems through ML and DL techniques into the table. In this context, multi-domain audience is strongly engaged in the TEL processes and urged to close collaboration with the co-mates. In fact, this was the case during the ML sessions, presented here, where post-graduate students and researchers/PhD candidates were collaborating towards the construction of the ground-truth dataset (i.e., image-samples annotated with movement labels) to train and test the artificial recognition modules [12,13]. Table 5 lists the different categories/classification schemes that were formed in order Educ. Sci. 2019, 9, 198 17 of 22 to verify the initial proof of concept (i.e., to estimate the algorithmic style detection performance) and the associated evaluation metrics/results. The ratio of the correctly classified samples to the total population is defined as the Pattern Recognition accuracy (PR), while partial scores (PRi = A,B,C) refer to the intrinsic measures within each one of the classes i (=A, B, C). Table 5. Overall (PR%) and partial (PRi %) Performance Rates of the various classification schemes. # Class A PRA Class B PRB Class C PRC PR 1 Surrealism 86.67 Fauvism 100 93.33 2 Futurism 81.25 Surrealism 73.33 77.42 3 Meta-Impressionism 50.00 Cubism 66.67 56.67 4 Impressionism 80.00 Fauvism 80.00 80.00 5 Meta-Impressionism 66.67 Futurism 62.50 64.52 6 Abstract Expressionism 66.67 Surrealism 60.00 63.64 7 Cubism 73.33 Expressionism 72.22 72.73 8 Fauvism 86.67 Cubism 73.33 80.00 9 Meta-Impressionism 93.33 Surrealism 93.33 93.33 10 Meta-Impressionism 73.33 Surrealism 80.00 Futurism 81.25 78.26 11 Fauvism 86.67 Impressionism 80.00 Surrealism 66.67 76.00 12 Fauvism Futurism 76.67 Cubism Surrealism 80.65 78.69 13 Expressionism Impressionism Abstract Expressionism Meta-Impressionism 69.01 Cubism 62.30 65.91 Several machine-learning techniques were applied and compared (Support Vector Machines, k-Nearest Neighbors, Regressions, etc.), which balanced the overall and partial accuracies, before concluding to the utilization of Neural Network models. After iterative trial and error tests, the final neural structure that was involved the input layer (i.e., system input), one intermediate/hidden layer (with 40 neurons and sigmoid trigger functions), and a linear output layer (adapted to the number of classes). The standard k-fold validation method (k = 8) was adopted, which divided the input samples into k clusters; k-1 groups were used for model training and the remaining one for estimating the system generalization performance, which measured the classification scores in all of the formed k-combinations. Based on the data depicted in Table 5, high recognition rates have succeeded in quite a few schemes, while the overall preliminary results seem to be very encouraging (also accounting the difficulty of the problem and the small dataset used for training). Elaborating on this initial experimentation, useful insights have been highlighted and further discussed in the next section, which indicated potential future directions of machine-assisted art recognition and their implication in demanding media learning scenarios. 6. Limitations The current project has some unique characteristics that cannot be easily accommodated to other studies. Nevertheless, technological and methodological principles concerning the engaging aspects of the pursued TEL services could be propagated to the broader media training domain. In the same context, multidisciplinary demands may be significantly different in other challenging/real-world scenarios, so that the deployed modalities and solutions might not be so applicable. However, best practices can be extracted and tested in other/linked disciplines and related research collaborations. Concerning the two main set goals, i.e., the implementation of the multimedia guide to augment the learning experience and the machine-assisted art recognition approach for stimulating interdisciplinary engagement; both undertakings feature certain difficulties and limitations. Thus, formative and final evaluation outcomes need to be enhanced with the participation of wider audiences, by massively Educ. Sci. 2019, 9, 198 18 of 22 deploying the developed utilities in featured classrooms and/or as self-learning toolsets. Enlarging the received feedback would provide more concrete and reliable assessment insights, thus eliminating the associated design uncertainties during system maintenance. Regarding the anticipated AR modalities that rely on artificial movement detection, as already stated, the purpose was to make an initial proof of concept. The limited number of training samples and the formed classification schemes need to be elaborated with further experiments and more sophisticated analysis. These limitations do not affect the targets stated in this study in all cases, while future research directions can be listed based on this first experimentation, as discussed in the following section. 7. Discussion Starting with the stated hypotheses and questions, the conducted analysis verified that the targeted audience approves the art-guide idea of Istoriart with the envisioned TEL services. Specifically, most of the participants answered that they would be interested in extending their knowledge on the topic (>87%), seeing the usefulness of such an environment as very important (>75%). These percentages are incredibly high, even if the associated results for web searching and retrieval (of art and modern-art information) have significantly lower levels (46% and 24%, respectively). The majority is positive on the multimedia character of the webpage (~93%), as well as in the offline browsing option, although with some small variation (40% thinks of this facility as very important, 40% as important, and the remaining 10% as irrelevant). The above findings become even more significant when combined with the query on the availability of similar applications in the Greek language (the 39% is certain or almost certain that such a site is missing, while a 38% declares ignorance). Despite the restricted sample of 47 participants, the preliminary statistical values revealed correlations between the users’ gender, professions, and their interests/expectations about Istoriart. Overall, the achieved integration and evaluation results point in the direction that the project succeeded in combining many of the wanted/positive attributes that have been analyzed during the review of related works (Table 1). These initial conclusions will be further investigated, aiming to optimize the platform modalities for supporting the suitable and effective adaptation to users’ characteristics (art background, media literacy, technology familiarity, etc.), thus facilitating pleasant navigation with engaging interactions and an overall rich user experience. The bundle of the implemented TEL services can be utilized for forming innovative education activities, both for in-class tutoring and self-training support, through the use of comprehensive material (e.g., showing items of specific movements, artists, artworks, their temporal evolution/timelines, etc.). Hence, the values of blended learning [26] are extended with the incorporation of representative examples, experiential assignments, simulations, and exercises that can be addressed to individuals or as team-work projects, thus transforming the schooling routine into an augmented interactive experience [30,31,45–49]. Specifically, photo galleries can be used to highlight the resemblances and dissimilarities of the different species; quizzes can examine the levels of understanding and knowledge, triggering class discussion; puzzles can be exploited to organize personal collections and team-contests [26–28]. Gaming components can also provide indications of the salient visual parameters that facilitate artwork recognition, which urge trainees to get involved in associated AR-driven TEL services [12,13,27–39] (i.e., to identify the parts of the paintings, the revealing of which triggers higher/empirical cognition, thus helping to detect the descriptive features behind the different styles and their classification properties). Feedback queries allow for the harvesting of users’ responses (while playing the corresponding movement recognition games), thus receiving useful insights on subjective discrimination difficulties, which could help to better adapt and direct the teaching procedures. At the same time, a dedicated repository is constructed and annotated with the necessary meta-information, which can be further elaborated through semi-automated crowdsourcing processes. These labelled datasets could be valuable in the direction of learning by example, both for humans and machines (i.e., use the associated samples as inputs in sophisticated ML/DL artificially-trained recognition systems, as it was attempted in the current project, making the first proof of concept). Educ. Sci. 2019, 9, 198 19 of 22 Extending the above, it is essential to evaluate the achieved audience engagement perspectives, which form the second research question to be answered. Based on the usability assessment outcomes and the associated qualitative validation sessions, which were conducted in featured target groups and related post-graduate classes, the received feedback verifies the powerfully engaging character of the Istoriart TEL services. Excluding all of the above-stated advantages, it worth elaborating on the example of the ML-training procedures. The corresponding research was deployed with the close collaboration of multi-domain authorities, which revealed the highly multidisciplinary nature of the undertaking. Specifically, subject-matter experts from the broader area of audiovisual industries, who had previous knowledge and/or interests in modern art, cooperated with technologists, specialists on signal processing, and artificial intelligence. The common targets were to form the initial dataset with representative painting styles and to detect the applicable visual properties for their automated discrimination. The classification schemes, as presented in Table 5, were investigated and finally formed based on the joint empirical observations of the involved teams. It should be noted that direct movement recognition experiments were initially attempted, which aimed at separating all of the samples in each painting category at once, but reduced performance rates were achieved, which pointed out the increased complexity of the problem [12,13]. This was a good “lesson” for media disciplines to comprehend that ML algorithms cannot provide any solution by their own, an experience that can be portrayed on similar trendy topics (i.e., Semantic Web, Internet of Things, Big Data applications in media/communications). In this context, their digital literacy was expanded in practice, which allowed for them to contribute the team effort, i.e., by participating in the investigation and configuration of the remaining tasks of visual feature extraction and ranking. On the other hand, technologists took the opportunity to exploit Istoriart resources, to get basic training on the necessary artistic background. Overall, both teams were attracted by the mutual challenge and they enjoyed their active engagement in such a demanding project. In future perspectives, the conducted semantic analysis could be reinforced/optimized with the collection of extended ground-truth data (for more adequate movement representation), or even with the integration of hierarchical modules, which support the classification process in sequential layers of coarse/hybrid schemes. In all cases, this style recognition approach serves as an introductory guide and a key element for Istoriart, fulfilling educational purposes either for art related users or technology-oriented ones, while addressing the wanted interdisciplinary skillset. Furthermore, the extracted conclusions regarding movement discrimination and the explicitly-described ML know-how could initiate interesting discussions in media classes, motivating the collaborative actions between the co-mates. Finally, the already-trained modules can be exploited, along with the Istoriart gaming components, to enhance the initially constructed repository, which could propel future DL/ML works, targeting significantly improved generalization performance. When such mature solutions become broadly available, more sophisticated TEL services (like the augmented-reality painting recognition of Figure 4) would further heighten the schooling procedures to even higher levels of audience excitement and engagement. Such systems seem among the few prosperous solutions to support digital literacy and life-long learning in the dizzying media society transformation, where unawareness and the uncertainty of the technological capacities are dominant to considerable amounts of population. 8. Conclusions Summarizing the above observations, someone would argue with confidence about the strong impact of technology-enhanced learning in supporting highly multidisciplinary topics, prioritizing audience engagement. Based on the feedback that was received during the implementation of the Istoriart case-study, the adopted practices can help to improve the teaching experience (in multiple levels), promoting digital literacy support, for both the specific topic and broader training processes. More specifically, the engaging, experiencing, and innovative TEL aspects can be ingested within the international theoretical framework regarding highly multidisciplinary/demanding fields. As already stated, we are living in a world of data, in which the pervasive nature of information exchange and Educ. Sci. 2019, 9, 198 20 of 22 communication have dominated today’s ubiquitous society. The media landscape has drastically altered during the last years, while it continues to elaborate with strong dependencies on the use of technology. In this context, the previously mentioned contemporary trends of the Semantic Web and Big Data are considered as the next big thing in the media world and require the close collaboration of multi-domain experts. Hence, following the experience gained in this project, TEL services could propel such cooperative models, in favor of all the involved parties. Author Contributions: Conceptualization, E.C. and C.D.; methodology, E.C. and C.D.; software, N.T. and L.V.; validation, R.K., investigation, E.C.; data curation, R.K., L.V. and N.T.; writing—original draft preparation, E.C., R.K. and C.D.; writing—review and editing, L.V., N.T. and C.D.; visualization, N.T. and C.D.; supervision, C.D. Funding: This research received no external funding. Acknowledgments: The authors acknowledge the valuable contribution and guidance of G. Paschalidis, during the methodological preparation and implementation of the conducted work. Conflicts of Interest: The authors declare no conflict of interest. References 1. Dimoulas, C.; Veglis, A.; Kalliris, G. Semantically enhanced authoring of shared media. In Encyclopedia of Information Science and Technology, 4th ed.; Khosrow-Pour, M., Ed.; IGI Global: Hershey, PA, USA, 2018. 2. Spyridou, L.P.; Matsiola, M.; Veglis, A.; Kalliris, G.; Dimoulas, C. Journalism in a state of flux: Journalists as agents of technology innovation and emerging news practices. Int. Commun. Gaz. 2013, 75, 76–98. [CrossRef] 3. Ntalakas, A.; Dimoulas, C.A.; Kalliris, G.; Veglis, A. Drone journalism: Generating immersive experiences. J. Media Crit. JMC 2017, 3, 187–199. [CrossRef] 4. Katsaounidou, A.; Dimoulas, C.; Veglis, J. Cross-Media Authentication and Verification: Emerging Research and Opportunities: Emerging Research and Opportunities; IGI Global: Hershey, PA, USA, 2018. 5. Katsaounidou, A.; Dimoulas, C. Integrating Content Authentication Support in Media Services. In Encyclopedia of Information Science and Technology, 4th ed.; Khosrow-Pour, M., Ed.; IGI Global: Hershey, PA, USA, 2018. 6. Katsaounidou, A.; Dimoulas, C. The Role of media educator on the age of misinformation Crisis. Presented at the EJTA Teachers’ Conference on Crisis Reporting, Thessaloniki, Greece, 19–20 October 2018. 7. Matsiola, M.; Spiliopoulos, P.; Kotsakis, R.; Nicolaou, C.; Podara, A. Technology-Enhanced Learning in Audiovisual Education: The Case of Radio Journalism Course Design. Educ. Sci. 2019, 9, 62. [CrossRef] 8. Walker, R.; Jenkins, M.; Voce, J. The rhetoric and reality of technology-enhanced learning developments in UK higher education: Reflections on recent UCISA research findings (2012–2016). Interact. Learn. Environ. 2018, 26, 858–868. [CrossRef] 9. Psomadaki, O.; Dimoulas, C.; Kalliris, G.; Paschalidis, G. Digital storytelling and audience engagement in cultural heritage management: A collaborative model based on the Digital City of Thessaloniki. J. Cult. Herit. 2019, 36, 12–22. [CrossRef] 10. Dimoulas, C.; Kalliris, G.; Chatzara, E.; Tsipas, N.; Papanikolaou, G. Audiovisual production, restoration-archiving and content management methods to preserve local tradition and folkloric heritage. J. Cult. Herit. 2014, 15, 234–241. [CrossRef] 11. Groshans, G.; Mikhailova, E.; Post, C.; Schlautman, M.; Carbajales-Dale, P.; Payne, K. Digital Story Map Learning for STEM Disciplines. Educ. Sci. 2019, 9, 75. [CrossRef] 12. Kotsakis, R. Application of Machine Learning Algorithms for Extracting and Classifying Content Information in the Media. Ph.D. Thesis, School of Journalism & Mass Communications, Aristotle University of Thessaloniki, Thessaloniki, Greece, 2015. Available online: http://ikee.lib.auth.gr/record/270162 (accessed on 5 May 2019). (In Greek) 13. Chatzara, E. Design and Implementation of a Greek Modern-Art Guide. Master ’s Thesis, School of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessaloniki, Greece, 2013. Available online: http://ikee.lib.auth.gr/record/133778 (accessed on 5 May 2019). (In Greek) 14. Lugmayr, A.; Stockleben, B.; Risse, T.; Kaario, J.; Pogorelc, B. (Eds.) New business, design and models to create semantic ambient media experiences. Multimed. Tools Appl. 2013, 63, 1–159. http://dx.doi.org/10.1177/1748048512461763 http://dx.doi.org/10.17349/jmc117317 http://dx.doi.org/10.3390/educsci9010062 http://dx.doi.org/10.1080/10494820.2017.1419497 http://dx.doi.org/10.1016/j.culher.2018.07.016 http://dx.doi.org/10.1016/j.culher.2013.05.003 http://dx.doi.org/10.3390/educsci9020075 http://ikee.lib.auth.gr/record/270162 http://ikee.lib.auth.gr/record/133778 Educ. Sci. 2019, 9, 198 21 of 22 15. Veglis, A.; Dimoulas, C.; Kalliris, G. Towards intelligent cross-media publishing: Media practices and technology convergence perspectives. In Media Convergence Handbook; Lugmayr, A., Dal Zotto, C., Eds.; Media Business and Innovation; Springer: Berlin/Heidelberg, Germany, 2016; Volume 1, pp. 131–150. 16. Vryzas, N.; Sidiropoulos, E.; Vrysis, E.; Avraam, E.; Dimoulas, C. A Mobil Cloud Computing Collaborative Model for the Support of on-site Content Capturing and Publishing. J. Media Crit. JMC 2018, 4, 349–364. 17. Sidiropoulos, E.; Vryzas, N.; Vrysis, E.; Avraam, E.; Dimoulas, C. Collaborative collection and Mapping Multimedia Crisis Semantics. Presented at the EJTA Teachers’ Conference on Crisis Reporting, Thessaloniki, Greece, 19–20 October 2018. 18. Dimoulas, C.; Veglis, A.; Kalliris, G. Chapter 17: Application of mobile cloud-based technologies in news reporting: Current trends and future perspectives. In Mobile Networks and Cloud Computing Convergence for Progressive Services and Applications; Rodrigues, J., Lin, K., Lloret, J., Eds.; IGI Global: Hershey, PA, USA, 2014; pp. 320–343. 19. Pressman, S.R. Software Engineering: A Practitioner’s Approach, 7th ed.; McGraw-Hill: New York, NY, USA, 2010. 20. Sommerville, I. Software Engineering, 9th ed.; Addison-Wesley: London, UK, 2011. 21. Vaughan, T. Multimedia: Making It Work, 9th ed.; McGraw-Hill Osborne Media: New York, NY, USA, 2014. 22. Dimoulas, C.A. Multimedia. In The SAGE International Encyclopedia of Mass Media and Society; Merskin, D.L., Ed.; SAGE Publications, Inc.: Saunders Oaks, CA, USA, 2019. 23. Dimoulas, C.A. Multimedia Authoring and Management Technologies: Non-Linear Storytelling in the New Digital Media; Association of Greek Academic Libraries: Athens, Greece, 2015. Available online: https: //repository.kallipos.gr/handle/11419/4343 (accessed on 5 May 2019). (In Greek) 24. Veglis, A.; Maniou, T. The Mediated Data Model of Communication Flow: Big Data and Data Journalism. KOME Int. J. Pure Commun. Inq. 2018, 6, 32–43. [CrossRef] 25. Veglis, A.; Maniou, T. Chatbots on the Rise: A new Narrative in Journalism. Stud. Media Commun. 2019, 7, 1–6. [CrossRef] 26. Hussein, B. A blended learning approach to teaching project management: A model for active participation and involvement: Insights from Norway. Educ. Sci. 2015, 5, 104–125. [CrossRef] 27. Naz, N.; Sayyed, A.; Dal Sasso, G.; Khanum, S.; de Souza, M. SavingLife: An Educational Technology for Basic and Advanced Cardiovascular Life Support. Educ. Sci. 2018, 8, 78. [CrossRef] 28. Chen, M.H.M.; Tsai, S.T.; Chang, C.C. Effects of Game-Based Instruction on the Results of Primary School Children Taking a Natural Science Course. Educ. Sci. 2019, 9, 79. [CrossRef] 29. Hinojo-Lucena, F.J.; Aznar-Díaz, I.; Cáceres-Reche, M.P.; Romero-Rodríguez, J.M. Artificial Intelligence in Higher Education: A Bibliometric Study on its Impact in the Scientific Literature. Educ. Sci. 2019, 9, 51. [CrossRef] 30. Kaasinen, A. Plant Species Recognition Skills in Finnish Students and Teachers. Educ. Sci. 2019, 9, 85. [CrossRef] 31. Badea, M.; Florea, C.; Florea, L.; Vertan, C. Can we teach computers to understand art? Domain adaptation for enhancing deep networks capacity to de-abstract art. Image Vis. Comput. 2018, 77, 21–32. [CrossRef] 32. Florea, C.; Gieseke, F. Artistic movement recognition by consensus of boosted SVM based experts. J. Vis. Commun. Image Represent. 2018, 56, 220–233. [CrossRef] 33. Wechsler, H.; Toor, A.S. Modern art challenges face detection. Pattern Recognit. Lett. 2018. [CrossRef] 34. Abry, P. Image Processing for Digital Art Work (special issue). Signal Process. 2013, 93, 525–620. [CrossRef] 35. Berezhnoy, I.E.; Postma, E.O.; Van Den Herik, H.J. Automatic extraction of brushstroke orientation from paintings: PPPOET: Prevailing orientation extraction technique. Mach. Vis. Appl. 2009, 20, 1–9. [CrossRef] 36. Gartus, A.; Klemer, N.; Leder, H. The effects of visual context and individual differences on perception and evaluation of modern art and graffiti art. Acta Psychol. 2015, 156, 64–76. [CrossRef] [PubMed] 37. Hughes, J.M.; Mao, D.; Rockmore, D.N.; Wang, Y.; Wu, Q. Empirical mode decomposition analysis for visual stylometry. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 2147–2157. [CrossRef] [PubMed] 38. Li, C.; Chen, T. Aesthetic visual quality assessment of paintings. IEEE J. Sel. Top. Signal Process. 2009, 3, 236–252. [CrossRef] 39. Li, J.; Yao, L.; Hendriks, E.; Wang, J.Z. Rhythmic brushstrokes distinguish van Gogh from his contemporaries: Findings via automated brushstroke extraction. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 1159–1176. [PubMed] https://repository.kallipos.gr/handle/11419/4343 https://repository.kallipos.gr/handle/11419/4343 http://dx.doi.org/10.17646/KOME.2018.23 http://dx.doi.org/10.11114/smc.v7i1.3986 http://dx.doi.org/10.3390/educsci5020104 http://dx.doi.org/10.3390/educsci8020078 http://dx.doi.org/10.3390/educsci9020079 http://dx.doi.org/10.3390/educsci9010051 http://dx.doi.org/10.3390/educsci9020085 http://dx.doi.org/10.1016/j.imavis.2018.06.009 http://dx.doi.org/10.1016/j.jvcir.2018.09.015 http://dx.doi.org/10.1016/j.patrec.2018.02.014 http://dx.doi.org/10.1016/j.sigpro.2012.09.018 http://dx.doi.org/10.1007/s00138-007-0098-7 http://dx.doi.org/10.1016/j.actpsy.2015.01.005 http://www.ncbi.nlm.nih.gov/pubmed/25700235 http://dx.doi.org/10.1109/TPAMI.2012.16 http://www.ncbi.nlm.nih.gov/pubmed/23289129 http://dx.doi.org/10.1109/JSTSP.2009.2015077 http://www.ncbi.nlm.nih.gov/pubmed/22516651 Educ. Sci. 2019, 9, 198 22 of 22 40. Barry, C.; Lang, M. A survey of multimedia and web development techniques and methodology usage. IEEE Multimed. 2001, 8, 52–60. [CrossRef] 41. Nielsen, J. Usability Engineering; Academic Press: Cambridge, MA, USA, 1993. 42. Nielsen, J. Heuristic evaluation. In Usability Inspection Methods; Nielsen, J., Mack, R.L., Eds.; John Wiley & Sons: New York, NY, USA, 1994. 43. Dick, W.; Carey, L.; Carey, J.O. The Systematic Design of Instruction, 8th ed.; Longman: New York, NY, USA, 2015. 44. Zapartas, P.; Startsenko, I. Design and Development of a Web Environment for Audiovisual Content Management. Master’s Thesis, School of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessaloniki, Greece, 2012. Available online: http://search.lib.auth.gr/Record/ikee-130613 (accessed on 5 May 2019). (In Greek) 45. Tsipas, N.; Zapartas, P.; Vrysis, L.; Dimoulas, C. Augmenting social multimedia semantic interaction through audio-enhanced web-tv services. In Proceedings of the Audio Mostly 2015 on Interaction with Sound, Thessaloniki, Greece, 7–9 October 2015; p. 34. 46. Predescu, A.D.; Triantafyllidis, G. New Forms of Creative Artistic Expression Through Technology: An Alternative Perspective to Education. In Interactivity, Game Creation, Design, Learning, and Innovation; Springer: Cham, Switzerland, 2017; pp. 500–509. 47. Billeskov, J.A.; Møller, T.N.; Triantafyllidis, G.; Palamas, G. Using Motion Expressiveness and Human Pose Estimation for Collaborative Surveillance Art. In Interactivity, Game Creation, Design, Learning, and Innovation; Springer: Cham, Switzerland, 2018; pp. 111–120. 48. Vrysis, L.; Tsipas, N.; Dimoulas, C.; Papanikolaou, G. Crowdsourcing Audio Semantics by Means of Hybrid Bimodal Segmentation with Hierarchical Classification. J. Audio Eng. Soc. 2016, 64, 1042–1054. [CrossRef] 49. Vrysis, L.; Vryzas, N.; Sidiropoulos, E.; Avraam, E.; Dimoulas, C.A. jReporter: A Smart Voice-Recording Mobile Application. In Proceedings of the Audio Engineering Society Convention 146, Dublin, Ireland, 20–23 March 2018. 50. MathWorks. Computer Vision Toolbox: User’s Guide. 2012. Available online: mathworks.com/help/pdf_ doc/vision/vision_ug.pdf (accessed on 5 May 2019). © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). http://dx.doi.org/10.1109/93.917971 http://search.lib.auth.gr/Record/ikee-130613 http://dx.doi.org/10.17743/jaes.2016.0051 mathworks.com/help/pdf_doc/vision/vision_ug.pdf mathworks.com/help/pdf_doc/vision/vision_ug.pdf http://creativecommons.org/ http://creativecommons.org/licenses/by/4.0/. Introduction Background—Review or Related Work Research Aims and Project Motivation Materials and Methods Analysis Design Development Evaluation Results Implemented Guide and Offered Services Analysis and Usability Evaluation Results Machine-Driven Movement Recognition: Classification Schemes and Performance Evaluation Limitations Discussion Conclusions References