key: cord-0975881-u4dd9dpa
authors: Pornpongtechavanicha, Phisit; Wuttidittachotti, Pongpisit; Daengsi, Therdpong
title: QoE modeling for audiovisual associated with MOBA game using subjective approach
date: 2022-04-22
journal: Multimed Tools Appl
DOI: 10.1007/s11042-022-12807-1
sha: d37d9686cde441ee7e2064c00f0a17ff1dacffb2
doc_id: 975881
cord_uid: u4dd9dpa

With the advancement of multimedia communications and information technologies, online video games have become very popular recreational activities worldwide, while Multiplayer Online Battle Arena (MOBA) games have gained immense popularity from game players in recent times. Therefore, based on quality of experience, audiovisual quality evaluation models for MOBA games have been proposed in this study. The subjective data from 210 game players (both males and females) about audio quality during conversation, the video quality, the overall quality, and the weighted percentage of video and voice have been collected from small competition events of the Garena ROV (Arena of Valor) in Hua Hin, Thailand. The data from 160 game players have been used to create the models, while the data from 50 game players have been applied for model performance evaluation. After developing the models, Mean Absolute Percentage Error (MAPE) technique has been utilized for model evaluation. On completion of this study, it was found that two thirds of the proposed models provide better performance than the other previous works. All of them, called MOS(Audiovisual-MOBA1) model, MOS(Audiovisual-MOBA2) model and MOS(Audiovisual-MOBA3) model, show excellent performances with MAPE values of 4.95%, 5.92% and 4.75% respectively. Therefore, excellent performance of these proposed models becomes one of the contributions of this study. Also, this evidence of excellent model performance makes the proposed models ideal for utilization for assessment of audiovisual quality of the MOBA games. All audiovisual quality evaluation models provide excellent efficiencies.

Internet has been utilized widely for various purposes; for example, multimedia communications, information searching, business (e.g., e-commerce and Internet banking), education (e.g., e-learning), entertainment (e.g., Netflix and YouTube), social media networking (e.g., Facebook and Twitter) and Gaming. Furthermore, with the advancement of mobile communications, people can also use mobile Internet via mobile communication networks (e.g., 4G/ LTE and 5G). It has been forecasted that mobile communications networks will support several kinds of data traffic including video traffic that covers online video game [1] .

Because of the availability of mobile Internet at present, it has changed entertainment medium significantly, including video game industry [3] . A lot of video games, which has become online video games, face a constant increase in popularity. Similar to any other media, video game industry is now considered as one which is continuously growing and has one of the largest marketshare [38] . Particularly, in 2020-2021, Coronavirus (COVID-19) has spread to every country in the world. The virus changes human behavior from offline activity to online activity. It is one of the key factors driving the game industry. It increases interest in video gaming due to COVID-19-related lockdown measures. Especially, mobile gaming saw the most significant growth. Mobile games generated revenues of 77.2 billion USD approximately in total in 2020 (growing +13.3% compared to 2019) [14] . In Thailand, the expected growth value of the game industry will be increased from about 33.7 million USD in 2021 to 39 million in 2022 approximately. This information is consistent with the expected global growth values that will be about 174.2 billion USD and 187.8 billion USD in 2021-2022 respectively [14] . Nevertheless, it has been reported in 2019 that Thailand was the 19th biggest market in the world with about 18.3 million video game players [5] . That means video games are very popular in this country, particularly online video games that are played by not only regular teenagers but also Buddhist novice monks [40] . One of several reasons for gaming popularity is the support and promotion from the Tourism Authority of Thailand [43] .

For playing online games, particularly real-time multiplayer games, mobile phone's rendering efficiency (e.g., frame rate) and a high level of synchronism are required [30, 33] . Furthermore, it requires good network performance, otherwise network parameters (e.g., packet loss, latency or delay and jitter) impact the perception and response of game players. Consequently, perceived audio quality and video quality, called audiovisual quality, might be degraded, while playing online games with poor network quality. Therefore, this is very important for online game operators and/or service providers to understand how those factors affect gaming experience of game players [30] . One of the methods to understand the game players from each of their 'point of view' is using the Quality of Experience (QoE) metric, which can be applied for audiovisual quality measurement. Therefore, this research proposes a method to evaluate audiovisual quality of online video games provided by video game operators and/or service providers at the end-users point. However, this study is limited to a Multiplayer Online Battle Arena (MOBA) game that is very popular in Thailand, called ROV that stands for Realm of Valor [3, 12] .

This article, which is the extended version of [12] , has been arranged into seven sections. After the introduction, background information is described in Section 2. It covers three subsections, including Esports, MOBA and RoV, influence factors for gaming quality and audiovisual quality assessment methods, and prior research works. Section 3, describes the methodology, including data gathering from the event of online game competitions, proposed models that is the highlight of this article, and model evaluation and analysis. As for section 4 these discussions are presented. The conclusion with future work is eventually expressed in Section 5, the last section.

2 Background and related work 2.1 Esports, MOBA and RoV As described in [10] , the term 'esports' stands for 'electronic sports', which is considered to be a variant of online gaming and/or video gaming. 'Esports' can be defined as a multiplayer video game played competitively for spectators, usually by professional game players [10] . The esports association in South Korea defines it as a leisure activity within cyberspace in which participants matched their electronic game skills to compete with each other for winning [27] . In other words, this is a sort of sports within and through the cyberspace medium where a competitive game is played typically within accepted rules of tournaments and leagues on Internet [26, 48] . It can be a form of sports where the primary aspect of the sport is supported and/or facilitated by computer systems/networks and/or Internet and the input of the system are players and teams, while the output is mediated by human computer interfaces [18, 27] . It has been stated in [6] that esports is a type of sporting activity in which game players train and develop their hand-eye co-ordination skills and mental skills while using game-based multimedia communication and information technology, and where game players are virtually represented in an electronic sporting world [20, 46] . Esports involve a number of different video game genres including action games, racing games, sports games, first person shooters, fighting and strategy games. Furthermore, this is now accepted worldwide, given the popularity and growth of the esport industry. It has been estimated that these numbers will be grow to 234 million esports enthusiasts and more than 240 million occasional viewers, in total 474 million people approximately, in 2021 [41] . Furthermore, it was reported that the 2019 SEA Games hosted the tournaments for six games across mobile phones, consoles and personal computers, including Arena of Valor (AoV or called ROV in Thailand) and other five popular video games, in December 2019 [16] . Recently, The International Olympic Committee (IOC) announced that the Olympic Virtual Series organized competitions in five esports (e.g., cycling, auto racing, baseball and sailing) and staged ahead of the Summer Games 2021 in Japan [21] .

At present, while there are many genres of online video games (e.g., action-adventure, shooter, role-playing, sport, fighting and strategy games), one of the most popular game genres is MOBA. This is one of the largest subgenres of strategy video games [17, 34] . According to the game player motivation model [13, 51] , mastery is one of the major objectives for game players. It is also one of the most popular strategy game genres in eSports at present [2] . There are several MOBA games in the video game industry, for example, League of Legends (LoL), Defense of the Ancients (DotA) and Heroes of the Storms [3] . For ROV, it stands for Realm of Valor as it is called in Thailand, while it is called Arena of Valor and Honor of Kings for global and China respectively [12, 43] . This is one of the most popular and attractive MOBA games on smart phones in Asia since its gamification tool is applied to explore and analyze the main elements in ROV's competitive and entertainment experience [3] . Its popularity is evident from the 100 million downloads from the public [45] .

ROV is a Free-to-Play (F2P) online video game that can be downloaded, installed and played by game players. It provides the scenario that two teams of players can fight with each other to become the winner. It has a shorter game period, about 15-20 min. Also, it has easy controller for playing on smart phones; thus, it can provide entertainment to the game players [12] . Furthermore, this is a game of 5 versus 5 played by the Red and the Blue team [3] . This game supports both iOS and Android operating system [3] . The goal of each team, it is to destroy the opponent's tower located at the opponent's base, for winning [3] . While playing the game, each one in each team can talk or communicate to other team members via Voice over IP (VoIP) that is an ROV's characteristic. Nevertheless, game players may encounter technical problems (e.g., low video quality and unclear voice quality) if their network provides them poor Quality of Service (QoS). ROV is very popular in Thailand. It was reported that there were more than 31 million registered accounts in this country, 11 million watch hours and 55 million views of Season 3 videos in Q2/2019, while the Thai population was 69.6 million people approximately in the same year [5, 44] . Recently, the ROV team from Thailand has become the winner in the Arena of Valor World Cup 2021 (AWC 2021) that was organized with the theme 'Fight together rise together' (see Fig. 1 ) [4] .

ITU-T describes that there are several influence factors that can impact on the gaming QoE perceived by users or game players who play networked games or online games [22] . Those influence factors have been grouped into three main factors which are comprised of: 1) Human factors: these factors are described as variant or invariant properties or characteristics of game players. The characteristics can describe the demographic and backgrounds, the emotional state and/or the physical and mental constitution. These factors cover experience with gaming in general, experiences with a specific game, extrinsic and intrinsic motivation, dynamic, and static human factors, and human vision. 2) Context factors: they are things that embrace any situational property to describe the user's surrounding in terms of physical, task, social, and technical characteristics, for Context influence factors consist of physical environment factors, service factors, novelty, and social context that is one of the essential factors for online multiplayer games. 3) System factors: these factors are characteristics and properties that determine the produced quality of a service or an application in technical terms. They cover the factors and their parameters or components as shown in Table 1 [22] .

For Gaming QoE, it is a multidimensional construct related to the quality features, which were studied and proposed by [31] . Then, they have been applied to the ITU-T P. 809 standard [23] . For quality features, there are aesthetics and appeal, interaction quality, playing quality, positive and negative effect, player experience acceptability, and engagement which consist of involvement, immersion, presence, flow, absorption, and relationships between engagement concepts [23] .

Audiovisual quality evaluation methods can be categorized into subjective and objective methods [9] . They are referred to as QoE measurement methods. QoE directly focuses on user perception of services and/or applications. It has been described in [28] that QoE is the key determining factor in the success of multimedia services and/or applications. QoE covers a wider range of aspects and includes more factors than QoS. Furthermore, it has been described in [25] that QoE directly reflect users' acceptance of the services, this is therefore appropriate for the assessment of service quality in applications, networks and/or systems.

For QoE, Mean Opinion Score (MOS) is the key quality factor that presents the QoE level directly [7] . It is usually a five-point scale to quantify level of multimedia quality perception (5 

In 2018, Mora-Cantallops and Sicilia [32] presented a literature review by focusing on 23 MOBA game papers. They found that one of the main lines in research on MOBA games is based on modeling and prediction, covering strategy modelling. However, they did not mention about QoE modeling based on MOBA games. From other previous research works, there are several interesting works associated with QoE models, including objective-online game QoE models [37, 39, 42, 47] , but only the subjective QoE models are presented in Table 2 that can be briefly described as follows:

1) Dwyer & Finn [15] performed a study to assess MOS of gameplay under the controlled conditions based on latency. They recruited 32 volunteers to play the game 'Team Fortress 2', before the analysis and modeling two MOS models associated with delay and jitter as shown in (1)-(2). 2) Based on video telephony, ITU-T proposed the QoE model called 'Opinion model for video telephony applications as presented in (3) in [19] conducted a study based on audio and video (no games) to obtain MOS scores from the experiments with more than 36 participants within controlled environment (e.g., about 15 in. flat-screen and a sound insulated cabinet). He found that audio quality is weighted slightly higher than video quality for normal-motion content, whereas video quality is weighted significantly higher than audio quality for high-motion content. In addition, he performed the regression analysis and then proposed two QoE models as shown in (4)-(5). 4) Similar to [19] , Winkler and Faller [49] performed the tests with 17 in. LCD screen following ITU-T standards subjectively to investigate about perceived audiovisual quality based on low-bitrate multimedia content (e.g., high motion video of American football and the sounds of female commentator and crowd cheering). After conducting the tests with 24 participants and analysis. Then, two models as shown in (6)-(7) were proposed. 5) Belmudez and Moller [8] conducted the study with 21 and 48 participants in the passive and interaction experiments respectively. They studied with various scenarios before performing the analysis, and then proposed several mathematical models, including two models for the short conversation test as in (8) and the audiovisual short conversation test as shown in (8)-(9) respectively. 6) Lastly, Pornpongtechavanicha and Daengsi [35] conducted the study based on video telephony. This work was conducted with about 200 subjects who participated the subjective tests with short conversation scenario via two applications, LINE and Skype. After obtaining the MOS results for audio quality and video quality, the analysis was performed before proposing a simple mathematical model for video telephony quality evaluation as shown in (10). One can see in Table 2 that the QoE models as in (1)-(2) are associated with QoS parameters (e.g., latency or delay, jitter and packet loss), unlike the approach in this article that is based on subjective approach. While the models in (3)-(10) are mainly based on video telephony utilizing subjective approach but they are not designed or proposed for online game quality evaluation like the proposal in Section 3. Furthermore, it has been found in [36] that (3)-(5) are unreliable, while (6)-(7) provide lower performance than (8)- (10) . Therefore, the branch of research line based on QoE modeling and prediction for MOBA games covering audiovisual quality becomes a new challenge and a gap of proposing the approach as in this article.

This section presents the methodology that covered, game competition design and data gathering, proposed models, and model evaluation and analysis, following the processes as shown in Fig. 2 . More details are described in each sub-section.

This study focused on calling or conversation of game players while playing the online video game via smart phones or mobile phones. The online video game in this study was based on the Garena AOV or called ROV for short in Thailand. This online game is one of the very popular MOBA games in Thailand.

For creating the mathematical model, the subjective data from the game players is necessary. In order to gather a lot of game players at the same period, therefore the ROV competitions were designed by organizing as two competitive events in 2019 in Hua Hin, Prachuap Khiri Khan, Thailand. All game players were the students of Rajamangala University of Technology Rattanakosin (RMUTR), Wang Klai Kangwon Campus in Thailand, who applied to join the ROV competitions.

Two competition events or tournaments were organized in September and October 2019 for the subjective data gathering, Further information can be described as follows:

-There were four modes, consisting of 2v2, 3v3, 4v4 and 5v5 (see Fig. 3 ), that means there were 2, 3, 4, and 5 game players per team respectively. -Similar to [63], all of them had to vote their opinion in weighted percentage about the audio quality and video quality (e.g., audio/video = 90% / 10%, 80% / 20%, 70% / 30%, 60% / 40%, … or 10% / 90%).

After finishing the events, the gathered data from the game players were split into two parts. The first part was used for creating the audiovisual quality evaluation model, consisting of 103 male and 57 female participants, 40 participants per mode or 160 subjects in total, as presented in Table 3 . The average age of them was 19.71 ± 1.35 years old at that period. While, the For the weighted percentage of audio and video quality, after obtaining the votes about the ratio between audio quality and video quality from all participants, the weighted percentage for audio and video for each mode of the gameplay was calculated and presented in Table 5 . One can see that the weighted percentage of video quality is slightly higher than the percentage of audio quality. Furthermore, for the average from all modes of the gameplay, the weighted percentages of video quality and audio quality were 46.56% and 53.44% respectively. Nevertheless, the second part of the subjective data will be described in details in Subsection 3.3 for model performance evaluation.

This sub-section presents the highlight and the major contribution of this article. In order to create models, the proposed mathematical models for audiovisual -QoE model associated with MOBA game can be simply created using the concept of the Simple Additive Weighting (SAW) that is presented as follows [29] :

Where MOS SAW is the overall MOS based on SAW, while w j and MOS j are is the weight and the MOS of criteria j respectively. Furthermore, this proposed model also followed (10) in Table 2 , which is the QoE model for video telephony [35] . Nevertheless, there is advancement beyond [35] , since the gathered data of audiovisual quality have been also applied to the QoE model. As shown in Table 5 in the previous section, the weighted percentage of video quality and audio quality are 53.44% and 46.56% respectively. The calculation of them with MOS Video of 4.24 and MOS Audio of 3.94 as in the last column in Table 4 can be shown as follows: One can see that the calculated MOS Audiovisual of about 4.1 is higher than the average MOS Audiovisual of 4.00 shown at the last column and row in Table 4 . That means the difference in value of 4.10-4.0 = 0.10 between those MOS Audiovisual can be applied to create new models as well. Therefore, the weighted percentage of video quality and audio quality of 53.44% and 46.56% respectively, and the difference value of 0.1 can be utilized to the new model using a simple technique. However, this article aims to propose simple QoE models for MOBA games, thus the weighted coefficients should be simple values as well. Therefore, the coefficient a and b should be 0.47 and 0.53 instead of 46.56% and 53.44% respectively, while the coefficient c should be 0.10 (from the difference between the calculated MOS Audiovisual and the average MOS Audiovisual as the above mentioned. These three coefficients can be utilized in the proposed QoE models for MOBA game as shown in (12)- (14) .

After obtaining three equations of the proposed QoE models as in (12)- (14) , they were evaluated and compared with the previous QoE models. However, as mentioned in Subsection 2, (1)- (7) are not appropriate for model evaluation Moreover, after considering (10), it was found that the weighted coefficient of 0.65 for MOS Audio is significantly higher than the coefficient of 0.35 for MOS Video . This is inconsistent with the weighted coefficients in (12)- (14) where the coefficient of 0.53 for MOS Video is higher than the coefficient of 0.47 for MOS Audio . Thus, (10) has been also discarded from model evaluation because the principal criteria of the proposed model is predominantly audio quality, and not video quality as in (12)- (14) . Therefore, only two models as in (8) and (9) that have been used for model evaluation for comparison with (12)- (14) . For model evaluation, Mean Absolute Percentage Error (MAPE) that is a popular technique with high reliability has been selected for this article. It can be calculated using (15) , where MAPE is the mean absolute percentage error (%), S cal means the calculated score from the mathematical model, and S ref is the reference score from the test set [11, 35] . The MAPE value can be interpreted as follows:

-0% < MAPE <10% presents an excellent forecast -10% < MAPE <20% presents a good forecast -20% < MAPE <50% presents a fair forecast

For MAPE calculation in this article, test sets must be applied. In this article, three test sets were created from the 2nd part of the gathered data as in Table 6 , randomly. Each test set consisted of 6 records from each game mode. Thus, each test set is comprised of 24 records in total from 24 game players (both female and male). Those were 18-20 years old at that period.

The information about those test sets, called T1-T3 for short, is presented in Table 7 . After evaluating with T1-T3, one can see in Table 8 that MOS Audiovisual-MOBA3 as in (14) (9) or the MOS Audiovisual-BM2 model provide average MAPE of 5.24% and 7.47% respectively. Meanwhile (12)- (14) or MOS Audiovisual-MOBA1 -MOS Audiovisual-MOBA3 models provide average MAPE values of 4.95%, 5.92% and 4.79% respectively. That means MOS Audiovisual-BM2 model as in (9) is the worst model when compared to the other models, whereas MOS Audiovisual-MOBA3 as in (14) is the best model because it provides the highest performance with the lowest MAPE value (only 4.79%). Therefore, this can be used as the criteria for the best model selection. Nonetheless, all models under evaluation, as in (8)- (9) and (12)- (14) provide excellent forecasts since these MAPE values are less than 10%. The average MAPE values from all mathematical models don't seem different. Therefore, this is necessary to verify this by using a statistical technique called Analysis of Variance (ANOVA) as in [11] . For analysis using this technique, the raw data of MOS Audiovisual values from calculation with T1-T3 using (8)-(9) and (12)- (14) have been utilized based on the hypothesis as follows:

H 0 : The audiovisual quality calculated from (8)-(9) and (12)-(14) is the same. H 1 : The audiovisual quality calculated from (8)-(9) and (9)-(14) is not the same.

After performing the hypothesis tests within the confidence interval of 95%, the results are presented in Table 9 . The analysis results show that the p values from hypothesis testing with T1-T3 are 0.172, 0.295 and 0.156 respectively. Thus, the null hypothesis (H 0 ) can be accepted, while the alternative hypothesis (H 1 ) has been rejected, because those p values are less than the significant level of 0.05. That means there is no significant difference in every case with each test set among the model (8)-(9) and (12)- (14) , although (14) or the MOS Audiovisual-MOBA3 model provides the best performance when compared to the other models. (12)- (14) in the previous section. They have been created with a simple concept but it has been proven in the previous section that their performances are excellent. Of course, this issue is the major contribution of this article. 3) This is one of the audiovisual studies that has been conducted with a lot of participants.

With 210 subjects, this number is higher than other prior works [8, 19, 24, 49] , except [35] . Therefore, the proposed models can be applied to measure audiovisual quality for MOBA game with confidence. Moreover, this article is the first study based on MOBA game that was conducted with more than 200 subjects. 4) The first part of MOS Audiovisual from the game players as in Table 4 , called the training set, shows average MOS Audiovisual of 4.00 ± 0.51, while the MOS Audiovisual for the test set T1-T3, as in Table 7 , shows MOS Audiovisual of 4.00 ± 0.29, 4.08 ± 0.41 and 4.04 ± 0.36 respectively. There are slight differences between the training set and the test set (T1-T3). However, this issue is acceptable because both sets of data have been gathered from the students with the same criteria. 5) From the coefficient of 0.53 for MOS Video and the coefficient of 0.47 for MOS Audio , one can see that video quality slightly seems more important than audio quality for the game players. These coefficients are not conformed with the coefficients of the video telephony QoE as shown in [35] , since in that QoE model, audio quality is more important than video quality. 6) One of the proposed models, MOS Audiovisual-MOBA3 provides the lowest MAPE value of 4.79%, whereas MOS Audiovisual-MOBA1 , MOS BM1 , and MOS Audiovisual-MOBA2 provides the MAPE values of 4.95%, 5.24%, and 5.92% respectively. That means the MOS Audiovisual- [8] . Therefore, it should be the best option for subjective QoE assessment for gaming, while MOS Audiovisual-MOBA1 should be the second option. Furthermore, video game developers may apply these new models to help improving the audiovisual quality of their games, in order to increase users' perception and satisfaction. 7) The subjective test method as presented in this study might be utilized to study other game genres as described in Section 2 (e.g., FPS) or other games (e.g., Free Fire) on both PCs and mobile phones. Moreover, the proposed models in this article might be utilized to evaluate other MOBA games with high reliability. 8) Lastly, other techniques, such as Analytic Hierarchy Process (AHP) may be applied for finding the coefficients of the new QoE models for audiovisual or new gaming QoE models, in order to expand and enhance the discipline and related disciplines.

This article presents three audiovisual quality evaluation models with reference to MOBA games using subjective tests with ROV that is a popular game in Thailand. After obtaining the data from the study with 210 game players who joined the ROV game competition events, three QoE models based on MOBA game were proposed. Then, the model performance evaluation was performed. The evaluation results show that the proposed QoE models, consisting of MOS Audiovisual-MOBA1 -MOS Audiovisual-MOBA3 for MOBA game provide excellent performances, particularly MOS Audiovisual-MOBA3 that provides the highest performance when compared to MAPE values from pervious QoE models with MAPE value of 4.79%. Although, this study with subjective approach on mobile phones covers all modes in a MOBA game (e.g., 2v2, 3v3, 4v4 and 5v5), similar studies on PCs, other game genres and various other games should be conducted in future works.

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

5G Service & Use Cases

VisuaLeague: Player performance analysis using spatial-temporal data

Analysis of realm of valor and its business model on PC and mobile platform comparison

Arena of Valor World Cup 2021: Schedule, streams, and results

Garena touts Thai e-sports potential

The mediating effect of motivations between psychiatric distress and gaming disorder among esport gamers and recreational gamers

Delivering quality of experience in multimedia networks

Audiovisual quality integration for interactive communications

Characterizing voice and video traffic behaviour over the internet

Will esports result in a higher prevalence of problematic gaming? A review of the global situation

QoE modeling for voice over IP: simplified E-model enhancement utilizing the subjective MOS prediction model: a case of G.729 and Thai users

A proposed audiovisual quality assessment model associated with multiplayer online Battle arena game: a pilot study with ROV

Gamer Motivation Framework: Not all games are created equal

Depa (2021) Investment Quarterly Gaming Industry

Predicting the perceived quality of a first person shooter game: the team fortress 2 T-model

Genre breakdown of video game sales in the United States in

What is eSports and why do people watch it?

A basic multimedia quality model

International Olympic Committee (2021) IOC makes landmark move into virtual sports by announcing firstever Olympic Virtual Series

ITU-T Recommendation ITU-T G.1032 Influence factors on gaming quality of experience

ITU-T Recommendation ITU-T P.809 Subjective evaluation methods for gaming quality

ITU-T Recommendation Recommendation ITU-T G.1070 Opinion model for videotelephony applications

An IoT service aggregation method based on dynamic planning for QoE restraints

Electronic sport and its impact on future sport

Reconsidering esport: economics and executive ownership

Quality of experience-driven adaptation scheme for video applications over wireless networks

A vertical handover decision based context awareness guaranteeing the user perceived quality of service

Understanding gaming experience in Mobile multiplayer online Battle arena games

New ITU-T standards for gaming QoE evaluation and management

MOBA games: a literature review

Analysis of Gameplay in MOBA Games Under Different Network Conditions

Feature based multimedia data analysis in video games. School of Science & Technology, Master of Science (MSc) in Data Science

Video telephony -quality of experience: a simple QoE model to assess video calls using subjective approach

Audiovisual quality assessment: a study of video calls provided by social media applications

Empirical study of subjective quality for massive multiplayer games

Feature based Multimedia Data Analysis in Video Games

Empirical QoE study of in-home streaming of online games

Thai Buddhist monks triumph in e-sports event as some fans criticise winners for wearing orange robes

How to measure and model QoE for networked games?

Gaming-mad Thais ready to play

Video Games Stats (VGS) (2020) Honor of Kings / Arena of Valor

On the scientific relevance of esports

Predicting the perceived quality of a first person shooter: the quake IV G-model

Fulfilling the needs of eSports consumers: a uses and gratifications perspective

Perceived audiovisual quality of low-bitrate multimedia content

MOS estimation model development using ACR listening-opinion tests with Thai users referring to loss effects: a case of G.726 and G.729

The Gamer Motivation Model in Handy Reference

Acknowledgments Thanks to the Faculty of Industry and Technology, Rajamangala University of Technology Rattanakosin (Wang Klai Kangwon Campus), Faculty of Information Technology and Digital Innovation, King Mongkut's University of Technology North Bangkok, and Faculty of Engineering, Rajamangala University of Technology Phra Nakhon for supporting. Especially, thanks to all students, staff and lecturers who were participants or involved. Lastly, thanks to Mr. Bhaskar Laha (Bobby) for English editing.

Conflict of interest All authors declare that there is no any conflict of interest.