Sem-Fit: A semantic based expert system to provide recommendations in the tourism domain to provide recommendations Sem-Fit: A semantic based expert system in the tourism domain Ángel García-Crespo 1, José Luis López-Cuadrado 2, Ricardo Colomo-Palacios ⇑, Israel González-Carrasco 2, Belén Ruiz-Mezcua 3 Computer Science Department, Universidad Carlos III de Madrid, Av. Universidad 30, Leganés 28911, Madrid, Spain a r t i c l e i n f o a b s t r a c t Keywords: Semantic technologies Semantic labeling Fuzzy logic Recommender systems Hotels The hotel industry is one of the leading stakeholders in the tourism sector. In order to reduce the trav eler’s cost of seeking accommodations, enforce the return ratio efficiency of guest rooms and enhance total operating performance, evaluating and selecting a suitable hotel location has become one of the most critical issues for the hotel industry. In this scenario, recommender services are increasingly emerg ing which employ intelligent agents and artificial intelligence to ‘‘cut through’’ unlimited information and obtain personalized solutions. Taking this assumption into account, this paper presents Sem Fit, a seman tic hotel recommendation expert system, based on the consumer’s experience about recommendation provided by the system. Sem Fit uses the consumer’s experience point of view in order to apply fuzzy logic techniques to relating customer and hotel characteristics, represented by means of domain ontolo gies and affect grids. After receiving a recommendation, the customer provides a valuation about the rec ommendation generated by the system. Based on these valuations, the rules of the system are updated in 1. Introduction Tourism is one of the most powerful With roughly 11% of the world’s total emplo is often presented as the first global indust the first tourist continent (Longhi, 2007). Th nization (2006) predicts that by 2020, tou world will increase by over 200%. Because tion intensive business, there are opportun order to adjust the new recommendations to past user experiences. To test the validity of Sem Fit, the validation accomplished includes the interaction of the customer with the system and then the results are compared with the expert recommendation for each customer profile. Moreover, the values of preci sion and recall and F1 have been calculated, based on three points of view, to measure the degree of rel evance of the recommendations of the fuzzy system, showing that the system recommendations are on the same level as an expert in the domain. industries worldwide. yment or GDP, tourism ry, and Europe is by far e World Tourism Orga rist arrivals around the tourism is an informa merce (Cao & Schniederjans, 2006). Not in vain, to be able to com pete in an increasingly competitive and globalized market, companies need to be provided with new strategies that allow them to confront successfully the environment challenges (Acosta & Febles, 2010). Developments in search engines, carrying capacity and speed of networks, have influenced the number of travelers around the world that use technologies for planning and experi encing their travels (Buhalis & Law, 2008). Thus, according to ities to apply informa Kenteris, Gavalas, and Economou (2009), the convergence of IT tion technology (IT) to support tourism and tourists (Watson, Akselsen, Monod, & Pitt, 2004), and, pursuing these opportunities, and communications technologies and the rapid evolution of the Internet have been some of the most influential factors in tourism the tourism industry is leading eCommerce applications (Werthner & Ricci, 2004) and one of the fastest growing segments of eCom ⇑ Corresponding author. Tel.: +34 91 624 59 58; fax: +34 91 624 9129. E-mail addresses: angel.garcia@uc3m.es (Á. García-Crespo), joseluis.lopez. cuadrado@uc3m.es (J.L. López-Cuadrado), ricardo.colomo@uc3m.es (R. Colomo- Palacios), Israel.gonzalez@uc3m.es (I. González-Carrasco), belen.ruiz@uc3m.es (B. Ruiz-Mezcua). 1 Tel.: +34 91 624 94 17; fax: +34 91 624 9129. 2 Tel.: +34 91 624 91 17; fax: +34 91 624 9129. 3 Tel.: +34 91 624 99 68; fax: +34 91 624 9129. 1 that have changed travelers’ behavior. Indeed the Internet is cur rently the primary source of tourist destination information for travelers (Chiu, Yueh, Leung, & Hung, 2009). Buhalis (1998) pointed out that the use of the Internet in the tourism industry provides access to a large number of people, as well as offering the opportu nity to develop closer relationships with customers. ICTs have radically changed the efficiency and effectiveness of tourism organisations, the way that businesses are conducted in the marketplace, as well as how consumers interact with organisations (Buhalis, 2003). And in a highly competitive environment, mailto:angel.garcia@uc3m.es mailto:joseluis.lopez.cuadrado@uc3m.es mailto:joseluis.lopez.cuadrado@uc3m.es mailto:ricardo.colomo@uc3m.es mailto:Israel.gonzalez@uc3m.es mailto:belen.ruiz@uc3m.es referencia bibliográfica. Published in:Expert Systems with Applications, (15 September 2011), 38(10), 13310–13319 according to Luo, Feng, and Cai (2004), tourists who searched on the Internet tended to spend more at their destinations as com pared to those who consult other information sources. The hotel industry is one of the leading stakeholders in the tour ism sector. In order to reduce travelers’ cost of seeking accommo dations, enforce the return ratio efficiency of guest rooms and enhance total operating performance, evaluating and selecting a suitable hotel location has become one of the most critical issues for the hotel industry (Chou, Hsu, & Chen, 2008). Service quality in hotels can be regarded as a composite measure of various attri butes, including tangible attributes but also intangible/subjective attributes (safety, quietness, . . .) (Benítez, Martín, & Román, 2007). Consumers’ judgments towards a service depend basically on the strength of their beliefs or expectations about various fea tures or attributes associated with the service and the weight of attributes (Engel, Blackwell, & Miniard, 1995). And, since online travelers are enthusiastic about meeting other travelers who have similar attitudes, interests and way of life (Wang, Yu, & Fesenma ier, 2002), there is a strong connection between travelers’ judg ments and hotel recommendations. In this scenario, recommender services are increasingly emerging that employ intelligent agents and artificial intelligence to ‘‘cut through’’ unlim ited information and obtain personalized solutions (Ricci & Werth ner, 2006). Recommender systems are commonly defined as applications that e commerce sites exploit to suggest products and provide consumers with information to facilitate their deci sion making processes (Niininen, Buhalis, & March, 2007). How to deliver relevant information to both potential and existing cus tomers has become an important task for the hospitality industry (Xian, Kim, Hu, & Fesenmaier, 2007), which, in many cases, is based on artificial intelligence techniques (Schiaffino & Amandi, 2009). Destination recommendation systems are mostly fed with subjec tive information provided by the tourism industry itself (Goosen, Meeuwsen, Franke, & Kuyper, 2009), but most of the relevant infor mation for recommendation should come from customers. Indeed, according to Klein (1998), travel products in general are arguably ‘‘experience goods’’ in that full information on certain attributes cannot be known without direct experience. Taking this assumption into account, this paper presents Sem Fit, a semantic hotel recommendation expert system, based on the consumer’s experience about the recommendation provided by the system. The proposed expert system uses the consumer’s experience point of view in order to apply fuzzy logic techniques to relating customer and hotel characteristics. Hotel characteris tics are represented by means of a domain ontology. After receiving a recommendation, the customer provides a valuation about the recommendation generated by the system. Based on the customer’s valuations, the rules of the system are updated in order to adjust the new recommendations to the past user experiences. The paper consists of five sections and is structured as follows. Section 2 reviews the relevant literature. Section 3 discusses the main features of Sem Fit including the conceptual model, algorith mic and architecture. Section 4 describes the evaluation of the tool performed including a description of the sample, the method, re sults and discussion. Finally, the paper ends with a discussion of re search findings, limitations and concluding remarks. 2. Background It is a widely recognized fact that information and decision making have become the foundation for the world economy (Wang, 2008). Among many enterprise assets, knowledge is trea ted as a critical driving force for attaining enterprise performance goals because knowledge facilitates better business decision 2 making in a timely fashion (Han & Park, 2009). And due to the importance of tourism, recommender systems and decision sup port systems for tourism have been a field of study since the very beginnings of artificial intelligence. A recommender system can provide a set of solutions that best fit the user, depending on different factors concerning the user, the objective or the context where it is applied. Such systems can re duce search efforts (Liang, Lai, & Ku, 2006) and provide valuable information to assist consumers’ decision making process (Ricci, 2002) in order to solve the problem of information overload (Kuo, Chen, & Liang, 2009). Adomavicius and Tuzhilin (2005) pro vide a survey of recommender systems as well as describe various limitations of current recommendation methods, and discuss pos sible extensions that can improve recommendation capabilities and make recommender systems applicable to an even broader range of applications. Due to the importance of tourism, many efforts had been de voted to recommender systems for tourism (e.g. Castillo et al., 2008; Loh, Lorenzi, Saldana, & Licthnow, 2004; Ricci & Nguyen, 2007; Wallace, Maglogiannis, Karpouzis, Kormentzas, & Kollias, 2003), often based on artificial intelligence techniques, as was pre dicted in the early nineties by Crouch (1991): intelligent agents (e.g. Aciar, Serarols Tarres, Royo Vela, & De la Rosa i Esteva, 2007; Schiaffino & Amandi, 2009), fuzzy approaches (e.g. Lenar & Sobecki, 2007; Ngai & Wat, 2003), Bayesian networks (e.g. Huang & Bian, 2009; Jiang, Shang, & Liu, 2009), to cite just a few. Nor is the field of using semantics in tourism new. Fodor and Werthner (2004) presented Harmonise, a project that deals with business integration in tourism using ontologies for mediation. The SATINE project by Dogac et al. (2004) describes how to deploy semantically enriched travel Web services and how to exploit semantics through Web service registries. Niemann, Mochol, and Tolksdorf (2008) propose how to enhance hotel search with Semantic Web Technologies. Jakkilinki, Georgievski, and Sharda (2007) proposed an ontology based e Tourism Planner AuSTO that enables users to create an itinerary in one single application by this intelligent tool that builds on semantic web technologies. The LA_DMS project (Kanellopoulos, 2008) provides semantic based information for tourism destinations by combining the P2P para digm with semantic web technologies. In the specific field of using semantics to provide better infor mation for tourists there have been relevant and recent efforts in the literature. For example, García Crespo et al. (2009) proposed a semantically enriched recommendation platform for tourists on route, later expanded to Destination Management Organizations (García Crespo, Colomo Palacios, Gómez Berbís, Chamizo, & Rive ra, 2010a). Lee, Chang, and Wang (2009) used ontologies to provide recommendation in the context of the city of Tainan (Taiwan). Fi nally, and in the most similar contribution to the literature, Huang and Bian (2009) integrated Bayesian networks and semantics to provide personalized recommendations for tourist attractions over the Internet. According to Huang and Bian (2009), there are two challenges in developing a system for personalized recommendations for tourism. One is the integration of heterogeneous online travel information. The other is the semantic matching of tourist attractions with travelers’ preferences. Taking into account that all information will be given to the system by the users using this means, the challenge of Sem Fit will be the semantic match ing between attractions and preferences. To do so, Sem Fit will use the fuzzy logic paradigm in order to express the relationship between the hotel characteristics and the customer preferences. The Sem Fit’s fuzzy engine will recalculate this fuzzy relation ships based on the customer feeling about previous recommen dations, allowing the automatic adaptation of the recommender system to the customers’ preferences. Young people 10 20 30 40 Fig. 2. Membership function for the fuzzy set ‘‘young people’’. 3. Sem-Fit: fundamentals and internals Usually experts and customers express their knowledge and preferences in the form of imprecise terms such as ‘‘young’’ or ‘‘near’’. The fuzzy logic paradigm allows the representation of these imprecise terms in order to implement intelligent systems. Based on this paradigm, Sem Fit allows the representation of fuzzy vari ables in order to describe hotels and customers. Later on, Sem Fit uses these variables to perform recommendations. This section is structured as follows. First, an explanation on the fuzzy logic par adigm is given. Secondly, the process of capturing fuzzy knowledge in hotel recommendation is depicted. Later on, the recommenda tion process is defined and explained. Fourth, the customer feeling capturing procedure is portrayed. Finally, Sem Fit architecture and implementation is shown. 3.1. Fuzzy logic paradigm The fuzzy sets theory provides a framework for the representa tion of the uncertainty of many aspects of human knowledge (Zadeh, 1965). Classic sets theory establishes that the elements of the universe may belong to a set or may not belong to that set. Then, given the set of odd numbers: U f1; 2; 3; 4; 5; . . .g; we can affirm that ‘‘3’’ belongs to such a set. We can also affirm that the number ‘‘21’’ belongs to the set of numbers greater than 7, but number 3 does not belong to such a set. The membership function for a given set can be depicted as shown in Fig. 1. When the element belongs to the set, the function takes the value 1, and when the ele ment does not belong to the set, the function takes the value 0. For a given element, fuzzy sets theory proposes the use of inter mediate degrees of membership to a set. In this way, if we consider the set of young people we can consider that a person who is 15 years old belongs to such a set with a degree of 1 (belongs), a person who is 30 years old belongs in some degree (for example 0.7) and a person who is 80 years old does not belong to this set (degree of 0). We can represent this membership function as de picted in Fig. 2. Fuzzy sets provide a way for defining cases in which the mem bership to a given set is relative or the membership function is not defined at all, allowing the representation of imprecision or uncer tainty. Imprecision and uncertainty modeling by means of fuzzy sets allows solving problems which cannot be solved by means 1 0 1 2 3 4 5 6 7 8 9 10 11 … Fig. 1. Membership function for the set of numbers greater than 7. 3 of classic techniques. Some such domains are: classification prob lems, pattern recognition, signal processing, knowledge based sys tems or temporal reasoning. Using this fuzzy theory, the purpose of Sem Fit is to offer hotel recommendations based on expert criterion. This expert criterion will be represented by means of fuzzy sets based on the affect grid (Russell, Weiss, & Mendelsohn, 1989). However, the customer pref erences may be different from the expert point of view or may change as time goes by. For this reason it is necessary to update the fuzzy rules which represent the expert criterion using the cus tomers’ feedback. 3.2. Capturing the fuzzy knowledge about hotel recommendation Fig. 3 depicts the main elements in our proposal about captur ing knowledge in hotel recommendations: 1. The semantic descriptions of the hotels. 2. The fuzzy relationship between the characteristics of the cus tomers and the characteristics of the hotels. 3. The customer feeling about the recommendations. The first step of the recommendation process consists of repre sentation of knowledge about how the hotels are selected. This knowledge is expressed using fuzzy sets. In a first stage, the fuzzy sets are defined based on the expert knowledge. An expert defines by means of fuzzy sets the characteristics of the hotels and the characteristics of the customers. Some fuzzy sets defined are: lux ury hotel, relaxing trip or young people. After defining the fuzzy terms used by the expert to describe hotels and customers, the membership function is defined for each fuzzy set. This membership function allows for the transformation of the customer characteristics and the hotels characteristics into fuzzy values. Studies of emotion and affect in human beings have an estab lished history which originates in philosophy. As a result of this tradition, and using their own work as a basis (Russell, 1980), Russell et al. (1989) proposed a measure of affect which had a pro found impact on the field of social psychology. They termed the measure the affect grid, a scale designed as a quick means of assessing affect along the dimensions of pleasure displeasure and arousal sleepiness on a 1 9 scale. The affect grid may prove to be the instrument of choice when subjects are called on to make affective judgments in rapid succession or to make a large number of judgments, especially when those judgments are to be aggre gated (1989). Using this scale, researchers can collect emotion rat ings from stress and tension to calm, relaxation or serenity, and Fig. 3. System design. from ecstasy, excitement and joy to depression, melancholy, sad ness, and gloom. According to the studies of these authors, the affect grid is potentially suitable for any study that requires judgments about affect of either a descriptive or a subjective kind. Based on the cus tomer characteristics, the fuzzy system will estimate the custom ers affect grid for each hotel characteristic. Then, the fuzzy sets for the representation of the customer characteristics are related with the concepts of the hotel ontology by means of the affect grid. In this way, the results of the fuzzy recommender can be translated into concepts of the hotel ontology in order to obtain the recommendation. The measure of the affect grid will be translated to linguistic tags related to the hotel characteristics by mean of semantic annotations based on the rules defined by the expert options. Such tags are ade quate, very adequate, not adequate or never recommend. Each tag has a value related in order to rate each hotel based on the amount of positive or negative tags obtained from the fuzzy reasoning. Affect grid was previously employed by authors in previous works that combined semantic technologies with emotions (García Crespo, Colomo Palacios, Gómez Berbís, & García Sánchez, 2010b). 3.3. Recommendation process The main steps of the recommendation process are depicted in Fig. 4. The recommendation is primarily based on the customer char acteristics. The steps of the reasoning process are: 1. Obtaining the customer characteristics. The customer provides the information about his personal situation, preferences, etc. The information required is established based on the expert cri terion as mentioned in Section 3.1. 2. Fuzzification. The values provided by the customer are con verted into the fuzzy sets defined by the expert to describe the customer characteristics. The fuzzy value for each variable is obtained by means of the membership function related to each fuzzy set. 4 3. Once the customer profile has been fuzzified, the fuzzy rules are evaluated obtaining the set of fuzzy values for the hotel charac teristics, as well as its tag for defining the level of adequation based on the affect grid (Fig. 5). 4. The results obtained are defuzzified in order to obtain a set of concrete characteristics for the hotels. 5. With the defuzzified results, hotels with the obtained charac teristics are retrieved based on the hotels ontology and the annotations. 6. Next the fuzzy recommendation system will calculate the weight of each hotel based on the values of the suitability obtained from the decision matrix. The total weight can be configured in two modes, called ‘‘nor mal mode’’ and ‘‘sensible mode’’. The ‘‘normal’’ mode calculates the weight of the hotel as the summitry of all the suitabilities: Pproduct Xn i 1 Xm j 1 W ij; where Wij is the defuzzified value of the association between the customer characteristic i and the hotel characteristic j. The ‘‘sensible’’ mode calculates the weight of the hotel as the product of all the suitabilities. Pproduct Yn i 1 Ym j 1 W ij: The sensible mode allows greater differences between hotels, pro viding more differentiation between the recommendations. 7. The hotel which obtains the highest weight is the final recom mendation, called ‘‘Star recommendation’’. Also a set of alterna tive recommendations, called ‘‘suggested hotels’’ will be shown. These steps constitute the primary recommendation process. However, the expert criterion may not be adequate, because the customer tendencies can change. For this reason, the fuzzy rela tionships between the customer characteristics and the character istics of the hotels must be dynamically recalculated based on the customer feeling. Fig. 4. Recommendation process. Fig. 5. Affect grid for hotel characteristics. 3.4. Capturing the customer feeling As mentioned, the knowledge about the relationships between customer characteristics and hotel characteristics represented by means of the affect grid (Russell et al., 1989) has to be updated. There are two possibilities to update such values. On the one hand, the expert can update the values of the affect grid and, on the other hand, these values can be automatically updated by means of heu ristics. The first option is the classic approach in which an expert updates the knowledge after studying the customer tendencies. The second option allows the automatic adaption of the recom mender system based on the information acquired during the rec ommendation process. For this option it is necessary to take into account the level of agreement of the customer with the solution provided in order to make future recommendations. Such recom 5 mendations have an effect in two ways: on the one hand they al low the correction of the initial criterion of the expert because the negative valuation by the customers may imply the need to redefine the initial affect grid. On the other hand, the feedback of the customer may be stored in order to adjust the criterion of the expert to the customer preferences. For this reason, we propose the automatic reconfiguration of the fuzzy relationships using customer feedback. After receiving a rec ommendation the customer will rate his level of pleasure displea sure about the characteristics of the recommendation received. The objective of this question is to obtain the first impression of the customer about the recommendation. The fact that the rating of the customer is about his feeling about the recommendation in stead of his real experience is important. This is because the rec ommendation could be correct but the real experience of the customer could be negative, due to external factors (bad weather, personal problems, illness, etc.). The customer feelings are stored in the knowledge base. When the number of customer evaluations is greater than one hundred, the fuzzy system will evaluate the overall customer feelings based on fuzzy meta rules defined in order to determine when it will be necessary to update the affect grids. If the overall customer feeling is negative, then the affect grids related to the negative valued rec ommendations will be adjusted according to such meta rules. The updating process will be repeated every 100 customer valuations. 3.5. System architecture and implementation Fig. 6 depicts a three layer scheme that represents the recom mendation system architecture. The first layer corresponds to the user interface. There are two different elements in the user interface layer. On the one hand, the administrator GUI allows the administrator to define the fuzzy rules that represent knowledge about hotel recommen dation. The administrator GUI consists of a web based applica tion to allow the definition of the fuzzy sets as well as the membership function for each fuzzy set. This application also al lows the easy matching between customer characteristics and hotel characteristics by means of the affect grid. On the other hand, the GUI recommendation allows the customers to obtain intelligent recommendations about hotels. The GUI recommen dation consists of a web questionnaire in which the customer answers a set of questions proposed by the expert. Such ques tions determine the customer profile in order to determine the most suitable hotel. After the questions have been answered, the fuzzy engine evaluates the rules, and the most suitable hotel Fig. 6. System a 6 is shown to the customer. Optionally, a list of less suitable hotels can be displayed. After the recommendation, the customer pro vides his degree of pleasure displeasure about the recommenda tion received. This information will be used by the fuzzy engine for reconfiguring the fuzzy relationships between the customer characteristics and the hotel characteristics. The second layer represents the business logic. This layer con tains the fuzzy engine and the semantic engine. The fuzzy engine will evaluate the fuzzy rules defined by the expert in order to determine the most suitable hotel characteristics for a given cus tomer profile based on the affect grid. As mentioned, the fuzzy knowledge is defined by means of the GUI administrator. Once the hotel characteristics have been determined, the fuzzy engine retrieves the hotels with these characteristics by means of the semantic engine. The semantic engine, based on the Jena Frame work (Reynolds, 2006), will manage the hotels ontology and will provide the set of hotels with the characteristics determined by the fuzzy engine. Besides the recommendations, the fuzzy engine will recalculate the fuzzy affect grid based on the customer feed back. The fuzzy recalculator has been implemented as a daemon in the web server. Such a daemon monitors customer feedback, recalculates the fuzzy relationships between characteristics when the customer evaluation is significant and updates the affect grids according to such customer feedback. Finally, the persistence layer stores the knowledge about the hotel recommendation. As mentioned, on the one hand, the hotel ontology defines the relevant characteristics of each hotel. The concepts of the hotel ontology (Fig. 7) describe the category of each hotel based on stars, the room characteristics, the special equip ment of the hotel, etc. This ontology also describes special activi ties related to the hotel based on the season. All this information rchitecture. Fig. 7. Partial view of the hotels ontology. is used to describe the hotels available in the system. The hotels ontology has been defined using the Ontology Web Language (OWL) (Bechhofer et al., 2004). The OWL language has three vari ants: OWL Lite, OWL DL and OWL Full. OWL Lite provides a small set of features, while OWL DL is more expressive than OWL Lite providing decidability based on description logics. OWL Full allows full expressivity but decidability is not warranted. For this reason, we have used OWL DL for the ontology definition. The storage and ontology reasoning has been developed based on the Jena frame work. Besides the ontology specific storage, a database stores the information about the fuzzy sets and the fuzzy relationships with the ontology. On the other hand, the customer characteristics and their relationships with the hotel characteristics are repre sented by means of fuzzy sets and fuzzy relationships by means of affect grids and are stored in a database. Table 1 Comparison between system and expert recommendations. Precision Recall F1 Star recommendation vs. expert recommendation 0.58 0.58 0.58 Overall system suggestions vs. expert recommendation 0.19 0.96 0.32 4. Evaluation The subsequent section describes the empirical evaluation of the project. The final aim of this study is to work out if Sem Fit serves as a valid recommendation system in a controlled environment. 4.1. Research design Once the system has been developed and trained, it is necessary to prove the validity of our proposal; we must prove that the star recommendations provided by the system are accepted by the cus tomers. In case of disagreement with the star recommendation, it is of interest to know if the customer found a valid alternative in the list of suggested hotels. Additionally, given a set of customer profiles we have evaluated the similarity between the recommen dations provided by the fuzzy system and the recommendations provided by four experts in hotel recommendations. By means of this experiment we have evaluated the accuracy of the fuzzy sys tem comparing its recommendations to the recommendations of human experts. In case of disagreement between the expert crite ria and the fuzzy system’s star recommendation, it is of interest to know if the expert recommendation is included in the list of sug gested hotels. Finally, comparison between the expert recommen dation and the selection of the customer has been performed in order to validate the expert criterion. Precision, recall and F1 will be used in order to measure the de gree of relevance of the recommendations of the system. This tech nique has been employed based on two points of view: on the one hand, it is used to measure customer agreement with the recommendation received and, on the other hand, to measure the agreement between the system results and the expert recommen dations. As mentioned, this technique has been also employed to measure customer agreement with the expert’s recommendation. These measures had been used before to measure recommenda tions in semantic systems (e.g. García Crespo, Colomo Palacios, Gómez Berbís, & Ruiz Mezcua, 2010c, García Crespo, Rodríguez, Mencke, Gómez Berbís, & Colomo Palacios, 2010d, Paniagua Martín, García Crespo, Colomo Palacios, & Ruiz Mezcua, 2011). Precision, recall and F1 measures are defined as follows: 7 precision CorrectHotelsFound TotalHotelsFound ; recall CorrectHotelsFound TotalCorrectHotels ; F1 2 � precision � recall precision þ recall : The correct hotels will be determined by the expert criteria or the customer depending on the test. The following subsections describe the experimentation carried out and the results obtained. 4.2. Sample In order to determine the validity of the proposed approach, the designed evaluation was carried out. A set of 50 students (37 male and 13 female) in their final year of the Computer Science degree program at Universidad Carlos III de Madrid accessed the system in order to obtain a recommendation for their spring break. The system was trained with information on 10 hotels in Mallorca (Spain) based on the knowledge of an expert from a travel agency. With the results obtained from the system, each student selected the start preference or, in case of disagreement, one of the alterna tive recommendations provided or an empty alternative. Four experts in the travel domain recommend a hotel for each student in order to compare the recommendation of the system with the expert criteria. Recommendation generated by the fuzzy system was also compared with the expert recommendations in order to measure the validity of the star and alternative recommendations. 4.3. Results The experimentation was carried out in two separated tests that will be analyzed in the next subsections. 4.3.1. Test 1. Comparison between the system results and the expert recommendation Table 1 summarizes the precision and recall and F1 values ob tained by means of the comparison between the expert recommen dations and the results provided by the fuzzy system. This comparison has been divided into two stages. In the first stage, we have compared the star recommendation of the system with the expert recommendation. The star recommendation consists of a single recommendation (the top rated hotel for the customer characteristics) provided by the fuzzy system for each customer, and the expert recommendation is also a single recommendation suggested by the expert for each customer. The precision of the fuzzy system is 0.58. The value of recall is the same as precision be cause the number of returned categories is the same as the number of valid categories. We consider this value a promising result be cause more than 50% of the recommendations of the system are the same as the expert, and there is only one possible result. On the other hand, the fuzzy system provides a set of four alter natives for the star recommendation. We have compared the over all results of the system (five results, star recommendation and four suggested hotels) with the suggestion of the expert. In this scenario we have obtained a precision of 0.192, less than the 0.58 obtained for the star recommendation. It is logical conse quence because the number of returned values is greater (5 for each customer vs. only one star recommendation) and the number of correct values (recommendations of the expert) is the same. However the recall value achieves the value of 0.96. It means that in 96% of the cases the fuzzy system includes the recommendation of the expert in the set of results returned (either as star or sug gested recommendation). It is a good result because the system is capable of providing an expert recommendation in the set of suggestions in almost all of the cases. 4.3.2. Test 2. Comparison between the customer selection and the suggestions of both system and expert Test 1 compared the results of the system with the suggestions of the expert. But, are the expert suggestions a good basis? In order to answer this question, Test 2 compares the recommendations of the expert with the selection of the customer. Table 2 shows the precision, recall and F1 values obtained in this test. The precision value is 0.76, and the recall value is the same. It means that the 76% of expert recommendations are accepted by the customer. The second stage of the Test 2 consisted of the comparison be tween the system result and the final customer selection. In this stage we consider separately, on the one hand, only the star recom mendation and, on the other hand, the star recommendation plus the alternative suggestions. Precision and recall and F1 values for test 2 are presented in Table 3. The first row shows the comparison between the star recommendation and the customer selection. In this case, the precision value is 0.48. It means that the star recommendation of the system is the customer selection in the 48% of the cases. The value of recall is the same because the valid values are the same as the amount of values considered. The second row of Table 3 presents the comparison between the overall results of the system (star recommendation plus 4 sugges tions) and the final customer selection. As in Test 1, the precision value is smaller because the number of suggestions is greater, but the recall value is 0.96. It means that in 96% of the cases, the system offers a star recommendation or an alternative suggestion that satisfies the customer. 4.4. Discussion After the interaction of the customer with the system, the re sults were compared with the expert recommendation for each customer profile. We found that 24 customers found the star rec Table 2 Comparison between expert recommendation and customer selection. Precision Recall F1 Expert recommendation vs. customer selection 0.76 0.76 0.76 Table 3 Comparison between system recommendation and customer selection. Precision Recall F1 Star recommendation vs. customer selection 0.48 0.48 0.48 Overall system suggestions vs. customer selection 0.19 0.96 0.32 8 ommendation provided by the fuzzy system suitable and 48 cus tomers found a recommendation included in the star and suggested hotels suitable. Only two customers did not find a suit able recommendation. On the other hand, we found that 29 star recommendations coincide with the expert recommendation, and 48 expert recommendations were included in the sum of the star recommendations with the suggested recommendations. We can conclude that the sum of suggested and star recommendations provides an accurate set of recommendations in which the cus tomer can find a hotel. The system recommendations fit with the expert recommendation taking into account star and suggested ho tels. Finally the expert recommendation coincides with the cus tomer selection in 38 cases. However, the sum of star and suggested hotels of the fuzzy systems obtain better results than the expert recommendation. As shown in the previous subsection, the values of precision and recall and F1 have been calculated based on three points of view. Test 1 measured the similarity between the system recommenda tions and the expert recommendations. If we only take into ac count the star recommendation, both precision and recall values for the system recommendation in this scenario are 0.58. It means that 58% of the system star recommendations coincide with the ex pert recommendation. As mentioned, it is an acceptable margin be cause there is only one valid value (the expert recommendation which is highly subjective) and the system only offers one star rec ommendation. However, if we include in the study the four alter native suggestions of the system we can see that the precision decreases because the number of categories found is greater and the correct value is only one, but the recall value is 0.96. It is an excellent result because in 96% of the cases the system offers the expert recommendation. It means that the system recommenda tions are on the same level as an expert in the domain and the cus tomer can express his feelings in the same way that he does with the expert. Test 2 studies, on the one hand, the results of the expert vs. cus tomer selection. In this case, the precision and recall values ob tained are 0.76. We can see that the expert obtains better results with only one recommendation. This is natural, because the expert can take into account more parameters such as the corporal expression of the customer, non verbal behavior) in order to pro vide a recommendation. It is a good value, because the expert pro vides 3 good recommendations out of 4 recommendations. It also means that the knowledge represented in the fuzzy system is accu rate. On the other hand, Test 2 compared the star recommendation of the system with the customer selection. In this case, the preci sion value was 0.48. It means that the star recommendation is ac cepted by the customer in 48% of the cases. However, when we add the additional suggestions of the system, the precision is smaller because the number of suggestions is greater and the number of correct values is only one (the customer selection), but the recall is 0.96. It is an excellent result because it means that the customer finds a suitable recommendation in 96% of the cases. It is a high rate of positive recommendations considering that the selection of a hotel is a highly subjective decision. Cao and Li (2007) obtain an average recall of 83.82% for recommendations of 15 laptops in all 138 laptops for seven different customers; however in Cao and Li’s study the customer can select more than one recommen dation. Zenebe and Norcio (2009) present a comparative study of several techniques for recommending systems obtaining a recall of 38% for the fuzzy set theoretical method in a highly subjective domain such as movies selection. Zenebe and Norcio’s experiment includes a large number of customers and movies, and it is natural that the recall value was smaller. In our case of study the number of customers and products are smaller. Based on the obtained results, the proposed system is able to estimate customers’ feelings based on their profile and is capable of offering a set of recommendations that will be accepted most of the time. Future research may include a larger number of hotels and customers in order to measure the scalability of the proposed system. 5. Conclusions and future work We have presented Sem Fit, a semantic hotel recommendation expert system, based on consumers’ experience about the recom mendation provided by the system. The proposed expert system uses the consumers’ experience point of view in order to apply fuz zy logic techniques to relate customers and hotels characteristics, represented by means of domain ontologies and affect grids. After receiving a recommendation, the customer provides a valuation about the recommendation generated by the system. Based on the customers’ valuations, the rules of the system are updated in order to adjust the new recommendations to the past user experi ences. The validation accomplished shows that the sum of star and suggested hotels of the fuzzy systems obtains better results than the expert recommendation. Moreover, the values of precision and recall and F1 reveal that the Sem Fit recommendations are on the same level as an expert in the domain and the customer will be able to express his feelings in the same way that he does with the expert. As an extension of this paper, the knowledge base could be im proved including more products, such as suitable destinies or asso ciated services. Furthermore, other ways of collecting data on customer feeling about a recommendation with greater accuracy and clarity could be studied. Acknowledgements This work is supported by the Spanish Ministry of Industry, Tourism, and Commerce under the EUREKA project SITIO (TSI 020400 2009 148), SONAR2 (TSI 020100 2008 665) and GO2 (TSI 020400 2009 127). References Aciar, S. V., Serarols-Tarres, C., Royo-Vela, M., & De la Rosa i Esteva, J. L. (2007). Increasing effectiveness in e-commerce: Recommendations applying intelligent agents. International Journal of Business and Systems Research, 1(1), 81–97. Acosta, Z., & Febles, J. (2010). The organizacional management as instrument to overcome the resistance to the innovative process: An application in the canary company. International Journal of Human Capital and Information Technology Professionals, 1(2), 49–64. Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734–749. Bechhofer, S., van Harmelen, F., Hendler, J., Horrocks, I., McGuinness, D. L., Patel- Schneider, P. F., et al. (2004). OWLWeb ontology language reference. http:// www.w3.org/TR/owl-ref/. Benítez, J. M., Martín, J. C., & Román, C. (2007). Using fuzzy number for measuring quality of service in the hotel industry. Tourism Management, 28(2), 544–555. Buhalis, D. (1998). Strategic use of information technologies in the tourism industry. Tourism Management, 19(5), 409–421. Buhalis, D. (2003). eTourism: Information technology for strategic tourism management. London: Pearson. Buhalis, D., & Law, R. (2008). Progress in information technology and tourism management: 20 years on and 10 years after the Internet—The state of eTourism research. Tourism Management, 29(4), 609–623. Cao, Q., & Schniederjans, M. J. (2006). Agent-mediated architecture for reputation- based electronic tourism systems: A neural network approach. Information & Management, 43(5), 598–606. Cao, Y., & Li, Y. (2007). An intelligent fuzzy-based recommendation system for consumer electronic products. Expert Systems with Applications, 33(1), 230–240. Castillo, L., Armengol, E., Onaindía, E., Sebastiá, L., González-Boticario, J., Rodríguez, A., et al. (2008). SAMAP: An user-oriented adaptive system for planning tourist visits. Expert Systems with Applications, 34(2), 1318–1332. Chiu, D. K. W., Yueh, Y. T. F., Leung, H. F., & Hung, P. C. K. (2009). Towards ubiquitous tourist service coordination and process integration: A collaborative travel agent system architecture with semantic web services. Information Systems Frontiers, 11(3), 241–256. 9 Chou, T. Y., Hsu, C. L., & Chen, M. C. (2008). A fuzzy multi-decision model for international tourist hotels location selection. International Journal of Hospitality Management, 27(2), 293–301. Crouch, G. I. (1991). Expert computer systems in tourism: Emerging possibilities. Journal of Travel Research, 29(3), 3–10. Dogac, A., Kabak, Y., Laleci, G., Sinir, S., Yildiz, A., Kirbas, S., et al. (2004). Semantically enriched Web services for the travel industry. ACM Sigmod Record, 33(3), 21–27. Engel, J. F., Blackwell, R. D., & Miniard, P. W. (1995). Consumer behavior. Forth Worth, TX: The Dryden Press. Fodor, O., & Werthner, H. (2004). Harmonise: A step toward an interoperable e- tourism marketplace. International Journal of Electronic Commerce, 9(2), 11–39. García-Crespo, A., Chamizo, J., Rivera, I., Mencke, M., Colomo-Palacios, R., & Gómez- Berbís, J. M. (2009). SPETA: Social pervasive e-Tourism advisor. Telematics and Informatics, 26(3), 306–315. García-Crespo, A., Colomo-Palacios, R., Gómez-Berbís, J. M., Chamizo, J., & Rivera, I. (2010a). Intelligent decision-support systems for e-tourism: Using SPETA II as a knowledge management platform for DMOs and e-tourism service providers. International Journal of Decision Support System Technology, 2(1), 35–47. García-Crespo, A., Colomo-Palacios, R., Gómez-Berbís, J. M., & García-Sánchez, F. (2010b). SOLAR: Social link advanced recommendation system. Future Generation Computer Systems, 26(3), 374–380. García-Crespo, A., Colomo-Palacios, R., Gómez-Berbís, J. M., & Ruiz-Mezcua, B. (2010c). SEMO: A framework for customer social networks analysis based on semantics. Journal of Information Technology, 25(2), 178–188. García-Crespo, A., Rodríguez, A., Mencke, M., Gómez-Berbís, J. M., & Colomo- Palacios, R. (2010d). ODDIN: Ontology-driven differential diagnosis based on logical inference and probabilistic refinements. Expert Systems with Applications, 37(3), 2621–2628. Goosen, M., Meeuwsen, H., Franke, J., & Kuyper, M. (2009). My Ideal tourism destination: Personalized destination recommendation system combining individual preferences and GIS data. Information Technology & Tourism, 11(1), 17–30. Han, K. H., & Park, J. W. (2009). Process-centered knowledge model and enterprise ontology for the development of knowledge management system. Expert Systems with Applications, 36(4), 7441–7447. Huang, Y., & Bian, L. (2009). A Bayesian network and analytic hierarchy process based personalized recommendations for tourist attractions over the Internet. Expert Systems with Applications, 36(1), 933–943. Jakkilinki, R., Georgievski, M., & Sharda, N. (2007). Connecting destinations with an ontology-based e-tourism planner. In M. Sigala, L. Mich, & J. Murphy (Eds.), Information and communication technologies in tourism (pp. 21–32). Berlin, Germany: Springer Wien. Jiang, Y., Shang, J., & Liu, Y. (2009). Maximizing customer satisfaction through an online recommendation system: A novel associative classification model. Decision Support Systems, 48(3), 470–479. Kanellopoulos, D. N. (2008). An ontology-based system for intelligent matching of travellers’ needs for group package tours. International Journal of Digital Culture and Electronic Tourism, 1(1), 76–99. Kenteris, M., Gavalas, D., & Economou, D. (2009). An innovative mobile electronic tourist guide application. Personal and Ubiquitous Computing, 13(2), 103–118. Klein, L. R. (1998). Evaluating the potential of interactive media through a new lens: search versus experience goods. Journal of Business Research, 41(3), 195–203. Kuo, M. H., Chen, L. C., & Liang, C. W. (2009). Building and evaluating a location- based service recommendation system with a preference adjustment mechanism. Expert Systems with Applications, 36(2 Part 2), 3543–3554. Lee, C. S., Chang, Y. C., & Wang, M. H. (2009). Ontological recommendation multi- agent for Tainan City travel. Expert Systems with Applications, 36(1), 6740–6753. Lenar, M., & Sobecki, J. (2007). Using recommendation to improve negotiations in agent-based systems. Journal of Universal Computer Science, 13(2), 267–286. Liang, T. P., Lai, H. J., & Ku, Y. C. (2006). Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings. Journal of Management Information Systems, 23(3), 45–70. Loh, S., Lorenzi, F., Saldana, R., & Licthnow, D. (2004). A tourism recommender system based on collaboration and text analysis. Information Technology & Tourism, 6(3), 157–165. Longhi, C. (2007). Usages of the Internet and e-tourism. Towards a new economy of tourism. In Proceedings of the second international conference advances in tourism economics, Portugal. Luo, M., Feng, R., & Cai, L. A. (2004). Information search behavior and tourist characteristics: The internet vis-à-vis other information sources. Journal of Travel & Tourism Marketing, 17(2-3), 15–25. Ngai, E. W. T., & Wat, F. K. T. (2003). Design and development of a fuzzy expert system for hotel selection. Omega, 31(4), 275–286. Niemann, M., Mochol, M., & Tolksdorf, R. (2008). Enhancing hotel search with semantic Web technologies. Journal of Theoretical and Applied Electronic Commerce Research, 3(2), 82–96. Niininen, O., Buhalis, D., & March, R. (2007). Customer empowerment in tourism through consumer centric marketing (CCM). Qualitative Market Research: An International Journal, 10(3), 265–281. Paniagua-Martín, F., García-Crespo, Á., Colomo-Palacios, R., & Ruiz-Mezcua, B. (2011). SSAAMAR: Semantic annotation architecture for accessible multimedia resources. IEEE Multimedia, 18(2), 16–25. Reynolds, D. (2006). Jena rules. In Proceedings of the 2006 Jena user conference. Ricci, F. (2002). Travel recommender systems. IEEE Intelligent Systems, 17(6), 55–57. http://www.w3.org/TR/owl-ref/ http://www.w3.org/TR/owl-ref/ Ricci, F., & Nguyen, Q. N. (2007). Acquiring and revising preferences in a critique- based mobile recommender system. IEEE Intelligent Systems, 22(3), 22–29. Ricci, F., & Werthner, H. (2006). Recommender systems. International Journal of Electronic Commerce, 11(2), 5–9. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). The affect grid: A single-item scale of pleasure and arousal. Personality and Social Psychology, 57(3), 493–502. Schiaffino, S., & Amandi, A. (2009). Building an expert travel agent as a software agent. Expert Systems with Applications, 36(2), 1291–1299. Wallace, M., Maglogiannis, I., Karpouzis, K., Kormentzas, G., & Kollias, S. (2003). Intelligent one-stop-shop travel recommendations using an adaptive neural network and clustering of history. Information Technology & Tourism, 6, 181–193. Wang, J. (2008). Improving decision-making practices through information filtering. International Journal of Information and Decision Sciences, 1(1), 1–4. 10 Wang, Y., Yu, Q., & Fesenmaier, R. D. (2002). Defining the virtual tourist community: Implications for tourism marketing. Tourism Management, 23(4), 407–417. Watson, R. T., Akselsen, S., Monod, E., & Pitt, L. F. (2004). The open tourism consortium: laying the foundations for the future of tourism. European Management Journal, 22(3), 315–326. Werthner, H., & Ricci, F. (2004). E-Commerce and tourism. Communications of the ACM, 47(12), 101–105. World Tourism Organization (2006). Tourism market trends 2005: World overview & tourism topics. Madrid, Spain: World Tourism Organization. Xian, Z., Kim, S. E., Hu, C., & Fesenmaier, D. R. (2007). Language representation of restaurants: Implications for developing online recommender systems. Hospitality Management, 26(4), 1005–1018. Zadeh, L. (1965). Fuzzy sets. Information & Control, 8(3), 338–353. Zenebe, A., & Norcio, A. F. (2009). Representation, similarity measures and aggregation methods using fuzzy sets for content-based recommender systems. Fuzzy Sets and Systems, 160(1), 76–94.