Microsoft Word - kioskpaper.doc http://pro.sagepub.com/ Ergonomics Society Annual Meeting Proceedings of the Human Factors and http://pro.sagepub.com/content/51/5/484 The online version of this article can be found at: DOI: 10.1177/154193120705100512 2007 51: 484Proceedings of the Human Factors and Ergonomics Society Annual Meeting Jacob Solomon and Frank A. Drews Digital Photo Kiosk Evaluation Published by: http://www.sagepublications.com On behalf of: Human Factors and Ergonomics Society can be found at:Proceedings of the Human Factors and Ergonomics Society Annual MeetingAdditional services and information for http://pro.sagepub.com/cgi/alertsEmail Alerts: http://pro.sagepub.com/subscriptionsSubscriptions: http://www.sagepub.com/journalsReprints.navReprints: http://www.sagepub.com/journalsPermissions.navPermissions: http://pro.sagepub.com/content/51/5/484.refs.htmlCitations: What is This? - Oct 1, 2007Version of Record >> at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from at MICHIGAN STATE UNIV on September 21, 2013pro.sagepub.comDownloaded from http://pro.sagepub.com/ http://pro.sagepub.com/content/51/5/484 http://www.sagepublications.com http://www.hfes.org http://pro.sagepub.com/cgi/alerts http://pro.sagepub.com/subscriptions http://www.sagepub.com/journalsReprints.nav http://www.sagepub.com/journalsPermissions.nav http://pro.sagepub.com/content/51/5/484.refs.html http://pro.sagepub.com/content/51/5/484.full.pdf http://online.sagepub.com/site/sphelp/vorhelp.xhtml http://pro.sagepub.com/ http://pro.sagepub.com/ http://pro.sagepub.com/ http://pro.sagepub.com/ http://pro.sagepub.com/ http://pro.sagepub.com/ Digital Photo Kiosk Evaluation Jacob Solomon and Frank A. Drews University of Utah Salt Lake City, Utah Self-service modules have become an integral part of the economy throughout the world, replacing expensive human operators in many set- tings. However, usability issues continue to diminish the economic value of these modules. This experiment demonstrates how the application of sound usability principles can be applied to self-service settings to in- crease the usability of self-service modules. The study compared the us- ability of two versions of a self-service digital photo kiosk. In one version we replicated a kiosk presently in use and broadly available. The other version of the software incorporated several design principles, such as the use of a metaphor, intended to increase usability and learnability, more specifically to allow for easier navigation. Participant’s performance in completing tasks was measured as a function of speed, accuracy, and the need for human assistance. The results demonstrate that incorporating usability principles can improve usability of self-service modules. INTRODUCTION Self-service kiosks are an increasingly im- portant mechanism in the modern business world. Self-service devices are currently being used to re- trieve money from a bank account, to check out items from a grocery store, to rent movies, to buy event tickets, and many other uses. From a business point of view self-service allows for bypassing the need for expensive and potentially unreliable human employees. However, there is undoubtedly a con- cern that perhaps a new cost is created by the use of self-service. That cost is related to the issue of us- ability. If self-service machines have poor usability then they are economically unfeasible because of their low acceptance rates by customers. In fact, some machines have such poor usability that they require a human attendant to aid users. In such a circumstance, the technology has failed, because both the cost of the machine, the cost of the em- ployee, and the loss of business due to customer dissatisfaction and frustration must be considered. Digital photo kiosks are a particularly con- spicuous example of the problems which exist in self-service modules. As the photo industry transi- tions from film photography to digital photography, these kiosks are a vital means for photo labs to maintain the profitability of their expensive but high quality photo printing equipment. They are also meant to promote one of the fundamental advan- tages of digital photography, which is that one can take hundreds of pictures and view them before printing, allowing more control over what gets printed and the quality of prints. However, the com- plexity of digital photo editing and organization makes its application to a self-service setting chal- lenging for software developers. Self-service mod- ules have traditionally been reserved for more sim- ple tasks, as is the case with ATM’s, snack ma- chines, and gas pumps. Because the task involved in digital photo kiosks is much more complex than traditional self-service modules, there is danger that the mental models users take from more established and simpler applications can not be adequately ap- plied, with the consequence of poor usability for the customer. Unfortunately, many current photo kiosks use traditional self-service modules as a model for their own design. One example of a self service module that illustrated these problems is the Fuji Aladdin Digital Photo Kiosk. The Aladdin follows a “step-by-step” model of navigation. From the standpoint of a software engineer, this software is very efficient. Photos are loaded from a storage me- dium and processed quickly at the cost of allowing the user only very limited freedom to perform com- plicated or unusual tasks. However, the software PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 51st ANNUAL MEETING—2007 484 does not provide the user the vital information needed to confidently navigate through the applica- tion. This can by highlighted by the complete lack of information about previous and potential future steps in the procedure. Although the user is figura- tively put on a “track” from which they cannot de- viate, it is not quite clear which options will be ahead. This lack of preview leads to frequent mis- takes and requests for assistance from a human su- pervisor. The goal of the present study is to demon- strate how human factors design principles can be applied to a digital photo kiosk in order to compen- sate for the complexity of the task and maintain strong usability. Participants performed tasks on two simulated versions of a kiosk. One version was modeled directly after the Fuji Aladdin Digital Photo Center. The other version used a similar, step by step structure, however it also provided the nec- essary information needed to navigate successfully, and presented this information in a way consistent with good human factors design principles. The necessary information was provided up front, which is crucial to good interaction (Drews & Westen- skow, 2006). Finally, a simple metaphor was also used to help users apply a mental model to the task. Thus, we followed Davidson, Dove and Weltz (1998) recommendation that “Usability is strongly tied to the extent to which a user's mental model matches and predicts the action of a system.” To provide users with a familiar mental model we ap- plied the metaphor of a photo album to provide guidance. We predicted that participants using the redesigned software would complete their task more quickly and with fewer errors than in the control condition (Aladdin system). Finally, we hypothe- sized that the redesigned software would have greater learnability due to the application of these principles. METHODS The study took place at the University of Utah. 22 male and 19 female undergraduate stu- dents were randomly assigned to either the control condition, in which they used the kiosk modeled after the Aladdin, or experimental condition, where they used the redesigned kiosk. Two tasks had to be completed by the participants. All participants were instructed to complete the same two tasks, regard- less of experimental condition. The first and simple task consisted of ordering one 4x6 inch print of all eleven available pictures. After completion, partici- pants quit the kiosk and completed the NASA Task Load Index Survey. Next, participants returned to the kiosk to perform the second and more compli- cated, but still very common task. This task in- volved ordering a 4x6 inch print of five pictures containing athletic shoes, and one 8x10 inch print of the picture containing a horseshoe. The software would give the user an error message any time they attempted to do anything which would misdirect them from the path to com- pleting the specific task. Information about the number of error messages and time required for successful completion was collected. Also, partici- pants were given the option of asking for assistance from the experimenter in completing their task. The experimenter would record the number of requests for assistance. The design of the study was a 2 by 2 design with task difficulty (simple and difficult) as a re- peated subjects factors and version of the kiosk (Aladdin and redesigned version) as a between sub- jects factor. Because we were interested in the im- pact of learning we did structure the tasks in a fixed sequence, where the difficult task always followed the simpler task. Thus, the effectiveness of the re- designed version will be tested focusing on the in- teraction between task difficulty and version of the kiosk. As dependent variables, we used the time for successful completion of the task, the number of errors participants made before completion, and the number of requests for assistance. RESULTS Task Performance Time. The analyses revealed that there was no main effect of experimental condition. Thus, overall participants in both conditions did not differ in terms of their completion time for both tasks. However, consistent with our hypotheses we found a significant interaction effect between kiosk ver- sion and task difficulty (F(1,39) = 6.88; p < .05) in- dicating that participants benefited significantly PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 51st ANNUAL MEETING—2007 485 from the redesign when dealing with the second more difficult task. 143.38 172 214.1 124.25 0 50 100 150 200 250 Aladdin Model Redesigned Model M ea n T im e (s ) Simple Task Difficult Tas k Figure 1. Mean time Errors. The analysis revealed no significant interaction between task difficulty and kiosk ver- sion. However, the main effect of kiosk version re- vealed a trend (F(1,39) = 3.068; p < .1) indicating that overall, users made fewer errors on the redes- igned kiosk. 6.86 3.5 12.24 3.75 0 2 4 6 8 10 12 14 Aladdin Model Redes igned Model M ea n E rr o rs Simple Task Difficult Tas k Figure 2. Mean errors Help. The last analysis focused on the num- ber of requests for assistance made by participants. A significant main effect of kiosk version was found (F(1,39) = 5.673; p < .05). And consistent with the hypothesis, a significant interaction (F(1,39) = 9.066; p < .05) was also found between task difficulty and kiosk version. Users in general required less assistance on the redesigned kiosk, particularly on the difficult second task. 0.05 0.1 0.48 0.05 0 0.1 0.2 0.3 0.4 0.5 0.6 Aladdin Model Redesigned Model M ea n H el p Simple Task Difficult Tas k Figure 3. Mean errors NASA Task Load Index Survey No significant results were found from the self-reported NASA Task Load Index Survey on any of the six categories (mental workload, physical workload, temporal demand, effort, performance, frustration). DISCUSSION The most important conclusion that can be drawn from the data is that the redesigned kiosk clearly had greater learnability. Users of the redes- igned kiosk improved their performance on the sec- ond more difficult task. Performance decreased on the difficult task by users of the Aladdin kiosk. In particular, users moved much more quickly through the more complicated task after having used the software to perform a simpler task. It appears that the design principles incorporated into the new ki- osk design made the use and the navigation easier for users. Table 1 explains the fundamental differ- ences between the two kiosk designs. PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 51st ANNUAL MEETING—2007 486 Aladdin Kiosk Redesigned Kiosk 3 step structure- Edit, order, view order summary 3 step structure- Edit, add to album, view album No instructions Instructions, at begining and on each page No metaphor Photo album metaphor Navigation explained by vague button labels Navigation explained by precise button labels Table 1 Ordering photos from the kiosk is a three step process in both kiosk models. The first step is to prepare final images by cropping, adjusting color and fixing red eye. The next step is to select images to be printed in the desired size or sizes. The third step is to review the work and ensure that the order is satisfactory. If it is not, users can return to either of the other two steps. The Aladdin kiosk model does not explain this three step process; users are expected to figure it out as they go. Not only is this three step process not made explicit for the user, but buttons which guide users between steps are vague. Labeling a button which moves the user into the next step as “okay” is not effective when the user has no idea what the next step is (see figure 4). Figure 4. Opening screen in Aladdin kiosk model A logical solution to this lack of preview would be an instruction screen which lists the steps and describes the entire process. This was provided as the first screen in the redesigned kiosk. However, it was believed that merely a set of instructions, provided only once at the beginning, would not be sufficient, and that after a few minutes users would not remember everything from the instructions, and that often they would not read them carefully or in entirety. The photo album metaphor was intended to help users build / apply a mental model of this three step process, so that they would not only navigate the software but actually learn its structure, improv- ing future performance. In assembling a real photo album, one would prepare final versions of prints, assign prints to various albums or pages based on size, then view the album to ensure its completion. This is essentially the same three step process used by the Aladdin kiosk model. However, the applica- tion of a metaphor in addition to written instructions at the beginning of the process and on each subse- quent page allows users to more easily understand the order of subtasks (see also Drews et al, 2007). This is the mental model which the metaphor hopes to create. Users working with an effective mental model of the software’s structure can more success- fully operate the kiosk. The results of this study demonstrate that users, after completing a simple task, performed a more difficult task with greater ease than the original simple task. Performance of the second task on the Aladdin kiosk decreased. This demonstrates that the metaphor and instruc- tions provided in the redesigned kiosk led to signifi- cant improvements in learnability. The photo album metaphor was intended to replicate the success of the “desktop” metaphor in computers and the “shopping cart” metaphor common in online shop- ping. These metaphors describe an application’s virtual storage space, where users can set aside files or information for quick retrieval at a later point. This sense of virtual storage is crucial in a compli- cated task such as photo editing and printing, as us- ers typically work with more images than they can easily remember or will fit on one page. The virtual photo album of the redesigned kiosk is no different than the “order summary” of the Aladdin kiosk. But not only does the Aladdin kiosk not inform users that an order summary may be viewed until the end of the process, but the term “order summary” does not convey the same sense of location that a “photo album” conveys. We believe this can account for the decreased performance on the difficult task in the Aladdin kiosk, which requires users to pass through all three steps twice because two print sizes are required. Having presumably learned from per- PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 51st ANNUAL MEETING—2007 487 forming the simple task that the “order summary” is the last step, users can easily be confused when they cannot order the 8x10 without passing through the order summary and restarting the entire process. We believe the metaphor allowed users to feel confident about navigating between the subtasks, without re- stricting them to a specific order. In other words, we believe the metaphor made the kiosk more explor- able, which is crucial to good interaction (Tognazz- ini, 2007). This also allows the redesigned kiosk to have a more sophisticated and complex workspace than the Aladdin model. In the Aladdin model, every new option is accompanied by a new screen, presumably to simplify decision making and steer users to the correct locations. But since the meta- phor provides an identifiable location and structure for the users to work from, this is not necessary in the redesigned model. Instead, users can perform multiple functions from the same workspace, since they know where their work is going and how to review it. The kiosk in essence becomes more like a personal computer. Figure 5 Ordering prints on redesigned kiosk The most important conclusion that can be drawn from this study is that kiosk design must take into account task complexity in order to maintain usability. The controlled structure found in the Aladdin kiosk model may be efficient for helping users complete a simple task, but a complex task requires more information and freedom than is pro- vided. The design of self-service modules for com- plicated tasks should look at usability principles found in more complex software for personal com- puters. One of these principles, a metaphor, was ap- plied with great success in this study. Learnability is crucial to the success of self-service. If self-service intends to apply itself in more complex domains, the unavoidable consequence is that users will have to invest greater effort in learning these modules. However, this study shows that good design princi- ples can significantly increase a kiosk’s learnability. Users of both kiosks performed the simple task with the similar speed. But the data indicates that the time spent by users of the Aladdin kiosk wandering through the application by trial and error was spent learning the software on the redesigned kiosk. The difficult task was then performed quickly and eas- ily, while the on the Aladdin kiosk it was signifi- cantly more difficult. If designers of self-service modules will incorporate design principles from personal computer software design, which is al- ready experienced at designing interactive modules for complicated tasks, task complexity can be minimized and usability increased. References Davidson, M.J., Dove, L., and Weltz, J. (1999) Mental models and usability. Cognitive Psy- chology at DePaul University. Retrieved Feb- ruary 9, 2007, from http://www.lauradove.info/reports/mental%20 models.htm Drews, F.A., Picciano, P., Agutter, J., Syroid, N., Westenskow, D.R. & Strayer, D.L. (2007). Development and evaluation of a just-in- time support system. Human Factors, 49, 543-551. Drews, F.A., & Westenskow, D.R. (2006). Human-Computer Interaction in Healthcare. In: P. Carayon (Ed.) Handbook of Human Factors and Ergonomics in Healthcare and Pa- tient Safety. Lawrence Erlbaum Associates. Preece, J., Rogers, Y., and Sharp, H. (2002). Interaction Design: Beyond Human Computer Interaction. New York: John Wiley & Sons Tognazzini, B. (2007). First principles of interaction design. Retrieved February 9, 2007 from http://www.asktog.com/basics/firstPrinciples.h tml PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 51st ANNUAL MEETING—2007 488