Performance, workload, and usability in a multiscreen, multi-device, information-rich environment A peer-reviewed version of this preprint was published in PeerJ on 10 September 2018. View the peer-reviewed version (peerj.com/articles/cs-162), which is the preferred citable publication unless you specifically need to cite this preprint. Saleem JJ, Weiler DT. 2018. Performance, workload, and usability in a multiscreen, multi-device, information-rich environment. PeerJ Computer Science 4:e162 https://doi.org/10.7717/peerj-cs.162 https://doi.org/10.7717/peerj-cs.162 https://doi.org/10.7717/peerj-cs.162 Performance, workload, and usability in a multiscreen, multi- device, information-rich environment Jason J Saleem Corresp., 1, 2 , Dustin T Weiler 1, 3 1 Department of Industrial Engineering, University of Louisville, Louisville, Kentucky, United States 2 Center for Ergonomics, University of Louisville, Louisville, Kentucky, United States 3 Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, Wisconsin, United States Corresponding Author: Jason J Saleem Email address: jason.saleem@louisville.edu Potential benefits of multiscreen and multiple device environments were assessed using three different computing environments. A single factor, within-subject study was conducted with 18 engineering students in a laboratory experiment. Three levels for the computing environment factor included one with a desktop computer with a single monitor (control, condition A); one with a desktop with dual monitors, as well as a single tablet computer (condition B); and one with a desktop with a single monitor, as well as two tablet computers (condition C). There was no statistically significant difference in efficiency or workload when completing scenarios for the three computing environments. However, a dual monitor desktop with a single tablet computer (B) was the ideal computing environment for the information-rich engineering problem given to participants, supported by significantly fewer errors compared to condition C and significantly higher usability ratings compared to conditions A and C. A single desktop monitor with two tablet computers (C) did not provide any advantage compared to a single desktop monitor (A). PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.26826v1 | CC BY 4.0 Open Access | rec: 4 Apr 2018, publ: 4 Apr 2018 1 Performance, Workload, and Usability in a Multiscreen, Multi-device, Information-rich 2 Environment 3 4 Jason J. Saleem1,2 * and Dustin T. Weiler1,3 5 6 1Department of Industrial Engineering, J.B. Speed School of Engineering, University of 7 Louisville, Louisville, Kentucky, USA 8 2Center for Ergonomics, University of Louisville, Louisville, KY, USA 9 3Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, 10 Wisconsin, USA 11 12 13 Corresponding Author: 14 Jason J. Saleem, PhD, Department of Industrial Engineering, J.B. Speed School of Engineering, 15 University of Louisville, Louisville, KY, USA, 40292, Tel: + 1 502 852 2274, Fax: + 1 502 852 16 5633, Email: jason.saleem@louisville.edu 17 18 19 20 21 22 23 mailto:jason.saleem@louisville.edu 24 Abstract 25 Potential benefits of multiscreen and multiple device environments were assessed using 26 three different computing environments. A single factor, within-subject study was conducted 27 with 18 engineering students in a laboratory experiment. Three levels for the computing 28 environment factor included one with a desktop computer with a single monitor (control, 29 condition A); one with a desktop with dual monitors, as well as a single tablet computer 30 (condition B); and one with a desktop with a single monitor, as well as two tablet computers 31 (condition C). There was no statistically significant difference in efficiency or workload when 32 completing scenarios for the three computing environments. However, a dual monitor desktop 33 with a single tablet computer (B) was the ideal computing environment for the information-rich 34 engineering problem given to participants, supported by significantly fewer errors compared to 35 condition C and significantly higher usability ratings compared to conditions A and C. A single 36 desktop monitor with two tablet computers (C) did not provide any advantage compared to a 37 single desktop monitor (A). 38 39 40 41 42 43 44 45 46 47 1. Introduction 48 As having more than one computing device and/or monitors is becoming more feasible 49 for individuals, a future trend is the of adoption of a multiscreen and multiple device approach to 50 cope with distractions and multiple tasks. Although this may seem counterintuitive, more 51 screens and possibly more devices may help focus one’s attention rather than serve as a 52 distraction, making multiple tasks viewable at a glance across multiple device screens 53 (Thompson, 2014). Assuming each device has a different primary purpose, the additional 54 screens may begin to approximate some of the inherent affordances of paper. That is, spreading 55 out papers on a desk lets one’s eyes easily scan, which is a property hard to replicate on a single 56 computer screen. Thus, coordination of multiple computing devices and screens is a strategy that 57 may potentially improve one’s performance in an information-rich environment by focusing their 58 attention and reducing their mental workload. Combining multiple screens and information 59 devices has recently been studied qualitatively, in the field (Jokela, Ojala, & Olsson, 2015). 60 However, little quantitative experimentation has been done as to how a multi-device setup might 61 affect task performance, which is the main objective of this study. 62 The study described in this paper is a natural evolution of a previous study that involved 63 paper-based workarounds to using the electronic health record (EHR) (Saleem et al., 2009). In 64 this study, we found that paper served as an important tool and assisted healthcare employees in 65 their work. In other cases, paper use circumvented the intended EHR design, introduced potential 66 gaps in documentation, and generated possible paths to medical error. Investigating these paper 67 processes helped us understand how the current exam room computing and EHR were not 68 meeting the needs of the clinicians. The “forgotten” power of paper, including its ability to serve 69 as a reliable cognitive memory aid and to help focus attention on important information, were 70 lost as EHRs began to take shape. Today, a multiscreen and multiple device work environment 71 is becoming a trend. How to optimize the use and coordination of these multiple screens and 72 devices is not known. This type of environment may help simulate the forgotten power of paper 73 by replicating many of the lost affordances of paper-based processes, such as easy visual 74 attention switches across screens, as well as the display of the most important information, 75 separated by function or purpose across screens and devices. The objective of our study was to 76 understand how to optimize this type of multiscreen and multiple device environment for 77 improved user performance and satisfaction, and reduced mental workload. 78 There exists a large body of human-computer interaction (HCI) literature on the use of 79 multiple screens, screen sizes, and form factors (e.g., desktop, tablet, smartphone). Previous 80 studies in academic (Anderson, Colvin, Tobler, & Lindsay, 2004; Russell & Wong, 2005) and 81 hospital (Poder, Godbout, & Bellemare, 2011) settings have demonstrated that performance is 82 improved with the use of two monitors compared to one. For example, participants were quicker 83 on tasks, did the work faster, and performed more work with fewer errors in multiscreen (dual 84 screen) configurations than with a single screen (Anderson et al., 2004). Another study 85 demonstrated that users do not tend to treat a second monitor as additional space. That is, 86 participants reported rarely straddling a single window across two monitors. This is consistent 87 with the physical gaps that are often left between monitors. Instead, users typically maximize a 88 design to fill one monitor entirely, leaving the other monitor free for other uses (Grudin, 2001). 89 The visual and physical separation between displays requires that users perform visual attention 90 switches between displays (Rashid, Nacenta, & Quigley, 2012). In one study, the authors 91 utilized a divided attention paradigm to explore the effects of visual separation and physical 92 discontinuities when distributing information across multiple displays. Results showed reliable 93 detrimental effects (about a 10% performance decrement) when information is separated within 94 the visual field, but only when coupled with an offset in depth (Tan & Czerwinski, 2003). 95 The optimal monitor size and position has also been studied. One study compared 15-, 96 17-, 19-, and 21-inch monitors and found that while participants’ performance was most efficient 97 with the 21-inch monitor for Excel and Word tasks, users significantly preferred the 19-inch 98 monitor (Simmons, 2001). The majority (65%) of participants noted that the 21-inch monitor 99 was too large or bulky for the average workspace (Simmons & Manahan, 1999; Simmons, 2001). 100 A limitation of this study was that screen resolution was not controlled for across the four screen 101 sizes. Although there has also been experimentation with very large displays (e.g., 42-inch 102 monitor), there are several usability issues that are barriers to adopting larger displays, including: 103 losing track of the cursor, distal access to information, window management problems (e.g., 104 windows pop up in unexpected places), task management problems, configuration problems, and 105 failure to leverage the periphery (Czerwinski et al., 2006). Therefore, separate smaller displays 106 (e.g., 19-inch) seems to be advantageous as compared to a single, very large display. In terms of 107 user-preferred position of computer monitors, one study found that participants placed larger 108 displays farther and lower while maintaining the display top at or near eye height (Shin & 109 Hegde, 2010). Preferred position of the dual displays in landscape arrangement did not differ 110 from that of a single display. Therefore, it appears that the preferred display position varies with 111 the vertical dimension of the overall viewable area of the display (Shin & Hegde, 2010). 112 In addition to multiple monitors, handheld computers such as tablets and smartphones are 113 becoming much more accessible in the workplace. For example, in clinical care settings, one 114 research team noted that by making the most useful and appropriate data available on multiple 115 devices and by facilitating the visual attention switching between those devices, staff members 116 can efficiently integrate them in their workflow, allowing for faster and more accurate decisions 117 (De Backere F. et al., 2015). Research on the performance differences with the form factor of 118 handheld computers revealed a significant difference in completion times between the tablet and 119 smart phone screen sizes (17.8 vs. 7.1 cm), but no differences in errors or subjectively assessed 120 cognitive workload (Byrd & Caldwell, 2011). These previous studies were useful for 121 understanding how to blend a multiple monitor environment with additional devices, such as 122 tablet computers, for creating multiscreen environments to compare in our study. 123 124 2. Methods 125 2.1 Study Design 126 This research was approved by the Institutional Review Board (IRB) at the University of 127 Louisville (IRB # 16.0025). Informed consent was obtained from each participant. The study 128 was conducted in the Center for Ergonomics lab space at the University of Louisville to test the 129 three different computing work areas with 18 engineering students. We used a counterbalanced, 130 within-subject design, with ‘Computing Environment’ as the single independent variable. The 131 three levels of Computing Environment are shown in Figure 1. The presentation order of the 132 three work area computing conditions were counterbalanced across the 18 participants to control 133 for a potential carry over learning effect. Condition A had a single desktop computer with a 19- 134 inch monitor (baseline condition). Condition B had a desktop with dual 19-inch monitors, as 135 well as a single tablet computer with a 9.7-inch display. Condition C had a desktop with a 19- 136 inch monitor, as well as two tablet computers, with 9.7 inch displays. The 19-inch monitors 137 were in fixed positions; however, the tablet computers were not fixed or propped up and could be 138 moved based on users’ preferences. A standard keyboard and mouse were used as the input 139 devices for the monitors. The desktop had a Windows 7 operating system and the tablets were 140 iPad Air 2’s with the iOS 10 operating system. The input for the iPads were via touch screen and 141 electronic keyboard (no external input devices were connected to the iPads). The same 142 resolution (1920 x 1080 pixels) for the 19-inch monitors was used for each condition. The 143 resolution of the iPads was 1536 x 2048 pixels. These three conditions were chosen based on a 144 review of the literature to begin to understand how a multiscreen work area may affect 145 performance and satisfaction in an information-rich environment. 146 ---------------------------------------------- 147 Insert Figure 1 about here 148 ---------------------------------------------- 149 A previous study found that a 19-inch monitor is the optimal screen size based on 150 performance and preference (Simmons & Manahan, 1999; Simmons, 2001). Therefore, a single 151 19-inch monitor work area served as a baseline condition (A) for comparison with the 152 multiscreen conditions. Several studies have found increased performance for dual-screen users 153 (Anderson et al., 2004; Poder et al., 2011; Russell & Wong, 2005); a dual screen set up is part of 154 condition (B). The dual-screens were fixed on the horizontal plane from the user’s perspective 155 since varying screen position by depth was found to result in a performance decrement (Tan & 156 Czerwinski, 2003). Although pervious research supports the use of dual-screen monitors, it is 157 not known how performance and satisfaction is impacted with the availability of additional 158 screens from the use of mobile technologies. Tablet computers were introduced in conditions 159 (B) and (C) rather than other form factors such as smart phones since previous research 160 demonstrated a significant difference in task completion times between the tablet and smart 161 phone screen sizes (Byrd & Caldwell, 2011). One tablet computer was introduced in condition 162 (B) to use in conjunction with the dual monitor desktop and two tablets computers were 163 introduced in condition (C) to use on conjunction with a single monitor desktop. Conditions (B) 164 and (C) incorporated multiple screens across multiple form factors (desktop monitor and tablet 165 computer) to understand the if multiple screens can help focus (or distract) users’ attention in an 166 information-rich environment (Thompson, 2014). 167 2.2 Participants 168 For this study, 18 industrial engineering students (11 males, 7 females) participated 169 between March-June 2016. Industrial engineering students were chosen based on the flow- 170 charting tasks involved in the session; all students, except for one, had previously learned how to 171 use a process flow chart from an undergraduate course on work design. The one exception was a 172 graduate student with a mathematics undergraduate background. However, she was given an 173 overview of process flow charting technique prior to data collection. Participants were between 174 the ages of 19 and 26 years old; the median age was 23. All participants, with the exception of 175 one, reported little or no previous knowledge of race car driving, which was the application area 176 for the experimental tasks. One participant had a great deal of knowledge about race car driving. 177 Ten of the participants currently used a dual-monitor set-up for their personal workstations and 178 all but one participant had experience using tablet computers or ‘2 in 1’ computers (tablets that 179 convert to a laptop). Only one participant reported regularly using an iPad, which were the 180 tablets used as part of this study. 181 2.3 Dependent Measures 182 We used performance (efficiency and accuracy), workload, and usability as measures to 183 demonstrate improved work area computing. Specifically, improved efficiency and accuracy 184 using a certain work area computing condition (A, B, or C), or time to complete tasks and 185 reduction of errors, would suggest that the work area computing set-up better supports the users’ 186 ability to efficiently and effectively complete information-rich tasks. Similarly, through 187 improved work area computing set-up, a decrease in mental workload (and thus required 188 attentional resources) was predicted, as measured by the NASA Task Load Index (TLX) (Hart & 189 Staveland, 1998). We used unweighted TLX scores as the TLX dimensional weighting 190 procedure has been found to be of limited benefit (Hendy, Hamilton, & Landry, 1993; Nygren, 191 1991). Finally, an improved work area computing set-up would be expected to score higher on a 192 validated usability survey; we used the Computer Usability Satisfaction Questionnaire (CSUQ) 193 (Lewis, 1995). Each of these measures was used to compare the three experimental conditions 194 for work area computing. 195 2.4 Scenarios and Tasks 196 Participants were asked to use flow process charts to document the steps that members of 197 a National Association for Stock Car Auto Racing (NASCAR) team perform during a pit stop. 198 Participants documented different members of the pit crew for each of the three work area 199 computing conditions A, B, and C. The multiscreen/device conditions B and C can be described 200 as “related parallel use” conditions (Jokela et al., 2015), where participants work on completing a 201 single task using more than one device in parallel. The three members of the pit crew for this 202 experiment were front tire carrier, rear tire carrier, and jack man. Participants first watched a 203 demonstration / tutorial video that showed the roles of each member of the pit crew (Interstate 204 Batteries, 2012). After this orientation, participants experienced each work area computing 205 condition while completing a flow process chart to document a different pit crew member’s tasks 206 while watching an actual pit stop video (ArmyRanger241 [screen name], 2015). Solutions for 207 the flow process charts for each of the three roles were developed by one of the authors 208 (D.T.W.), who possessed extensive knowledge of NASCAR racing, prior to the first participant 209 (Appendices A-C). We chose this particular pit stop scenario as an example of an information- 210 rich task, where the use of multiple screens was potentially useful. Table 1 shows how the 211 information was partitioned across the screens and devices for each condition. 212 ---------------------------------------------- 213 Insert Table 1 about here 214 ---------------------------------------------- 215 2.5 Experimental Space 216 The laboratory in the Center for Ergonomics consisted of a participant room (134 sq. ft.) 217 within a larger, main laboratory space (848 sq. ft.). The participant room and main laboratory 218 space were connected with a door and a one-way mirror. The experimenter’s station was located 219 just outside of the participant room. Morae usability testing software connected the participant’s 220 computer and experiment’s computer and was used to display the tasks and instructions to the 221 participant. Morae was also used to video record the direct screen capture of the participant’s 222 interaction with the two desktop monitors. A Web cam was used to record the participant’s 223 interaction with the iPads, and was synced with the Morae screen capture recording. Time to 224 complete the scenarios was automatically captured by Morae. 225 2.6 Procedure 226 After completing a demographics form, participants were given a brief verbal overview 227 of the purpose of the experiment and then oriented to the experimental space. After watching the 228 pit stop demonstration (tutorial) video, participants completed a flow process chart for a member 229 of the pit crew with the work area computing conditions A, B, and C, (counterbalanced across 230 participants) with the information available to them listed in Table 1. Documents and 231 information needed to complete this task, including a blank flow process chart, were provided to 232 the participant by the experimenter though email. After accessing these information items 233 through email, participants could display them as they wished (split screen or toggle between 234 windows to view one at a time) as long as the information items were partitioned across the 235 monitors and devices as prescribed in Table 1. For all three conditions, the flow process chart 236 was always located on Monitor 1 as completing the chart was the primary activity. All other 237 information sources in Table 1 were supportive of completing the flow process chart. A 238 dimension sheet of the pit stop area was provided so that participants could estimate distance for 239 travel steps in the flow process chart. 240 Each of three pit crew roles (tire carrier, rear tire carrier, and jack man) were randomly 241 assigned to the three conditions for each participant. After completing the scenarios for a given 242 condition, participants were given the NASA TLX (computerized version) and CSUQ (paper- 243 based version) surveys for mental workload and usability, respectively. Thus participants 244 completed each survey a total of three times, one for each work area computing condition A, B, 245 and C. After completing the final condition, the debrief session commenced, with the 246 experimenter conducting a semi-structure interview to explore each participant’s experiences 247 with each condition (Appendix D). The debrief interview was audio recorded by Morae. 248 Participants received a $30 gift card at the completion of the debrief session as compensation for 249 their time. The entire participation time was scheduled for 1.5 hours for each volunteer. 250 2.7 Hypotheses 251 Based on a review of the literature supporting the use of multiscreen and multi-device 252 computing to improve performance in information-rich environments, as well as the possibility 253 that multiple screens may help focus one’s attention when the information and functions are 254 parsed distinctly across each screen, the following was predicted: 255 Hypothesis 1: Participants will perform the scenarios in significantly less time and with 256 significantly fewer errors with conditions B and C as compared to condition A (Figure 1). 257 Hypothesis 2: Participants will experience significantly less mental workload when completing 258 the scenarios with conditions B and C as compared to condition A. 259 Hypothesis 3: Participants will rate the usability of the work area computing set-up in conditions 260 B and C significantly higher as compared to condition A. 261 Hypothesis 4: There will be no significant differences for any of the dependent variables between 262 condition B and condition C. 263 2.8 Analysis 264 The simulation study followed a single factor, within-subject experimental design. The 265 single factor was ‘Computing Environment’, with three levels (A, B, C), depicted in Figure 1. 266 Analysis of variance (ANOVA) was planned to test for a main effect of ‘Computing 267 Environment’ on each dependent outcome measures. We planned to use non-parametric 268 statistical testing (i.e., Friedman Two-Way ANOVA) if the normality assumption of ANOVA 269 was violated for a dependent variable. A 0.05 level of significance was applied to all statistical 270 tests. Qualitative data collected from the debrief interview session were analyzed for recurrent 271 themes across participants. These qualitative data were collected to help explain the quantitative 272 results. 273 274 275 3. Results 276 3.1 Performance 277 3.1.1 Time. Mean scenario completion time, with the standard deviation in parentheses, 278 was 596.2 sec (163.4 sec) for condition A, 563.0 sec (213.8 sec) for condition B, and 589.7 sec 279 (195.5 sec) for condition C. ANOVA did not reveal a main effect of Computing Environment on 280 time. 281 3.1.2 Accuracy. Solutions were used to check the accuracy of each participants flow 282 process charts in terms of errors made. Errors included omission errors, incorrect classification 283 of events (e.g., operation vs. transportation), and errors involving the time or distance (for 284 transportation items) for each event. These error counts were treated as ordinal data; 6 median 285 errors were committed by participants when completing scenarios with condition A, 5 median 286 errors with condition B, and 7 median errors with condition C. A Friedman Two-Way ANOVA 287 revealed a main effect of Computing Environment on errors, X2(2) = 6.78, p = 0.034, unadjusted 288 for ties. Post-hoc analysis showed that the difference between conditions B and C was the only 289 significant difference (Wilcoxon Signed Ranks Test). 290 3.2 Workload 291 The NASA TLX data were not normally distributed for the overall composite score or for 292 any of the six subscales, with the exception of mental demand. Therefore, we used non- 293 parametric testing to analyze the workload data. The Friedman Two-Way ANOVA was used to 294 analyze the overall score and subscales and found no statistically significant differences in 295 workload across the three conditions. A summary of the NASA TLX scores is presented in 296 Table 2. 297 298 ---------------------------------------------- 299 Insert Table 2 about here 300 ---------------------------------------------- 301 3.3 Usability 302 The CSUQ is analyzed along an overall score and three subscales, shown in Table 3. 303 Item 9 related to error messages and was excluded from the analysis since there were no error 304 messages presented to participants as part of the study scenario. A copy of the complete CSUQ 305 survey is available in Appendix E. We used ANOVA to test for a main effect of Computing 306 Environment on the system usefulness and information quality subscales. However, the data for 307 overall satisfaction and interface quality failed the normality assumption and so we treated those 308 data as ordinal and used the Friedman Two-Way ANOVA for those two subscales. Statistically 309 significant results were found for overall satisfaction, X2(2) = 12.19, p = 0.002, unadjusted for 310 ties; system usefulness, F(2, 51) = 3.35, p = 0.043; and interface quality, X2(2) = 14.53, p = 311 0.001, unadjusted for ties. For system usefulness, post-hoc analysis (Tukey Pairwise 312 Comparisons) revealed that the significant difference is isolated between conditions A and B. 313 Condition C is not considered different than A or B. For both overall satisfaction and interface 314 quality, post-hoc analysis (Wilcoxon Signed Rank Test) revealed that B is significantly different 315 from A and C. However, A and C are not considered different. 316 ---------------------------------------------- 317 Insert Table 3 about here 318 ---------------------------------------------- 319 320 321 3.4 Qualitative Results 322 During the debrief interview, 15 of 18 participants expressed a clear preference for the 323 computing environment in condition B (dual monitors and one iPad); 3 participants expressed a 324 clear preference for condition A (single monitor); no participants expressed a preference for 325 condition C (single monitor and two iPads). Of the 15 participants who chose the layout in 326 condition B as best, 6 of them explicitly stated that the iPad was unnecessary. Conversely, 3 of 327 the 15 participants expressed a clear preference for the iPad in addition to the dual monitors. 328 When asked what an “optimal” computing environment for their work (i.e., not restricted to 329 choosing one of the three conditions), 16 participants indicated they would prefer two desktop 330 monitors. One participant would prefer a single desktop monitor. And one participant indicated, 331 “the more monitors the better”. Within these 18 responses, five participants expressed a desire 332 or noted an advantage for having a mobile device in addition to fixed monitors for portability of 333 information (three mentioned tablet computers, one mentioned a smart phone, and one 334 mentioned a laptop). 335 336 4. Discussion 337 The results of this investigation into the benefit of using multiple screens and multiple 338 devices were mixed; some of our hypotheses were not supported and others were partially 339 supported. Our first hypothesis was that participants would perform the scenarios in 340 significantly less time and with significantly fewer errors with conditions B and C as compared 341 to condition A. While participants, on average, completed scenarios in less time with condition 342 B, there was no statistically significant difference in time to complete scenarios for the three 343 computing environments. One statistically significant result for errors was isolated between 344 conditions B and C; participants committed significantly less errors when using condition B 345 compared to C. These results suggest marginal support for our first hypothesis, but only for 346 condition B. Condition C was not considered different than the baseline condition A for time 347 and errors. 348 Our second hypothesis was that participants would experience significantly less mental 349 workload when completing the scenarios with conditions B and C as compared to condition A. 350 This hypothesis was not supported. There was no statistically significant difference in the 351 NASA TLX scores when completing scenarios for the three computing environments. However, 352 it is worth noting that condition B was scored, on average, as better than the other conditions 353 especially on the ‘mental demand’ and ‘effort’ subscales. 354 The third hypothesis was that participants would rate the usability of the work area 355 computing set-up in conditions B and C significantly higher as compared to condition A. This 356 hypothesis was partially supported. Condition B was scored significantly higher for overall 357 usability and interface quality compared to both conditions A and C; as well as significantly 358 higher for system usefulness compared to condition A. However, condition C was not scored 359 significantly higher for any of the CSUQ scales compared to the baseline condition A. 360 Our final hypothesis was that there would be no significant differences for any of the 361 dependent variables between condition B and condition C. This hypothesis was not supported. 362 Participants committed significantly fewer errors with condition B compared to condition C. 363 They also rated the overall usability and interface quality as significantly better for the 364 computing environment condition B compared to condition C. 365 4.1 Key Findings 366 A dual monitor desktop with a single tablet computer (condition B) was the ideal 367 computing environment for the “information-rich” engineering problem given to participants. 368 This is supported by converging evidence from the dependent measures as well as the qualitative 369 debrief interviews. A single desktop monitor with two tablet computers (condition C) did not 370 provide any advantage compared to a single desktop monitor (condition A). Overall, these 371 findings provide only marginal support for the concept we set out to investigate, which was the 372 notion that more screens and possibly more devices may help focus one’s attention rather than 373 serve as a distraction, making multiple tasks viewable at a glance across multiple device screens 374 (Thompson, 2014). The finding of a performance and usability advantage of the dual monitors 375 in condition B is consistent with previous studies (Anderson et al., 2004; Poder et al., 2011; 376 Russell & Wong, 2005). A key difference in our study is that we provided a tablet computer in 377 addition to the dual monitors. However, the debrief interviews were mixed as to the usefulness 378 of the third screen provided by the tablet; some participants thought it was not helpful whereas 379 other did find it useful. The complete lack of performance, workload, and usability differences 380 between condition C (single monitor and two tablet computers) and condition A (single monitor) 381 does not support the notion that a multiscreen environment can help focus one’s attention. 382 Indeed, some participants noted that using multiple screens provided by the tablet computer(s) 383 was distracting. Others noted that while they did not hinder their tasks, they did not help. 384 4.2 Limitations and Future Research 385 Our study focused on engineering students completing flow process charts with a race car 386 pit stop scenario as an example of an information-rich task, where the use of multiple screens 387 was potentially useful. A more complex scenario or application area, with a clearer distinction 388 for parsing certain information across screens with distinctly different purposes, may be more 389 amenable to a multiscreen and multi-device environment. For example, a physician that needs to 390 integrate patient data and other information from multiple functions within an EHR and other 391 related clinical information systems may be a more beneficial example to investigate in a future 392 study. Also, our study used Apple iPad tablets; all but one of our participants had experience 393 using tablet computers but only one reported regularly using a iPad. Future research should 394 incorporate other types of tablets and mobile devices, as well as more advanced ones that may 395 better approximate the forgotten power of paper (e.g., Tarun et al, 2013). 396 397 5. Conclusion 398 We designed a study to investigate the potential benefit of multiscreen and multiple 399 device environments using three different computing environment conditions. Scenarios 400 completed with condition B, which included a desktop with dual 19-inch monitors, as well as a 401 single tablet computer with a 9.7-inch display, resulted in significantly less errors compared 402 condition C, which included a desktop with a with a 19-inch monitor, as well as two tablet 403 computers, with 9.7 inch displays. Condition B was also resulted in significantly higher usability 404 ratings compared to condition C and compared to a baseline condition A (single desktop 405 computer with a 19-inch monitor). Our findings are consistent with the literature that show 406 better performance using a dual screen set-up. However, our findings provide only marginal 407 support for the benefit of incorporating additional screens in the form of tablet computers during 408 information-rich, complex tasks. Based on these results, we recommend a computing work 409 environment with dual screen monitors, with an optional tablet computer, for complex and 410 information-rich computing tasks. 411 412 413 414 Acknowledgements 415 A portion of the results from this study was presented at the Human Factors and 416 Ergonomics Society (HFES) International Annual Meeting, Austin, TX, October 9–13, 2017. 417 This work was supported by the Department of Industrial Engineering, J.B. Speed School of 418 Engineering, and the Center for Ergonomics, University of Louisville. 419 420 References 421 Anderson, A., Colvin, J., Tobler, N., & Lindsay, D. (2004). Productivity and multi-screen 422 displays. Rocky Mountain Communication Review, 2, 31-53. 423 ArmyRanger241 [screen name]. (2015). NASCAR Pit Stop at Talledega 2015. Retrieved from 424 https://www.youtube.com/watch?v=GrhsOndLSJQ. 425 Byrd, K. S. & Caldwell, B. S. (2011). Increased memory load during task completion when 426 procedures are presented on mobile screens. Behaviour & Information Technology, 30, 427 643-658. 428 Czerwinski, M., Robertson, G., Meyers, B., Smith, G., Robbins, D., & Tan, D. (2006). Large 429 display research overview. Proceedings of CHI 2006, 69-74. 430 De Backere F., Vanhove, T., Dejonghe, E., Feys, M., Herinckx, T., Vankelecom, J. et al. (2015). 431 Platform for efficient switching between multiple devices in the intensive care unit. 432 Methods Inf.Med., 54, 5-15. 433 Grudin, J. (2001). Partitioning digital worlds: focal and peripheral awareness in multiple monitor 434 use. Proc.CHI 2001, 458-465. 435 Hart, S. & Staveland, L. (1998). Development of the NASA-TLX (Task Load Index): Results of 436 empirical and theoretical research. In P.A.Hancock & N. Meshkati (Eds.), Human Mental 437 Workload (pp. 139-183). North-Holland: Elsevier Science Publishers. 438 Hendy, K. C., Hamilton, K. M., & Landry, L. N. (1993). Measuring subjective workload: When 439 is one scale better than many? Human Factors, 35, 579-601. http://www.youtube.com/watch?v=GrhsOndLSJQ 440 Interstate Batteries. (2012). Inside a NASCAR Pit Stop with Joe Gibbs Racing Pit Crew Coach 441 Mike Lepp. Retrieved from https://www.youtube.com/watch?v=PDSkH7gE6XM. 442 Jokela, T., Ojala, J., & Olsson, T. (2015). A diary study on combining multiple information 443 devices in everyday activities and tasks. In Proceedings of the 33rd Annual ACM 444 Conference on Human Factors in Computing Systems (pp. 3903-3912). ACM. 445 Lewis, J. R. (1995). IBM Computer Usability Satisfaction Questionnaires: Psychometric 446 evaluation and instructions for use. International Journal of Human-Computer 447 Interaction, 7, 57-78. 448 Nygren, T. E. (1991). Psychometric properties of subjective workload measurement techniques: 449 Implications for their use in the assessment of perceived mental workload. Human 450 Factors, 33, 17-33. 451 Poder, T. G., Godbout, S. T., & Bellemare, C. (2011). Dual vs. single computer monitor in a 452 Canadian hospital Archiving Department: a study of efficiency and satisfaction. Health 453 Information Management Journal, 40, 20-25. 454 Rashid, U., Nacenta, M. A., & Quigley, A. (2012). Factors influencing visual attention switch in 455 multi-display user interfaces: A survey. Proceedings of the 2012 International 456 Symposium on Pervasive Displays, 1. 457 Russell, S. E. & Wong, K. (2005). Dual-Screen monitors: a qualitative analysis of their use in an 458 academic library. The Journal of Academic Librarianship, 31, 574-577. 459 Saleem, J. J., Russ, A. L., Justice, C. F., Hagg, H., Ebright, P. R., Woodbridge, P. A. et al. 460 (2009). Exploring the persistence of paper with the electronic health record. 461 Int.J.Med.Inform., 78, 618-628. http://www.youtube.com/watch?v=PDSkH7gE6XM 462 Shin, G. & Hegde, S. (2010). User-preferred position of computer displays: effects of display 463 size. Hum.Factors, 52, 574-585. 464 Simmons, T. (2001). What's the optimum computer display size? Ergonomics in Design, 9, 19- 465 24. 466 Simmons, T. & Manahan, M. (1999). The effects of monitor size on user performance and 467 preference. Proceedings of the Human Factors and Ergonomics Society 43rd Annual 468 Meeting, 1393. 469 Tan, D. S. & Czerwinski, M. (2003). Effects of visual separation and physical continuities when 470 distributing information across multiple displays. Proc.INTERACT 2003, 252-265. 471 Tarun, A., Wang, P., Girouard, A., Strohmeier, P., Reilly, D., & Vertegaal, R. (2013). PaperTab: 472 An Electronic Paper Computer with Multiple Large Flexible Electrophoretic Displays. In 473 Extended Abstracts of ACM CHI’13 Conference on Human Factors in Computing. ACM 474 Press, pp. 3131-3134. 475 Thompson, C. (2014, July 22). How working on multiple screens can actually help you focus. 476 Wired. Retrieved from http://www.wired.com/2014/07/multi-screen-life/ 477 Figure 1 Three experimental conditions for Computing Environment. Condition A had a single desktop computer with a 19-inch monitor (baseline condition). Condition B had a desktop with dual 19-inch monitors, as well as a single tablet computer with a 9.7-inch display. Condition C had a desktop with a 19-inch monitor, as well as two tablet computers, with 9.7 inch displays. Table 1(on next page) Information partition across screens and devices Participants were asked to use flow process charts to document the steps that members of a race car team perform during a pit stop for each of the three experimental conditions for Computing Environment. The table shows how the information needed to complete the scenario was partitioned across the screens and devices for each condition. 1 Table 1 2 Information partition across screens and devices 3 Screen/device Condition A Condition B Condition C 4 Monitor 1 Tutorial video Flow process chart Tutorial video 5 Pit stop video Flow process chart 6 Flow process chart Email access 7 Email access 8 Dimension sheet 9 Monitor 2 N/A Tutorial video N/A 10 Email access 11 Dimension sheet 12 iPad 1 N/A Pit stop video Pit stop video 13 iPad 2 N/A N/A Dimension sheet 14 15 Table 2(on next page) Summary of NASA TLX scores (mean, standard deviation) The table shows workload ratings for each of the six subscales and overall composite score for the NASA TLX. Cond., condition; MD, mental demand; PD, physical demand; TD, temporal demand; Perf., performance; Frust., frustration; total, total composite TLX score, unweighted. 1 Table 2 2 Summary of NASA TLX scores (mean, standard deviation) 3 Cond MD PD TD Perf. Effort Frust. Total 4 A 54.2 (22.1) 21.1 (14.8) 42.2 (17.8) 37.8 (22.7) 51.1 (18.2) 31.7 (25.0) 39.7 (13.3) 5 B 46.7 (21.9) 22.2 (19.7) 39.4 (21.1) 40.6 (28.1) 42.5 (22.2) 31.7 (26.2) 37.2 (14.7) 6 C 48.1 (18.4) 23.6 (18.0) 43.9 (21.7) 41.1 (24.3) 46.9 (20.8) 31.7 (23.8) 39.2 (13.5) 7 Note. Cond., condition; MD, mental demand; PD, physical demand; TD, temporal demand; Perf., performance; Frust., frustration; 8 total, total composite TLX score, unweighted. 9 Table 3(on next page) Usability scores fromthe Computer System Usability Questionnaire (CSUQ) The table shows the usability ratings from the CSUQ. Item 9 was excluded from the analysis as not applicable. Ratings are derived from 7-point Likert-type scales ranging from 1 = strongly disagree to 7= strongly agree. * p values indicate statistically significant findings ( p <0.05). p values reported for system usefulness and information quality are from analysis of variance (ANOVA). p values reported for overall satisfaction and interface quality are from the Friedman Two-Way ANOVA, unadjusted for ties. 1 Table 3 2 Usability scores from the Computer System Usability Questionnaire (CSUQ) 3 Score Condition A Condition B Condition C p value 4 Overall satisfaction (items 1-19) 5.0 (1.0) 5.8 (1.0) 5.3 (1.0) 0.002* 5 System usefulness (items 1-8) 5.0 (1.1) 5.9 (1.0) 5.2 (1.0) 0.043* 6 Information quality (items 10-15) 5.1 (1.0) 5.6 (1.0) 5.5 (0.9) 0.313 7 Interface quality (items 16-18) 4.7 (1.5) 5.9 (1.1) 5.1 (1.3) 0.001* 8 Note. Item 9 was excluded from the analysis as not applicable. Ratings are derived from 7-point Likert-type scales ranging from 1 = 9 strongly disagree to 7= strongly agree. *p values indicate statistically significant findings (p<0.05). p values reported for system 10 usefulness and information quality are from analysis of variance (ANOVA). p values reported for overall satisfaction and interface 11 quality are from the Friedman Two-Way ANOVA, unadjusted for ties. 12