key: cord-0133159-p0z329hr authors: Hombeck, Jan; Ji, Li; Lawonn, Kai; Perin, Charles title: A Study of Opacity Ranges for Transparent Overlays in 3D Landscapes date: 2020-09-02 journal: nan DOI: nan sha: 521b81ee9ad30f799b7e33cf162e17c802808014 doc_id: 133159 cord_uid: p0z329hr {When visualizing data in a realistically rendered 3D virtual environment, it is often important to represent not only the 3D scene but also overlaid information about additional, abstract data. These overlays must be usefully visible, i.e. be readable enough to convey the information they represent, but remain unobtrusive to avoid cluttering the view. We take a step toward establishing guidelines for designing such overlays by studying the relationship between three different patterns (filled, striped and dotted patterns), two pattern densities, the presence or not of a solid outline, two types of background (blank and with trees), and the opacity of the overlay. For each combination of factors, participants set the faintest and the strongest acceptable opacity values. Results from this first study suggest that i) ranges of acceptable opacities are around 20-70%, that ii) ranges can be extended by 5% by using an outline, and that iii) ranges shift based on features like pattern and density. choices. These choices are currently limited, as they are inherited from 2D GIS tools and constrained to a transparent layer of color being displayed on top of the realistically rendered landscape [9] . To think beyond this conventional design we can turn to the field of information visualization, which provides many empirical guidelines for representing abstract data, established through graphical perception studies [6] . Opacity is an effective visual variable to encode quantitative information [24] such as uncertainty [7] of the RoI, and has been the focus of studies on the perception of grids in 2D charts [5] . Graphical Perception studies in information visualization, however, make strong assumptions regarding the background and environment in which the abstract data is being represented, typically a uniform background or a 2D map and a static point of view. In contrast, 3D landscapes are colorful, contain complex geometries and textures, and can be looked at from infinite numbers of points of view. It makes it difficult to directly apply results from studies that consider standard 2D charts (e.g., [5, 13, 14, 20] ). To identify reasonable opacity ranges for overlays, we must consider the many other graphical properties of these overlays. Our first step was to identify promising ones to study. In the field of illustrative rendering, patterns that are filled [12] , striped [15, 26] and doted [15, 21] are widely used to render overlays. Outlines [17] are commonly used to increase the visibility of 3D shapes [18, 19] . Pattern density [10] and overlay opacity [5] also affect how information is perceived. With transparent overlays, the rendered image must contain sufficient visual cue to permit the Human Visual System (HVS) to recognize the separate layers, instead of a single layer of the blended colors [11, 23] . A sketching session with our research group and an interview with a visualization practitioner confirmed the usefulness of investigating the following rendering parameters, widely used in the literature: pattern style, pattern density, pattern outline, and scene background. We conducted a remote study in accordance to our research ethics board regulations during the covid-19 pandemic. A remote study means lower internal validity of the results [22] because the apparatus is less controlled than in a lab setting (e.g., varying display sizes, luminosity and contrast). To control for these factors we required participants to use a monitor with resolution of 1920x1080 px and Figure 2 : The three PAT T ERN for the two OUT LINE for the two DENSITY used in the study (only BKG is not shown in this Figure) . with the luminosity value of their screen set to its maximum, and we conducted the study during daylight hours only. On the other hand, a remote study increases the external validity of the results [22] . It is closer to the real-world context, where people use various devices and screens in various environments. Overall, our study loses some precision to increase its realism and generalizability [22] . We conducted the study with the TeamViewer remote desktop control software [3] . This allowed the experimenter to verbally communicate with the participants, answer their questions and observe their interactions. Participants interacted with the study software via mouse and keyboard. While we required a good internet connection from participants, the delay (time between the action from the participant and the response from the server) varied between participants and sometimes within a session. The study software showed views of a forest area in a professionally used virtual environment in Unity. To establish opacity ranges in which different configurations of overlay are considered useful, we created the two following tasks based on Bartram et al.'s [5] study of opacity values for grids: • T Faintest : Please adjust the color to be as faint as you think it can comfortably be to be still useful; any fainter and you would no longer be able to easily use it. • T Strongest : Please adjust the color to be as strongly visible as you think it can comfortably be before it interferes with or "comes in front of" the environment; any stronger and it would be too obtrusive. Participant used the left and right arrow keys to change the opacity value and pressed the space bar to confirm and go to the next trial. We studied the four following factors (see Figures 2 and 3 ): • PAT T ERN was either P Fill (a solid color pattern), P Stripes (a stripped pattern). or P Dots (a dotted pattern). • Both P Stripes and P Dots were presented with two different values for DENSITY . With P Stripes at low density D Low , each stripe as a 240-pixel width and a 120-pixel width at high density D High . With P Dots each dot has a 240-pixel diameter at D Low and a 120-pixel diameter at D High . • The overlay OUT LINE (a 12-pixel width solid stroke around the RoI) was either present (O On ) or not (O ∅ ). • The scene BKG was either with trees B Trees or without B ∅ . Overall the experiment consisted of [2×PAT T ERN (P Stripes , Each participant performed the experiment in two task blocks, one for T Faintest and one for T Strongest (80 trials per task block in random order). The order of task block was balanced across participants. . The overlay is rendered in a post-processing step that maps its shape on top of the underlying surface (that includes objects present in the scene). The experimenter introduced the study and let the participant read the instructions explaining the tasks. The participant then performed a series of training trials until they understood the tasks and controls. After having asked any question they had, they started the recorded trials, either with T Faintest or with T Strongest first. Once they had completed the first task block, they were asked to take a 5-minute break before completing the second task block. Finally they filled out the study questionnaire and participated in a short debriefing. The questionnaire asked demographic questions (age, gender, level of education, experience in visualization, experience in 3D, usage of video games and of computers) and study-related questions on Likert scales. Each session lasted approximately one hour. We hypothesize that the denser a pattern, the easier to perceive the overlay as a conceptually separate layer for the HSV, and thus the less saliency the overlay will need. For instance, T Faintest will lead to higher opacity values for D Low , because a low-density pattern will require more emphasis through higher opacity, for the HVS to recognize the overlay as a conceptually separate layer. We also hypothesize that the outline will facilitate the HVS to more easily recognize that the overlay belongs to a separate shape, and therefore belongs to a conceptually separate transparent layer. Therefore, the outline will make a wider range of opacity recognizable as transparent overlays instead of a single layer of blended color. More formally, our hypotheses were as follows: • H DensityFaint : D Low will lead to higher average opacity values than D High for T Faintest . Considering that P Fill has the highest density, we therefore hypothesize that for . Indeed, with D Low , the pattern shapes are larger and there are fewer of them, resulting in relatively large empty spaces in the pattern. • H DensityStrong : D Low will lead to higher average opacity values than D High for T Strongest . With P Fill having infinite density, we hypothesize that for T Strongest , we will obtain O( We recruited sixteen participants (six male, ten female) with normal or corrected vision through mailing lists and social networks. Most participants were between 19-25 years of age with on average 1-2 years of experience with 3D environments and data visualization. Detailed participant information is provided in supplemental material. Participants received $15 via email for their participation. We base our analyses on estimation using bootstrapped confidence intervals instead (CI) of p-values, following recommendations from APA [4] . A 95% CI contains the true mean 95% of the time and conveys effect sizes [8] . We pre-specified all analyses before running the study and tested on pilot data (scripts for parsing the data, computing CIs, and generating drafts of figures). Here we revisit our hypotheses against the results shown in Figure 4 . H DensityFaint is partially confirmed. Indeed, P Fill has consistently lower opacity values than the other configurations of PAT T ERN and DENSITY . However, P Stripes × D Low has a significantly lower opacity value than P Stripes × D High ; and P Dots × D Low has a significantly lower opacity value than P Dots × D High -these results are opposite to what we hypothesized. H DensityStrong is not confirmed, with only P Fill and P Stripes × D High leading weakly to greater opacity values than P Dots × D High (around 5% greater). Overall, lower density leads to lower values for the lower bound of the opacity range, resulting in larger usable opacity ranges (around 5% larger for P Stripes and 10% larger for P Dots ). H Outline is not confirmed for T Faintest with B ∅ and T Strongest , with no notable difference based on OUT LINE. However, there are differences for T Faintest and with B Trees , for which P Fill , P Stripes × D Low , P Stripes × D High and P Dots × D Low have smaller opacity values (around 5% smaller) with O On than with O ∅ , i.e. the opacity ranges are extended by around 5% (except for P Dots × D High ). H Background is not confirmed as there is no strong difference between any of the conditions based on BKG. The results reveal additional aspects beyond hypotheses checking. First, different conditions lead to different opacity ranges. Opacity ranges for P Fill are consistently slightly larger than for P Stripes × D Low , which are larger than P Dots × D Low , which are larger than P Stripes × D High , which are larger than P Dots × D High . Second, the CIs for T Strongest are wider than for T Faintest , i.e. there is higher variance and uncertainty with T Strongest than with T Faintest . Third, Figure 4 shows the deviation from participant average completion time for both tasks (instead of absolute completion times that are meaningless with the remote setup). A configuration led to faster completion if its CI falls on the left side of the 0 baseline, and to slower completion if it falls on the right side. We notice that BKG strongly affects participant completion time, with nearly all conditions with B Trees resulting in longer completion times than average. We also observe that P Fill and P Stripes × D Low might result in slightly shorter times than P Dots × D High . 14/16 participants selected P Fill as their favorite pattern. P Fill also received the strongest scores for performing both T Faintest (12 participants strongly positive) and T Strongest (13 strongly positive), and for confidence in their answers (14 strongly positive). P Stripes received mitigated scores for T Faintest (7 positive, 6 negative) but was better ranked for T Strongest (10 positive, 3 negative), and made participants quite confident in their answers (12 positive, 2 negative). P Dots was the least preferred pattern, with 7 positive and 6 negative for T Faintest , 7 positive and 6 negative for T Strongest , and 6 positive and 5 negative for confidence in their answers. Participants found that O On helped for completing both T Faintest (13 positive) and T Strongest (11 positive), and increased their confidence in their answers (13 positive). We did not expect that increasing the density of a pattern would also increase the required opacity value, with the effect being stronger for T Faintest . It is possible that a more complex pattern makes it more challenging to mentally reconstruct the information about two layers (the overlay and the 3D scene underneath) from a single image, thus requires stronger visual stimuli and salience, i.e. higher overlay opacity value. Completion times conform with this observation. The time required to set the opacity for D High is in almost all cases longer than for D Low , which may be due to longer times spent to mentally reconstruct the overlay. Since for T Strongest the pattern is more visible the effect is not as strong as for T Faintest . Some overlay configurations have wider opacity ranges than others. P Dots has relatively smaller, shifted upwards ranges, compared to P Fill and P Stripes (i.e. higher minimum and maximum bounds). We think this is again due to the complexity of the pattern, and that higher opacity values for the overlays with P Dots make the distinction of the two layers (overlay and 3D scene underneath) easier. Figure 5 shows differences in distributions of opacity values used for each PAT T ERN. All three plots have two inflexion points, for their minimum and maximum bound of opacity range, but their overall shapes differ. The extent of these opacity ranges, combined with their degree of bimodality (how separable the minimum bound is from the maximum bound) can guide the selection of rendering parameters according to the data to visualize. For example, an overlay configuration with a small extent and a low degree of bimodality would be appropriate to visualize a binary data dimension, e.g enable or disable the overlay; a high degree of bimodality is appropriate for a quantitative data dimension; and a large extent is appropriate for a quantitative data dimension that requires a high resolution. Our results show that an outline can help increase the acceptable opacity range of an overlay, especially on the lower opacity bound. With an outline the pattern is no longer solely responsible for establishing the structure of the overlay, it serves as an additional internal support and can be less salient. Outlines are particularly useful when there are objects in the scene, increasing the range of acceptable opacity values (lower bound) by around 5% in that case. Outlines are less useful for T Strongest , because the region of interest is already sufficiently visible due to the salience of the pattern itself. We found that participants were slower with objects (trees) in the background, for all conditions we tested. We explain this result because the rendering of an overlay is partially occluded by the objects inside. It results in a more complex overlay shape with uncertain boundaries that takes more time to reconstruct mentally. Several participants found that P Stripes helped identify the structure of the terrain. However, we noticed that this happened mostly when the stripes were parallel to the slope of the terrain. One way to improve structural visibility would be to use grid-like patterns that would assist in understanding the gradient in two dimensions. Our empirical results provide a starting point to establish how to design overlaid visualizations of abstract data in realistically rendered 3D environments. This first step necessarily comes with limitations in terms of generalizability of the results. First, the study measures people's preferences for opacity ranges instead of visual performance. This is a subjective measure that allows us to derive the most comfortable opacity ranges for the user. In the future we plan to conduct quantitative perceptual studies that will help us measure more objectively visual performance. Second, we used a landscape with and without trees, but other aspects of the terrain need to be investigated: what if the texture of the terrain is light, like snow, or dark, like grass or forest? What if it is uniform, like an ocean, or has many topological features, like a mountain? There are also many more possible design variations of the overlays to study. These include the color of the overlay, the contrast between the dominant color of the terrain and the color of the overlay, the orientation of the patterns, the distribution of points and lines in the patterns and the point shape and radius. The world of computer graphics offers a wide array of additional rendering parameters that might not be traditionally used when visualizing abstract data on 2D charts (or even avoided by fear of creating chart junk), but that could be relevant for overlays in 3D environments. This includes the impact of color, contrast, shading or reflections. These additional considerations may be valuable to make it easier to discriminate the overlay from the background when they are of similar color. In our study we kept the rendering as simple as possible to avoid artifacts or distractions, but one could for example look at casting shadows and incorporating reflections. Another important aspect to study is that of intersecting overlays. Having multiple (overlapping) RoI to analyze simultaneously is common within industrial applications. How to display multiple data sets is a current challenge that also calls for additional research. We have provided the first set of empirically determined acceptable (subjective) opacity ranges to use when adding overlays on top of a photorealistic 3D environment. Through our study we observed participants set the lower and upper limits for opacity values for three different patterns (filled, striped, and dotted patterns), two pattern densities, the presence or not of a solid outline, and two types of background (blank and with trees). We found that the range of acceptable opacity values is roughly between 20-70%, with ranges shifting up or down slightly according to the overlay configurations. While patterns like dots are mostly considered as distracting, we found that striped patterns can help better understand the curvature of the underlying terrain. How to coalesce data visualization with realistic and interactive rendering in a unified visual context is an open research question with practical industrial impacts. Rather than providing definite answers to this open question, our initial inquiry provides many directions worth studying to better understand how to represent overlays on top of complex, realistic 3D scenes. We look forward to continuing this exploration at the junction of information and scientific visualization, and realistic computer graphics rendering. ESRI ArcGIS Mapping and Analytics System TeamViewer Remote Desktop Software The Publication manual of the American psychological association Whisper, don't scream: Grids and transparency Graphical perception: Theory, experimentation, and application to the development of graphical methods Visualization of uncertainty in lattices to support decision-making Inference by eye: Confidence intervals and how to read pictures of data Geographic Visualization: Concepts, Tools and Applications Visual learning in the perception of texture: Simple and contingent aftereffects of texture density Color transparency A non-photorealistic lighting model for automatic technical illustration Meteorology meets computer graphicsa look at a wide range of weather visualisations for diverse audiences Just noticeable differences in 2d and 3d bar charts: A psychophysical analysis of chart readability Illustrating transparent surfaces with curvature-directed strokes Points of significance: error bars Feature lines for illustrating medical surface models: Mathematical background and survey A survey on multimodal medical data visualization A survey of surfacebased illustrative rendering for visualization A baseline study of emphasis effects in information visualization A survey of digital stippling Methodology matters: Doing research in the behavioral and social sciences The perception of transparency Visualization Analysis and Design Shaping the digital twin for design and production engineering View direction, surface orientation and texture orientation for perception of surface shape This work was funded by NSERC, the Carl Zeiss Foundation and the Federal Ministry for Economic Affairs and Energy of Germany.