Embodied Navigation in Immersive Abstract Data Visualization: Is Overview+Detail or Zooming Better for 3D Scatterplots?
Yalong Yang, Maxime Cordeil, Johanna Beyer, Tim Dwyer, Kim Marriott, Hanspeter Pfister
EEmbodied Navigation in Immersive Abstract Data Visualization:Is Overview+Detail or Zooming Better for 3D Scatterplots?
Yalong Yang, Maxime Cordeil, Johanna Beyer, Tim Dwyer, Kim Marriott and Hanspeter Pfister
Fig. 1. Conditions tested in our user study: a) Room-sized interface (or Rm ). b) Room-sized interface with an overview (or RmO ). c)Zooming interface (or Zm ): users zoom in and out with a “pinch” gesture. d) Zooming interface with an overview (or ZmO ). Abstract — Abstract data has no natural scale and so interactive data visualizations must provide techniques to allow the user to choosetheir viewpoint and scale. Such techniques are well established in desktop visualization tools. The two most common techniques arezoom+pan and overview+detail. However, how best to enable the analyst to navigate and view abstract data at different levels of scalein immersive environments has not previously been studied. We report the findings of the first systematic study of immersive navigationtechniques for 3D scatterplots. We tested four conditions that represent our best attempt to adapt standard 2D navigation techniquesto data visualization in an immersive environment while still providing standard immersive navigation techniques through physicalmovement and teleportation. We compared room-sized visualization versus a zooming interface, each with and without an overview.We find significant differences in participants’ response times and accuracy for a number of standard visual analysis tasks. Both zoomand overview provide benefits over standard locomotion support alone (i.e., physical movement and pointer teleportation). However,which variation is superior, depends on the task. We obtain a more nuanced understanding of the results by analyzing them in terms ofa time-cost model for the different components of navigation: way-finding, travel, number of travel steps, and context switching.
Index Terms —Immersive Analytics, Information Visualization, Virtual Reality, Navigation, Overview+Detail, Zooming, Scatterplot
NTRODUCTION
Abstract data has no natural scale. That is, data that is not based in aphysical reference space can be freely re-scaled and viewed from anyangle that best supports the analysis task. However, this freedom alsorepresents a challenge to the design of interactive navigation methods:how do we allow people to move freely through an abstract informationspace without confusing them? Designers of desktop visualizationtools have grappled with this problem for decades and have evolvedsophisticated techniques for navigation. For example, it is common indata visualizations, such as scatterplots, time-series, and so on, to allowusers to zoom in, making a specific region larger to access more detail.They can then pan around at the new zoom level or zoom out againto reorient themselves in the full dataset. Another common approachis to provide a minimap window to provide an overview of the wholedataset at all times. As discussed in Section 2.1, these methods are wellstudied and widely accepted in desktop data visualization.Recent developments in technology has seen renewed interest in us-ing immersive environments for data visualization, but this requires usto reconsider and possibly adapt navigation methods that have become • Yalong Yang, Johanna Beyer and Hanspeter Pfister are with the School ofEngineering and Applied Sciences, Harvard University, Cambridge, MA,USA. E-mail: { yalongyang, jbeyer, pfister } @g.harvard.edu• Maxime Cordeil, Tim Dwyer and Kim Marriott are with the Department ofHuman-Centred Computing, Faculty of Information Technology, MonashUniversity, Melbourne, Australia. E-mail: { max.cordeil, tim.dwyer,kim.marriott } @monash.eduManuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publicationxx xxx. 201x; date of current version xx xxx. 201x. For information onobtaining reprints of this article, please send e-mail to: [email protected] Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx standard on the desktop. As virtual and augmented reality headsetscontinue to improve — e.g., in resolution, field-of-view, tracking sta-bility, and interaction capabilities — they present a viable alternativeto traditional screens for the purposes of data visualization. A recentstudy has found a clear benefit to an immersive display of 3D clusteredpoint clouds over more traditional 2D displays [32]. However, in thatstudy the only way for participants to change their point of view wasthrough physical movement. There are questions about how one mightnavigate in situations where the participant cannot move around (e.g.,they are seated), or where the ideal zoom-level for close inspection ofthe data is too large to fit in the physically navigable space.In Virtual Reality (VR) gaming “teleporting” is a standard way tonavigate a space that is too large to fully explore through one-to-onescale physical movement. The equivalent of the desktop minimapoverview+detail in VR is World In Miniature (WIM) navigation. Zoom-ing is also possible in immersive environments through a latched gestureto scale the information space around the user. However, the effec-tiveness of such immersive navigation techniques for point cloud datavisualization applications in VR has not previously been tested. The in-herent differences between such abstract data visualizations and typicalVR worlds, namely, the freedom to move between different scales andviewpoints, makes it unclear whether standard VR navigation methodsare sufficient to support analysis tasks.Our overarching research question, therefore, is: how can we bestenable the analyst to navigate and view abstract data at differentlevels of scale in immersive environments? To address this questionwe first thoroughly review standard data visualization navigation tech-niques from desktop environments and the initial work that has begunto provide navigable data spaces in immersive environments (Sec. 2).We find that existing literature in immersive data visualization has notfully explored how to adapt navigation techniques for data visualiza-tion from the desktop and/or to adapt VR navigation techniques to the a r X i v : . [ c s . H C ] A ug pplication of data visualization. Therefore, in Section 3, we look atthe space of possible ways in which users can navigate immersive datavisualizations and from this, in Sec. 4.1, we choose and study four basicnavigation possibilities: room-sized data at fixed-scale with physicalnavigation versus a smaller data display with zooming (dynamic scale),each with and without an overview WIM display.Specifically, we compare zooming to overview+detail. We are par-ticularly interested in how to ensure that navigation remains embodiedthrough physical movements and gestures that are as natural as possi-ble [22]. The contributions of this paper are:1. The first systematic exploration of overview and detail interactionparadigms in immersive scatterplot visualization, Sec. 3.2. A study comparing four navigation conditions (as above), Sec. 4and Sec. 5. This finds that: Participants significantly preferredeither zoom or overview over standard locomotion support alone.Adding an overview also improved accuracy in some tasks. Room-sized egocentric views were generally faster in our study.3. A more nuanced interpretation of our study results using a navi-gation time-cost model , Sec. 6.4. Recommendations for use of zoom and overview in combinationwith physical navigation for immersive data visualization, Sec. 7. ELATED W ORK
A foundation of visual data exploration is the Shneiderman
Informa-tion Seeking Mantra of “Overview first, filtering and details on de-mand” [60]. Several paradigms have emerged to support navigationfollowing this mantra, such as overview+detail, focus+context [41] andzooming [17]. Many 2D visualization techniques provide such interac-tions, such as Google Map (zooming), PowerPoint (overview+detail),and fish-eye views (focus+context). In an extensive literature review,Cockburn et al. [19] conclude that each technique has issues — zoom-ing impacts memory [20]; focus+context distorts the information space;and overview + detail views also make it difficult for users to relateinformation between the overview and the detailed view [9, 29].Navigation has also been studied on mobile displays. Burigat etal. [14] found that interactive overviews were particularly advantageouson small-screen mobile devices for map navigation tasks [14,15]. How-ever, in the context of information visualization, adding an overviewto a zoomable scatterplot on a PDA was found to be ineffective anddetrimental compared to a zoomable user interface only [16].With screens and projectors becoming cheaper, larger and easierto tile, researchers have explored the benefits of large display sizesfor navigating visualizations. Ball and North [4] studied the effect ofdisplay size on 2D navigation of abstract data spaces and found thatlarger displays and physical navigation outperformed smaller displays.They also found that virtual navigation (i.e., pan and zoom) was morefrustrating than physically navigating a large tiled-display. Ball andNorth [5] also investigated the reason for improved performance ofinformation space navigation on large displays and found that physicalmovements were the main factor, compared to field of view. An-other study by Ronne and Hornbaek [55] investigated focus+context,overview+detail and zooming on varying display sizes. They found thatsmall size display was least efficient, but found no differences betweenmedium and large screens and that overview+detail performed the best.In our study, we investigate the use of overview+detail and contin-uous zoom, as they are most suited to our tasks. We do not considerfocus+context as it is not well suited to our tasks (i.e., counting pointsand evaluating distances between pairs of points), and spatial distortionwould introduce misinterpretation of distances.
Research in 3D User Interfaces (3DUI) and Virtual Reality has exploreddifferent ways for users to navigate 3D immersive scenes. Laviola etal. [37, Ch. 8] provide a taxonomy of these techniques. They describefour navigation metaphors — walking, steering, selection-based traveland manipulation based travel.Walking in VR can be either real walking (i.e., the user walks witha VR headset in the real world) or an assisted form of walking (e.g.,using a treadmill [26] or more dedicated devices to emulate walking in place [31]). While those walking devices provide a realistic sensationof walking in the virtual world, they are bulky and not affordable fora wide audience. Abtahi et al [1] recently investigated mechanisms toincrease users perceived walking speed for moving in a large virtualenvironment. However, it is not trivial on how to apply their techniquesfor visualization navigation. In our research, we focus on physicalwalking available through HMD VR devices, however restricted to acommon office size area. Walking has been found to be more efficientthan using a joystick for orientation in VR [18], and has generally beenfound to be beneficial for perception and navigation tasks compared tocontroller-based interactions [35,56,57]. Walking has also been provento be more efficient than standing at a desk for tasks involving visualsearch and recalling item locations in space [53].In contrast to walking, steering metaphors allow the user to navigatethe space without physically movement. They are usually gaze- and/orcontroller-based. Teleportation is a very common locomotion techniquewhich consists of casting a ray from a tracked controller to the desiredlocation [44] and a button push to travel there. Bowman et al. [12] foundthat hand-based steering was more efficient than gaze-based travel,but that instant teleportation (with no transition between initial andfinal location) was detrimental for orientation compared to continuousviewpoint change. Laviola et al. [37] argue that continuous uncontrolledmovements of the viewpoint induce cybersickness, but that it can bemitigated if the transition is very fast. Modern teleportation techniquesuse fade or blinking metaphors where the screen transitions to blackbefore the move and is restored at the destination position. In all theconditions of our experiment we allow this type of teleportation toallow the user to fast track travel in the virtual environment if needed.An alternative VR navigation technique is WIM [43, 63], whichintroduces a “minimap” to allow the user to teleport by selecting atarget position in the miniature. Recent refinements offer scalable,scrollable WIMs (SSWIMs) [71], or support for selecting the optimalviewpoint in dense occluded scenes [65]. We designed an embodiedplacement for WIM, and two different teleportation methods with WIM.In summary the effects of spatial memory, cybersickness and overallusability of such techniques have been studied in general purposeVR applications. In the context of abstract visualization their relativedrawbacks and benefits are unclear.
3D scatterplots have received significant attention in Immersive Ana-lytics research [2, 21, 25, 32, 52]. Raja et al. [54] found that using bodymotion and head-tracking in a CAVE-like environment was beneficialcompared to non-immersive environments for low-level scatterplottasks (i.e., cluster detection, distance estimation, point value estima-tion and outlier detection). Kraus et al. [32] explored the effect ofimmersion for identifying clusters with scatterplots. They explored a2D scatterplot matrix and 3D on-screen, and immersive table-size androom-size spaces. Overall their found that the VR conditions were wellsuited to the task. However they also pointed out that their room-sizecondition may not have been optimal due to the lack of an overview.Body movements have been observed in Immersive Analytics stud-ies involving 3D scatterplots, which may indicate potential benefits fordata exploration and presentation. For example, Prouzeau et al. [52]observed that some participants tended to put their heads inside 3Dscatterplots to explore hidden features; Batch et al. [7] observed par-ticipants organize their visualizations in a gallery-style setup and walkthrough them to report their findings. Simpson et al. [61] informallystudied walking versus rotating in place for navigating a 3D scatterplot,and their preliminary result indicated that participants with low spatialmemory were more efficient in the walking condition.Other alternatives to walking in a room-size environment have alsobeen explored. Filhio et al. [66] found that a seated VR setup for 3Dscatterplots performed better than on a desktop screen. Satriadi etal. [59] tested different hand gesture interactions (user standing still) topan and zoom 2.5D maps in Augmented Reality, but the visualizationswere within a fixed 2D viewport in the 3D space.Flying is a steering metaphor that has been used as an alternative towalking to obtain a detailed view of an immersive visualization. Withflying, benefits of immersion were found compared to a 2D view for ig. 2. Our study compares two main effects, leading to four conditions. distance evaluation tasks in scatterplots [67]. Sorger et al. [62] designedan overview+detail immersive 3D graph navigation metaphor that usesflying to gain an overview, and a tracked controller-based teleport to aselected node position to obtain details from a node-centric perspective.The main takeaway of their informal expert-study is that overviewswere perceived as important to keep the user oriented in the graph.To our knowledge, WIMs have been underexplored in immersivedata visualization. Nam et al. [46] introduced the Worlds-in-Wedgestechnique that combines multiple virtual environments simultaneouslyto support context switching for forest visualization. The study ofDrogemuller et al. [23] was the first to formally evaluate one andtwo-handed flying vs. teleportation and WIM in the context of largeimmersive 3D node-link diagrams [23]. They found that the flyingmethods were faster and preferred compared to teleportation and WIMs,for node finding and navigation tasks in the 3D graph. However, theydid not test the condition where the user only walks around the roomwithout instrumented metaphors for overview+detail.Finally, scaling the visualization as a 3D object in VR via bimanualinteraction has been used in Immersive Analytics systems [30, 68] butnot formally studied against other VR navigation techniques. In sum-mary, there is evidence that overview+detail and zooming are beneficialfor navigating 2D visualizations. However, their relative benefit in thecontext of immersive 3D scatterplots is still underexplored.
TUDY R ATIONALE AND D ESIGNS
Our study is intended to address the gaps in the literature describedabove in terms of how to carry over well understood 2D navigationtechniques to immersive visualizations (see Fig. 2). We decided tonot include 2D interfaces for 3D scatterplots as testing conditions, asthe 2D alternatives (i.e., scatterplot matrix and 3D scatterplot on a 2Dscreen) have been tested to be less effective than representing them as3D scatterplots in an immersive environment [32].We focus on adapting zoom and overview+detail techniques. Wedid not include focus+context in our study, as it has been shown thatsuch spatial distortion techniques are not effective for large 2D displayspaces [55], and we expect similar results in the immersive environ-ments. However, formal confirmation of this is worthwhile future work.Adapting overview+detail and zooming techniques to immersive envi-ronments from 2D interfaces requires many design decisions. In thefollowing, we describe our design considerations and choices.
The idea of the overview+detail is to provide two separate displayspaces representing the same information but at different scales. Thebasic design of our overview is straightforward: the whole informationspace in which the user is standing is represented in a cube – with alldata glyphs scaled down appropriately. However, three essential designchoices remain unanswered: placement of the overview; Point of View(PoV) indicator in the overview; and overview teleportation.
Placement of the overview:
In 2D display environments, the overviewis commonly placed at a global fixed position relative to the display:either at a corner of the detail view (e.g., some on-line maps) or outsidethe detail view (e.g., some text editors, PowerPoint). When using theoverview on a 2D display, the user’s Field of View (FoV) is consistentand can cover the full display space at all times. Thus, the user caneasily access the overview at any time in this placement. Unlike 2Ddisplays, in an immersive environment, the user is expected to performmore body movement. As a result, the user’s FoV is constantly chang-ing. If the overview is placed at a fixed global position in immersiveenvironments, the user may forget its location or it may not be reachablewhen required. In general, the overview needs to be easily accessibleso it can be brought into focus for close inspection, but by default itshould not occlude the users’ view of the main scene. A logical designchoice is to place the overview somewhere at the periphery of the users’view by default, but allow the user to grab it with the controller to bring
Fig. 3. Enlarging and shrinking the overview with arm movement.Fig. 4. Demonstration of point-and-click teleportation. p current (the greendot) is the current position of the user; p AoI (the blue dot) is the positionof Area of Interest (AoI) and the clicked position; and p teleport (the yellowdot) is the position the user will teleport to. it up for close inspection. But there remains the question of what is thebest default location relative to the user.Our first idea was to place the overview at a fixed position relative tothe FoV, i.e., the overview follows the user’s movement, e.g., to alwaysappear at the bottom right corner of the user’s FoV. We tested it withtwo participants; both of them reported the overview to be distractingand cluttered. They also found it difficult to access (grab) the overview.We then attached the overview to the user’s off-hand controller. Thiswas inspired by WIM where the miniature is associated with a trackedphysical board [43, 63], and Mine et al. [45] who attached widgets tovirtual hands in VR. To further reduce visual clutter, we shrink theoverview by default and allow the user to enlarge the overview bytouching it with the other controller. This embodied design allowsusers to enlarge or shrink the overview by simple hand movements (seeFig. 3). We tested it with another participant, and explicitly asked ifshe felt the overview caused visual clutter, was distracting, or difficultto access. The participant was highly positive about the new design. PoV indicator in the overview:
In 2D overview+detail interfaces, aFoV box is used to indicate which portion of the overview is presentedin the detail view. Similarly, in WIM, it is also important to indicatethe view position, but in addition the direction the user is facing [63].Following their designs, we use a cube to represent the tracked headset.The position and rotation of the cube are synchronized with the headsetin real-time. We also use a semi-transparent cone attached to the cubeto represent the user’s view direction (see Fig. 3).
Overview teleportation:
Tight coupling between the overview anddetail views is standard in 2D interfaces (e.g., [29, 72]). The aim isto allow the user to change the portion of the scene presented in thedetail view by interacting with the overview. There are two widelyused implementations: drag-and-drop and point-and-click interactions.Changing the detail view in 2D interfaces is equivalent to changing theviewing point and direction in immersive environments. We adaptedthe 2D interactions for immersive environments:
Drag-and-drop teleportation:
On 2D interfaces, the user can selectthe FoV box in the overview then drag-and-drop it at a new position inthe overview. The detail view will then switch to the new position. Inimmersive environments, WIM allows participants “pick themselvesup” , i.e., the user can pick up the PoV indicator and drag it to a newposition in the overview to teleport to the dropped location in the detailview. We implemented the same mechanism in our visualizations.
Point-and-click teleportation:
The user can also directly choosethe destination and explicitly trigger a command to change the detailview. On 2D interfaces, the operation involves pointing a cursor at thedestination position in the overview and then clicking to translate thedetail view. In immersive environments, in addition to changing theposition of the user, we also need to adjust the user’s orientation. Weimplemented a mechanism that will determine the user’s position andorientation (see Fig. 4). Basically, the teleport target position is on thestraight line connecting the current position and the Area of Interest ig. 5. The gesture to zoom and rotate simultaneously. From left to right:scale up the object and rotate clockwise; from right to left: scale downthe object and rotate anti-clockwise. (AoI), but slightly away from the AoI to ensure it is within the user’sFoV. The orientation is the direction of this straight line.Drag-and-drop teleportation is expected to give users more control oftheir position and orientation and possibly give them a better estimationof their position and orientation after the teleportation. However, thismulti-step operation is not welcomed by all users [33]. Point-and-clickteleportation requires less steps, but may increase the gap between theexpected and actual teleported position and rotation. Like many other2D interfaces, we include both mechanisms in our visualizations, to letusers choose their preferred option.
Although 2D desktop interfaces usually use the scroll-to-zoommetaphor, pinch-to-zoom is the standard zooming gesture on mostmulti-touch devices. The idea of pinch-to-zoom is to re-scale accordingto the distance between two touch points [28]. Immersive systemscan naturally be considered as multi-touch systems as they are capableof tracking at least two hand-held controllers. Therefore, we use thepinch-to-zoom gesture in our visualizations.The naive implementation of pinch-to-zoom in immersive environ-ments (i.e., only scaling the size of the object) shifts the positions ofthe controllers relative to the object. As a result, inconsistency is intro-duced between the interaction and the displayed information, whichconfuses the user. To address this issue, we integrate rotation into thesame gesture, i.e., the rotation of the manipulated object is based on thedirection between the two touch points while zooming. Additionally,we apply adjusted 3D translation to the object to ensure the positionsof the controllers relative to the zoomable object are preserved in theinteraction (i.e., latching ). The simultaneous zoom and rotate gesturein the 3D immersive environment is demonstrated in Fig. 5. A similarconcept is also widely used on 2D multi-touch zooming interfaces, e.g.,photo editing interfaces on mobile phones.
SER S TUDY
We pre-registered our user study at https://osf.io/ycz5x . We alsoinclude results of statistical tests as supplementary materials. Testconditions are also demonstrated in the supplemental video.
In addition to the two main effects discussed in Sec. 3, we summarizethe characteristics and parameters of test conditions in Tab. 1. Rm : To allow the user to explore finely-detailed data, we take advantageof the large display space in immersive environments and create a room-sized visualization, see Fig. 1(a). Room-sized design in immersiveenvironments is considered to be more immersive than table-sizedvisualizations [32, 73]. A few types of room-sized visualizations havebeen explored, e.g., node-link diagrams [34], egocentric globes [73],and scatterplots [32, 52]. Room-sized visualizations are expected toscale better for representing finely detailed data. However, only sub-parts of the visualization can be seen by the viewer at one time, so thatthe user may lose context during exploration.We use a 3D cube of 2 × × Factor
Rm RmO Zm ZmO
Overview+Detail ✘ ✔ ✘ ✔ Zooming ✘ ✘ ✔ ✔
Visualization
Number of views 1 2 1 2 Size 2 m
Overview : 40 cm
Detail view : 2 m 60 cm
Overview : 40 cm
Detail view : 60 cm Resize ✘ Overview : ✘ Detail view : ✘ ✔ Overview : ✘ Detail view : ✔ Grab ✘ Overview : ✔ Detail view : ✘ ✔ Overview : ✔ Detail view : ✔ Locomotion
Physical movement ✔ ✔ ✔ ✔
Pointer teleportation ✔ ✔ ✔ ✔
Overview teleportation ✘ ✔ ✘ ✔ Table 1. Characteristics and parameters of test visualizations. enable this standard functionality in all our tested conditions. This isthe only support for navigation in this condition.
RmO : In this condition, in addition to Rm , we provide an overview ofthe display space, see Fig. 1(b). The detail view of RmO is the same as Rm . The overview is a 3D cube of 40 × ×
40 centimeters when it isenlarged (see Fig. 3). The default size of the overview is 10 × × Zm : The visualization is initially table-sized (60 × ×
60 centimeters),and participants can use the pinch-to-zoom gesture (described in Sec.3.2) for resizing, see Fig. 1(c). The small initial size allows participantsto have an overview of the information first, which is recognized as thestandard analytic workflow. We also allow participants to grab the viewand manipulate it with a hand-held controller.
ZmO : In this condition, in addition to Zm , we provide an overview of thezooming view, see Fig. 1(d). The detail view of ZmO is the same as Zm (with an initial size of 60 × ×
60 centimeters), and the overview is thesame as it is in
RmO (with an enlarged size of 40 × ×
40 centimetersand default size of 10 × ×
10 centimeters).
We used a Samsung Odyssey virtual reality headset with a 110° fieldof view, 2160 × × ).Following feedback from our pilot user study and as a commonpractice for reducing motion sickness in VR [42, 73], we created anexternal reference frame, which is a 4 × Rm and the detail view of RmO shared the same center of the physicalroom. Their orientations were identical in the whole study. Zm and thedetail view of ZmO were placed 50 centimeters in front of the user’shead position and 50 centimeters below it. This setup allows users toeasily reach the visualization and keep the full visualization within theirFoV at the beginning of each trial. We repositioned and resized thevisualization at the beginning of every trial. We also asked participantsto move back to the center before the start of every trial.
We used MNIST [38], a real-world handwritten digits database to gen-erate point cloud data. For each data set, we first randomly sampled5,000 images as data points , and then used t-SNE [40] to calculate theirprojected 3D positions (i.e., 5,000 points per scatterplot). The t-SNEtechnique projects high-dimensional data into two or three dimensions.In our case, we used TensorFlow’s projection tool [64] as the implemen-tation of t-SNE to project 784 dimensions (28 ×
28 pixels) per imageto three dimensions. We kept the default parameters and executed 800iterations per data set, which has been tested to be sufficient for getting ig. 6. Example stimuli for three task conditions. The labels are only for demonstration purposes. (a,b) Distance: participants had to estimate whichpair of colored points (yellow or red) has the larger spatial distance. The distance within each pair varies in (a) and (b) to be relatively close and farrespectively. (c) Count: participants had to find which group of colored points (yellow, red or blue) has the largest number of points. a stable layout. We used different data sets for all trials. In total, wegenerated 60 data sets (4 for visualization training trials, 16 for tasktraining trials, and 40 for study trials). All points are colored gray,except the red-, yellow-, and blue-colored targets in tasks. Points inthe scatterplot were rendered as spheres with 3 cm diameter in Rm andthe detail view of RmO . The size of the points was proportional in otherrepresentations. Sample data is shown in Fig. 6.
Sarikaya and Gleicher proposed a task taxonomy for scatterplots. Thisidentified three types of high-level tasks: object-centric, browsing, andaggregate-level [58]. Instead of investigating fine detailed informa-tion, browsing and aggregate-level tasks are looking at general patterns,trends or correlations. These types of tasks require relatively less navi-gation effort, and participants preferred small-sized display spaces forsome of these tasks [32]. To better understand the navigation perfor-mance, we intended to select tasks that require relatively significantnavigation efforts. For our user study, we chose two object-centric tasksthat require the participant to navigate to the fine detailed data.
Distance:
Which of the point pairs are further away from each other:the red pair or the yellow pair?
Comparing distance is representativeof a variety of low-level visualization tasks, e.g., identifying outliersand clusters. These tasks are also essential parts of high-level analy-sis processes, e.g., identifying misclassified cases and understandingtheir spatial correlation with different nearby classes in the embeddingrepresentation of a machine-learning system. Variations of this taskhave been studied in most of the studies we reviewed [2, 32, 52, 67, 70].Among them, we directly adopted this task from Bach et al [2]. Similarto their study, we had two target pairs: one pair was colored red, andthe other pair was colored yellow (see Fig. 6 (a,b)). The point cloudwas dense yet sparse enough that the target points could be identifiedwithout the need for interactions other than changing the viewing direc-tion. In all conditions, the participants had to first search for the targetsand then compare their 3D distance by moving/teleporting around thespace and/or rotating the visualization when available (see Tab. 1).Participants needed to choose from two choices: red or yellow.Whether the two points in a pair can be both presented in the FoVcan be a key factor affecting the performance in immersive visualiza-tions [73]. We investigated this factor by creating two categories ofdistance:
Close and
Far . In
Close , the larger distance of the pair wascontrolled to be 25% of side length of the view (e.g., 0.5 meters in Rm ).In Far , we controlled this parameter to be 75% (e.g., 1.5 meters in Rm ).We also controlled the difference between the distances of the two pairsto be 10%. We expect a small difference can potentially encourageparticipants to verify their answers from different viewing positions anddirections and thus increase the number of navigations. For the samereason, we placed each pair far away from the other pair (i.e., they had adistance of 75% of the side length of the view, for example, 1.5 metersin Rm ). We developed an automatic strategy to select the points to meetall controlling requirements. We first repeatedly randomly select a pairof points from the 5000 points in a data set until the pair meets thedistance requirement for Close or Far and then keep randomly selectingthe other pair until all other requirements are met.
Count:
Which group has the largest number of points: the red group,the yellow group, or the blue group?
This task is essential for pro-cesses that require the understanding of numerosity, e.g., counting thenumber of misclassified cases or cases with specific properties for aclassification system. Again, variations of this task have been studiedin some of the studies we reviewed [2, 67, 70]. We created three groupsof points, which were colored in red, yellow, and blue (see Fig. 6 (c)).The points in a group were close to each other to form a small cluster.Points could partially overlap with each other, but we made sure thatthe number of points is unambiguous. The participant has to first searchfor the groups and then sequentially get close to each group to countthe number of points within that group. In all conditions, participantscan move or teleport around the space. In Zm and ZmO , participants alsoneed to enlarge the view to count the points. Participants are not ableto count the points in the overview in
RmO and
ZmO due to its small size.Participants need to choose from three choices: red, yellow, or blue.The number of points in a group varied from 5 to 10. Again, toincrease the potential number of navigation steps needed to completethis task, we placed groups far apart from each other (approximate 50%of the side length of the view, that is, e.g., one meter in Rm ). Unlike inthe Distance tasks where we used an automatic process to select targets,in the Count task, such method may produce ambiguous overlappingcases. Instead, we manually selected groups of points for each trial. We recruited 20 participants (14 females and six males) from HarvardUniversity. All had a normal or corrected-to-normal vision, were right-handed, and all were college students. All participants were withinthe age range of 20 −
30. VR experience varied: three participantshad no experience before this user study; ten participants had 0 − −
20 hours experience, andthree participants had more than 20 hours of experience. Most ofour participants do not play computer/video/mobile games frequently:17 participants reported they played less than 2 hours of games perweek, and the other three participants played 2 − The experiment followed a full-factorial within-subject design. Weused a Latin square (4 groups) to balance the visualizations, but keptthe ordering of tasks consistent:
Distance then
Count . The experimentlasted 1.5 hours on average. Each participant completed 40 study trials:4 VR conditions × (3 Distance-Close + Distance-Far + Count )Participants were first given a brief introduction to the experimentand VR headset. After putting on the VR headset, we asked themto adjust it to see the sample text in front of them clearly. We thenconducted a general VR training session to teach participants how tomove in VR space and how to manipulate a virtual object. First, weasked participants to move for a certain distance physically. Thenwe told them to touch the touchpad to enable the pointer and clickthe touchpad to teleport. We asked participants to get familiar withthe pointer teleportation with a few more teleportations. At the finalstage of the pointer teleportation training, we asked them to teleporto a place marked with a green circle on the floor. We then askedthe participants to grab a green cube by putting the controller insidethe cube and holding the trigger button. The participants finished thetraining session by placing the green cube at a new indicated positionwith a specific rotation. All participants completed the training andreported that they were familiar with the instructed interactions. Thetraining session took around 5 minutes.We conducted a visualization training session every time a partic-ipant encountered a visualization condition for the first time. In thetraining session, we introduced the available interactions and askedparticipants to get familiar with them with no time limit. Each con-dition (visualization × task) started with 2 training trials followed bytimed study trials. Before each trial, we re-positioned participants tothe room’s center and faced them in a consistent direction. In the train-ing trials, participants were not informed about specific strategies forcompleting the task but were encouraged to explore their own strategies.The correct answer was presented to them after they had selected ananswer in the training trials. If a participant answered incorrectly, weasked the them to review the training trial and verify their strategies.After each task, participants were asked to fill in a questionnaireregarding their strategies in each visualization, their subjective ratingsof confidence, mental and physical demands of each visualization, andto rank the visualizations based on their preference. We had a 5-minutebreak between two tasks. After completing two tasks, participants wereasked to fill another survey rating the overall usability and discussingthe pros and cons of each visualization. The demographic informationwas collected at the end of the user study. The questionnaire listedvisualizations in the same order as presented in the experiment. We measured time from the first rendering of the visualization to adouble-click of the controller trigger. After the double-click, the visual-ization was replaced by a multiple-choice panel with task descriptionand options. Participants’ choice was compared to the correct answerfor their accuracy . We recorded the position and rotation of the headset,controllers, and visualizations every 0.2 seconds. We also recordedthe number of different interactions participants conducted in eachstudy trial, including teleportation and zooming . We also collectedthe overview usage percentage , which is the percentage of time theparticipant was looking at the overview of each study trial. The sizeof both Zm and ZmO were also collected every 0.2 seconds. In the pilotstudy, we also asked participants to report the level of motion sicknessthey experienced in each condition. All participants reported the mini-mal level of motion sickness for all conditions. This could be becausethat the participant’s FoV was not fully occupied at any time, and theparticipant could easily access the visual reference (the floor). Thus,we decided to not record the motion sickness level in the formal study.
For dependent variables or their transformed values that can meetthe normality assumption, we used linear mixed modeling to evaluatethe effect of independent variables on the dependent variables [8].Compared to repeated measure ANOVA, linear mixed modeling iscapable of modeling more than two levels of independent variables anddoes not have the constraint of sphericity [24, Ch. 13]. We modeled allindependent variables and their interactions as fixed effects. A within-subject design with random intercepts was used for all models. Weevaluated the significance of the inclusion of an independent variableor interaction terms using log-likelihood ratio. We then performedTukey’s HSD post-hoc tests for pair-wise comparisons using the leastsquare means [39]. We used predicted vs. residual and Q — Q plots tographically evaluate the homoscedasticity and normality of the Pearsonresiduals respectively. For other dependent variables that cannot meetthe normality assumption, we used a
Friedman test to evaluate theeffect of the independent variable, as well as a Wilcoxon-Nemenyi-McDonald-Thompson test for pair-wise comparisons. Significancevalues are reported for p < . ( ∗ ) , p < . ( ∗∗ ) , and p < . ( ∗ ∗ ∗ ) ,respectively, abbreviated by the number of stars in parenthesis. Fig. 7. Results for time (seconds) and accuracy by task. Confidence in-tervals indicate 95% confidence for mean values. A dashed line indicatesstatistical significance for p < . .Fig. 8. Camera movement distance per task trial. A dashed line indicatesstatistical significance for p < . . ESULTS
In this section, we first summarize self-reported strategies, then providea pairwise comparison of performance (task time and accuracy) withthe different visualization conditions. Finally we discuss interactions,user preference, and qualitative feedback.
We asked participants to describe their strategies after each task. Wefound participants’ strategies were relatively consistent.
The distance task: In Rm , most participants (14 out of 20) stayedwithin the visualization space, within which, four participants explicitlymentioned that they used the pointer to “ jump ” to the center of a pairof points as well as the center of two pairs. Six other participants statedthey teleported outside the visualization space to have a better overviewfirst, and then teleported back to the visualization space.In RmO , most participants (13 out of 20) mentioned they mainly usedthe overview to find points. Eight participants used the overview toestimate the distance first, and then confirmed the answer in the detailview. 11 participants reported that they used the overview to teleport.Four participants used the same strategy they used in Rm to teleport tothe center of a pair of points as well as the center of two pairs.In Zm , most participants (17 out of 20) tried to find points in thesmall-sized view, and then compared the distance with an enlargedview. Other participants completed tasks with only small-sized views.In ZmO , most participants (18 out of 20) used the most popularstrategy in Zm , i.e., using the small-sized view to find points, andcomparing distances with an enlarged view. In which, seven participantsmentioned that they sometimes used the overview to teleport when thevisualization was enlarged. Two other participants kept the visualizationsmall all the time to answer the questions. The count task: In Rm , most participants (14 out of 20) mainly phys-ically walked to the place of interest in the space while the other sixparticipants reported they mainly used pointer teleportation.In RmO , most participants (17 out of 20) mainly used the overview toteleport. Two other participants mainly used pointer teleportation, andone participant mainly walked. ig. 9. Number of interactions per task trial. The number of interactions is considered to be the sum of number of teleportations and number ofzoomings. A dashed line indicates statistical significance for p < . .Fig. 10. Average usage of the overview per trial. A dashed line indicatesstatistical significance for p < . . In Zm , most participants (16 out of 20) zoomed in and out to reachdifferent groups. Three other participants first enlarged the visualiza-tion, then grabbed and moved the view to get to groups. One participantfirst zoomed in and then used the pointer to teleport to groups.In ZmO , most participants (14 out of 20) zoomed in and out to com-plete the task and stated that they did not use the overview much. Sixother participants first enlarged the visualization, and then used theoverview to teleport to groups. Rm vs. RmO : We found Rm was faster in the Count task ( ∗∗ ). Rm alsotended to be faster in the other tasks, but the differences were notstatistically significant. We also found RmO was more accurate in theDistance-Close task ( ∗ ), see Fig. 7. We believe the improved accuracymay come from the fact that the overview provided a different perspec-tive as well as a different scale for the participants to confirm theiranswers. Eight participants explicitly reported using the overview forthe distance comparison (see Sec. 5.1). We also found participants feltmore confident with RmO than Rm in the Distance task ( ∗∗ , see Fig. 14)which aligned with their higher accuracy in RmO . Zm vs. ZmO : We found similar performance between Zm and ZmO , exceptthat
ZmO was slower than Zm in the Distance-Far task condition ( ∗ , seeFig. 7). The very similar performance may because participants onlyuse the overview occasionally in ZmO . Apart from the overview, Zm and ZmO share the same view and interactions. The limited use ofthe overview in
ZmO can be confirmed from both the users’ strategy(see Sec. 5.1), and the fact that participants only spent around 10% onaverage of their time looking at the overview (see Fig. 10). This findingaligned with the results from Nekrasovski et al. [47] where they foundthat presence of an overview did not affect the performance of a 2Dzoomable hierarchical visualization (rubber sheet).
Summary:
Overall, adding an overview can increase task accuracy insome tasks, but may also introduce extra time cost. We also found anoverview seems to be unnecessary in Zm , and adding one may even bedistracting or disturbing for difficult tasks (e.g., the Distance-Far task).Due to the very similar performance between Zm and ZmO , we did notinclude
ZmO in the following pair-wise comparisons explicitly. Zm vs. Rm ) We found Zm was slower than Rm in the Count task ( ∗∗∗ ). It tended to beslower in the Distance-Close task but not significantly. Zm and Rm hadsimilar time performance in the Distance-Far task. Our results partiallyalign with the results from Lages and Bowman [35]. They found that theperformance of walking and object manipulation in VR was affectedby the gaming experience of participants. In essence, participantswithout significant gaming experience performed better with physicalmovement, while 3D manipulation enabled higher performance forparticipants with gaming experience. Most of our participants reported having no significant gaming experience (17 out of 20 play less than2 hours of games per week). However, due to the limited number ofparticipants with gaming experience in our user study, we are unable todraw statistical conclusions about this effect. Zm vs. RmO ) We found Zm and RmO had similar time performance in the easier tasks(i.e., the Distance-Close and Count tasks), and Zm was faster than RmO in the difficult task (i.e., the Distance-Far tasks, ∗ , see Fig. 7). Previoususer studies on 2D displays also found a similar time performance be-tween these two types of interfaces (e.g., [51, 55]) in simple navigationtasks. We also found RmO was more accurate in the Distance-Close task( ∗ ). This finding is partially aligned with the study by Plumlee andWare [51] where they also found overview+detail increased the accu-racy compared to a zooming interface on a 2D display. They suggestedthat the benefit may be due to reduced visual working memory load byhaving an extra view in the overview+detail interface. We found that participants had significantly more camera movementin Rm and RmO than in Zm and ZmO (all ∗ ∗ ∗ ). Rm also required morecamera movement than RmO in the Distance-Far task ( ∗ ∗ ∗ ) (see Fig. 8).Motion parallax is a likely explanation. It is key to depth perceptionin immersive environments: a stronger cue than stereopsis, as well asbeing key to resolving occlusion. As the size of visualization increases,you have to move further to get the same motion parallax benefits.
Fig. 11. Size of the zoomable visual-ization per trial.
We also found participantsteleported significantly more in Rm and RmO than in Zm and ZmO (all ∗ ∗ ∗ ). Rm also requiredmore camera movement than RmO in the Distance-Close andDistance-Far tasks ( ∗ ∗ ∗ ). Par-ticipants also performed signif-icantly more zooming interac-tions in Zm than ZmO in theCount task ( ∗ ∗ ∗ ). The size ofthe zooming interface was shown in Fig. 11.We also added up the number of teleportation and zooming stepsas the number of interactions. We found participants performed moreinteractions in Rm than in RmO ( ∗ ) and Zm ( ∗ ∗ ∗ ) in the Distance-Fartask. We also found Zm and ZmO required more interactions than Rm and RmO in the Count task (all ∗ ∗ ∗ ).In summary, we found that overview or zoom reduced the number ofrequired movements and teleportation compared to standard locomotionsupport alone. We also found Zm and ZmO needed a significant amountof pinch-to-zoom interactions in the Count tasks.
We asked participants to rank visualizations according to their prefer-ence for each task (see Fig. 12). For both the Distance and Count tasks,we found participants preferred Zm ( ∗ ∗ ∗ ) and RmO ( ∗ ) over Rm . We alsofound Zm was preferred over RmO ( ∗ ). Participants also rated the overallusability (see Fig. 13). We found the Rm was considered to have lowerusability than RmO ( ∗∗ ), Zm ( ∗ ∗ ∗ ) and ZmO ( ∗ ). Zm tended to be the most preferred visualization in our user studywith more than 50% of participants ranking it best in both tasks (seeFig. 12). Zm was also reported to be less demanding (see Fig. 14). Rm was not preferred by our participants, even though it generally ig. 12. User preference ranking of each condition for two tasks (Distanceand Count). Dashed lines indicate p < . .Fig. 13. Overall usability ratings. Dashed lines indicate p < . . Percent-age of negative and positive rankings is shown next to the bars. performed well (see Fig. 7). There could be two possible reasons: First ,participants felt Rm was more physical demanding (see Fig. 14) andthe recorded movement data confirmed their subjective feeling (seeFig. 8). Second , with a fixed large scale single-view visualization, Rm was expected to have a high visual working memory load. Thehigher number of interactions and movements in Rm partially supportsthis assumption. Subjectively, participants also rated Rm to be morementally demanding than Zm ( ∗ ) and ZmO ( ∗ ) in the Count task. We asked participants to give feedback on the pros and cons of eachdesign. We clustered comments into groups for each visualization.In this subsection, we demonstrate representative ones along with thenumber of participants mentioned these similar comments. Rm was mentioned to be “close to real life” by 11 participants.Among them, four participants explicitly reported it to be “immersive” ,three participants felt “more engaged” , and three participants liked itsfixed view: “it is easier to remember the points, whether in front orbehind me.” However, ten participants also reported “it is difficult tofind the points sometimes.”
RmO was considered an improvement over Rm . 15 participants re-ported that “the minimap was really useful for finding the points andmoving around.” Three participants also stated that “it is really good tohave two scales [of views] at the same time.”
However, two participantsfelt “overwhelmed” by the interactions. Another two complained that “it breaks spatial continuity [when teleporting with the overview].” Zm was found to be “intuitive” and “easy to use” by 11 participants.Among them, three participants mentioned that they like the “flexibil-ity” of the interaction and feel they have “more control” . One alsocommented that “it solves the problem of distance in a continuous way.Without losing the reference.” However, three commented it to be “notfeeling real [comparing to Rm and RmO ].”
Two others mentioned: “Imay lose perspective after I move or zoom the view multiple times.”
ZmO was mainly compared to Zm . Six participants found “the min-imap was good to jump around.” However, 12 participants commentedthat “I did not use the minimap much.”
Five participants also com-plained that “it can be confusing as you have too many choices.”
Of the four navigation methods that we tested, we found some signifi-cant differences in participant performance across the different tasks.However, there was no one navigation method that was best for ev-ery task. The overview increased accuracy for the Rm condition in theDistance-Close task, however, the overview seemed to be an unneces-sary distraction in the other tasks, and provided no benefit to the Zm condition. Zm was faster in the most difficult task (i.e., the Distance-Fartask). Participants also clearly did not like the Rm condition. ISCUSSION AND N AVIGATION T IME -C OST M ODEL
Our study did not find a single best navigation method. In particular,analysis of the time data did not reveal a clear winner: for instance, Zm was faster in the Distance-Far task but slower in the Count task.Furthermore, we found that Rm performed generally well in terms of Fig. 14. Confidence, mental demand, and physical demand ratings ina five-point-Likert scale for two tasks. Dashed lines indicate p < . .Percentage of negative and positive rankings is shown next to the bars. time, but participants, for instance, complained about the difficulty offinding targets. This should have introduced extra time costs, but thiswas not reflected in the overall time.To provide a more nuanced understanding of these mixed results,we now provide an initial exploratory analysis of the timing re-sults based on previously suggested models of time cost for naviga-tion [13, 48, 50, 51] and interactive visualization [36, 69]. Navigationis a complex process with multiple components and the relevance ofthese components will vary in different tasks. By considering the costof each component separately, we hope to better explain our results.For example, while participants might spend more time identifyingthe targets with Rm , they may spend less time on other components,thereby compensating for this loss. Based on the literature, the fourmost essential components in models of navigation time-cost are: Wayfinding (term from [13, 48]): This is the process of finding thedestination. Similar to decision costs to form goals in [36].
Travel (term from [13, 48]): This is the process of “moving” to thedestination. It can be walking, teleportation or manipulating the visual-ization to the desired form. Similar to physical-motion costs to executesequences in [36] and transit between visits in [50, 51]
Number-of-travels (term from [50,51]): Due to limited visual workingmemory [69], completing a task can involve more than one travel toaccess information or to confirm the answer.
Context-switching (term from [69]): When the perceived viewchanges (either through physical movement or manipulating the vi-sualization), the user must re-interpret it based on expectation. Similarto view-change costs to interpret perception in [36].In the rest of this section, we first analyze our visualization condi-tions with our navigation time-cost model. We then discuss the relativeimportance of the components in our two tasks. Finally, we summarizeour discussion by demonstrating how we can use our model to suggestvisualization techniques for different tasks. We also demonstrate howwe can use this model to identify the specific performance bottleneckof a visualization, and then propose potential strategies to improve it.
In this subsection, we analyze the time cost (or performance) of ourvisualization conditions in terms of the above components. The resultsof our analysis are demonstrated in Fig. 15.
Wayfinding:
The overview in
RmO could better facilitate the processof identifying targets compared to Rm . Participants’ comments con-firmed this, where 12 out of 20 mentioned “it is much easier to findthe points [in RmO ] with the minimap [, rather than in Rm ].” For thesame reason, we believe that the overview in
ZmO can better supportidentifying targets compared to Zm , especially when the visualizationis enlarged. Zm supports searching targets in its small-sized state well.This is confirmed by the users’ strategy where most participants locatedtargets with size-reduced (or zoomed-out) Zm (see Sec. 5.1). However,participants lose the overview once they enlarge the visualization. Par-ticipants clearly had difficulties in finding the targets in Rm (see Sec. 5.7).In summary, we suggest that the time-cost of our tested visualizationsin wayfinding is ordered as: RmO < ZmO < Zm < Rm . ig. 15. Navigation time-cost of our tested conditions broken down intofour navigation components. Positions are relative and qualitative, notbased on precise metrics. Travel:
We believe, in our relatively small-sized testing environment,physical movement in Rm might take less time than Zm . Familiaritywith natural walking could make Rm outperform the relatively unnat-ural pinch-to-zoom gesture of Zm (i.e., people are unable to rescale aphysical object in real life). This assumption was partially confirmedby previous studies (e.g., the ones reviewed by Ball et al. [6]) wherephysical movement gives better time performance using visualizationson large tiled displays. In Rm , apart from physical movement, we alsoprovide pointer teleportation. However, pointer teleportation can onlyease transit if the destination is within the users’ FoV. In RmO , the userscan teleport to a place outside their FoV using the overview. Comparedto Rm , RmO has a more flexible teleportation mechanism, which shouldresult in a faster transit. For the same reason, we believe
ZmO couldoutperform Zm in travel. In summary, we suggest that the time-cost ofour tested visualizations in travel is ordered as: RmO < Rm < ZmO < Zm . Number-of-travels:
We use recorded interaction data as a proxy mea-sure of the number-of-travels. Overall, Rm clearly required significantlymore physical movement and teleportations than other visualizations(see Sec. 5.5). RmO also required more physical movement and telepor-tation than Zm (see Sec. 5.5). Although Zm required a large number ofzooming in the Count task, we believe RmO required an overall largernumber of travels. We found that the camera movement was similarin Zm and ZmO . Meanwhile,
ZmO required more teleportation, whilethe number of performed zooming interactions was more in Zm . Insummary, we suggest that our tested visualizations in number-of-travelsis ordered as: Zm ≈ ZmO < RmO < Rm . Context-switching:
Physical movement is a spatial-continuous activ-ity, which we consider to induce minimal context-switching costs. Wealso consider the “pinch-to-zoom” gesture a spatial-continuous trav-eling method which has similar performance compared to physicalmovement. Instant movement by teleportation introduces spatial dis-orientation and discontinuity [3, 11]. Furthermore, compared to themore predictable pointer teleportation where the destination is usuallywithin the FoV, teleportation with the overview is expected to have ahigher cost. Apart from teleportation, a user can also move based on theinformation in the overview (e.g., a user can identify the target is rightof the current viewing direction in the overview and then turn right tofind the target). This operation is also expected to be high-cost, as theuser needs to visually link two separate display spaces. In Rm , partici-pants performed more teleportations than in Zm (see Fig. 9), which weconsider inducing greater context-switching costs. Participants alsospent more time with the overview in RmO than in
ZmO (see Fig. 10).In summary, we suggest the time-cost of our tested visualizations incontext-switching is ordered as: Zm < Rm < ZmO < RmO . Based on the quantitative interaction data, qualitative feedback, and ourobservations in the user study, we infer the relative importance of thetime-cost model components for our tested tasks (see Tab. 2).In the Count task, the targets are groups of points, so they are easyto find. Therefore, we believe that participants required minimal effortin wayfinding, and the number of travels is near identical across testedconditions. The context-switching effort should also be relatively low,as points in one group were all within the FoV. Thus, participants didnot require frequent switching of views. Participants still needed toswitch views when moving to the next target group, but the number ofsuch switches is relatively low.
Task
Wayfinding Travel
Medium High Medium Medium
Distance-Far
High High High High
Count
Low High Low Low
Table 2. Relative importance of time-cost components for tasks.
For the Distance task, there are only four colored targets, so thetarget points were more difficult to locate. Participants had to keepchanging their viewing position and direction to find them. Moreover,the targets were mostly not within the participant’s FoV at the sametime, so participants had to switch views frequently to perform thecomparison. The recorded interaction data shows that the Distance-Far task required more physical movement than the other two taskconditions in Rm ( ∗ ∗ ∗ ), RmO ( ∗ ∗ ∗ ), and ZmO ( ∗ ).In summary, we suggest that the effort required for wayfinding,number-of-travels, and context-switching was higher in the Distancetask than in the Count task. Within the Distance task, the Far conditionrequires more effort in these three components than the Close condition.Travel is an essential part in all navigation tasks, and we consider it tohave a high weight in all conditions. We demonstrate the potential of our navigation time-cost model torecommend visualization techniques for different tasks. We do this byexplaining the overall time performance using the analysis results fromSec. 6.1 and 6.2.For tasks that are less demanding on wayfinding and number-of-travels (e.g., the Distance-Close and Count tasks in our study), Rm isexpected to have a good performance. This is because although Rm is not good at these two components, they have a limited amount ofinfluence. On the other hand, Rm shows an overall good performanceon the more important component (i.e., travel). For tasks that requirea significant effort in number-of-travels and context-switch (e.g., theDistance-Far task in our study), Zm is a good choice, as it outperformsother conditions on these two components. RmO has its advantages for wayfinding and travel, but the high cost incontext-switching significantly influences its performance. For futurestudies, we should consider techniques that reduce the effort for context-switching in
RmO , e.g., animated teleportation [10, 63], or instead ofalways teleporting forwards, allowing users to interactively choose theirviewing direction of the teleportation destination [27]. We also proposea preliminary idea that when the user selects a target in the overview,a visual indicator will appear in the detail view to guide the user tothe target. This is inspired by the work from Petford et al. [49], whichreviewed guiding techniques for out-of-view objects.
ONCLUSION AND F UTURE W ORK
We would recommend that developers of immersive visualization sys-tems provide a variety of navigation methods to suit different tasks andenvironments. For example, if the user has the capability to operatein a large open space, then there are definitely tasks (such as DistanceClose) that will benefit from room-size navigation. However, in seatedVR, the zoom is going to be essential. Our adaptation of the traditional overview technique may be useful in room-size navigation for tasksthat require operation at different scales, but such an overview shouldbe easy to hide until required, to prevent distraction.For future studies, a larger tracking space would support greaterphysical navigation but may also cause significant fatigue. Larger,more complex data may benefit the overview more. We also suggestthat designs that can reduce context-switching cost in overview+detailinterfaces are likely to improve its performance. We also would like todesign studies to verify our navigation time-cost model systematically. A CKNOWLEDGMENTS
This research was supported in part under KAUST Office of SponsoredResearch (OSR) award OSR-2015-CCF-2533-01 and Australian Re-search Councils Discovery Projects funding scheme DP180100755.Yalong Yang was supported by a Harvard Physical Sciences and Engi-neering Accelerator Award. We also wish to thank all our participantsfor their time and our reviewers for their comments and feedback.
EFERENCES [1] P. Abtahi, M. Gonzalez-Franco, E. Ofek, and A. Steed. I’m a Giant: Walk-ing in Large Virtual Environments at High Speed Gains. In
Proceedingsof the 2019 CHI Conference on Human Factors in Computing Systems- CHI ’19 , pp. 1–13. ACM Press, Glasgow, Scotland Uk, 2019. doi: 10.1145/3290605.3300752[2] B. Bach, R. Sicat, J. Beyer, M. Cordeil, and H. Pfister. The Hologramin My Hand: How Effective is Interactive Exploration of 3D Visualiza-tions in Immersive Tangible Augmented Reality?
IEEE Transactions onVisualization and Computer Graphics , 24(1):457–467, Jan. 2018. doi: 10.1109/TVCG.2017.2745941[3] N. H. Bakker, P. O. Passenier, and P. J. Werkhoven. Effects of Head-Slaved Navigation and the Use of Teleports on Spatial Orientation inVirtual Environments.
Human Factors: The Journal of the Human Factorsand Ergonomics Society , 45(1):160–169, Mar. 2003. doi: 10.1518/hfes.45.1.160.27234[4] R. Ball and C. North. Effects of tiled high-resolution display on basicvisualization and navigation tasks. In
CHI’05 extended abstracts onHuman factors in computing systems , pp. 1196–1199, 2005.[5] R. Ball and C. North. The effects of peripheral vision and physical navi-gation on large scale visualization. In
Proceedings of graphics interface2008 , pp. 9–16, 2008.[6] R. Ball, C. North, and D. A. Bowman. Move to improve: promotingphysical navigation to increase user performance with large displays. In
Proceedings of the SIGCHI Conference on Human Factors in ComputingSystems - CHI ’07 , pp. 191–200. ACM Press, San Jose, California, USA,2007. doi: 10.1145/1240624.1240656[7] A. Batch, A. Cunningham, M. Cordeil, N. Elmqvist, T. Dwyer, B. H.Thomas, and K. Marriott. There is no spoon: Evaluating performance,space use, and presence with expert domain users in immersive analytics.
IEEE transactions on visualization and computer graphics , 26(1):536–546,2019.[8] D. Bates, M. M¨achler, B. Bolker, and S. Walker. Fitting linear mixed-effects models using lme4.
Journal of Statistical Software , 67(1), 2015.doi: 10.18637/jss.v067.i01[9] P. Baudisch, N. Good, V. Bellotti, and P. Schraedley. Keeping things incontext: a comparative evaluation of focus plus context screens, overviews,and zooming. In
Proceedings of the SIGCHI conference on Human factorsin computing systems , pp. 259–266, 2002.[10] J. Bhandari, P. MacNeilage, and E. Folmer. Teleportation without SpatialDisorientation Using Optical Flow Cues. In
Proceedings of the 44thGraphics Interface Conference , GI ’18, pp. 162–167. Canadian Human-Computer Communications Society, Toronto, Canada, June 2018. doi: 10.20380/GI2018.22[11] D. A. Bowman, D. Koller, and L. Hodges. Travel in immersive virtualenvironments: an evaluation of viewpoint motion control techniques. In
Proceedings of IEEE 1997 Annual International Symposium on VirtualReality , pp. 45–52,. IEEE Comput. Soc. Press, Albuquerque, NM, USA,1997. doi: 10.1109/VRAIS.1997.583043[12] D. A. Bowman, D. Koller, and L. F. Hodges. Travel in immersive virtualenvironments: An evaluation of viewpoint motion control techniques. In
Proceedings of IEEE 1997 Annual International Symposium on VirtualReality , pp. 45–52. IEEE, 1997.[13] D. A. Bowman, E. Kruijff, J. J. LaViola, and I. Poupyrev.
3D UserInterfaces: Theory and Practice . Addison Wesley Longman PublishingCo., Inc., USA, 2004.[14] S. Burigat and L. Chittaro. On the effectiveness of overview+ detailvisualization on mobile devices.
Personal and ubiquitous computing ,17(2):371–385, 2013.[15] S. Burigat, L. Chittaro, and E. Parlato. Map, diagram, and web pagenavigation on mobile devices: the effectiveness of zoomable user interfaceswith overviews. In
Proceedings of the 10th international conference onHuman computer interaction with mobile devices and services , pp. 147–156, 2008.[16] T. B¨uring, J. Gerken, and H. Reiterer. Usability of overview-supportedzooming on small screens with regard to individual differences in spatialability. In
Proceedings of the working conference on Advanced visualinterfaces , pp. 233–240, 2006.[17] M. Card.
Readings in information visualization: using vision to think .Morgan Kaufmann, 1999.[18] S. S. Chance, F. Gaunet, A. C. Beall, and J. M. Loomis. Locomotion modeaffects the updating of objects encountered during travel: The contributionof vestibular and proprioceptive inputs to path integration.
Presence ,7(2):168–178, 1998. [19] A. Cockburn, A. Karlson, and B. B. Bederson. A review of overview+detail, zooming, and focus+ context interfaces.
ACM Computing Surveys(CSUR) , 41(1):1–31, 2009.[20] A. Cockburn and J. Savage. Comparing speed-dependent automatic zoom-ing with traditional scroll, pan and zoom methods. In
People and Comput-ers XVIIDesigning for Society , pp. 87–102. Springer, 2004.[21] M. Cordeil, A. Cunningham, T. Dwyer, B. H. Thomas, and K. Marriott.ImAxes: Immersive Axes as Embodied Affordances for Interactive Mul-tivariate Data Visualisation. the 30th Annual ACM Symposium on UserInterface Software and Technology, pp. 71–83, Oct. 2017. doi: 10.1145/3126594.3126613[22] P. Dourish.
Where the action is . MIT press Cambridge, 2001.[23] A. Drogemuller, A. Cunningham, J. Walsh, B. H. Thomas, M. Cordeil, andW. Ross. Examining virtual reality navigation techniques for 3D networkvisualisations.
Journal of Computer Languages , 56:100937, Feb. 2020.doi: 10.1016/j.cola.2019.100937[24] A. Field, J. Miles, and Z. Field.
Discovering statistics using R . Sagepublications, 2012.[25] A. Fonnet, T. Vigier, Y. Prie, G. Cliquet, and F. Picarougne. Axesand coordinate systems representations for immersive analytics of multi-dimensional data. In , pp. 1–10. IEEE, 2018.[26] J. Fung, C. L. Richards, F. Malouin, B. J. McFadyen, and A. Lamontagne.A treadmill and motion coupled virtual reality system for gait trainingpost-stroke.
CyberPsychology & behavior , 9(2):157–162, 2006.[27] M. Funk, F. Mller, M. Fendrich, M. Shene, M. Kolvenbach, N. Dobbertin,S. Gnther, and M. Mhlhuser. Assessing the Accuracy of Point & TeleportLocomotion with Orientation Indication for Virtual Reality using CurvedTrajectories. In
Proceedings of the 2019 CHI Conference on HumanFactors in Computing Systems - CHI ’19 , pp. 1–12. ACM Press, Glasgow,Scotland Uk, 2019. doi: 10.1145/3290605.3300377[28] K. Hinckley, M. Czerwinski, and M. Sinclair. Interaction and modelingtechniques for desktop two-handed input. In
Proceedings of the 11thannual ACM symposium on User interface software and technology -UIST ’98 , pp. 49–58. ACM Press, San Francisco, California, United States,1998. doi: 10.1145/288392.288572[29] K. Hornbæk, B. B. Bederson, and C. Plaisant. Navigation patterns andusability of zoomable user interfaces with and without an overview.
ACMTransactions on Computer-Human Interaction (TOCHI) , 9(4):362–389,2002.[30] C. Hurter, N. H. Riche, S. M. Drucker, M. Cordeil, R. Alligier, andR. Vuillemot. Fiberclay: Sculpting three dimensional trajectories to revealstructural insights.
IEEE Transactions on Visualization and ComputerGraphics , 25(1):704–714, 2018.[31] H. Iwata, H. Yano, and F. Nakaizumi. Gait master: A versatile locomotioninterface for uneven virtual terrain. In
Proceedings IEEE Virtual Reality2001 , pp. 131–137. IEEE, 2001.[32] M. Kraus, N. Weiler, D. Oelke, J. Kehrer, D. A. Keim, and J. Fuchs. TheImpact of Immersion on Cluster Identification Tasks.
IEEE Transactionson Visualization and Computer Graphics , pp. 1–1, 2019. doi: 10.1109/TVCG.2019.2934395[33] H. P. Kumar, C. Plaisant, and B. Shneiderman. Browsing hierarchical datawith multi-level dynamic queries and pruning.
International Journal ofHuman-Computer Studies , 46(1):103–124, Jan. 1997. doi: 10.1006/ijhc.1996.0085[34] O.-H. Kwon, C. Muelder, K. Lee, and K.-L. Ma. A Study of Layout,Rendering, and Interaction Methods for Immersive Graph Visualization.
IEEE Transactions on Visualization and Computer Graphics , 22(7):1802–1815, July 2016. doi: 10.1109/TVCG.2016.2520921[35] W. S. Lages and D. A. Bowman. Move the Object or Move Myself?Walking vs. Manipulation for the Examination of 3D Scientific Data.
Frontiers in ICT , 5:15, July 2018. doi: 10.3389/fict.2018.00015[36] H. Lam. A Framework of Interaction Costs in Information Visualization.
IEEE Transactions on Visualization and Computer Graphics , 14(6):1149–1156, Nov. 2008. doi: 10.1109/TVCG.2008.109[37] J. J. LaViola Jr, E. Kruijff, R. P. McMahan, D. Bowman, and I. P. Poupyrev.
3D user interfaces: theory and practice . Addison-Wesley Professional,2017.[38] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learningapplied to document recognition.
Proceedings of the IEEE , 86(11):2278–2324, Nov. 1998. doi: 10.1109/5.726791[39] R. V. Lenth. Least-squares means: The R Package lsmeans.
Journal ofStatistical Software , 69(1), 2016. doi: 10.18637/jss.v069.i01[40] L. v. d. Maaten and G. Hinton. Visualizing Data using t-SNE.
Journal ofachine Learning Research , 9(Nov):2579–2605, 2008.[41] J. D. Mackinlay, G. G. Robertson, and S. K. Card. The perspective wall:Detail and context smoothly integrated. In
Proceedings of the SIGCHIconference on Human factors in computing systems , pp. 173–176, 1991.[42] M. McGill, A. Ng, and S. Brewster. I Am The Passenger: How VisualMotion Cues Can Influence Sickness For In-Car VR. In
Proceedings ofthe 2017 CHI Conference on Human Factors in Computing Systems - CHI’17 , pp. 5655–5668. ACM Press, Denver, Colorado, USA, 2017. doi: 10.1145/3025453.3026046[43] M. R. Mine. Isaac: A virtual environment tool for the interactive construc-tion of virtual worlds. Technical report, USA, 1995.[44] M. R. Mine. Virtual environment interaction techniques.
UNC ChapelHill CS Dept , 1995.[45] M. R. Mine, F. P. Brooks, and C. H. Sequin. Moving objects in space:exploiting proprioception in virtual-environment interaction. In
Proceed-ings of the 24th annual conference on Computer graphics and interactivetechniques - SIGGRAPH ’97 , pp. 19–26. ACM Press, 1997. doi: 10.1145/258734.258747[46] J. W. Nam, K. McCullough, J. Tveite, M. M. Espinosa, C. H. Perry,B. T. Wilson, and D. F. Keefe. Worlds-in-Wedges: Combining Worlds-in-Miniature and Portals to Support Comparative Immersive Visualization ofForestry Data. In , pp. 747–755. IEEE, Osaka, Japan, Mar. 2019. doi: 10.1109/VR.2019.8797871[47] D. Nekrasovski, A. Bodnar, J. McGrenere, F. Guimbretire, and T. Munzner.An evaluation of pan & zoom and rubber sheet navigation with and withoutan overview. In
Proceedings of the SIGCHI conference on Human Factorsin computing systems - CHI ’06 , pp. 11–20. ACM Press, Montreal, Qubec,Canada, 2006. doi: 10.1145/1124772.1124775[48] N. C. Nilsson, S. Serafin, F. Steinicke, and R. Nordahl. Natural Walkingin Virtual Reality: A Review.
Computers in Entertainment , 16(2):1–22,Apr. 2018. doi: 10.1145/3180658[49] J. Petford, I. Carson, M. A. Nacenta, and C. Gutwin. A Comparisonof Notification Techniques for Out-of-View Objects in Full-CoverageDisplays. In
Proceedings of the 2019 CHI Conference on Human Factorsin Computing Systems - CHI ’19 , pp. 1–13. ACM Press, Glasgow, ScotlandUk, 2019. doi: 10.1145/3290605.3300288[50] M. Plumlee and C. Ware. Zooming, multiple windows, and visual workingmemory. In
Proceedings of the Working Conference on Advanced VisualInterfaces - AVI ’02 , p. 59. ACM Press, Trento, Italy, 2002. doi: 10.1145/1556262.1556270[51] M. D. Plumlee and C. Ware. Zooming versus multiple window interfaces:Cognitive costs of visual comparisons.
ACM Transactions on Computer-Human Interaction (TOCHI) , 13(2):179–209, June 2006. doi: 10.1145/1165734.1165736[52] A. Prouzeau, M. Cordeil, C. Robin, B. Ens, B. H. Thomas, and T. Dwyer.Scaptics and Highlight-Planes: Immersive Interaction Techniques forFinding Occluded Features in 3D Scatterplots. In
Proceedings of the 2019CHI Conference on Human Factors in Computing Systems - CHI ’19 , pp.1–12. ACM Press, Glasgow, Scotland Uk, 2019. doi: 10.1145/3290605.3300555[53] R. R¨adle, H.-C. Jetter, S. Butscher, and H. Reiterer. The effect of egocentricbody movements on users’ navigation performance and spatial memory inzoomable user interfaces. In
Proceedings of the 2013 ACM internationalconference on Interactive tabletops and surfaces , pp. 23–32, 2013.[54] D. Raja, D. Bowman, J. Lucas, and C. North. Exploring the benefitsof immersion in abstract information visualization. In
Proc. ImmersiveProjection Technology Workshop , pp. 61–69, 2004.[55] M. Rønne Jakobsen and K. Hornbæk. Sizing up visualizations: effectsof display size in focus+ context, overview+ detail, and zooming inter-faces. In
Proceedings of the SIGCHI Conference on Human Factors inComputing Systems , pp. 1451–1460, 2011.[56] R. A. Ruddle and S. Lessels. The benefits of using a walking interface tonavigate virtual environments.
ACM Transactions on Computer-HumanInteraction (TOCHI) , 16(1):1–18, 2009.[57] R. A. Ruddle, E. Volkova, and H. H. B¨ulthoff. Walking improves yourcognitive map in environments that are large-scale and large in extent.
ACM Transactions on Computer-Human Interaction (TOCHI) , 18(2):1–20,2011.[58] A. Sarikaya and M. Gleicher. Scatterplots: Tasks, Data, and Designs.
IEEETransactions on Visualization and Computer Graphics , 24(1):402–412,Jan. 2018. doi: 10.1109/TVCG.2017.2744184[59] K. A. Satriadi, B. Ens, M. Cordeil, B. Jenny, T. Czauderna, and W. Willett.Augmented reality map navigation with freehand gestures. In , pp. 593–603.IEEE, 2019.[60] B. Shneiderman. The eyes have it: A task by data type taxonomy forinformation visualizations. In
Proceedings 1996 IEEE symposium onvisual languages , pp. 336–343. IEEE, 1996.[61] M. Simpson, J. Zhao, and A. Klippel. Take a walk: Evaluating movementtypes for data visualization in immersive virtual reality. In
Workshop onImmersive Analytics, IEEE Vis , 2017.[62] J. Sorger, M. Waldner, W. Knecht, and A. Arleo. Immersive analytics oflarge dynamic networks via overview and detail navigation. arXiv preprintarXiv:1910.06825 , 2019.[63] R. Stoakley, M. J. Conway, and R. Pausch. Virtual reality on a WIM: inter-active worlds in miniature. In
Proceedings of the SIGCHI conference onHuman factors in computing systems - CHI ’95 , pp. 265–272. ACM Press,Denver, Colorado, United States, 1995. doi: 10.1145/223904.223938[64] TensorFlow, Google. Embedding Projector. https://projector.tensorflow.org/ , 2020. Online; accessed March 2020.[65] R. Trueba, C. Andujar, and F. Argelaguet. Complexity and occlusion man-agement for the world-in-miniature metaphor. In
International Symposiumon Smart Graphics , pp. 155–166. Springer, 2009.[66] J. A. Wagner Filho, C. Freitas, and L. Nedel. VirtualDesk: A Comfortableand Efficient Immersive Information Visualization Approach.
ComputerGraphics Forum , 37(3):415–426, June 2018. doi: 10.1111/cgf.13430[67] J. A. Wagner Filho, M. F. Rey, C. M. D. S. Freitas, and L. Nedel. ImmersiveVisualization of Abstract Information: An Evaluation on Dimensionally-Reduced Data Scatterplots. In , pp. 483–490. IEEE, Mar. 2018. doi: 10.1109/VR.2018.8447558[68] J. A. Wagner Filho, W. Stuerzlinger, and L. Nedel. Evaluating an Im-mersive Space-Time Cube Geovisualization for Intuitive Trajectory DataExploration.
IEEE Transactions on Visualization and Computer Graphics ,26(1):514–524, 2019. doi: 10.1109/TVCG.2019.2934415[69] M. Q. Wang Baldonado, A. Woodruff, and A. Kuchinsky. Guidelines forusing multiple views in information visualization. In
Proceedings of theworking conference on Advanced visual interfaces - AVI ’00 , pp. 110–119.ACM Press, Palermo, Italy, 2000. doi: 10.1145/345513.345271[70] Y. Wei, H. Mei, Y. Zhao, S. Zhou, B. Lin, H. Jiang, and W. Chen. Eval-uating Perceptual Bias During Geometric Scaling of Scatterplots.
IEEETransactions on Visualization and Computer Graphics , 26(1):321–331,Jan. 2020. doi: 10.1109/TVCG.2019.2934208[71] C. A. Wingrave, Y. Haciahmetoglu, and D. A. Bowman. Overcomingworld in miniature limitations by a scaled and scrolling wim. In
3D UserInterfaces (3DUI’06) , pp. 11–16. IEEE, 2006.[72] L. Woodburn, Y. Yang, and K. Marriott. Interactive Visualisation ofHierarchical Quantitative Data: An Evaluation. In , pp. 96–100. IEEE, Vancouver, BC, Canada, Oct. 2019.doi: 10.1109/VISUAL.2019.8933545[73] Y. Yang, B. Jenny, T. Dwyer, K. Marriott, H. Chen, and M. Cordeil. Mapsand Globes in Virtual Reality.