Comparative Layouts Revisited: Design Space, Guidelines, and Future Directions
CComparative Layouts Revisited:Design Space, Guidelines, and Future Directions
Sehi L’Yi, Jaemin Jo, and Jinwook Seo
Abstract — We present a systematic review on three comparative layouts— juxtaposition , superposition , and explicit-encoding —whichare information visualization (InfoVis) layouts designed to support comparison tasks. For the last decade, these layouts have served asfundamental idioms in designing many visualization systems. However, we found that the layouts have been used with inconsistentterms and confusion, and the lessons from previous studies are fragmented. The goal of our research is to distill the results fromprevious studies into a consistent and reusable framework. We review 127 research papers, including 15 papers with quantitative userstudies, which employed comparative layouts. We first alleviate the ambiguous boundaries in the design space of comparative layoutsby suggesting lucid terminology (e.g., chart-wise and item-wise juxtaposition). We then identify the diverse aspects of comparativelayouts, such as the advantages and concerns of using each layout in the real-world scenarios and researchers’ approaches toovercome the concerns. Building our knowledge on top of the initial insights gained from the Gleicher et al.’s survey [19], we elaborateon relevant empirical evidence that we distilled from our survey (e.g., the actual effectiveness of the layouts in different study settings)and identify novel facets that the original work did not cover (e.g., the familiarity of the layouts to people). Finally, we show the consistentand contradictory results on the performance of comparative layouts and offer practical implications for using the layouts by suggestingtrade-offs and seven actionable guidelines. Index Terms —Comparative layout, visual comparison, literature review, juxtaposition, superposition, explicit-encoding
NTRODUCTION
A decade ago, Gleicher et al. [19] suggested three primitive informa-tion visualization (InfoVis) layouts that support comparison tasks— juxtaposition , superposition , and explicit-encoding —based on theirliterature survey on more than 100 research papers (Fig. 1). Theselayouts have served as fundamental idioms for designing comparativevisualizations in diverse areas such as radiology [54], biology [63], andgeology [1]. The layouts have also been popular in academia, as thenumber of papers citing the comparative layouts shows rapid growth.To develop a better understanding of comparative layouts, re-searchers have attempted to study the effectiveness of the three layoutsby conducting user studies and adopting the layouts to specific do-mains. Gleicher et al. [19] initially discussed the potential strength andweakness of comparative layouts in terms of scalability, cognitive cost,and task performance, followed by many user studies in the Human–Computer Interaction (HCI) field [26, 37, 39, 44, 47, 52, 53, 56, 60].Ondov et al. [47], for example, compared the variants of juxtapositionand superposition, such as using adjacent, symmetric, and animatedarrangements, in identifying the maximum change and correlationbetween two visualizations.However, we find the lessons and practical findings from those pre-vious studies fragmented, sometimes even with inconsistent terms. Forexample, we encounter several visualizations techniques (e.g., variantsof bar charts or heatmaps) that are inconsistently regarded as eitherjuxtaposition or superposition [2, 31, 47, 52, 56, 73]. Moreover, contraryto the general consensus that superposition is more effective for smalldifferences [9, 13, 24], recent studies show that juxtaposition can bemore effective for some tasks, such as comparing global characteristicsbetween two bar charts [26, 47].We present a systematic review on three comparative layouts with127 research papers, including 15 papers with quantitative user studies,which employed the layouts. There are several studies that explored the • Sehi L’Yi is with Harvard Medical School. E-mail:sehi [email protected].• Jaemin Jo is with Sungkyunkwan University. E-mail: [email protected].• Jinwook Seo is with Seoul National University. E-mail: [email protected] received xx xxx. 201x; accepted xx xxx. 201x. Date of Publicationxx xxx. 201x; date of current version xx xxx. 201x. For information onobtaining reprints of this article, please send e-mail to: [email protected] Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx design space of visualization arrangement. Javed et al. [27] classifiedfour design choices for composite visualizations, and Chen et al. [10]suggested a conceptual framework for overlaying visualizations. Themain difference of our work compared with the prior studies is that webase our work on researchers’ empirical findings from a wider-range ofresearch papers. We combine and systematize the insights researchersgained in the wild , for example, during a visualization design process incollaboration with data analysts or in evaluation with the actual users,and distill these studies into a consistent and reusable framework.We first alleviate the unambiguous boundaries between comparativelayouts using lucid classification to give implications in a more system-atic and precise manner (e.g., chart-wise and item-wise juxtaposition ;Fig. 2, Fig. 3, and Fig. 4). We then identify the diverse aspects ofeach layout (Table 1), such as the advantages and concerns of usingthem in the real-world scenarios and the researchers’ approaches toovercome the concerns. Building our knowledge on top of the initialinsights gained from the Gleicher et al.’s survey [19], we suggest rele-vant empirical evidence we discovered from our survey (e.g., the actualeffectiveness of layouts in different study settings) and identify novelfacets that the original study did not discover (e.g., the familiarity oflayouts to people). Moreover, we show the consistent and contradictorystudy results of comparative layouts in terms of different study factorsby agglomerating pairwise relations of layouts from 15 study papers(Fig. 5). Going a step further, we suggest seven actionable guidelinesfor comparative layouts and promising research directions based onour literature review as well as the results of the 15 papers. Finally,we provide a web-based visualization gallery to support designers inexploring the complex design space of comparative layouts using aflexible visualization grammar. LEICHER ET AL .’ S C OMPARATIVE LAYOUT
Throughout the paper, we will use the terms from Gleicher et al. [19] torefer to comparative layouts: juxtaposition , superposition , and explicit-encoding . The three designs describe the arrangement of two or morevisualizations to support comparison tasks. First, juxtaposition refersto placing visualizations next to each other (Fig. 1A). It is sometimescalled spatial juxtaposition to distinguish it from temporal juxtaposi-tion , which temporally separates visualizations, for example, switchingfrom one to another or using animated transition [23]. Superpositionrefers to placing visualizations on top of the other, such as overlayingone line chart on another (Fig. 1B). Finally, explicit-encoding focuseson revealing the predefined relationship between visualizations. For a r X i v : . [ c s . H C ] S e p xample, if the difference between two trends is of interest, one canexplicitly draw the difference on a line chart (Fig. 1C). Note that theexplicit-encoding layout is not limited to creating a new visualizationwith aggregated values but also includes visual elements overlaid on theoriginal visualization (e.g., lines connecting the corresponding pointsin two scatterplots [34]). Designers can also combine multiple layouts(i.e., hybrid layout ), such as overlapping two node-link diagrams (su-perposition) with the common edges and nodes highlighted using adifferent color (explicit-encoding) [44]. Although animated transitionand explicit-encoding are not technically ‘layouts’ (i.e., visualizationarrangement), we call them comparative layouts for the consistencywith Gleicher et al.’s work. ITERATURE R EVIEW
We reviewed 127 research papers that employed the comparative lay-outs to expand our understanding about the layouts.
We reviewed two types of research papers in our survey: (1)
Gen-eral papers that employed and/or discussed comparative layouts buthave not conducted any quantitative user studies and (2)
Study papersthat conducted quantitative studies to compare the performance of thelayouts.
General papers.
We firstly looked into all the 401 publications thatcited the work of Gleicher et al. [19] until March 11, 2020 using GoogleScholar. We then excluded irrelevant papers using the following crite-ria: (1) papers which do not explicitly use comparative layouts and donot present any discussions about them (e.g., some papers mentionedcomparative layouts only to provide high-level contexts of comparativevisualization in introduction), (2) duplicate publications (e.g., thesispapers), and (3) papers written in languages other than English. Lastly,we excluded (4) papers which mainly focusing on scientific visual-ization (e.g., 3D blood flow simulation [63]) to stick to the originalfocus of the comparative layouts [19], that is information visualization(InfoVis). However, we did not exclude scientific visualization papersif they suggested any InfoVis techniques with comparative layouts (e.g.,juxtaposed line charts [69]). After the filtering process, we obtained aset of 112
General papers.We surveyed the following factors from the selected papers, whichwere the main factors discussed in previous papers [18, 19]:• The type of visualizations placed using the layouts• The number of visualizations to compare at once [18]• How each of the comparative layouts [19] is used• How researchers describe the advantages and concerns of usingeach layout• Researchers’ approaches to overcome the concernsTo avoid ambiguity in collecting the usage of the comparative lay-outs, we mainly based our data collection on the authors’ justificationsdescribed in the papers. Even though visualizations are placed adja-cently as in many general visualization systems, we have not regardedthis as using a comparative layout (i.e., juxtaposition) unless the authorsexplicitly stated the proper rationale because it is unclear whether thelayout is used for visual comparison or not. We have not also consid-ered the cases where the different visualization types are placed usingthe comparative layouts because comparison tasks are most likely to betaken with same visualizations. One typical example in our review isthe difference (explicit-encoding) overlaid on top of a grouped bar chart(juxtaposition) [56] (Fig. 2H). In this case, consistent to the authors’explanation, we did not consider it as using an additional superpositionlayout between the juxtaposed bar chart (Fig. 2F) and the explicit-encoding chart (Fig. 2C) because these two charts are not arranged forcomparing the two.
Study papers.
To collect the empirical evidence on the performanceof comparative layouts, we additionally surveyed the papers that (1)
Fig. 1. Three primitive information visualization (InfoVis) layouts for visualcomparison tasks [19]: (A) placing visualizations next to each other ( jux-taposition ), (B) placing visualizations on top of the other ( superposition ),and (C) directly showing the relationship of interests such as subtractionvalues ( explicit-encoding ). presented quantitative user studies, (2) directly compared the usefulnessof the layouts (i.e., comparative layouts being an independent variable),and (3) included visual comparison tasks (e.g., [18]).To complement the relatively small number of papers with quan-titative user studies found from the original target papers (i.e., 401publications that cited Gleicher et al.’s work [19]), we additionallylooked into two more sets of publications after reviewing the originalset. First, we looked into the 110 papers that Gleicher et al. [19] origi-nally reviewed. Second, we reviewed publications that are published atthe following two venues for the last ten years: ACM CHI Conferenceon Human Factors in Computing Systems ( CHI ) and IEEE Conferenceon Information Visualization (
InfoVis ). We chose these two venuessince we found the majority (87.5%) of papers with quantitative userstudies were from these venues. Through this process, we were able tofind 15
Study papers.From the study papers, we collected study conditions (e.g., com-parative layouts used as independent variables, tasks of studies, andthe number of participants) and results, as well as the main factors wecollected from the
General papers (e.g., advantages and concerns ofusing each layout).
OMPARATIVE L AYOUTS IN T HE W ILD
Overall, we found 245 visualization examples from 127 papers thatused comparative layouts (about 1.9 examples per paper). The mostwidely used layout is juxtaposition (106), while superposition (39) andexplicit-encoding (38) are frequently used as well. We also found 55examples that used multiple layouts at once (i.e., hybrid layout). Themost widely used visualization types include bar charts (64), heatmaps(41), node-link diagrams (40), line charts (39), scatterplots (15), andmap visualizations (13). More than half of the examples (57.4%) usedcomparative layouts for comparing a pair of visualizations (i.e., 1:1comparison). The 127 papers have been published at 50 venues; themajority of papers were from IEEE Transactions on Visualization andComputer Graphics (TVCG) (42), ACM CHI Conference on HumanFactors in Computing Systems (CHI) (11), and Computer GraphicsForum (CGF) (10).
Visual comparison tasks are finding and understanding relationshipbetween target visual elements [18, 19], such as their similarity or dif-ference, and they usually involve first identifying the target elementsin visualizations and then retrieving the relationship between them [3].For a more comprehensive examination of the 15 papers with quan-titative user studies, we classified comparison tasks (total 40 tasks)by their scope— global (6 papers) and local (12 papers)—because wefind that these two types of tasks are quite distinguishable in termsof how people perform visual comparison [26].
Global tasks refer tocomparing the overall characteristics of individual visualizations, suchas comparing the correlation of each bar chart. In contrast, local tasks refer to the comparison between visual items, such as comparing thelength of bars in two bar charts. The main characteristic of local taskscompared to global ones is that people must link the correspondingvisual elements between visualizations before actually comparing them(e.g., finding bars of the same category in two distant bar charts) unlessa system explicitly highlights them. On the other hand, global tasks ig. 2. Examples of comparative layouts observed in our literature survey: (A-C) the three primitive comparative layouts (i.e., juxtaposition,superposition, and explicit-encoding); (D) visual linking between adjacently placed bar charts; (E) symmetrically stacked bar charts; (F-G) a groupedbar chart and a stacked bar chart using item-wise juxtaposition; (H) explicit-encoding overlay for showing subtraction values on top of a grouped barchart; (I-K) variants of chart-wise and item-wise juxtaposition for heatmaps; (L) superposed heatmaps using different cell sizes. Note that the designspace of comparative layouts in our classification is not limited to the 12 examples shown in this figure because multiple layouts can be used at oncein different combinations, and they can be used in any visualization types. require global perspectives that people seem to use more diverse per-ceptual heuristics in taking the comparison tasks [26]. We used thistask categorization to explore the consistent and contradictory studyresults of comparative layouts (Fig. 5).
We found that the same arrangement of visualizations are often calleddifferently. One common case is to call a chart with juxtaposed visualmarks (e.g., a grouped bar chart; Fig. 2F) either juxtaposition or su-perposition. For example, Srinivasan et al. [56] called a grouped barchart the juxtaposition layout in that the chart places bars side-by-side.In contrast, Ondov et al. [47] treated the same chart as a superposi-tion layout, considering the chart as multiple bar charts overlaid withdifferent offsets. Similar problems occur in the case of matrix visual-
Fig. 3. A summary of comparative layouts found from 127 papers. Thelayouts were classified into five exclusive categories: chart-wise juxtaposi-tion (CJ), item-wise juxtaposition (IJ), superposition (S), explicit-encoding(E), and animated transition (A) in addition to an extra hybrid layout (H).The number of layouts are counted exclusively (e.g., superposition usedin a hybrid layout is counted only for the (H) hybrid layout). The bar charton the left shows the number of layouts by the number of visualizationsplaced together for visual comparison. The chart on the right shows thefrequency of arrangements used for juxtaposition. izations [2, 73]. Temporal juxtaposition, animated transition betweenmultiple charts, is sometimes considered as superposition in that itshows multiple visualizations in the same space [31, 52]. Superpositionand explicit-encoding are also ambiguous for specific visualizationdesigns. For example, in the case where two node-link diagrams areshown in a single view with common edges and nodes highlighted,one can consider it as either a single union node-link diagram withexplicit-encoding [52] or superposition of two node-link diagrams withexplicit-encoding [44].
To more systematically organize the insights gained in the literaturereview and provide implications for comparative layouts in a moreprecise manner without confusion, we found it is necessary to alleviatethe ambiguous boundaries between the layouts. We propose to classifythe three comparative layouts into five exclusive categories: (1) chart-wise juxtaposition, (2) item-wise juxtaposition, (3) animated transition,(4) superposition, and (5) explicit-encoding and (6) an extra hybridlayout. Fig. 2 shows the examples of each category observed in ourtarget papers, and Fig. 3 summarizes the overall distribution of eachcategory.To reflect the diverse variants of juxtaposition layouts, we suggesttwo subcategories for juxtaposition with six different ways of arrange-ments. We classified original juxtaposition into chart-wise and item-wise juxtaposition , distinguishing the type of targets that are arrangedusing juxtaposition (i.e., chart or visual elements). For example, placingtwo bar charts side-by-side (i.e., concatenating two bar charts) is chart-wise juxtaposition (Fig. 2A), while arranging bars next to each other(i.e., grouped bar charts) is item-wise juxtaposition (Fig. 2F). Distin-guishing item-wise juxtaposition from chart-wise juxtaposition can beespecially useful to alleviate possible confusion when discussing theircontrasting effectiveness. For example, we found many papers consider“juxtaposition” as the least effective layout for finding small differences,which seemed to refer to chart-wise juxtaposition. However, item-wisejuxtaposition is found to be much more effective than chart-wise jux-taposition according to user study results [26, 47]. In chart-wise andtem-wise juxtaposition, we discovered six different ways of arrangingvisualizations or visual elements— adjacent , stacked , grid , mirrored , diagonal , and free-form (Fig. 2)—where three of the terms are broughtfrom the recent study [47]. Adjacent and stacked arrangements referto placing charts or visual elements in horizontal and vertical axesrespectively, constructing either a grouped or a stacked bar chart inthe item-wise version (Fig. 2F and G). In our survey, several matrixvisualizations used diagonal arrangements for these layouts (Fig. 2Iand K). The free-form arrangements are supported when people caninteractively rearrange the visualizations without any restrictions. Themirrored arrangement is placing visualizations symmetrically (Fig. 2E),which can be used with another arrangement where the adjacent ar-rangement is most frequently used with the mirrored layout. Superposition refers to the designs that combine multiple visualiza-tions into one visualization with a unified coordinate system. In contrastto chart-wise or item-wise juxtaposition, visual elements can overlapin superposition (e.g., nodes and links can overlap if two node-linkdiagrams are superposed [2]). While juxtaposition and superpositionrefer to static designs, the animated transition category refers to thedesigns that use the temporal transition from one chart to another tohighlight the difference between multiple charts. The transition usuallytakes place on the same visualization space, showing a single chart ata time which distinguishes animated transition from juxtaposition orsuperposition.
Explicit-encoding refers to the use of extra visual ele-ments that help comparison. For example, one can draw lines betweentwo scatterplots to connect the corresponding points [30]) or highlightcommon edges or nodes between two network diagrams with a differ-ent color [44]. Explicit-encoding can be used without juxtaposition orsuperposition; for example, if the difference between two bar chartsis of interest, one can draw a separate bar chart that only shows thedifference without the original bars (Fig. 2C).In practice, two layouts from different categories can be used to-gether, which refers to hybrid layout . For example, to help peoplemore easily find the related bars in juxtaposed bar charts, systems canvisually link them using a difference color (explicit-encoding) uponuser interaction (Fig. 2D). A separate visualization that is constructedusing explicit-encoding can be also overlaid on top of juxtaposed barcharts (Fig. 2H) to support accessing both the difference and the orig-inal information. Highlighting common or unique visual elements insuperposed node-link diagrams also belongs to this layout.
The distribution of comparative layouts depending on visualizationtypes and the number of visualizations placed together is shown inFig. 4. For almost all visualization types, chart-wise juxtaposition is themost frequent. This may be because of its strong advantages: It is easyto implement [19] and supports straight-forward comparison [52] forsmall number of visualizations and for comparing non-complex visualstimuli [30]. An exception to this is when designing node-link diagrams.Since this type of visualizations is generally more complex comparedto bar and line charts, researchers prominently used hybrid layouts toalleviate visual clutters and show differences more clearly [51, 52, 70].Unlike chart-wise juxtaposition, the use of item-wise juxtapositionseems to be restricted by visualization types. In item-wise juxtaposition,to be able to place corresponding visual elements next to each other,at least one of the x and y axes should encode categorical values. Thisseems to be the reason why item-wise juxtaposition is actively usedfor bar charts and heatmaps while never used in scatterplots and linecharts. Superposition is adopted in the highest proportion comparedto other layouts when designing line charts, placing multiple lines ina single view with different colors. Overall, chart-wise juxtapositionand hybrid layouts are used in higher proportion for comparing morethan two visualizations (Fig. 3), compared to that of placing onlytwo visualizations, which seems to be a way to overcome the limitedscalability that other layouts have. In this section, we reflect on the advantages and concerns of usingeach layout suggested in the papers to develop our understanding of the
Fig. 4. The frequency of comparative layouts grouped by the five mostfrequent visualization types and the number of visualizations placedtogether for comparison. The abbreviations for comparative layouts areidentical to that in Fig. 3 comparative layouts in the real-world scenarios (Table 1).
The advantages of chart-wise juxtaposition mainly stem from its char-acteristic that it does not significantly change the original visualiza-tion [11, 37, 39, 41], which is sometimes the main reason for choos-ing chart-wise juxtaposition over other layouts [41]. Another relatedadvantage is its ability to support separate analyses of individual vi-sualizations [11, 48, 52], which is an important factor for professionalanalysts in network analysis [52]. Researchers also advocate its appli-cability to any visualizations [4] or its simplicity in implementation:“[Juxtaposition is] simple, even trivial” [5]. When two visualizationsare juxtaposed and mirrored, it is known that the human perceptionsystem effectively recognizes the symmetry between two visual repre-sentations [62] which facilitates comparison between the two. A recentwork [47] provided practical evidence that juxtaposing two charts ina mirror manner was more efficient than using animated transition oritem-wise juxtaposition for comparing the correlation of individual barcharts.On the other hand, six papers have commonly claimed that thekey concern of chart-wise juxtaposition is its limited scalability[19, 39, 53, 56, 66, 74]. For example, it is challenging to juxtaposea large number of visualizations simultaneously in the limited screenspace; as an extreme case, it is sometimes impossible to place even twovisualizations at the same time in a mobile environment [74]. Anotherconcern regarding chart-wise juxtaposition lies in its effectiveness incomparison. Tominski et al. [60] described this problem as “eyes haveto move from one part to the other part,” which consequently leadspeople to rely on the mental image of the first part to compare it with theother part. In this sense, chart-wise juxtaposition has been criticized for able 1. Various aspects of comparative layouts discussed by researchers in 127 papers: advantages and concerns of using the layouts andapproaches to overcome the concerns. These aspects (66 categories in total) are categorized into three sub-categories in comparison with thediscussions in Gleicher et al.’s work [19]: (N) newly found from our survey (35 categories), (E) not new but confirmed by empirical evidence** (15categories), and (G) neither new nor confirmed* (16 categories).
Juxtaposition Superposition
Advantages no visual interference* 8 limited scalability** 12 effective comparison 4 visual interference* 17supporting other tasks 3 difficulty in comparison** 7 suitable for subtle difference** 4 limited scalability* 3simple implementation* 2 cognitive burden* 6 easy comparison 3 visual separation* 1straight-forward comparison 2 ineffective comparison** 5 effective interpretation 2 lacking intuitiveness 1suitable for large difference 2 unsuitable for subtle difference 4 minimize eye movement* 2 unfamiliar 1easy comparison 2 difficulty in linking items** 4 suitable for large difference 2 precluding other tasks 1familiar 1 managing consistency 4 less cognitive burden* 2high preference 1 unsuitable for complex stimuli 2 suitable for spatial data* 1
Approaches convenient comparison 1 long eye movement* 2 high preference 1 using hybrid layout** 2unsuitable for large difference 1 aggregating visual elements* 1managing opacity* 1
Approaches filter 1using hybrid layout** 8managing consistency 6shortening distance** 4
Animated Transition filter 2
Advantages change arrangement* 1 suitable for temporal data* 2 no concurrent comparison* 2
Explicit-Encoding effective comparison** 2 cognitive burden* 2
Advantages structural change 1 unsuitable for large difference 2suitable for subtle difference 4 information loss** 4 showing constancy 1 ineffective comparison** 2effective comparison** 4 precluding other tasks** 3 showing causality 1 requiring constant attention 1reasonable scalability 3 unfamiliar 1 showing narratives 1 difficulty in comparison** 1high preference 1 supporting other tasks 1
Approaches using hybrid layout** 1 staging change 1 such cognitive cost [37, 44, 45, 60] and considered as the least effectivelayout for comparison tasks compared with other layouts [2, 4, 13].Specifically, researchers claimed that the subtle difference betweenjuxtaposed visualizations is especially difficult to recognize [13, 47,55, 68]: “Spot the difference games, in which observers try to detectsmall changes ..., illustrate the difficulty of [comparing between tworegions]” [47]. Comparing complex visualizations (e.g., two node-link diagrams) is also claimed to be inefficient [30, 73] since peoplehave to temporally remember a complicated representation. Anotherconcern on chart-wise juxtaposition is that it is difficult to couple thecorresponding visual elements from two distant visualizations [11, 24,38, 58]. For example, Correll et al. [11] found that people often makemistakes when identifying relevant cells in two heatmaps with chart-wise juxtaposition. Emphasizing this issue, Lobo et al. [38] claimed thatchart-wise juxtaposition can be effective “only if objects can easily bematched.” Many researchers also added that, to be effective, designersshould carefully optimize the consistency between visualizations [6,15, 30, 31], such as using the same range for the axes in chart-wisejuxtaposition or placing relevant visual elements in the same logicalposition in juxtaposed node-link diagrams.
Superposition has been advocated for supporting comparison tasks[2, 29, 44, 45, 52], allowing a “quick and easy” comparison [12]. Subtledifference, which is challenging to recognize in chart-wise juxtaposi-tion, can be visually salient in superposition [9, 13, 24] because targetvisual elements are arranged closely. Wang et al. [68] argued thatsuperposition is “especially useful when the spatial location is a keycomponent of the comparison,” such as in geographical visualizations.The key concern on superposition is visual interference (i.e., visual ele-ments being overlapped challenge people in interpreting visualizations)which can lead to a scalability issue [31,37,44,60,65,66,74]. For exam-ple, Viola et al. [64] mentioned the complexity of this concern: “[T]hedisplay of several data attributes quickly leads to visual clutter. Thereis thus no general methodology on how to design effective integratedmulti-attribute visualizations.” In this context, Caruso et al. [9] assertedthat superposition can be useful only when target visualizations aresimilar enough. A qualitative study by Tominski et al. [60] showed thatit is hard to compare two superposed heatmaps because of the blendedcolor of each cell.
The main advantage of explicit-encoding is that it allows direct accessto the predefined relationship [45, 59, 73]: “[T]he viewer does not needto make a mental comparison or find the difference, as it has alreadybeen calculated” [45]. For this reason, explicit-encoding can be usedfor designs where visualizing subtle difference is of importance [32].Its second advantage is the scalability in terms of the number of targetvisualizations since it usually focuses only on showing the predefinedrelationships without showing the original visualization. For example,in a mobile environment, explicit-encoding can be more effective thanjuxtaposition or superposition [74] since the screen space is limited.Based on user studies, researchers also found explicit-encoding is usefulwhen overlaid with other layouts (i.e., hybrid layouts). The hybridlayouts allowed a faster and more accurate comparison between node-link diagrams [44] and were more preferred by people [56] comparedwith using a single layout.However, it can be ineffective if people can only see a specificrelationship without the original information: “Ideally, we would like tosee the entire dataset without missing any detail, but explicit-encodingconcedes this design goal ... in favor of others” [31]. This seems aconsiderable drawback as data analysts described in a research paper[15] did not like such information abstraction: “[D]ue to informationloss, scientists were not comfortable with the idea of smoothing bycomputation of average.”A relevant problem of explicit-encoding is called decontextualiza-tion , which involves losing contexts of data in visual representations:“The user sees the result of a comparison but cannot interpret it withoutadditional visualization of the original data. This increases the complex-ity of the visualization” [66]. Another concern for explicit-encodingis its unfamiliarity. A study with treemap visualizations [35] showedthat people occasionally misinterpreted a novel textual representationthat encodes the direction of value changes. Similarly, participantsfrom another study had difficulties in interpreting explicitly encodeddifferences (Fig. 2F), and they rated explicit-encoding least effectivecompared with item-wise juxtaposition or hybrid designs [56].
Animated transition is especially useful for recognizing a small localdifference between two visualizations, as it outperformed item-wise andchart-wise juxtaposition in finding the maximum difference betweena pair of bar charts or donut charts [47]. Because animated transitionhows visualizations separately in time, it allows people to take inde-pendent analyses on each visualization [52]. However, the drawback ofthis layout is that people cannot see target visualizations at once [31,56],which is known to be less effective than comparing concurrently visiblerepresentations [43] especially when the number of target visualiza-tions increases. Moreover, animation requires constant attention andinteraction (e.g., switching between views repeatedly) [2,47,73], which“may increase the time requirement” [2]. The performance of animatedtransition on comparison tasks is controversial; while animated transi-tion showed outstanding performance in a study [47] with an local task,it resulted in inaccurate comparison even with confusion with node-linkdiagrams [52]. Similarly, experts who used animated scatterplots tosee multiple t-SNE results mentioned that watching animated transitionwas cognitively challenging: “[T]racking the nodes in an animatedmanner requires a mental map comparison, which is demanding” [34].
In this section, we discuss researchers’ previous attempts to overcomethe concerns of each layout to develop deeper insights of comparativelayouts with diverse design options.
We found four main approaches for chart-wise juxtaposition to over-come its limited scalability and ineffectiveness in comparison tasks.
Using Hybrid Layout.
Explicit-encoding is frequently used to com-plement chart-wise juxtaposition [11, 24, 31, 32, 65]. We identifiedtwo major purposes of this approach: (1) assisting to couple the cor-responding visual elements and (2) improving the effectiveness in therecognition of difference. For example, egoComp [36] used lines con-necting visual elements in multiple visualizations “to decrease theuser’s memory cost.” Heimerl et al. [24] suggested explicitly showingbin boundaries in multi-class scatterplots to “[h]elp with mapping binsacross different plots.” To address the difficulty in comparing a largenumber of heatmap visualizations in chart-wise adjacent arrangements,BayesPiles [65] allowed people to select a reference heatmap to tempo-rally color-encode differences (i.e., subtraction values) in the rest of thematrices. Results from user studies [44, 56] support the effectiveness ofhybrid layouts as using explicit-encoding overlays with chart-wise anditem-wise juxtaposition in bar charts and node-link diagrams showedbetter performance compared with solely relying on the juxtapositionlayouts (Fig. 5A and E).
Shortening Distance.
Juxtaposing visualizations or visual elementsas close as possible is one of the simplest but effective methods. A bodyof studies showed empirical evidence that comparison is easier whenvisual representations are closer together [33,49,57]. We identified fourstudies that explicitly mentioned using similar approaches [7,56,59,60]:“When the two stimuli are far away from each other, the subject hasto frequently move the eyes to switch the focus. Therefore, ... wehave placed the stimuli as close to each other as possible” [7]. Withuser interaction, Tominski et al. [60] allowed people to crop and bringthe rectangular part of a visualization close to the area to which theywant to compare it with. We also found two studies that used item-wise juxtaposition for this purpose; for example, Srinivasan et al. [56]“opted to use a grouped bar chart instead of a concatenated bar chart(bar charts with chart-wise juxtaposition) since comparisons are likelyto be more accurate with no distracting bars in between correspondingvalues.” In a geographical visualization, CompaRing [59] brings afew regions of comparison candidates near a reference region uponuser selection. Study results support the effectiveness of item-wisejuxtaposition [44, 47, 52] in enhancing comparison performance interms of time and accuracy, especially in local tasks.
Maintaining Consistency.
Gleicher et al. [19] mentioned the impor-tance of maintaining the consistency of visual properties in chart-wisejuxtaposition to minimize cognitive burden. This is relevant to consis-tency management in multiple coordinated views [50], such as determin-ing whether to use shared or independent data domains and ranges onthe screen for individual visual channels (e.g., color, size, and the x and y axes). Likewise, Kim et al. [31] mentioned, “[Keeping visualizationsconsistent] seems to be particularly useful for juxtaposition because they provide a common context to link the data instances.” Examplesinclude arranging categories in the same order between heatmaps [72]or using a constant height for all visualizations [25]. We also foundthat almost all studies that employed chart-wise juxtaposition usedthis approach by using a constant color scheme [59], size [60], andcoordinate systems [71]. Filter.
The number of items or visualizations being compared simul-taneously is known to determine the difficulty in comparison tasks [18].For example, CompaRing [59] automatically selects a few number ofcomparison targets to reduce the complexity, and Zaman et al. [70]proposed “subtractive encoding” which removes common nodes andedges from network visualizations to highlight the differences.
We discuss two approaches to alleviate the main drawback of superpo-sition, visual interference.
Using Clutter Reduction Methods.
To manage the visual interference,clutter reduction methods can be employed, which can be categorizedinto Ellis et al.’s taxonomy of clutter reduction techniques [16]. Forexample, Dasgupta et al. [15] aggregated multiple lines as a band toprevent them from being a “spaghetti plot.” Many studies controlledthe transparency [60] or size [2] of visual elements, while filteringvisual elements [70] is also a popular method. Other methods includejittering or adding offsets along axes in line charts [14] and node-linkdiagrams [44].
Using Hybrid Layout.
Although not commonly suggested, com-plementing superposition using explicit-encoding seems promising toovercome the visual interference and further enhance its performancein comparison tasks. For example, inspired by natural behaviors withprinted papers, one study [60] allowed people to peek at the summaryof occluded regions through a folding interaction and found that thiskind of explicit-encoding on demand complements the weakness ofsuperposition. Similarly, VAICo [53] used explicit-encoding in super-posed images to summarize and show the clusters of inconstant regionswith user interactions. Another result shows that highlighting commonor unique nodes and edges in superposed node-link diagrams outper-formed a single layout with few exceptions [44] and were preferred byprofessionals [52].
In explicit-encoding, researchers used hybrid layouts to complementthe weaknesses of explicit-encoding (i.e., decontextualization and unfa-miliarity). One study [42] discussed this issue and suggested using ad-ditional layouts as a remedy: “To avoid decontextualization using onlyexplicit-encoding ..., we also use juxtaposition.” A similar approachwas evaluated in a study [56] that using a single explicit-encoding chartshowed least preference by the unfamiliarity, but when used with anitem-wise juxtaposed visualization, the preference became the best com-pared to other variants of bar charts. For animated transition, the use ofstaged changes between spatial locations is advocated, as the animationoften confused people when transition between two visualizations witha large amount of difference took place [52].
To better help designers systematically explore the design options ofcomparative layouts, we provide a web-based visualization gallery. Itshows diverse designs that are observed in the literature review, such asthe visualizations in Fig. 2, and enables people to flexibly change thelayout based on the following visualization grammar:
Layout := Type, Unit, Arrangement, MirroredType := Juxtaposition | Superposition | Explicit-EncodingUnit := Chart | ItemArrangement := Adjacent | Stacked | Diagonal | AnimatedMirrored := True | False
People can select one of three comparative layouts (i.e., jux-taposition, superposition, and explicit-encoding) and test thediverse ways of arranging visualizations in juxtaposition: the unitof comparison targets (chart-wise or item-wise), arrangements ig. 5. A visual summary of the consistent and contradictory pairwise relations of comparative layouts found in
Study papers [8, 26, 28, 37, 44, 46, 47,51, 52, 56, 67]. The diagrams (A-I) are arranged in terms of six visualization types, two task types ( local and global tasks), and three dependentvariables ( completion time , accuracy , and precision ) where the precision refers to tier values in a staircase method [26, 47]. Arrows indicate thatthe source is significantly more effective than the destination. The comparative layouts, represented as nodes, are arranged in a way that layoutswith better performances (not significantly better without arrows) are placed from top to bottom and left to right. The blue and red edges indicatethe consistent and contradictory relations in a certain combination of visualizations, tasks, and dependent variables, respectively, and gray edgesindicate that the relation between the two layouts is discovered only once in that combination. The abbreviations of comparative layouts used in thisfigure are identical to that in Fig. 3 except that subscripts in juxtaposition layouts indicate the different arrangements: adjacent, stacked, diagonal,and mirrored. The full summarization of study results can be found at https://sehilyi.github.io/comparative-layout-explorer/ . adjacent, stacked, diagonal, or animated), and the use of mir-rored arrangements. Since visual consistency and interferenceare important factors for comparative layouts according to oursurvey results, we allow users to configure them as well, suchas using shared, independent, or distinct color palettes betweenjuxtaposed bar charts or using a different size for cells in superposedheatmaps. This visualization gallery can be accessed via https://sehilyi.github.io/comparative-layout-explorer/ . ISCUSSION
We offer practical implications for comparative layouts by suggestingtheir trade-offs, actionable guidelines, and promising directions forfuture research.
To assist designers in selecting comparative layouts, we summarize andreshape our findings discussed in the previous sections into trade-offsof using the four most frequently used layouts—chart-wise juxtaposi-tion (CJ), item-wise juxtaposition (IJ), superposition (S), and explicit-encoding (E)—in terms of four main themes: scalability, effectivenessin recognizing a predefined relationship, familiarity, and supportingvisualization tasks other than comparison. We present a general con-sensus made by researchers in the parentheses next to the name of eachtheme, where “
T (L1 > L2) ” represents that the L1 layout is commonlysaid to be better than L2 in terms of the T theme, and ≈ represents thattheir effectiveness depends on situations. Scalability (E > CJ ≈ IJ ≈ S) . Explicit-encoding is commonlyregarded as the most scalable layout for the increasing number oftarget visualizations because it focuses only on a specific relationship.This seems a strong advantage for explicit-encoding since the otherthree layouts are commonly complained of their limited scalability.For this reason, explicit-encoding was favored by researchers whendealing with small screen space or a large number of visualizations[17, 59, 74]. However, the scalability of the rest seems to depend onother factors such as screen space availability and visual representationcomplexity, leading to the consideration between space efficiency andvisual interference. Effectiveness in Recognizing a Relationship (E > S ≈ IJ ≈ CJ) .Researchers commonly claimed that recognizing a specific relationshipis most effective with explicit-encoding because it directly calculatesand represents the relationship [46, 56]. Between the rest, though thegeneral consensus is that shorter distance between comparison targetsis more effective, we found chart-wise juxtaposition is sometimes moreeffective in global tasks compared with item-wise juxtaposition [26,47].Therefore, their effectiveness may split depending on what relationshippeople are dealing with.
Familiarity (CJ > IJ ≈ S > E) . Although it may depend on thevisualization types used, chart-wise juxtaposition seems to provide themost familiar visualization to people because it does not require anysignificant modification to individual visualizations. Between item-wisejuxtaposition and superposition, neither seems to entirely outperformthe other as we find both the familiar and unfamiliar examples foreach layout: Grouped bar charts and multi-class scatterplots can beconsidered as familiar visualizations of using item-wise juxtapositionand superposition respectively, while variants of heatmaps [73] andnode-link diagrams [2] are the unfamiliar ones. Explicit-encodingis likely to provide the least familiar outcomes because it frequentlyemploys novel visual representations with data aggregation, which isknown to be unfamiliar to InfoVis novices [21]. Supporting Other Types of Tasks (CJ > IJ ≈ S > E) . Because visualanalytics involves performing a series of multiple tasks, the importanceof supporting other types of tasks, as well as comparison tasks, isemphasized by many researchers [44, 56, 59]. The consensus in thisrespect is that explicit-encoding is least effective since it generallyeliminates the original visualizations [15, 56]. On the other hand, chart-wise juxtaposition is commonly claimed to support general tasks thebest by separately showing individual visualizations. We do not yethave any clear understanding between item-wise juxtaposition and superposition in supporting independent analysis, which hence requiresadditional studies. When looking into the studies with local tasks [44,47,52,56], chart-wisejuxtaposition has almost never outperformed any other layouts in termsof accuracy, completion time, and precision [26, 47] (see the yellow CJ nodes in Fig. 5). Considering the diverse factors used in the studies(e.g., visualization types, stimuli complexity, data size, and amountof difference), these consistent results give a very strong implicationthat if detecting local differences is the main task, designers must useother layouts, such as item-wise juxtaposition or superposition. Thisimplication is aligned with other existing studies [33, 49, 57], but weconfirm it in the context of comparative layouts by categorizing tasksinto global and local comparison. In the study results for local comparison tasks [39, 44, 47, 52, 56], wefound a few exceptions where chart-wise juxtaposition showed compa-rable results to item-wise juxtaposition or superposition. The first casewas when target visual elements are highlighted (i.e., using explicit-encoding additionally) so that people did not have to manually linkthem [44] (e.g., the last diagram in Fig. 5E). The second case was whendealing with geographical visualizations of showing dense regions sothat some kinds of landmarks already existed, for example, buildingsand roads, which people can use when identifying the correspondingvisual elements [39]. Therefore, it is desirable to provide landmarks,such as grids or reference lines, or further using explicit-encoding toenhance the performance; however, please note that providing land-marks in chart-wise juxtaposition did not make dramatic performanceimprovements to outperform item-wise juxtaposition and superposition.
Aligning comparison targets looks to be the fundamental design choiceto take advantage of the effectiveness that superposition potentially has.This is beyond simply using consistent value ranges for x and y axes inmultiple visualizations [50] (e.g., using an identical date period for twotrends in a superposed line chart) but is to more actively align ‘visualmarks’ that need to be compared. The study results with line chartswell illustrate this tendency (Fig. 5F-G): Although superposed linecharts overall showed superior performance than chart-wise versions,comparing values between different time points for different trendsmade the superposed line charts significantly ineffective compared tothe chart-wise one (see the “Discrimination” task). If the comparisontargets can be predefined, they can be aligned in the visualizationby default, or interactive designs that allow people to use variousalignments on demand can be adopted [20]. According to an observation study [60], using superposition inheatmaps was not effective because people had difficulties in distin-guishing the blended color of cells. Consistent with the observation,we found only a few examples of using superposition for heatmapswithout any techniques for reducing the visual interference. To preventsuch a blending problem, visualization designers can use one of sixalternative methods that we discovered in our review. For comparinga pair of heatmaps, first, designers can simply use glyph visualiza-tions [60], such as encoding the radius of circles rather than their color.Second, if two quantitative values are orthogonal (e.g., value and uncer-tainty), designers can consider using different color channels, such ashue and saturation, following a successful design in uncertainty visu-alizations [11]. Third, superposing heatmaps with different cell sizescan be an effective design as several studies showed [2, 73] (Fig. 2L).Fourth, instead of superposition, variants of item-wise juxtapositioncan be used with stacked or diagonal arrangements [2, 73] (Fig. 2J andFig. 2K). Lastly, when the number of visualizations becomes larger,weaving techniques [22, 24, 40] or using explicit-encoding to revealhe accurate difference in chart-wise juxtaposed heatmaps [65] can beused.
Explicit-encoding seems to be the most delicate layout, which hasstrong advantages and weaknesses at the same time. Despite its ef-fectiveness in recognizing the predefined relationship as advocated bymany researchers, others suggested its strong drawbacks. One exam-ple is its unfamiliarity, which can affect InfoVis novices in learningand interpreting visualizations [21]. Explicit-encoding has commonlyreceived low preference to InfoVis novices [52, 56] and often showedpoor performance by the unfamiliarity [35]. Moreover, by the decon-textualization, it is often criticized by professionals in the real-worldscenarios [15, 52]. Since many researchers suggested strong rationaleswhen they had to use explicit-encoding in their paper (e.g., perceptualadvantages [65] or scalability [74]), we think it should be used when itsadvantages are certain and surpass its diverse shortcomings. One suchexample would be using explicit-encoding for alleviating perceptualdistortions in superposed line charts such as Playfair’s chart [61].
According to our review, a hybrid layout seems to well complement thedisadvantages that a single layout has, and it was one of the most fre-quently used layout in our target papers. All the user studies (four out of15) that used hybrid layouts showed superior performance for specifictasks compared with a single layout, such as effectiveness in detectingand measuring local changes [44, 52, 53], high preference [52, 56], andbetter scalability [53]. The study results with bar charts are good exam-ples that illustrate the ability of hybrid layouts for complementing otherlayouts (Fig. 5A-B): Solely using an explicit-encoding layout (Fig. 2C)was least preferred by people and sometimes showed the worst per-formance depending on tasks, but placing it on a familiar grouped barchart (i.e., item-wise juxtaposition; Fig. 2H) made it most preferred bypeople and was never the worst. We think that to protect comparativevisualizations from the strong weaknesses that explicit-encoding have,designers should consider using other layouts together.
One study showed that animated transition showed the best performancefor detecting a small difference in item-wise comparison, outperform-ing all other layouts (i.e., chart-wise and item-wise juxtaposition) [47].However, its performance seems very sensitive to tasks, visualiza-tion types, and visual complexity (Fig. 5E). For example, in globaltasks such as identifying the maximum correlation [47] and structuralchanges [44], the performance became weaker. Moreover, a largeamount of changes between two node-link diagrams [52] confusedpeople, leading to poor task performance in accuracy. As a remedy,designers can consider using staged animation [23], which was help-ful for large changes. However, we still identify many unexploredareas for this layout in visual comparison tasks (e.g., task types, visualrepresentations, and data complexity). As it showed relatively largeperformance variations across different designs, designers should useanimated transition with care and refrain from using it for detectinglarge differences.
Researching Human Factors in Chart-wise Comparison.
As we lackempirical results for the performance of global comparison tasks (sixout of 15 papers), exploring comparative layouts with diverse globaltasks seems a promising direction to expand our understanding of thelayouts. When we looked into the study results with local compar-ison tasks, we were able to find relatively consistent results amongcomparative layouts. However, it seems that for global tasks, the taskperformance is much more sensitive, even sensitive to different waysof using chart-wise juxtaposition (adjacent, stacked, and mirrored inFig. 5D). For example, using mirrored and adjacent chart-wise juxta-position showed the best performance in correlation tasks [47], but forcomparing mean of individual visualizations [26], a stacked arrange-ment was the best. As recent work suggested [26], different perceptual heuristics of people seem to greatly influence the performance, resultingin varying performance by target relationships or visual representations.
Investigating the Effectiveness with Varying Difference.
In our re-view, one of the factors that researchers most frequently discussed fortheir designs was the amount of difference in terms of size or com-plexity. For example, chart-wise juxtaposition is generally regardedas least effective for detecting a small difference because of the longdistance between visual elements. However, the performance mightdepend on what kinds of small difference users are dealing with, eithera global or a local difference, as chart-wise juxtaposition performedbetter than item-wised juxtaposition for a certain task [26, 47]. As wefind none of the user studies in our survey directly confirmed theseaspects by varying size or complexity of difference, it looks worthexploring research topic.
Investigating the Scalability of Comparative Layouts.
Most userstudies (nine out of 15) focused only on one-to-one comparison. How-ever, in the real world, more than two visualizations are frequentlycompared together [18]. Juxtaposition and superposition are consid-ered to suffer from limited scalability, compared to explicit-encoding[31, 39, 60, 65, 66, 74]. Therefore, although an independent use ofexplicit-encoding was the least preferred design for comparing only thesmall number of visualizations [56], it might show the opposite resultswhen the number of visualizations increases to some extent. To developa better understanding of comparative layouts in the real world, it lookspromising to investigate the ability in terms of scalability.
The main reason that we focused on the papers that cited Gleicher etal.s work [19] was to make the data collection process reproducible,accurate, and efficient. By this choice, we could clearly define thescope of target papers with reasonable quantity and coverage ( N =127, Venues =50). Furthermore, since these papers often explicitly discussedthe strengths or weaknesses of the layouts through the terminology thatis consistent with the work of Gleicher et al., we could also gather andamalgamate their insights more accurately. For example, there are anumber of visualization interfaces that place visualizations side-by-side,but not all of them may be designed for visual comparison. However,we believe that it would also be interesting and worth looking intothe ‘outside’ of our scope by reviewing a broader set of papers. Forexample, animated transition is relatively less discussed in our paperdue to the limited number of papers that employed this layout. Sincethere might be many research papers that employ animated transition incomparison contexts without citing the Gleicher at al.’s work, it wouldbe worth to further explore this layout to strengthen our understandingof comparative layouts.
ONCLUSION
We presented a systematic review of 127 research papers, including15 papers with quantitative user studies, to better understand the threecomparative layouts for visual comparison: juxtaposition, superposi-tion, and explicit-encoding. Combining and systematizing the insightspreviously gained in the wild , we offered a consistent and reusableframework for using comparative layouts. We explored the diverseaspects of comparative layouts, including the advantages and concernsof each layout, approaches to overcome the concerns, and trade-offsbetween them. We also proposed seven actionable guidelines andunexplored research area to reveal promising future directions. A CKNOWLEDGMENTS
This work was supported by the National Research Foundation ofKorea (NRF) grant funded by the Korea government (MSIT) (No.NRF-2019R1A2C2089062). The ICT at Seoul National Universityprovided research facilities for this study. Jaemin Jo was affiliated withSeoul National University at the time of this research and is currentlyaffiliated with Sungkyunkwan University.
EFERENCES [1] J. Alcalde, C. E. Bond, and C. H. Randle. Framing bias: The effect offigure presentation on seismic interpretation.
Interpretation , 5(4):T591–T605, 2017.[2] B. Alper, B. Bach, N. Henry Riche, T. Isenberg, and J.-D. Fekete. Weightedgraph comparison techniques for brain connectivity analysis. In
Proceed-ings of the SIGCHI conference on human factors in computing systems ,pp. 483–492. ACM, 2013.[3] N. Andrienko and G. Andrienko.
Exploratory analysis of spatial andtemporal data: a systematic approach . Springer Science & BusinessMedia, 2006.[4] N. Andrienko and G. Andrienko. State transition graphs for semanticanalysis of movement behaviours.
Information Visualization , 17(1):41–65,2018.[5] F. Beck, M. Burch, S. Diehl, and D. Weiskopf. A taxonomy and survey ofdynamic graph visualization. In
Computer Graphics Forum , vol. 36, pp.133–159. Wiley Online Library, 2017.[6] J. Bernard, A. V¨ogele, R. Klein, and D. W. Fellner. Approaches andchallenges in the visual-interactive comparison of human motion data. In
VISIGRAPP (3: IVAPP) , pp. 217–224, 2017.[7] ˚A. Birkeland, C. Turkay, and I. Viola. Perceptually uniform motion space.
IEEE transactions on visualization and computer graphics , 20(11):1542–1554, 2014.[8] G. Carenini, C. Conati, E. Hoque, B. Steichen, D. Toker, and J. Enns.Highlighting interventions and user differences: informing adaptive infor-mation visualization support. In
Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems , pp. 1835–1844, 2014.[9] V. Caruso, A. Cattaneo, and J.-L. Gurtner. Creating technology-enhancedscenarios to promote observation skills of fashion-design students.
Form@re-Open Journal per la formazione in rete , 17(1):4–17, 2017.[10] M. Chen, S. Walton, K. Berger, J. Thiyagalingam, B. Duffy, H. Fang,C. Holloway, and A. E. Trefethen. Visual multiplexing. In
ComputerGraphics Forum , vol. 33, pp. 241–250. Wiley Online Library, 2014.[11] M. Correll, D. Moritz, and J. Heer. Value-suppressing uncertainty palettes.In
Proceedings of the 2018 CHI Conference on Human Factors in Com-puting Systems , p. 642. ACM, 2018.[12] L. Crissaff, L. W. Ruby, S. Deutch, R. L. DuBois, J.-D. Fekete, J. Freire,and C. Silva. Aries: enabling visual exploration and organization of artimage collections.
IEEE computer graphics and applications , 38(1):91–108, 2017.[13] A. Cruz, J. P. Arrais, and P. Machado. Interactive and coordinated visual-ization approaches for biological data analysis.
Briefings in bioinformatics ,2018.[14] A. Dasgupta, S. Burrows, K. Han, and P. J. Rasch. Empirical analysis ofthe subjective impressions and objective measures of domain scientists’visual analytic judgments. In
Proceedings of the 2017 CHI Conference onHuman Factors in Computing Systems , pp. 1193–1204. ACM, 2017.[15] A. Dasgupta, J. Poco, Y. Wei, R. Cook, E. Bertini, and C. T. Silva. Bridgingtheory with practice: An exploratory study of visualization use and designfor climate model comparison.
IEEE transactions on visualization andcomputer graphics , 21(9):996–1014, 2015.[16] G. Ellis and A. Dix. A taxonomy of clutter reduction for informationvisualisation.
IEEE transactions on visualization and computer graphics ,13(6):1216–1223, 2007.[17] A. Fofonov and L. Linsen. Projected field similarity for comparative visu-alization of multi-run multi-field time-varying spatial data. In
ComputerGraphics Forum , vol. 38, pp. 286–299. Wiley Online Library, 2019.[18] M. Gleicher. Considerations for visualizing comparison.
IEEE transac-tions on visualization and computer graphics , 24(1):413–423, 2017.[19] M. Gleicher, D. Albers, R. Walker, I. Jusufi, C. D. Hansen, and J. C.Roberts. Visual comparison for information visualization.
InformationVisualization , 10(4):289–309, 2011.[20] P. Goffin, P. Isenberg, T. Blascheck, and W. Willett. Interaction techniquesfor visual exploration using embedded word-scale visualizations. In
CHI2020-Conference on Human Factors in Computing Systems , 2020.[21] L. Grammel, M. Tory, and M.-A. Storey. How information visualizationnovices construct visualizations.
IEEE transactions on visualization andcomputer graphics , 16(6):943–952, 2010.[22] H. Hagh-Shenas, S. Kim, V. Interrante, and C. Healey. Weaving versusblending: a quantitative assessment of the information carrying capacitiesof two alternative methods for conveying multivariate data with color.
IEEE transactions on visualization and computer graphics , 13(6):1270– 1277, 2007.[23] J. Heer and G. Robertson. Animated transitions in statistical data graphics.
IEEE transactions on visualization and computer graphics , 13(6):1240–1247, 2007.[24] F. Heimerl, C.-C. Chang, A. Sarikaya, and M. Gleicher. Visual de-signs for binned aggregation of multi-class scatterplots. arXiv preprintarXiv:1810.02445 , 2018.[25] D. Herr, F. Beck, and T. Ertl. Visual analytics for decomposing temporalevent series of production lines. In , pp. 251–259. IEEE, 2018.[26] N. Jardine, B. Ondov, N. Elmqvist, and S. Franconeri. The perceptualproxies of visual comparison.
IEEE transactions on visualization andcomputer graphics , 2019.[27] W. Javed and N. Elmqvist. Exploring the design space of compositevisualization. In , pp. 1–8.IEEE, 2012.[28] W. Javed, B. McDonnel, and N. Elmqvist. Graphical perception of multipletime series.
IEEE transactions on visualization and computer graphics ,16(6):927–934, 2010.[29] M. Keck, D. Kammer, and R. Groh. Visual version comparison of multidi-mensional data sets using glyphs. In . IEEE, 2018.[30] J. Kehrer, H. Piringer, W. Berger, and M. E. Gr¨oller. A model for structure-based comparison of many categories in small-multiple displays.
IEEEtransactions on visualization and computer graphics , 19(12):2287–2296,2013.[31] K. Kim, J. V. Carlis, and D. F. Keefe. Comparison techniques utilizedin spatial 3d and 4d data visualizations: A survey and future directions.
Computers & Graphics , 67:138–147, 2017.[32] R. Krueger, G. Sun, F. Beck, R. Liang, and T. Ertl. Traveldiff: Visualcomparison analytics for massive movement patterns derived from twitter.In , pp. 176–183.IEEE, 2016.[33] A. Larsen and C. Bundesen. Effects of spatial separation in visual patternmatching: Evidence on the role of mental translation.
Journal of Ex-perimental Psychology: Human Perception and Performance , 24(3):719,1998.[34] Q. Li, K. S. Njotoprawiro, H. Haleem, Q. Chen, C. Yi, and X. Ma. Embed-dingvis: A visual analytics approach to comparative network embeddinginspection. arXiv preprint arXiv:1808.09074 , 2018.[35] D. Limberger, M. Trapp, and J. D¨ollner. In-situ comparison for 2.5 dtreemaps. In
Proceedings of the 10th International Conference on Infor-mation Visualization Theory and Applications 2019 (IVAPP/VISIGRAPP) ,2019.[36] D. Liu, F. Guo, B. Deng, H. Qu, and Y. Wu. egocomp: A node-link-based technique for visual comparison of ego-networks.
InformationVisualization , 16(3):179–189, 2017.[37] X. Liu and H. W. Shen. The effects of representation and juxtapositionon graphical perception of matrix visualization. In
Proceedings of the33rd annual ACM conference on Human factors in computing systems , pp.269–278. ACM, 2015.[38] M. J. Lobo, C. Appert, and E. Pietriga. Animation plans for before-and-after satellite images.
IEEE transactions on visualization and computergraphics , 25(2):1347–1360, 2018.[39] M.-J. Lobo, E. Pietriga, and C. Appert. An evaluation of interactivemap comparison techniques. In
Proceedings of the 33rd annual ACMconference on human factors in computing systems , pp. 3573–3582. ACM,2015.[40] M. Luboschik, A. Radloff, and H. Schumann. A new weaving techniquefor handling overlapping regions. In
Proceedings of the InternationalConference on Advanced Visual Interfaces , pp. 25–32, 2010.[41] A. Maries, N. Mays, M. Hunt, K. F. Wong, W. Layton, R. Boudreau,C. Rosano, and G. E. Marai. Grace: A visual comparison framework forintegrated spatial and non-spatial geriatric data.
IEEE transactions onvisualization and computer graphics , 19(12):2916–2925, 2013.[42] G. Marti, P. Donnat, F. Nielsen, and P. Very. Hcmapper: An interactivevisualization tool to compare partition-based flat clustering extracted frompairs of dendrograms. arXiv preprint arXiv:1507.08137 , 2015.[43] T. Munzner.
Visualization Analysis and Design . AK Peters VisualizationSeries. CRC Press, 2015.[44] Y. Naragino and K. Misue. Visualization techniques for the comparativeanalysis of weighted free trees. In , pp. 45–51. IEEE, 2017.45] C. Niederer, H. Stitz, R. Hourieh, F. Grassinger, W. Aigner, and M. Streit.Taco: Visualizing changes in tables over time.
IEEE transactions onvisualization and computer graphics , 24(1):677–686, 2017.[46] C. Nothelfer and S. Franconeri. Measures of the benefit of direct encodingof data deltas for data pair relation perception.
IEEE transactions onvisualization and computer graphics , 26(1):311–320, 2019.[47] B. Ondov, N. Jardine, N. Elmqvist, and S. Franconeri. Face to face:Evaluating visual comparison.
IEEE transactions on visualization andcomputer graphics , 25(1):861–871, 2018.[48] G. S. Paton and J. Henderson. Visualization, interpretation, and cognitivecybernetics.
Interpretation , 3(3):SX41–SX48, 2015.[49] M. D. Plumlee and C. Ware. Zooming versus multiple window interfaces:Cognitive costs of visual comparisons.
ACM Transactions on Computer-Human Interaction (TOCHI) , 13(2):179–209, 2006.[50] Z. Qu and J. Hullman. Keeping multiple views consistent: Constraints,validations, and exceptions in visualization authoring.
IEEE transactionson visualization and computer graphics , 24(1):468–477, 2017.[51] S. Rufiange and M. J. McGuffin. Diffani: Visualizing dynamic graphswith a hybrid of difference maps and animation.
IEEE Transactions onVisualization and Computer Graphics , 19(12):2556–2565, 2013.[52] R. R. Sambasivan, I. Shafer, M. L. Mazurek, and G. R. Ganger. Visualiz-ing request-flow comparison to aid performance diagnosis in distributedsystems.
IEEE transactions on visualization and computer graphics ,19(12):2466–2475, 2013.[53] J. Schmidt, M. E. Gr¨oller, and S. Bruckner. Vaico: Visual analysis forimage comparison.
IEEE transactions on visualization and computergraphics , 19(12):2090–2099, 2013.[54] H. Song, J. Lee, T. J. Kim, K. H. Lee, B. Kim, and J. Seo. Gazedx:Interactive visual analytics framework for comparative gaze analysis withvolumetric medical images.
IEEE transactions on visualization and com-puter graphics , 23(1):311–320, 2016.[55] F. Spechtenhauser and H. Piringer. Diffpin: Interactive specification ofreferences for comparative small-multiple displays. In
EuroVis (ShortPapers) , pp. 107–111, 2018.[56] A. Srinivasan, M. Brehmer, B. Lee, and S. M. Drucker. What’s thedifference?: Evaluating variations of multi-series bar charts for visualcomparison tasks. In
Proceedings of the 2018 CHI Conference on HumanFactors in Computing Systems , p. 304. ACM, 2018.[57] J. Talbot, V. Setlur, and A. Anand. Four experiments on the perceptionof bar charts.
IEEE transactions on visualization and computer graphics ,20(12):2152–2160, 2014.[58] H. Tarner, V. Frick, M. Pinzger, and F. Beck. Exploring visual comparisonof multivariate runtime statistics.
COLLECTIONS , 429:2, 2008.[59] C. Tominski. Comparing: reducing costs of visual comparison. In
Pro-ceedings of the Eurographics/IEEE VGTC Conference on Visualization:Short Papers , pp. 137–141. Eurographics Association, 2016.[60] C. Tominski, C. Forsell, and J. Johansson. Interaction support for visualcomparison inspired by natural behavior.
IEEE Transactions on Visualiza-tion and Computer Graphics , 18(12):2719–2728, 2012.[61] E. R. Tufte.
The visual display of quantitative information , vol. 2. Graphicspress Cheshire, CT, 2001.[62] C. W. Tyler.
Human symmetry perception and its computational analysis .Psychology Press, 2003.[63] R. Van Pelt, R. Gasteiger, K. Lawonn, M. Meuschke, and B. Preim. Com-parative blood flow visualization for cerebral aneurysm treatment assess-ment. In
Computer Graphics Forum , vol. 33, pp. 131–140. Wiley OnlineLibrary, 2014.[64] I. Viola and T. Isenberg. Pondering the concept of abstraction in (illus-trative) visualization.
IEEE transactions on visualization and computergraphics , 24(9):2573–2588, 2017.[65] A. Vogogias, J. Kennedy, D. Archambault, B. Bach, V. A. Smith, andH. Currant. Bayespiles: visualisation support for bayesian network struc-ture learning.
ACM Transactions on Intelligent Systems and Technology(TIST) , 10(1):5, 2018.[66] T. von Landesberger. Insights by visual comparison: The state and chal-lenges.
IEEE computer graphics and applications , 38(3):140–148, 2018.[67] J. Walker, R. Borgo, and M. W. Jones. Timenotes: a study on effec-tive chart visualization and interaction techniques for time-series data.
IEEE transactions on visualization and computer graphics , 22(1):549–558, 2015.[68] Y. Wang, H. Haleem, C. Shi, Y. Wu, X. Zhao, S. Fu, and H. Qu. Towardseasy comparison of local businesses using online reviews. In
ComputerGraphics Forum , vol. 37, pp. 63–74. Wiley Online Library, 2018. [69] J. Weissenb¨ock, B. Fr¨ohler, E. Gr¨oller, J. Kastner, and C. Heinzl. Dy-namic volume lines: Visual comparison of 3d volumes through space-filling curves.
IEEE transactions on visualization and computer graphics ,25(1):1040–1049, 2018.[70] L. Zaman, W. Stuerzlinger, and C. Neugebauer. Mace: A new interfacefor comparing and editing of multiple alternative documents for genera-tive design. In
Proceedings of the 2017 ACM Symposium on DocumentEngineering , pp. 67–76. ACM, 2017.[71] H. Zeng, H. Haleem, X. Plantaz, N. Cao, and H. Qu. Cnncomparator:Comparative analytics of convolutional neural networks. arXiv preprintarXiv:1710.05285 , 2017.[72] Q. Zhang, A. Slingsby, J. Dykes, J. Wood, M.-J. Kraak, C. A. Blok, andR. Ahas. Visual analysis design to support research into movement and useof space in tallinn: A case study.
Information Visualization , 13(3):213–231,2014.[73] J. Zhao, Z. Liu, M. Dontcheva, A. Hertzmann, and A. Wilson. Matrixwave:Visual comparison of event sequence data. In
Proceedings of the 33rdAnnual ACM Conference on Human Factors in Computing Systems , pp.259–268. ACM, 2015.[74] M. Zhao, Y. Su, J. Zhao, S. Chen, and H. Qu. Mobile situated analyticsof ego-centric network data. In