Integrated Visualization Editing via Parameterized Declarative Templates
IIntegrated Visualization Editing viaParameterized Declarative Templates
Andrew McNutt
University of ChicagoChicago, IL
Ravi Chugh
University of ChicagoChicago, IL
ABSTRACT
Interfaces for creating visualizations typically embrace one of sev-eral common forms.
Textual specification enables fine-grained con-trol, shelf building facilitates rapid exploration, while chart choosing promotes immediacy and simplicity. Ideally these approaches couldbe unified to integrate the user- and usage-dependent benefits foundin each modality, yet these forms remain distinct.We propose parameterized declarative templates , a simple ab-straction mechanism over JSON-based visualization grammars,as a foundation for multimodal visualization editors. We demon-strate how templates can facilitate organization and reuse by fac-toring the more than 160 charts that constitute Vega-Lite’s examplegallery into approximately 40 templates. We exemplify the pliabil-ity of abstracting over charting grammars by implementing—as atemplate—the functionality of the shelf builder Polestar (a simu-lacra of Tableau) and a set of templates that emulate the GoogleSheets chart chooser. We show how templates support multimodalvisualization editing by implementing a prototype and evaluatingit through an approachability study.
CCS CONCEPTS • Human-centered computing → Visualization systems andtools ; Graphical user interfaces . KEYWORDS
Information Visualization, Declarative Grammars, Templates, UserInterfaces, Ivy, Systems
ACM Reference Format:
Andrew McNutt and Ravi Chugh. 2021. Integrated Visualization Editing viaParameterized Declarative Templates. In
CHI Conference on Human Factorsin Computing Systems (CHI ’21), May 8–13, 2021, Yokohama, Japan.
ACM,New York, NY, USA, 18 pages. https://doi.org/10.1145/3411764.3445356
Every user interface design involves compromise. Which tasksshould be made easy at the expense of making other tasks cumber-some or even impossible?There are several common user interface modalities for creatingvisualizations, each with distinct trade-offs [20].
Chart choosers
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from [email protected].
CHI ’21, May 8–13, 2021, Yokohama, Japan © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.ACM ISBN 978-1-4503-8096-6/21/05...$15.00https://doi.org/10.1145/3411764.3445356 (as in Excel) allow users to rapidly construct familiar visualizationsat the expense of flexibility.
Shelf builders (as in Tableau) facilitatedynamic exploration but can obstruct the construction of specificchart forms or the addition of visual nuances.
Textual program-ming is highly expressive but can impede rapid exploration.A modality which may be well suited to the initial stages of a task,may become cumbersome in subsequent phases. For example, whilea low-configuration approach for rapid data exploration might suf-fice at the beginning of their work, the user might subsequentlyrequire a more flexible—even if higher friction—interface that al-lows them to tune specific details of their chart. Graphical userinterface (GUI) systems, such as chart choosers and shelf builders,can leave experts wanting more precise control over the chart cre-ation process, while textual systems can leave novices in need ofassistance. Without the ability to move between modalities, usersare stuck with each interface’s shortcomings. None of the existingsingle-mode interfaces simultaneously achieve each of several goals(as in Fig. 2): ease of use ( G1 ), explorability ( G2 ), flexibility ( G3 ),and ease of reuse ( G4 ). These deficits can force users to switch toolsacross their analysis [32], or compel proficient users to seek ad hocsolutions that are difficult to repeat in different contexts.Ideally, interfaces of varying complexity could be integratedsuch that both novice users (for whom chart choosers are oftenbest suited [21]) and experts (whose most profitable interface willvary) obtain the benefits of each modality as their tasks require. Un-fortunately this territory remains under-explored, as visualizationsystems tend to prefer one-size-fits-all designs. Declarative visu-alization grammars are an enticing starting point as they providesignificant flexibility for specifying visualizations as text ( G3 ). How-ever, they lack the abstraction mechanisms found in full-featuredprogramming languages. This paper considers the question: Can we extend declarative grammars with abstraction mechanismsfor reuse ( G4 ), in a way that facilitates explorability ( G2 ) as in shelfbuilders and ease of use ( G1 ) as in chart choosers? To answer to this question, we propose an abstraction mecha-nism called parameterized declarative templates , or simply tem-plates , which extend textual, declarative grammars—specifically,those in which specifications are defined using JavaScript Object No-tation (JSON) [15]—with mechanisms for reusing chart definitions.As depicted in Fig. 1h, templates abstract “raw” declarative specifica-tions with parameters that specify data fields for a visual encoding(e.g.
Color ) and design parameters (e.g. height and width ), makingthem more easily reused ( G4 ). Templates can be rapidly instantiatedfor different data sets, shared among communities, and modified totaste—alleviating barriers to opportunistically [8] leveraging therich body of grammar-based charts found online. Templates, whichare essentially functions in a simple programming language withvariables and conditionals, are described formally in Sec. 4.2. a r X i v : . [ c s . H C ] J a n HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
Views Ribbon G Data FieldsData FiltersCode Pane Template Parameter Controls D Code Example
ABC
Related Views E Chart Area F Generalizes toGenerates H λ (Color: DataTarget, height: Num, width: Num,...) = {"height": "[height]", "width": "[width]", ..., "encoding": {"color": {"field": "[Color]", ...}, ...}} Ivy Template {"height": 757, "width": 940, ..., "encoding": {"color": {"field": "Race", ...}, ...}}
Vega-Lite Specification
Figure 1: The Ivy visualization editor combines textual specification with GUI-based shelf building and chart choosing. Here,an “Isotype Waffle” template is used visualize the people executed in the USA under the death penalty by race since 1977 [9].Templates are functions with typed parameters that abstract JSON specifications in declarative visualization grammars.
We systematically apply this idea in a prototype visualizationeditor, called Ivy, in which templates are created and instantiatedthrough text- and GUI-based manipulation ( G1 ). Templates abstractover arbitrary JSON-based visualization grammars—our implemen-tation currently supports Vega, Vega-Lite, Atom [62], and a toy datatable language—allowing users to easily move between them ( G3 ).We implemented an Ivy template that recreates the functionalityof the shelf builder system Polestar [97, 98], that serves as the de-fault view of the system, facilitating explorability ( G2 ). Our userinterface design is described in Sec. 4.3.The systematic application of templates furthermore enables twonotable interaction features beneficial to exploration ( G2 ). Catalogsearch utilizes standard type-based compatibility checks to imple-ment an extensible—if simple—recommendation system.
Fan out facilitates rapid exploration within a template by juxtaposing mul-tiple chart configurations on demand. These features are describedin Sec. 4.4.To evaluate how templates may serve as a foundation for multi-modal visualization editing, we performed two investigations. Weconsidered how they might usefully reproduce and compress extantfamilies of charts, first by factoring the 166 unique examples thatconstitute the Vega-Lite gallery into 43 templates, and then by re-constructing the 32 charts of the Google Sheets chart chooser as 16templates. We then conducted a small approachability study of ourprototype Ivy system which demonstrated that, with some trainingand guidance, users are able to create and instantiate templates bymixing the available modalities. We present these results in Sec. 5. Although the main ideas regarding templates—abstraction overvalue definitions—and their instantiations are familiar concepts,the contribution of this paper is to explore and evaluate how theseideas may help integrate what are currently disparate approachesfor creating visualizations. The result of this exploration is ourIvy prototype, which exhibits the promise of this approach. Thiscomposition of modalities provides several potential opportunities,including borrowing and adapting external examples (which manyshelf builders lack), rapid exploration of parameter combinations(which programmatic interfaces often lack) and self-service chartcreation (which chart choosers typically lack). Compared to textualprogramming in existing declarative grammars, the abstractionlayer provided by templates simplifies the process of chart reuse andtherein reduces code clones (as in our chart gallery reproductions).Our prototype is available at https://ivy-vis.netlify.app/. Theappendix further details our evaluation and implementation, whileother supplementary materials can be found at https://osf.io/cture/.
Software systems for creating visualizations can be classified into avariety of interaction modalities [20, 59]. Among these, we aim tobridge three of the most common: chart choosing, shelf building,and textual programming. We now discuss declarative grammarsand visualization editors, as they relate to the key ideas in thispaper: (1) to endow declarative visualization grammars with basicabstraction mechanisms, and (2) to design a multimodal UI basedon templates for creating and editing visualizations. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan G1 Ease of Use 🙂☹😫😐 ☹☹😫🤩 G2 Explorability ☹🙂🤩😐 G3 Flexibility n/a 😐🤩☹ G4 Ease of ReuseChart choosersText (declarative)Text (procedural)Shelf builders 😫 ☹ 🙂 🤩😐
Figure 2: Chart making modalities have distinct strengthsand weaknesses. Using parameterized declarative templates ,Ivy strives to combine the strengths of several commonmodalities in this design space.
Declarative grammars of graphics have proven extremely popular[23, 24, 42, 45, 49, 53, 62, 64, 66, 76–78, 80, 81, 83, 91, 92, 94, 95], withdifferent approaches providing more expressiveness than othersfor particular tasks. Compared to full-featured, procedural visual-ization programming languages [6, 7, 35], declarative visualizationlanguages trade fine-grained control over how to render a viewfor concise, expressive means to specify what to render. The ideabehind parameterized declarative templates is to tilt these specifi-cation languages slightly back towards the full-featured languages,by adding some basic programming abstraction mechanisms.Many declarative grammars [42, 45, 49, 62, 76–78, 81] adoptJSON as their specification format, which allows chart definitionsto be easily consumed by various environments and tools. Vega andits surrounding ecosystem exemplify these benefits, which we seekto amplify by making JSON-based grammar specifications easierto reuse and explore. A related effort is Harper et al.’s [25] systemfor converting D3 charts [7] into reusable Vega-Lite specifications,which they also dub templates. Their templates provide abstractionsrelated to ours, but are restricted to a limited subset of D3 charts. Ivytemplates abstract arbitrary JSON and thus support any JSON-basedvisualization grammar.
This prevalent technique starts with select-ing a desired chart form from a (potentially large) set of chart types,and then customizing it through a (usually limited) set of options.They are ubiquitous among analytics tools, such as spreadsheets,and are often found in visual analysis (VA) environments [2, 13, 60,82]. This workflow facilitates an approach to charting held by manynovices [21], but can also lead to premature commitment [22].Ivy provides a gallery of templates, some of which constitutesingle distinct chart forms, as in conventional chart choosers. Somesystems, like Ivy, provide APIs for extending the selection of charts[17, 56], but, unlike Ivy, they tend not to allow these changes fromwithin the tool. Some systems [13, 56, 90] provide social features,wherein users can create, share, and modify charts. Ivy users canpublish, fork, and remix templates through a shared template server.
In shelf builders, users map data columns tovisual attributes, typically in a manner that is motivated by a princi-pled visualization framework, such as VizQL [24] or the
Grammar of Graphics [95]. Tableau [80] and Charticulator [67] are prominentexamples of this paradigm. Data fields of the current dataset are of-ten represented as draggable “pills" that can be placed onto “shelves,”each of which denotes different aspects of the visualization (suchas horizontal position or color).We designed the GUI controls in Ivy for choosing argumentsfor template parameters to resemble the visual conventions ofPolestar [97, 98], itself a facsimile of Tableau [80]. Shelf builderssupport a range of tasks, including presentation [30, 34, 73, 74] andexploration [80, 97, 98]. We facilitate the latter by constructing anIvy template—IvyPolestar—that emulates and expands upon thePolestar application. This “default” template provides a familiarshelf builder interface upon application startup.Many VA tools are backed by JSON-based declarative grammars(such as Lyra [74, 102] or Voyager [97, 98]), yet do not allow usersto modify the underlying specifications. Restricting modificationsto what can be created within such tools misses a significant op-portunity, as there is a wealth of online knowledge and examplesthat ought to be utilized [8]. In commercial VA environments (suchas Tableau), the disconnect from the underlying specification cancause fine-grained editing capabilities to be relegated to deeply-nested drop-down menus, which can preclude feature utilization.We address these issues by making our templates malleable, so thatusers can adapt the interface to their needs.
Shelf builders often promi-nently feature affordances for rapid data exploration, often in theform of recommendation systems [97, 98]. Lee describes the currentstate of the art of these subsystems [46]. Our template-based archi-tecture gives rise to simplified analogs to these “smart” features,but which still facilitate view exploration goals.Our catalog search is an extensible variation of Tableau’s ShowMe feature [54], enabling users to create examples that are availablefor subsequent recommendation. Like Show Me, catalog search usesdata-role based matching to identify potential alternatives. UnlikeShow Me, however, catalog search does not internalize domain-specific knowledge of the underlying grammars—the recommenda-tions do not consider, for example, known relationships betweenhuman perception and chart configuration.Our fan out enables users to rapidly search across both data anddesign space by simultaneously juxtaposing parameter selectionsof interest. Similarly to catalog search, this feature is simple butachieves many of the same goals as additional prior systems [1, 11,52] that often include sophisticated domain-specific knowledge, byenabling users to rapidly explore alternatives in a low cost manner(i.e. in a simple design gallery [55]) and to manipulate collections ofexamples simultaneously (à la Juxtapose [26]). This technique mostclosely resembles René’s [18] combinatorial design exploration,although extended to encompass data-based variations.
There is a long history of UIs that com-bine textual chart programming with a system for rendering thosecharts. Computational notebooks, such as Jupyter Notebook [44],facilitate chart construction through tight feedback loops betweencode and rendered charts. In a similar manner to Ivy templates,papermill [61] allows analysts to parameterize computational note-books which can then be run in a non-notebook environment. Ob-servable [5] provides a reactive-programming platform for creating
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh rich web-based visualizations, which users are encouraged to forkand remix. Wood et al. [99] take a literate programming approachto the visualization design process by enabling authors to buildVega and Vega-Lite charts in a Markdown document.Most closely related to our work is the Vega-IDE [86], whichaugments textual specification of Vega and Vega-Lite charts withdebugging tools [31], and Chart Builder [12], which allows users toedit Vega-Lite charts through a GUI or text, but not both. We bor-row much of Vega-IDE interface design in the text-editing portionof our system; for instance, we make use of Microsoft Monaco’sJSON-Schema [63] based support for JSON grammars as a way toprovide linting and validation. Beyond standard text-editing fea-tures, we also implement extensible heuristic rewrite rules that(automatically) suggest abstractions of JSON values into parame-ters. This eases the flow of converting a declarative specificationinto a reusable template.
Several prior works have explored com-binations of user interface modalities for creating visualizations.Liger [72] mixes together shelf-based chart specification and visu-alization by demonstration . Hanpuku [4], Data-Driven Guides [41],and Data Illustrator [50] combine visual editor-style manipula-tion with visual chart specification or textual programming. Vic-tor [89] explored a prototype that combined a spreadsheet withdirect manipulation and manual view specification. His systemenabled highly expressive visualization construction, whereas wefocus on supporting analytic tasks.Tools such as Jupyter Widgets [38] and Shiny [70] provide mech-anisms for parameterizing analysis code in an ad hoc manner, how-ever these lack many of the graphical affordances for explorationfound in many visual analytics systems.Systems including mage [40], Wrex [14], and B2 [100] expandon these ideas by intermingling text and graphical specificationin computational notebooks. Sketch-n-Sketch [28] takes a bidirec-tional approach to combining (non data-driven) visual editing withtextual programming (in a full-featured, procedural language). Re-Vize [32] seeks to support multiple modalities by chaining togetheranalysis tools though a Vega-Lite-based API.Our investigation in this paper—to integrate chart choosing,shelf building, and textual specification—is complementary to theseefforts. This combination of modalities offers a rich feature-space(such as our catalog search and fan out features), enables educationalopportunities (by presenting a synchronized view between GUIand perhaps unfamiliar textual grammars), and supports a varietyof data exploration tasks.
Here we describe how two hypothetical users—Axel, a novice userof visual analytics systems, and Tabitha, a visual analytics poweruser—carry out two example workflows in Ivy. A motivation forour work is to allow users like Axel and Tabitha to collaborateand benefit from each other’s efforts by working within a commonsystem—aiming to supplant single-modality tooling that contributesto designer-developer breakdowns [48].
Axel wants to make a chart showing death penalty executions in theUSA for a report he is writing. After loading a relevant dataset, Axelbrowses templates that have been created by others. Axel benefitsfrom how other members of his team can easily grow the collectionof templates to adapt to the team’s changing visualization needs.Unsure of what values are in the dataset, he selects a univariatebar chart template and uses fan out (Sec. 4.4.2) to view all of thedimension fields simultaneously. As he views these summaries,he gradually removes fan out options that are not interesting ordo not serve his task. He eventually selects the option to display "race" . He notices that there are 16 related templates (Sec. 4.4.1)that he could use. The “Isotype Waffle” catches his eye. He selectsit to view a fully formed Isotype colored by "race" . The chartdoes not have quite the right dimensions so he adds width and height variables to the template, which allows him to make smallincremental adjustments from the GUI to produce the final version(Fig. 1f). He takes a screenshot and adds it to his report.
Tabitha is interested in exploring the Gapminder dataset [69], so sheloads the data and selects the IvyPolestar template. She explores thedata using the familiar row and column abstractions (as in Fig. 5) toinvestigate iterative hypotheses. After viewing several scatter plotsand bar charts, Tabitha wants to see a hierarchical representationof the part-to-whole relationships between region, country, andGDP; a Sunburst chart comes to mind.Unable to find a satisfactory chart in the Ivy gallery, she browsesthe Vega gallery and finds one, but the way that it works with datadoes not quite match her intended usage. She creates a blank tem-plate in Ivy and pastes in the example from the Vega gallery. She isthen presented with a series of automatically generated suggestions(Sec. 4.3.3) on how she might templatize her chart—clicking throughthese creates new shelves as appropriate. She text-edits the datatransformation logic to accommodate her desired functionality. Sheuses the built-in debugging tools to view the results of the currentdata state, iteratively developing her transformations. She addsparameters for controlling width, height, color scheme, and otheraesthetic values, at which point she is happy with the templateand decides to share it. She clicks the publish button to make thetemplate available on the community server, ensuring that she andothers can reuse her work in the future. Finally, she instantiatesthe Sunburst for her dataset and takes a screenshot.
We now describe the design and implementation of Ivy. First, inSec. 4.1, we introduce to the components of our system with a nar-rative walkthrough. Next, in Sec. 4.2, we present a rigorous formu-lation of parameterized declarative templates —a grammar-agnosticmechanism for abstracting JSON-based specifications—in a smallprogramming language. Then, in Sec. 4.3, we describe our UI designfor selecting and instantiating templates, designed to obtain thebenefits and familiar aesthetics of chart choosing and shelf building.Finally, in Sec. 4.4, we describe how the systematic application oftemplates enables beneficial forms of view exploration. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan { "templateAuthor": "Ivy Authors", "templateName": "Aggregate Bar Chart", "templateDescription": "An aggregating bar chart that can by sorted.", "templateLanguage": "vega-lite", "parameters": [ {"name": "yDim", "type": "DataTarget", "config": {"allowedTypes": ["DIMENSION"], "required": true}}, {"name": “sort", "type": "Switch", "config": {"active": "true", "inactive": "false", "defaultsToActive": true}}, {"name": “year", "type": "List", "config": { "allowedValues": ["2000", "1990", "1980", "1970", "1960", "1950", "1940", "1930", "1920", "1910", "1900", "1890"] }}, {"name": "xDim", "type": "DataTarget", "config": {"allowedTypes": ["MEASURE"], "required": true}}, {"name": “color", "type": "List", "config": {"allowedValues": ["\"purple\"", "\"steelblue\"", "\"coral\""]}} ], "body": "{ '$schema': 'https://vega.github.io/schema/vega-lite/v4.json', 'height': { 'step': 17 }, 'data': { 'name': 'myData' }, 'transform': [{ 'filter': 'datum.year == [year]' }], 'mark': { 'type': 'bar', 'color': ‘[color]' }, 'encoding': { 'y': {'field': '[yDim]', 'type': 'ordinal', 'sort': {'$if': 'sort.includes(`true`)'', 'true': '-x'}}, 'x': {'aggregate': 'sum', 'field': '[xDim]', 'type': 'quantitative'} } }"}123456789101112131415161718192021222324252627{ "$schema": "https://vega.github.io/schema/vega-lite/v4.json", "height": {"step": 17}, "data": {"url": "data/population.json"}, "transform": [{"filter": "datum.year == 2000"}], "mark": "bar", "encoding": { "y": {"field": "age", "type": "ordinal", "sort": "-x"}, "x": { "aggregate": "sum", "field": "people", "title": "population"} }}12345678910111213 B The corresponding Vega-Lite spec for generating that chart. A conditional which consists of a JavaScript predicate ($if), as well as (optional) true and false branches.A reference to the xDim parameter.Parameter declaration for variable "xDim", including type and configuration options. C Ivy template for an optionally sortable aggregate bar chart. A fan out across a subset of the allowed years. E agepeople Fan out control, the darked icon indicates that there is a fan out across that variable.This color shows the data roles specified by the template designer for this data target. This specification enables typed-based matching, but can be overridden or ignored by the user, as here in xDim. Color indicates the automatically inferred, or manually set, data role for this data column. D The resulting user interface for the template. A The original chart of interest gathered from the Vega-Lite documentation. yearxDimyDimcolorsort
Figure 3: Users can transform example chart specifications into reusable templates. Here we show the process of taking (A) abar chart of interest and (B) its corresponding Vega-Lite spec, and transforming it into (C) an Ivy template. Using (D) thecorresponding GUI shelf builder pane, the user can (E) fan out across parameters of interest to see alternatives simultaneously.
We now introduce the technical components of our system with anarrative description of our hypothetical user Tabitha constructinga reusable template from an example.She begins by loading a population dataset in Ivy. She then copiesin the “Aggregate Bar Chart” [84] from the Vega-Lite documentation(Figure 3b). Figure 3a shows the output visualization.After pasting in the code into the template body (via the Bodytab of Fig. 1c), several automated suggestions are provided on howthe data fields could be abstracted as template parameters . Clickingthrough the suggestions replaces the "age" and "people" datafields (Figure 3b, lines 8,10) with two new
DataTarget parameters(Figure 3c, lines 7,13). Tabitha renames the generated parameters to xDim and yDim which automatically replaces their uses (enclosedby “escape” brackets) with [xDim] and [yDim] (Figure 3c, lines 23-24). She uses the settings popover (Fig. 6) to specify their alloweddata roles—she could have also done so using the Params text box(Fig. 1c). These configurations result in a shelf builder style userinterface which she uses to explore her data set. She then usesthese shelves to specify values for the xDim and yDim parameters(Figure 3d). These value settings are then applied to the templatebody to produce a JSON specification, which is then transformedinto a visualization by a language-specific rendering function, inthis case, by Vega-Lite.Next, she wants to chart populations for different years, so shereplaces the constant value (Figure 3b, line 5) with [year] (Figure 3c, line 20), referring to a new template parameter thatabstracts over the choice of year (lines 9-12). Intending to apply thistemplate only to datasets with decennial measurements, Tabithaspecifies that year should be chosen from among a new
Enum type,called "allowedValues" , comprising census years (Figure 3c, line11). (She chooses not to abstract the filtering predicate, which wouldhave resulted in a both more general and more complex template.)Then, Tabitha considers whether and how to sort the bars. Know-ing the appropriate Vega-Lite option, she adds a "sort" field witha conditional to the template body (Figure 3c, line 23). She thencreates a new template parameter, sort (line 8), and configuresit such that if sort is set to true , then the "sort" value in theresulting Vega-Lite specification is set to "-x" , which, in renderedVega-Lite, sorts the bars in order of increasing value. If sort is setto false , then the resulting spec contains no "sort" field—as inthe original specification (Figure 3b).Lastly, Tabitha wants to enable control of the color of the bars inher chart. To do so, she first defines a color parameter (Figure 3c,line 14), and then adds a reference to this parameter on line (Fig-ure 3c, line 21). This allows her, or other users of this template, topick between colors preferred by her organization.Her template now satisfactorily prepared, she is ready to exploreher dataset by specifying parameter values through the GUI (asin Figure 3d). In addition to specifying individual value settings,she can explore multiple options simultaneously using fan out (Fig-ure 3e), which juxtaposes multiple views to consider (Sec. 4.4.2).
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
Templates 𝑡 :: = (cid:40) func = 𝜆 ( 𝑥 : 𝑇 , . . . ). 𝑒 lang = L , metadata = · · · , symbols = [ 𝑦 , . . . ] , Atomic Values 𝑎 :: = 𝑠 | 𝑛 | 𝑏 JSON Values 𝑗 :: = 𝑎 | { 𝑠 : 𝑗 , . . . } | [ 𝑗 , . . . ] Expressions 𝑒 :: = 𝑎 | { 𝑠 : 𝑒 , . . . } | [ 𝑒 , . . . ] Variables | 𝑥 | 𝑦 Conditionals | if 𝑝 then 𝑒 ( else 𝑒 ) Predicates 𝑝 :: = 𝑠 ( where ⟦ 𝑠 ⟧ JS = 𝑏 ) Settings 𝑆 :: = 𝑥 ↦→ 𝑎 , . . . Data Roles 𝑅 :: = Measure (cid:117) | Dimension (cid:117) | Time (cid:117)
Param. Types 𝑇 :: = DataTarget args | MultiDataTarget args | String args | Number args | Boolean args | Enum args | Text args | Section args
Spec. Lang. L :: = Vega | Vega-Lite | Atom | Table | · · ·
Views 𝑣 :: = ( L -language specific rendering) Template Application 𝑡 ( 𝑆 ) = 𝑣𝑡. lang = L 𝑡. func = 𝜆 ( 𝑥 : 𝑇 , . . . ). 𝑒 𝑆 = 𝑥 ↦→ 𝑎 , . . . A. Substitute Arguments (cid:122) (cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32) (cid:123) 𝑆𝑒 = 𝑒 ′ B. Evaluate Conditionals (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ⟦ 𝑒 ′ ⟧ Ivy = 𝑗 C. Render Visualization (cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123) ⟦ 𝑗 ⟧ L = 𝑣 Evaluation of JSON Expressions ⟦ 𝑒 ⟧ Ivy = 𝑗 Atomics ⟦ 𝑎 ⟧ Ivy = 𝑎 (1) Objects ⟦ { 𝑠 : 𝑒 , . . . , 𝑠 𝑛 : 𝑒 𝑛 } ⟧ Ivy = ∪ 𝑖 ⟦( 𝑠 𝑖 : ⟦ 𝑒 𝑖 ⟧ Ivy )⟧ Ivy (2)
Lists ⟦ [ 𝑒 , . . . , 𝑒 𝑛 ] ⟧ Ivy = [ ⟦ 𝑒 ⟧ Ivy , . . . , ⟦ 𝑒 𝑛 ⟧ Ivy ] (3) Evaluation of Conditionals ⟦ 𝑒 ⟧ Ivy = 𝑗 or ⊥⟦ if 𝑝 then 𝑒 ( else 𝑒 )⟧ Ivy = ⟦ 𝑒 ⟧ Ivy if ⟦ 𝑝 ⟧ JS = true (4) ⟦ if 𝑝 ( then 𝑒 ) else 𝑒 ⟧ Ivy = ⟦ 𝑒 ⟧ Ivy if ⟦ 𝑝 ⟧ JS = false (5) ⟦ if 𝑝 then 𝑒 ⟧ Ivy = ⊥ if ⟦ 𝑝 ⟧ JS = false (6) Evaluation of Conditional Fields ⟦( 𝑠 : 𝑒 )⟧ Ivy = { 𝑠 : 𝑒 ′ } or ∅⟦( 𝑠 : 𝑒 )⟧ Ivy = ∅ if ⟦ 𝑒 ⟧ Ivy = ⊥ (7) ⟦( 𝑠 : 𝑒 )⟧ Ivy = { 𝑠 : ⟦ 𝑒 ⟧ Ivy } otherwise (8) Figure 4: The Ivy template language is composed of an abstract syntax grammar (left) and evaluation rules (right).
Templates provide a simple set of abstractions over JSON-basedgrammars. Put simply, a template is a function specified in a su-perset of JSON, which includes variables and simple control flowoperators, that when applied to arguments produces a chart in aparticular visualization grammar. Templates are grammar-agnosticas they abstract arbitrary JSON specifications. We provide a fulldescription of templates to highlight exactly how they make declar-ative grammar specifications reusable ( G4 )—by combining multiplespecifications into a single template—and easier to use ( G1 )—bydemarcating the arguments for manipulation.Formally, we define a template 𝑡 as a function that applies a setof 𝑁 template parameters , 𝑥 𝑖 with type 𝑇 𝑖 , to a template body 𝑒 to generate a view 𝑣 . In addition to the func tion itself, a template isa record that defines: the output JSON specification lang uage L ofthe template body, metadata , and a list of zero or more template-specific constant symbols that the body may refer to (Sec. 4.2.2). Oursimplified model of JSON [15] values 𝑗 comprises atomic values 𝑎 —literal strings 𝑠 of type String , numbers 𝑛 of Number , and booleans 𝑏 of type Boolean —records of string-value pairs, and lists of values.The abstract syntax of templates is defined in Fig. 4.
Beyond JSON literals, a template body, or expression 𝑒 , may employ two basic programming constructs. A variable refers to a template function parameter 𝑥 or a templatesymbol 𝑦 , wherever a JSON literal value might normally appear. A conditional expression , written if 𝑝 then 𝑒 ( else 𝑒 ) , is dependenton the result of predicate expression 𝑝 , which is a “raw” JavaScript code string that evaluates to a boolean value. The else-branch isoptional, which supports optionally-defined fields, per Sec. 4.2.3.Fig. 3c shows the concrete syntax for variables and conditionalsin Ivy, which implements the abstract syntax of Fig. 4. Variables are“escaped” with square brackets (e.g. “ [year] ”). Following similarsupport in other tools [36], conditionals are written as shown inFig. 3c line 23. Full-featured languages typically provide more exten-sive abstraction mechanisms, but—as we show in Sec. 5—even justvariables and conditionals suffice for a wide variety of use cases. The 𝑁 parameter declarations 𝑥 𝑖 withtypes 𝑇 𝑖 serve two purposes: to abstract over data fields and stylis-tic choices in the definition of a visualization and to define GUIelements that allow users to specify argument values for theseparameters. For example, a “switch” widget is drawn for each pa-rameter of type Boolean , a dropdown menu for each
Enum , anda Tableau-style “shelf” for each
DataTarget (Fig. 3d). Fig. 4 de-fines several parameter types 𝑇 . The data target types DataTarget and
MultiDataTarget range over data columns (identified withstrings) in the current dataset as well as template-specific symbols(also encoded with strings in our implementation). Symbols aretemplate-specific values that can be used to instantiate parameters.For example, IvyPolestar defines a count symbol that induces atargeted channel to take on a count aggregation.Each type 𝑇 𝑖 contains type-specific arguments args 𝑖 , for example,the minimum and maximum allowed values for a Number or thevalues comprising an
Enum . The data parameter types—
DataTarget and
MultiDataTarget —carry a configuration field to define a datarole 𝑅 , discussed further in Sec. 4.3. Each type 𝑇 𝑖 also contains an ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan optional boolean expression 𝑝 𝑖 that determines whether the GUIshould display controls for that parameter 𝑥 𝑖 —for example, display-ing a sort direction widget only when a boolean sort parameter isset to true . Fig. 4 elides details of these type-specific arguments.As parameters are arguments to a function, they must also havevalues. We refer to a user’s choice of argument values 𝑎 𝑖 as settings 𝑆 . Settings are typically set using the GUI, but can also be specifiedas text (Fig. 1c, “Settings” tab). Finally, we producevisualizations by applying the function described by template 𝑡 to the parameter settings 𝑆 , following the three steps found inFig. 4 A-C. First (Fig. 4A), the argument values 𝑎 𝑖 are substitutedfor the parameters 𝑥 𝑖 in the template body 𝑒 using straightfor-ward substitution. For example, this transforms a snippet of thebody in Fig. 3c, {"y": {"field": "[yDim]"}, ...} with settings {"yDim": "age", ...} to be {"y": {"field": "age", ...}} . Second (Fig. 4B), the resulting expression is evaluated to producea JSON value, transforming any conditionals into JSON values. Thisis done by a straightforward recursive traversal, following Eqns(1)-(3). Whenever a conditional is encountered, it is executed byevaluating the JS-based predicate ( ⟦·⟧ JS referring to JS evaluation),and then replacing the conditional with either the correspondingthen- or else-branch as appropriate. Eqns (4) and (5) handle the twousual cases for conditionals. Eqn (6) handles the case when the pred-icate evaluates to false but no else-branch is provided; returninga ⊥ value, indicating that the conditional is to be deleted. Eqns (7)and (8) explicitly provide a mechanism to delete conditionals thatreturn ⊥ in record values. This is how, for example, Fig. 3c line 23optionally defines a "sort" field. Our implementation generalizesthis rule to arrays by viewing them as numerically indexed records.The third (Fig. 4C) and final step is to use the generated JSON torender a visualization. This is done by, for a specific visualizationlanguage L , using the corresponding rendering function ⟦ 𝑗 ⟧ L tointerpret the JSON value and render the resulting view—typicallyas an HTML Canvas or SVG. Our current implementation supportsfour languages L , namely, Vega, Vega-Lite, Park et al.’s Atom [62],and a simple table language. However, as we discuss in Sec. 6.1.1,Ivy’s language support is designed to be extensible. Equipped with the notion of templates, we next describe the userinterface design of Ivy. As shown in Fig. 1, the application consists oftwo panes, one for chart editing and another for chart viewing. Thechart editing pane contains a data column filled with Tableau-style“pills” representing data columns, and an encoding column with“shelves” for those pills to be placed upon. This encoding column canbe used to instantiate (i.e. provide arguments for) the parameters ofthe template, or to edit the GUI of the current template. The editingpane also includes a code editor which can manipulate the currenttemplate and UI state textually. Per Saket et al. [72], we facilitatemultimodal interaction by tightly synchronizing these views, suchthat changes in one modality are instantly reflected in the other.Below, we describe how Ivy supports the creation , selection , and application of templates to produce charts. The root of Ivy is atemplate gallery, which aims to achieve Satyanarayan et al.’s [78]vision for a system “allowing users to browse through designs forinspiration, or adapt them for their own visualizations.” This avoidsthe blank canvas problem [75] and supports ease of use ( G1 ).The gallery is populated with a library of system-provided tem-plates, as well as templates created by Ivy users (stored on a com-munal server). Simpler templates allow users to jump quickly tofamiliar visual forms (such as line charts or bar charts), while moresophisticated templates privilege thinking with their data [68] (asin IvyPolestar). The gallery is present both as a homepage for theapplication, independent from the visualization editor, as well asan intermediate view while creating new view tabs.Each template is accompanied by a set of user-defined examples,namely, settings chosen by users to instantiate the template, withdata bindings and output renderings with respect to a collection ofpredefined datasets. These examples serve as “crowd-sourced docu-mentation” for how individual templates operate. Furthermore, thisadds an element of opportunistic programming: to create templates,users can borrow small snippets—such as a well-formatted list ofcolor schemes—and use them in their own creations. After selecting atemplate and uploading a dataset of interest, the user is presentedwith a shelf builder-style GUI for setting the template parametersand specifying basic data filters. We choose this GUI design seekingto exploit the same affordances that drive the explorability ( G2 ) ofshelf builders. Specifically, our design closely follows that of thePolestar shelf builder system, which Wongsuphasawat et al. [97, 98]constructed as a simulacra of Tableau to serve as a baseline compari-son in the development of their recommendation-based explorationsystems. While emulating Polestar is a relatively small thresholdto overcome in the context of visualization systems in general, itdemonstrates the promise of our template-based approach. Systemssuch as Tableau or PowerBI possess features that—although sub-stantially larger and more complex—are not substantially different than those in Polestar.To select data parameters (i.e. DataTarget or MultiDataTarget )of interest, users drag-and-drop from a list of data fields, color-coded according to their data roles , onto encoding shelves, as inFig. 1a, d. Following prior work [1, 80, 97] roles include
Measure (cid:117) (quantitative fields),
Dimension (cid:117) (nominal or ordinal fields), and
Time (cid:117) (temporal fields). When a dataset is loaded, we make heuris-tic guesses about the role for each column, which the user canlater modify. We use roles in Ivy to construct a naive automatic
Add to Shelf feature (akin to Tableau’s Add to Sheet [54]), exceptours is simply based on order and data role. If a template has three
DataTarget s, the first of which allows only a
Measure (cid:117) while thelatter two allow anything, clicking
Add to Shelf on a
Dimension (cid:117) will add it to the second parameter.In addition to the visual aesthetics of Polestar (and hence thatof Tableau), we also emulate the functionality of its shelf-buildinginterface through a “default” library template called IvyPolestar(shown in Fig. 5). The only features not replicated are the AutomaticMark Type—implementation of which, though possible in Ivy, wasbeyond the scope of the paper—and the chart bookmarks—whichwe replaced with a notion of view tabs.
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
Template defined symbols. This enables both the Polestar-style COUNT field, as well more advanced features, such as row and column shown here, which enable a Tableau-style multi-shelf.
Figure 5: Our IvyPolestar template reproduces and extendsthe functionality of the Polestar shelf builder. Here tem-plate specific symbols are used to make a SPLOM.
The config of each parameter can be modified through a type specific popover (here a slider).Automatically generated suggestions for templatizing the current specification.Users can edit the template metadata here, including the name and template language. Parameters can be reordered by dragging. Their display status in view mode is shown here as SHOWN or HIDDEN based on the current param settings.
Figure 6: In edit mode, users can modify templates boththrough the shelf builder GUI and the textual code pane.
A notable feature in Polestar is a count symbol that is visu-ally similar to normal data fields and induces count aggregationson channels without a selected data column. To model this fea-ture, we define a template-specific symbol (cf. Sec. 4.2.2), count ,which IvyPolestar uses to implement the corresponding functional-ity. IvyPolestar introduces two additional symbols, row and column ,that enable faceting by data columns in the manner of Tableau’smulti-shelves. Placing row or column symbols on any shelf createsa MultiDataTarget which acts as a wrapper around Vega-Lite’sjuxtaposition operators [76]. Fig. 5 highlights this feature.
Templates can be createdor modified in two ways, either by modifying the textual representa-tion (Fig. 3c) or through GUI interactions (Fig. 6). The textual repre-sentation facilitates both small tweaks (Sec. 3.1), as well as creatingnew templates. For instance, users may copy code snippets foundonline—such as in language documentation or Stack Overflow—andtemplatize them to suit their task (Sec. 3.2). Templates can also be created by “freezing” and refining the GUI state when interact-ing with an existing template. For example, a user might apply afull-featured template, such as IvyPolestar, to construct somethingresembling their desired chart, fork the text output as a new tem-plate (as in Fig. 1d), and then provide fine textual grained updates.As discussed previously, exposing textual representation to the enduser furthers our flexibility ( G3 ) and reusability ( G4 ) goals.To ease the construction of templates, Ivy uses domain-specificpattern matching and rewrite rules to suggest potential transforma-tions to users. For instance, if a user were to find a chart in the Vegadocumentation that they wanted to copy, they would simply starta new template and paste the code into Ivy. The code pane thensuggests ways to transform the code. For example, if a value in aVega-Lite spec (such as in Fig. 3b) is used where a data reference isexpected (e.g. "field": "age" ), then Ivy suggests swapping "age" with a reference to a new parameter. Fig. 6 shows an example ofsuch rules. Rules are defined by Ivy developers, rather than Ivyusers. We implemented rewrite rules for Vega-Lite, Vega, and Atom. The systematic formulation and application of templates allows usto emulate recommendation and exploration ( G2 ) features foundin a variety of existing charting systems as a consequence of ourdesign. Here we highlight two such features that follow naturallyfrom the use of templates: one arises by fixing the arguments andvarying the template, and the other by fixing the choice of templateand varying the arguments. The het-erogeneity of user needs is often addressed in chart choosers byoffering large and often diverse sets of charting options [54], whichcan be intimidating or difficult to utilize due to their volume.The gallery in Ivy is equipped with catalog search , which allowsusers to search across the set of available templates based on com-patibility with a set of specified columns of interest—specifically,by using a simple type-compatibility algorithm that compares tem-plate parameter types with the data roles of selected columns. Thisfeature exists both in the gallery—where it acts as a type-basedsearch mechanism—as well as ambiently throughout the systemin the related templates tab (Fig. 1f)—where it acts as a simplealternative recommendation system. The current template is com-pared with each other template, yielding a partial match , a completematch , or no match . A partial match occurs when the selected datacolumns can be mapped to a template’s parameters. A match iscomplete if all required parameters are mapped. A complete matchcan translate the current selection without additional specification.This heuristic is further detailed in the appendix. When a match isselected, the current columns are mapped onto the new template,using a similar mechanism as our
Add to Shelf , and the resultingchart is shown immediately.
Comparisonsin visual analytics are often made temporally, requiring the analystto hold mental reference to each of the values under consideration.To reduce this cognitive burden, Ivy users can fan out a templateby applying multiple settings and rendering their output simulta-neously. Fig. 3e and Fig. 7 display examples of this interaction. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan
End the fan out and use this value.Remove this value from the fan out. D e s i g n V a r i a t i o n B Fan out across the yDim of a horizon chart exploring the happiness dataset from our user studyFan out across projections in a time-zone choropleth showing the distribution of students in a socially-distanced course.
Fan out suggestions are inferred based on data role. Just as with other data role inferences these can be ignored as the user wishes. D a t a V a r i a t i o n A Figure 7: Users rapidly explore design and data alternativesvia fan outs. After selecting parameter values of interest,they are shown all variations simultaneously.
To begin a fan out a user specifies sets of values that they wishto compare for each parameter of concern. This can be applied toboth design and data parameters. For data parameters we providesuggestions of appropriate values based on the inferred data roleand specified parameter role (Fig. 7a). We then compute a cartesianproduct of these sets and render a separate instance of the currenttemplate for each combination of values. Users then browse theresulting gallery, and can modify all of the combinations at oncethrough the shelf-building UI, as well as remove values or select acombination to view in isolation (thus collapsing the fan out).This approach allows users a low-stakes way to consider alterna-tive chart configurations and rapidly explore the space of availabledesign and data parameters.
In the previous section, we discussed how our system design in-tegrates and augments UI capabilities provided by existing chartchoosers, shelf builders, and text-based editors. Here, we assess howwell our template-based approach may work in practice by consid-ering two questions:
Do templates facilitate organization andreuse of existing visualizations? (Sec. 5.1), and
Is Ivy’s multi-modal UI approachable by real users? (Sec. 5.2)
Wrapped Time SeriesWaterfall TemplateTime Series with HalosTemporal Bar ChartSinusoidal WavesSeries-based Time SeriesScatterplot Mini DashboardRank over timePre-Calculated Box PlotParallel CoordinatesMulti Layer HistogramHeatmap DashboardDumbbell ChartDiverging Stacked Bar ChartDiscretizing scalesConnected ScatterplotColumn-based Line ChartCandlestick ChartBrushable HistogramBar Chart With MinimapSPLOMMosaic PlotIsotype WaffleHorizon ChartGantt Chart
Some examples fell outside our data model and had to be skipped
Template Name Skipped Examples C o un t o f un i q u e ex a m p l e s c o v e r e d b y t e m p l a t e B a r C h a r t S c a tt e r p l o t L i n e C h a r t S m a ll M u l t i p l e s U n i v a r i a t e S u mm a r y H e a t m a p A r e a C h a r t C a l c u l a t i n g B a r C h a r t P i e C h a r t A d v a n c e d U I S c a tt e r p l o t S c a tt e r M a p C a t e g o r i c a l E rr o r C h a r t D o t P l o t C o m b i n e d P l o t C h o r o p l e t h A d v a n c e d U I T i m e s e r i e s P o pu l a t i o n P y r a m i d C r o ss f il t e r S i n g l e Ex a m p l e T e m p l a t e s ( O r ph a n s ) wheat_wageslookuplayer_line_windowlayer_likertlayer_falkenseelayer_bar_annotationsinteractive_query_widgetsinteractive_multi_line_pivot_tooltipgeo_rulegeo_linegeo_layer_line_londonfacet_bulletconcat_layer_voyager_resultbar_layered_weatherairport_connections Figure 8: We reproduced the 166 examples found in the Vega-Lite gallery using 43 templates. Some examples had to beexcluded (right) as they fell outside our data model, usuallybecause they utilize multiple data sources, which Ivy doesnot currently support.
We considered two chart corpora, the example gallery of Vega-Lite [88] and the chart chooser found in Google Sheets [19], look-ing for opportunities to factor related visualizations into templates,which yielded 3.5x and 1.8x compression ratios , respectively. Com-pression is the number of examples constructible by a given tem-plate. The reported average compression are computed by | Examples | − |
Excluded by Data Model || Templates | (9)In an example gallery, a larger ratio means less duplicated code(text) among examples. In a chart chooser, it means fewer chartforms with possibly more parameters. These results demonstratethat the simple template abstraction mechanisms enhance the flexi-bility ( G3 ) of existing declarative grammars while improving theirreusability ( G4 ) by serving a variety of use cases.While even just the basic abstraction mechanisms in templates—namely, variables and conditionals—are sufficient for merging allexamples of each corpus into single templates, we strove to con-struct templates that are factored in reasonable ways, for example,by not depending on datasets having particular columns or fea-tures, and by considering what might plausibly be found in a chartchooser. This process prompted a number of minor system improve-ments, such as simplifying the conditional syntax and improvingerror handling, and suggested directions for future work on Ivy,such as a more sophisticated data model. Additional details of thisanalysis can be found in the appendix. This gallery consists of 166 dis-tinct specifications, reflecting years of iterative refinement anddevelopment by the Vega-Lite community. These examples high-light a breadth and depth of features exposed by the library, thusforming an ideal testbed for the utility of templates. We recreatedthis corpus with 43 templates, skipping 14 examples due to incom-patibility with our data model (a single, flat input data table) for a3.5x compression. Fig. 8 reports the frequency of each template.We aimed to capture both the core content of each example (themajor feature being demonstrated) as well as the resulting image.However, this was not always possible. We allowed minor text mod-ifications and the inclusion of properties not present in the originalexample, as long as they did not affect the core of the example
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh (such as styling). It was sometimes necessary to forgo or modifysome examples to accommodate our current implementation. Forinstance, a scatter-map of zip-code centroids colored by their firstdigit [87] needed to be modified because Ivy currently lacks a cus-tom calculation feature. As a way to guide this design, we strove tomaximize the concatenation ratio , or the ratio between the size of atemplate body and that of the concatenated examples it captures,which is computed for a particular measure of size 𝛾 , (cid:205) 𝑥 ∈ covered examples 𝛾 ( 𝑥 ) 𝛾 ( template ) (10)The resulting templates have an average lines of code concatenationratio of 1.48x (a proxy for simplification) and an average abstractsyntax tree concatenation ratio of 1.80x (a proxy for UI complexityminimization). In conjunction with the average example compres-sion ratio of 3.5x, this suggests that our templatizations are betterthan merely concatenating the examples. The set of charts found in thischooser consists of 32 distinct chart forms. We focused on GoogleSheets as a representative chart chooser because it possesses asimilar set of chart options as other choosers, such as Excel andLibreOffice. We reproduced 29 of these charts through 16 templatesfor a 1.8x compression factor. We skipped 3 examples for the fol-lowing reasons: “3D Pie” charts require a 3D visualization grammar,“Org. Charts” fall outside of our tabular data model, and “Timelines”require annotations, and therein modifications to the underlyingdataset, which is not currently supported in Ivy.
We conducted a study in order to evaluate whether the multimodaluser interface in Ivy is approachable by real users. During pre-study pilot sessions, we found that users who were more familiarwith Vega-Lite learned more quickly how to integrate Ivy’s fullcapabilities to address tasks than those unfamiliar with Vega-Lite.Therefore, we aimed to recruit users familiar with these paradigmsin order to evaluate the approachability of interacting with multiplecharting modalities, rather than that of the underlying grammars.The target expertise for our study can be characterized roughly asbetween that of the hypothetical users from Sec. 3.
We solicited 5 participants from a recent vi-sualization course in a computational public policy masters pro-gram, all of whom are now employed as professional data analysts.Based on their participation in the course, we were confident thatthese now-graduated students were familiar with Vega-Lite andTableau. Participants, denoted P1 through P5, self-reported a meanfamiliarity with Vega/Vega-Lite of 𝜇 = . 𝜇 = . Following a brief introduction to Ivy through in-application documentation, participants completed a series of 8tasks that covered data exploration, chart specification, and tem-plate construction problems (additional details in the appendix).For instance, one task asked participants to templatize a particular Vega-Lite-based box plot [85], while another asked them to find thenumber of countries in a multi-year version of the World HappinessReport [27]. We focused on these tasks because they are similarto tasks supported in related systems and (with the exception oftemplate building) were addressable with any of the modalities indi-vidually or in conjunction. Correctness was judged by comparisonwith a solution set prepared prior to the study. We engaged subjectsin an informal think-aloud protocol during the session, which led toa number of the reported observations in the next section. Subjectswere then asked to fill out an exit survey on the usability of Ivy andtemplates. Sessions were held over video conference software andlasted a mean of 𝜇 =
95 minutes, although 2 hours were allotted.Participants received a $50 Amazon gift certificate for their work.
All participants were able to complete all taskswithin the allotted time, although all users required some assis-tance at various stages, typically due to implementation bugs orlearnability hurdles. A variety of strategies were used to accomplishthe data exploration tasks, some using only shelf building, someonly chart choosing, or a mixture of both. All users were able tocomplete the template construction tasks and then use the tem-plates to address data exploration tasks. We believe this indicatesthat real users, once acclimated, are able to produce non-trivialtemplates to accomplish varying goals.Ths system was generally seen as usable. Participants mostlyagreed with the statement “I think that I would like to use thissystem frequently" ( 𝜇 = . 𝜇 = . “feels moreuseful than just coding” . P4 noted that their organization had re-cently switched a major Tableau-based dashboard to a Shiny-baseddashboard because of a lack of precise data controls in Tableau. Incontrast to the push for visual analytics tools to completely shedties to text (embracing a “no-code” approach), we believe this sug-gests that systems that straddle the boundary between code andGUI specification can offer a valuable mixture of affordances thatsupport real use cases. We observed that Ivy held some challenges forparticipants. Participants sometimes struggled to understand whatthe templatization suggestions would do, indicating a closeness ofmapping [22] failure. P1 and P2 suggested that visual design couldbe improved to aid in feature discovery. While some participants(P2, P4) agreed that templates are good for creating rapidly reusablecharts, P2 noted that templates are “not very portable.”
This could beaddressed by embedding Ivy in tools like Shiny [70] or Jupyter [38]. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan
The learnability of the system and mixed-modality remained aprimary difficulty. Users typically struggled to figure out how tobridge the gap between text and GUI at first, however, by the end ofthe session, all users were competent in both regimes. For instance,most subjects iteratively refined their solutions to the box plottask, modifying both code and GUI values to address developinghypotheses. P3 noted that the system required a non-trivial level ofcomputational and visualization literacy. Fortunately, users couldseemingly bootstrap their knowledge to overcome these hurdles. P4noted that the integration between the “code body and the point-and-clickable GUI is really tight and also good for reinforcing learning” and that “If you know how to do something in one form, you can doit and watch how it changes the other side of the tool.”
The scope of this study was small. We merely sought to demon-strate that real users of similar systems could approach the mixed-modality UI found in Ivy. While these results suggest that thiscombination is promising, further investigation is required to un-derstand its utility in the context of more developed interfaces.Once our system reaches maturity, we intend to conduct a studycomparing it with standard analytics tools, such as Tableau or Excel.
In this paper, we described how parameterized declarative tem-plates —a typed abstraction layer over JSON specifications—canserve as a basis for a multimodal UI to create and explore visu-alizations. Ivy-style templates may help in the organization andreuse ( G4 ) of existing visualization corpora (per Sec. 5.1). Vega andVega-Lite have garnered ample popularity, and new declarativevisualization grammars are being actively developed [42, 49, 96].As the availability and use of these grammars continues to prolifer-ate, there is opportunity for shared platforms and tooling betweenlanguages, which we explore in our grammar-agnostic templates.The integration of features in our prototype appears to be accessi-ble to users with modest experience in both visual analytics systemsand Vega/Vega-Lite (per Sec. 5.2). Users were able to make effectiveuse of affordances for exploration found in our shelf building UIand fan out ( G2 ), and were able to utilize the capability of templatesto improve the ease of use ( G1 ) and reuse ( G4 ) of declarative chartspecifications while maintaining their flexibility ( G3 ).We believe that this multimodal approach has value for a varietyof use cases. Exposing a connection between GUI and programmaticAPI may enable analysts to self-serve their chart creation needs.If a particular chart form is not available (but is constructible byone of the supported grammars) then they can create it for them-selves, rather than requiring reliance on engineering resources.This connection between text and GUI appears to help users learnand comprehend JSON-based charting grammars, which may beunfamiliar or difficult to understand. The repeatable customizationfound in templates might also, for example, enable practitioners(e.g. data journalists) to explore designs in a structured manner thatdoes not violate their organization’s visual identity. The version of multimodal visual analytics found in our prototypehas its share of limitations. The strength of each modality in Ivyis only as good as its implementation, which can render artificial barriers between what users expect and what is supported (e.g. P3expected a pivot table). And while Ivy encompasses chart choosing,shelf building, and textual specification, it does so at the cost of anincreased learning curve. However, we believe that this difficultyis not endemic to multimodal systems, and that through attentivedesign the experience of using the system can be made easier.In making our template-based approach more viable for practicaluse, it is easy to imagine a variety of system improvements—suchas additional template parameter types (e.g. color schemes or inlinedata fields), drag-and-drop interactions for refactoring and abstract-ing specifications (e.g. [29, 47]), as well as enriching the ways inwhich changes made in one modality are reflected and explainedin the others. Beyond these, we highlight below several avenuesfor future research.
Ivy is designed to be extensible: sup-port for each specification language is defined through a standard-ized interface, which includes a JSON Schema describing the syntax,a JavaScript rendering function for the language, and rewrite rulesto help users abstract specifications. Our implementation currentlysupports a small set of languages (Vega, Vega-Lite, Atom , and asimple table language), which, in future work, we would like toincrease so as to support a greater variety of tasks.Our grammar-agnostic template framework provides a standardset of abstraction and data manipulation mechanisms—which mayreduce the need for grammar designers to define their own—andnovel UI features for exploring candidate templates (catalog search)and encodings (fan outs)—which may facilitate more efficient andconsistent exploration. Our system, furthermore, hoists the burdenof data transformation out of the rendering grammar (albeit witha currently-limited set of transforms), which would otherwise re-quire each grammar to implement its own data manipulation logic.As users move between templates (specified in possibly differentgrammars), their settings (including filters) are mapped from onetemplate to the next via role and order-based heuristics. Futurework could enable translation between supported grammars, whichcould yield opportunities for education and portability.Despite the benefits of language-independent functionality, thereare also benefits to taking domain-specific knowledge into account.Language-specific rewrite rules—part of the extension interface,described above—are one such example. Language-specific knowl-edge could further be used for recommendation, as well as datamanipulation and presentation concerns. For instance, when a datafield is dragged to a drop zone in Lyra [74] the appropriate type ofscale is automatically inferred [75]. Lyra is able to offer this func-tionality because it has a model of the grammar being manipulated,a functionality which our approach currently lacks.A lack of context and content-aware automated guidance is akey limitation of our design. Yet, it should be possible to identify aricher extensibility API, while still allowing each language to benefitfrom the abstraction and UI concerns shared by all. Such an APIwould enable us to combine domain-specific chart recommendation(Sec. 2.2.3) with Ivy’s domain-independent type-based exploration(Sec. 4.4), as well as embrace new interaction modalities. HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
An important question forsystem designers is how to help users conduct safe visual analy-sis [101]. Analysts can deceive themselves with statistical traps [65],visualization hallucinations [43], or false graphical inferences [93].We believe that our type-based template search will dovetail witha metamorphic testing [58] based validation approach: by varyingthe parameters of a template and comparing the resulting images,a validation system could automatically identify errors at the inter-section of data and encoding. Similarly, we suggest that templateslikely offer an opportune medium for applying visualization lint-ing [33, 57] to a visual analytics context, as the types expose thespecific arguments over which analysis could be conducted. Fur-thermore, the fan out interaction could be extended to allow notonly juxtaposition of variants, but also their layering [51], enablingvisual sanity checks to the robustness of parameter selections [10].
A primary focus in this pa-per has been supporting data exploration tasks, but there are aconstellation of other tasks that users perform in visualizationtools. Users must carry out tasks “before” exploration (such asmodel building and data cleaning) and “after” exploration (such asannotation and presentation). Although the designs of many visualanalytics systems assume that data has been cleaned and processedprior to analysis [75], in practice these tasks are often interleavedand iterated.Integrating these tasks within a unified system may thus reappotential benefits. Data manipulation tasks, for instance, might bebetter facilitated by the incorporation of ideas from dataflow pro-gramming and spreadsheets, as well as programming-by-example techniques to help with data wrangling [16, 37, 39]. To more fullysupport presentation tasks, it would be fruitful to extend our com-bination of modalities to include visual builders [20]—which offer avariety of direct manipulation features [71, 79] for creating charts—and visualization by demonstration [72, 73, 91, 102].For each of these tasks, developing rich graphical interactions—while maintaining a “bidirectional” connection to the underlyingtextual representation—is an exciting research challenge: to inte-grate what is typically a vast divide between text and GUI basedanalytics systems. The approach pursued in this paper provides astep in this longer-term direction.
We thank our anonymous reviewers for their helpful comments, aswell as our study participants for their participation and invaluableinsights. We also thank Will Brackenbury, Michael Correll, GalenHarrison, Brian Hempel, Gordon Kindlmann, and Katy Koenig fortheir commentary and support.
REFERENCES [1] Mallika Agarwal, Arjun Srinivasan, and John T. Stasko. 2019. VisWall: Vi-sual Data Exploration Using Direct Combination on Large Touch Displays. In . IEEE, 26–30. https://doi.org/10.1109/VISUAL.2019.8933673[2] Christopher Ahlberg. 1996. Spotfire: an Information Exploration Environment.
ACM SIGMOD Record
25, 4 (1996), 25–29.[3] Aaron Bangor, Philip Kortum, and James Miller. 2009. Determining what indi-vidual SUS scores mean: Adding an adjective rating scale.
Journal of usabilitystudies
4, 3 (2009), 114–123. [4] Alex Bigelow, Steven Drucker, Danyel Fisher, and Miriah Meyer. 2016. IteratingBetween Tools to Create and Edit Visualizations.
IEEE Transactions on Visualiza-tion and Computer Graphics (2016). https://doi.org/10.1109/TVCG.2016.2598609[5] Mike Bostock. 2018. A Better Way to Code. https://medium.com/@mbostock/a-better-way-to-code-2b1d2876a3a0[6] Michael Bostock and Jeffrey Heer. 2009. Protovis: A Graphical Toolkit forVisualization.
IEEE Transactions on Visualization and Computer Graphics
15, 6(2009), 1121–1128. https://doi.org/10.1109/TVCG.2009.174[7] Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3 Data-DrivenDocuments.
IEEE Transactions on Visualization and Computer Graphics
17, 12(2011), 2301–2309. https://doi.org/10.1109/TVCG.2011.185[8] Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R.Klemmer. 2009. Two Studies of Opportunistic Programming: InterleavingWeb Foraging, Learning, and Writing Code. In
Proceedings of the SIGCHI Con-ference on Human Factors in Computing Systems . ACM, 1589–1598. https://doi.org/10.1145/1518701.1518944[9] Seth J. Chandler. 2018. Executions in the United States. https://datarepository.wolframcloud.com/resources/Seth-J.-Chandler_Executions-in-the-United-States Wolfram Data Repository.[10] Michael Correll, Mingwei Li, Gordon Kindlmann, and Carlos Scheidegger. 2018.Looks Good to Me: Visualizations as Sanity Checks.
IEEE Transactions onVisualization and Computer Graphics
CHI ’20: CHI Conference on Human Factorsin Computing Systems . ACM, 1–12. https://doi.org/10.1145/3313831.3376442[15] ECMA. 2017.
The JSON Data Interchange Syntax, ECMA-404
Proceedings of the 2020 Conferenceon Empirical Methods in Natural Language Processing: Findings . 1996–2005.[17] Flourish. 2020. Flourish | Data Visualization & Storytelling. https://flourish.studio/ https://flourish.studio/.[18] Jon Gold. 2016. Declarative Design Tools. https://jon.gold/2016/06/declarative-design-tools/ Accessed December 16, 2020.[19] Google. 2020. Types of charts & graphs in Google Sheets. https://support.google.com/docs/answer/190718 Accessed August 19, 2020.[20] Lars Grammel, Chris Bennett, Melanie Tory, and Margaret-Anne D. Storey. 2013.A Survey of Visualization Construction User Interfaces. In
EuroVis (Short Papers) .https://doi.org/10.2312/PE.EuroVisShort.EuroVisShort2013.019-023[21] Lars Grammel, Melanie Tory, and Margaret-Anne Storey. 2010. How InformationVisualization Novices Construct Visualizations.
IEEE Transactions on Visualiza-tion and Computer Graphics (2010). https://doi.org/10.1109/TVCG.2010.164[22] Thomas RG Green. 1989. Cognitive dimensions of notations.
People and com-puters V (1989), 443–460.[23] Li Guozheng, Min Tian, Qinmei Xu, Michael McGuffin, and Xiaoru Yuan. 2020.GoTree: A Grammar of Tree Visualizations.
Proceedings of the 2020 CHI Confer-ence on Human Factors in Computing Systems (2020). https://doi.org/10.1145/3313831.3376297[24] Pat Hanrahan. 2006. VizQL: A Language for Query, Analysis and Visualization.In
Proceedings of the 2006 ACM SIGMOD international conference on Managementof data . ACM, 721–721.[25] Jonathan Harper and Maneesh Agrawala. 2018. Converting Basic D3 Chartsinto Reusable Style Templates.
IEEE Transactions on Visualization and ComputerGraphics
24, 3 (2018), 1274–1286. https://doi.org/10.1109/TVCG.2017.2659744[26] Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R Klem-mer. 2008. Design as Exploration: Creating Interface Alternatives ThroughParallel Authoring and Runtime Tuning. In
Proceedings of the 21st annualACM symposium on User interface software and technology . 91–100. https://doi.org/10.1145/1449715.1449732[27] John F Helliwell, Richard Layard, and Jeffrey Sachs. 2012. World happinessreport. (2012).[28] Brian Hempel, Justin Lubin, and Ravi Chugh. 2019. Sketch-n-Sketch: Output-Directed Programming for SVG. In
Proceedings of the 32nd Annual ACM Sym-posium on User Interface Software and Technology . 281–292. https://doi.org/10.1145/3332165.3347925[29] Brian Hempel, Justin Lubin, Grace Lu, and Ravi Chugh. 2018. Deuce: A Light-weight User Interface for Structured Editing. In
International Conference onSoftware Engineering (ICSE) . https://doi.org/10.1145/3180155.3180165[30] Jane Hoffswell, Wilmot Li, and Zhicheng Liu. 2020. Techniques for FlexibleResponsive Visualization Design.
Proceedings of the 2020 CHI Conference onHuman Factors in Computing Systems (2020). https://doi.org/10.1145/3313831. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan
Proceedingsof the 2018 CHI Conference on Human Factors in Computing Systems . ACM.https://doi.org/10.1145/3173574.3174106[32] Marius Hogräfer and Hans-Jörg Schulz. 2019.
ReVize: A Library for VisualizationToolchaining with Vega-Lite . Eurographics. https://doi.org/handle/10.2312/stag20191375 ISSN: 2617-4855.[33] Aspen K Hopkins, Michael Correll, and Arvind Satyanarayan. 2020. VisuaLint:Sketchy In Situ Annotations of Chart Construction Errors. In
Computer GraphicsForum . https://doi.org/10.1111/cgf.13975[34] Kevin Hu, Diana Orghian, and César Hidalgo. 2018. DIVE: A Mixed-InitiativeSystem Supporting Integrated Data Exploration Workflows. In
Proceedings ofthe Workshop on Human-In-the-Loop Data Analytics . ACM, 1–7. https://doi.org/10.1145/3209900.3209910[35] John D. Hunter. 2007. Matplotlib: A 2D Graphics Environment.
Computing inScience & Engineering
Proceedings of the 2017 ACMInternational Conference on Management of Data (SIGMOD ’17) . Association forComputing Machinery, New York, NY, USA, 683–698. https://doi.org/10.1145/3035918.3064034[38] jupyter. [n.d.]. Interactive Widgets. https://jupyter.org/widgets AccessedDecember 16, 2020.[39] Sean Kandel, Andreas Paepcke, Joseph Hellerstein, and Jeffrey Heer. 2011. Wran-gler: Interactive Visual Specification of Data Transformation Scripts. In
Pro-ceedings of the SIGCHI Conference on Human Factors in Computing Systems .3363–3372. https://doi.org/10.1145/1978942.1979444[40] Mary Beth Kery, Donghao Ren, Fred Hohman, Dominik Moritz, Kanit Wong-suphasawat, and Kayur Patel. 2020. mage: Fluid Moves Between Code andGraphical Work in Computational Notebooks. In
Proceedings of the 33rd AnnualACM Symposium on User Interface Software and Technology . ACM.[41] Nam Wook Kim, Eston Schweickart, Zhicheng Liu, Mira Dontcheva, Wilmot Li,Jovan Popovic, and Hanspeter Pfister. 2016. Data-Driven Guides: SupportingExpressive Design for Information Graphics.
IEEE Transactions on Visualizationand Computer Graphics
23, 1 (2016), 491–500. https://doi.org/10.1109/TVCG.2016.2598620[42] Younghoon Kim and Jeffrey Heer. 2021. Gemini: A Grammar and RecommenderSystem for AnimatedTransitions in Statistical Graphics.
IEEE Transactions onVisualization and Computer Graphics (2021).[43] Gordon Kindlmann and Carlos Scheidegger. 2014. An Algebraic Process forVisualization Design.
IEEE Transactions on Visualization and Computer Graphics
20, 12 (2014), 2181–2190. https://doi.org/10.1109/TVCG.2014.2346325[44] Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian E. Granger,Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica B. Hamrick, JasonGrout, and Sylvain Corlay. 2016. Jupyter Notebooks-a publishing format forreproducible computational workflows.. In
ELPUB . 87–90. https://doi.org/10.3233/978-1-61499-649-1-87[45] Kari Lavikka, Jaana Oikkonen, Rainer Lehtonen, Johanna Hynninen, Sakari Hi-etanen, and Sampsa Hautaniemi. 2020. GenomeSpy: Grammar-Based InteractiveGenome Visualization. (2020).[46] Doris Jung-Lin Lee. 2020. Insight Machines: The Past, Present, and Futureof Visualization Recommendation. https://medium.com/multiple-views-visualization-research-explained/insight-machines-the-past-present-and-future-of-visualization-recommendation-2185c33a09aa Multiple ViewsVisualization Research Explained.[47] Yun Young Lee, Nicholas Chen, and Ralph E. Johnson. 2013. Drag-and-DropRefactoring: Intuitive and Efficient Program Transformation. In
InternationalConference on Software Engineering (ICSE) . https://doi.org/10.1109/ICSE.2013.6606548[48] Germán Leiva, Nolwenn Maudet, Wendy Mackay, and Michel Beaudouin-Lafon.2019. Enact: Reducing Designer-Developer Breakdowns When PrototypingCustom Interactions.
ACM Transactions on Computer-Human Interaction (TOCHI)
26, 3, Article 19 (May 2019), 48 pages. https://doi.org/10.1145/3310276[49] Jianping Kelvin Li and Kwan-Liu Ma. 2021. P6: A Declarative Language for Inte-grating Machine Learning in Visual Analytics.
IEEE Transactions on Visualizationand Computer Graphics (2021).[50] Zhicheng Liu, John Thompson, Alan Wilson, Mira Dontcheva, James Delorey,Sam Grigg, Bernard Kerr, and John Stasko. 2018. Data Illustrator: AugmentingVector Design Tools with Lazy Data Binding for Expressive Visualization Author-ing. In
Proceedings of the 2018 CHI Conference on Human Factors in ComputingSystems . ACM, 1–13. https://doi.org/10.1145/3173574.3173697[51] Aran Lunzer, Amelia McNamara, and Robert Krahn. 2014. LivelyR: Making Rcharts livelier. In useR! Conference .[52] Kwan-Liu Ma. 2000. Visualizing visualizations. User interfaces for managing andexploring scientific visualization data.
IEEE Computer Graphics and Applications
20, 5 (2000), 16–19. [53] Jock Mackinlay. 1986. Automating the Design of Graphical Presentations ofRelational Information.
ACM Trans. Graph.
5, 2 (1986), 110–141. https://doi.org/10.1145/22949.22950[54] Jock Mackinlay, Pat Hanrahan, and Chris Stolte. 2007. Show Me: Automatic Pre-sentation for Visual Analysis.
IEEE Transactions on Visualization and ComputerGraphics
13, 6 (2007), 1137–1144.[55] Joe Marks, Brad Andalman, Paul A Beardsley, William Freeman, Sarah Gibson,Jessica Hodgins, Thomas Kang, Brian Mirtich, Hanspeter Pfister, Wheeler Ruml,et al. 1997. Design galleries: A general approach to setting parameters forcomputer graphics and animation. In
Proceedings of the 24th annual conferenceon Computer graphics and interactive techniques . 389–400.[56] Michele Mauri, Tommaso Elli, Giorgio Caviglia, Giorgio Uboldi, and MatteoAzzi. 2017. RAWGraphs: A Visualisation Platform to Create Open Outputs. In
Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter . ACM.https://doi.org/10.1145/3125571.3125585[57] Andrew McNutt and Gordon Kindlmann. 2018. Linting for Visualization: To-wards a Practical Automated Visualization Guidance System. In
VisGuides: 2ndWorkshop on the Creation, Curation, Critique and Conditioning of Principles andGuidelines in Visualization .[58] Andrew McNutt, Gordon Kindlmann, and Michael Correll. 2020. SurfacingVisualization Mirages.
Proceedings of the 2020 CHI Conference on Human Factorsin Computing Systems (2020). https://doi.org/10.1145/3313831.3376420[59] Honghui Mei, Yuxin Ma, Yating Wei, and Wei Chen. 2018. The design space ofconstruction tools for information visualization: A survey.
Journal of VisualLanguages & Computing
44 (2018), 120–132. https://doi.org/10.1016/j.jvlc.2017.10.001[60] nteract. 2020. data-explorer. https://github.com/nteract/data-explorer[61] nteract. 2020. papermill. https://github.com/nteract/papermill[62] Deokgun Park, Steven M. Drucker, Roland Fernandez, and Niklas Elmqvist. 2018.Atom: A Grammar for Unit Visualizations.
IEEE Transactions on Visualizationand Computer Graphics
24, 12 (2018), 3032–3043. https://doi.org/10.1109/TVCG.2017.2785807[63] Felipe Pezoa, Juan L. Reutter, Fernando Suarez, Martín Ugarte, and DomagojVrgoč. 2016. Foundations of JSON Schema. In
Proceedings of the 25th InternationalConference on World Wide Web . https://doi.org/10.1145/2872427.2883029[64] Ate Poorthuis, Lucas van der Zee, Grace Guo, Jo Hsi Keong, and Bianchi Dy. 2020.Florence: a Web-based Grammar of Graphics for Making Maps and LearningCartography.
Cartographic Perspectives (2020).[65] Xiaoying Pu and Matthew Kay. 2018. The Garden of Forking Paths in Visu-alization: A Design Space for Reliable Exploratory Visual Analytics. In .IEEE, 37–45. https://doi.org/10.1109/BELIV.2018.8634103[66] Xiaoying Pu and Matthew Kay. 2020. A Probabilistic Grammar of Graphics. In
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems .https://doi.org/10.1145/3313831.3376466[67] Donghao Ren, Bongshin Lee, and Matthew Brehmer. 2018. Charticulator: Inter-active construction of bespoke chart layouts.
IEEE transactions on visualizationand computer graphics
25, 1 (2018), 789–799.[68] Hugo Romat, Nathalie Henry Riche, Ken Hinckley, Bongshin Lee, CarolineAppert, Emmanuel Pietriga, and Christopher Collins. 2019. ActiveInk: (Th)Inking with Data. In
Proceedings of the 2019 CHI Conference on Human Factorsin Computing Systems . 1–13. https://doi.org/10.1145/3290605.3300272[69] Hans Rosling and Zhongxing Zhang. 2011. Health advocacy with Gapminderanimated statistics.
Journal of Epidemiology and Global Health
1, 1 (2011), 11–14.https://doi.org/10.1016/j.jegh.2011.07.001[70] RStudio. [n.d.]. Shiny. https://shiny.rstudio.com/ Accessed December 16, 2020.[71] Bahador Saket, Samuel Huron, Charles Perin, and Alex Endert. 2019. Inves-tigating Direct Manipulation of Graphical Encodings as a Method for UserInteraction.
IEEE Transactions on Visualization and Computer Graphics (2019).https://doi.org/10.1109/TVCG.2019.2934534[72] Bahador Saket, Lei Jiang, Charles Perin, and Alex Endert. 2019. Liger: CombiningInteraction Paradigms for Visual Analysis. arXiv (July 2019), arXiv:1907.08345.https://ui.adsabs.harvard.edu/abs/2019arXiv190708345S/abstract[73] Bahador Saket, Hannah Kim, Eli T. Brown, and Alex Endert. 2016. Visualizationby Demonstration: An Interaction Paradigm for Visual Data Exploration.
IEEETransactions on Visualization and Computer Graphics
23, 1 (2016), 331–340.https://doi.org/10.1109/TVCG.2016.259883[74] Arvind Satyanarayan and Jeffrey Heer. 2014. Lyra: An Interactive VisualizationDesign Environment. In
Eurographics Conference on Visualization , Vol. 33. 10.https://doi.org/10.1111/cgf.12391[75] Arvind Satyanarayan, Bongshin Lee, Donghao Ren, Jeffrey Heer, John Stasko,John Thompson, Matthew Brehmer, and Zhicheng Liu. 2020. Critical Reflectionson Visualization Authoring Systems.
IEEE Transactions on Visualization andComputer Graphics
26, 1 (2020), 461–471. https://doi.org/10.1109/TVCG.2019.2934281[76] Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and JeffreyHeer. 2016. Vega-Lite: A Grammar of Interactive Graphics.
IEEE Transactionson Visualization and Computer Graphics (2016). https://doi.org/10.1109/TVCG.
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
IEEE Transactions on Visualization and Computer Graphics
22, 1(2016), 659–668. https://doi.org/10.1109/TVCG.2015.2467091[78] Arvind Satyanarayan, Kanit Wongsuphasawat, and Jeffrey Heer. 2014. Declar-ative Interaction Design for Data Visualization. In
Proceedings of the 27th an-nual ACM symposium on User interface software and technology . ACM, 669–678.https://doi.org/10.1145/2642918.2647360[79] Ben Shneiderman. 1983. Direct Manipulation: A Step Beyond ProgrammingLanguages.
Proceedings of the Joint Conference on Easier and More ProductiveUse of Computer Systems.(Part-II): Human Interface and the User Interface-Volume1981 (1983).[80] Chris Stolte, Diane Tang, and Pat Hanrahan. 2002. Polaris: A System for Query,Analysis, and Visualization of Multidimensional Relational Databases.
IEEETransactions on Visualization and Computer Graphics
8, 1 (2002), 52–65.[81] Wenbo Tao, Xinli Hou, Adam Sah, Leilani Battle, Remco Chang, and MichaelStonebraker. 2020. Kyrix-S: Authoring Scalable Scatterplot Visualizations of BigData.
IEEE Transactions on Visualization and Computer Graphics
Jour-nal of Open Source Software
3, 32 (2018), 1057. https://doi.org/10.21105/joss.01057[84] Vega. 2020. Aggregate Bar Chart. https://vega.github.io/vega-lite/examples/bar_aggregate.html[85] Vega. 2020. Box Plot with Min/Max Whiskers. https://vega.github.io/vega-lite/examples/boxplot_minmax_2D_vertical.html[86] Vega. 2020. Editor/IDE for Vega and Vega-Lite. https://vega.github.io/editor/.[87] Vega. 2020. One Dot per Zipcode in the U.S. https://vega.github.io/vega-lite/examples/geo_circle.html[88] Vega. 2020. Vega-Lite Documentation. https://vega.github.io/vega-lite AccessedAugust 19, 2020.[89] Bret Victor. 2013. Drawing Dynamic Visualizations. https://vimeo.com/66085662[90] Fernanda B. Viegas, Martin Wattenberg, Frank Van Ham, Jesse Kriss, and MattMcKeon. 2007. Many Eyes: A Site for Visualization at Internet Scale.
IEEETransactions on Visualization and Computer Graphics
13, 6 (2007), 1121–1128.https://doi.org/10.1109/TVCG.2007.70577[91] Chenglong Wang, Yu Feng, Rastislav Bodik, Alvin Cheung, and Isil Dillig. 2019.Visualization by Example.
Proceedings of the ACM on Programming Languages
4, POPL (Dec. 2019), 49:1–49:28. https://doi.org/10.1145/3371117[92] Hadley Wickham. 2010. A Layered Grammar of Graphics.
Journal of Computa-tional and Graphical Statistics
19, 1 (2010), 3–28. https://doi.org/10.1198/jcgs.2009.07098[93] Hadley Wickham, Dianne Cook, Heike Hofmann, and Andreas Buja. 2010.Graphical Inference for Infovis.
IEEE Transactions on Visualization and ComputerGraphics
16, 6 (2010), 973–979. https://doi.org/10.1109/TVCG.2010.161[94] Hadley Wickham and Heike Hofmann. 2011. Product Plots.
IEEE Transactionson Visualization and Computer Graphics
17, 12 (2011), 2223–2230. https://doi.org/10.1109/TVCG.2011.227[95] Leland Wilkinson. 2013.
The Grammar of Graphics . Springer.[96] Krist Wongsuphasawat. 2020. Encodable: Configurable Grammar for Visualiza-tion Components.
IEEE Transactions on Visualization and Computer Graphics (2020).[97] Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay,Bill Howe, and Jeffrey Heer. 2015. Voyager: Exploratory Analysis via FacetedBrowsing of Visualization Recommendations.
IEEE Transactions on Visualizationand Computer Graphics
22, 1 (2015), 649–658. https://doi.org/10.1109/TVCG.2015.2467191[98] Kanit Wongsuphasawat, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk,Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2017. Voyager 2:Augmenting Visual Analysis with Partial View Specifications. In
Proceedingsof the 2017 CHI Conference on Human Factors in Computing Systems . ACM,2648–2659. https://doi.org/10.1145/3025453.3025768[99] Jo Wood, Alexander Kachkaev, and Jason Dykes. 2019. Design Exposition withLiterate Visualization.
IEEE Transactions on Visualization and Computer Graphics
25, 1 (2019), 759–768. https://doi.org/10.1109/TVCG.2018.2864836[100] Yifan Wu, Joseph M. Hellerstein, and Arvind Satyanarayan. 2020. B2: BridgingCode and Interactive Visualization in Computational Notebooks. In
UIST ’20:The 33rd Annual ACM Symposium on User Interface Software and Technology .ACM, 152–165. https://doi.org/10.1145/3379337.3415851[101] Zheguang Zhao, Emanuel Zgraggen, Lorenzo De Stefani, Carsten Binnig, EliUpfal, and Tim Kraska. 2017. Safe Visual Data Exploration. In
Proceedings of the2017 ACM International Conference on Management of Data - SIGMOD ’17 . ACM,1671–1674. https://doi.org/10.1145/3035918.3058749[102] Jonathan Zong, Dhiraj Barnwal, Rupayan Neogy, and Arvind Satyanarayan.2021. Lyra 2: Designing Interactive Visualizations by Demonstration.
IEEE Transactions on Visualization and Computer Graphics (2021). ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan
Users can search the gallery for templates that are compatible with a list of data fields specified in this multi-data parameter.They can organize the templates by a variety of schemes. Each card gives a minimal description it’s the template.
Figure 9: Users can search for templates via text or catalog search in the gallery, which is a set of user created templates (hostedon a communal server) and system templates. Q u e r y Complete MatchComplete MatchNo MatchNone selected
DimensionMeasureTime
Partial MatchComplete MatchComplete Match Partial MatchNo MatchNo Match Partial MatchPartial MatchComplete Match11 1
Templates
Figure 10: Catalog search (Sec. 4.4.1) compares templates to a set of data fields, which can yield a partial match (no conflict butunrenderable), a complete match (renderable), or no match (conflict).
A APPENDIX
In this appendix we expand upon several elements of the design,implementation, and evaluation of Ivy discussed in the main paper.
A.1 Catalog Search Matching Heuristic
Fig. 9 shows the gallery of Ivy templates in our current implemen-tation. Here we provide a more precise description of the heuristicused to perform the template matching in our catalog search.For a template 𝑡 (among the set of templates 𝑇 ) with a set of dataparameters 𝑑 , (where each parameter has a set of allowed types 𝑑 𝑖 𝜏 ), and a search 𝑆 consisting of a set of data columns { 𝑐 𝑖 } whicheach have a single type 𝑐 𝑖 𝜏 , then 𝑆 is a partial match for 𝑡 if there isan injective mapping 𝑚 between them such that (∃ 𝑚 )(∀ 𝑐 𝑖 ∈ 𝑆 )(∃ 𝑑 𝑗 ∈ 𝑑 )( 𝑚 : 𝑐 𝑖 → 𝑑 𝑗 : 𝑐 𝑖 𝜏 ∈ 𝑑 𝑗 𝜏 ) (11)Correspondingly 𝑆 is a full match for 𝑡 if (∃ 𝑚 )(∀ 𝑟𝑑 𝑖 ∈ 𝑟𝑑 )(∃ 𝑐 𝑗 ∈ 𝑆 )( 𝑚 : 𝑐 𝑖 → 𝑑 𝑗 : 𝑐 𝑖 𝜏 ∈ 𝑟𝑑 𝑗 𝜏 ) (12)where we define the required set of data param in 𝑡 as 𝑟𝑑 ⊆ 𝑑 . Thischeck is bounded by the size of 𝑆 and 𝑇 , so a search across thetemplates will take O(| 𝑆 || 𝑇 |) . We illustrate this in Fig. 10. A.2 Implementation Details
Ivy is a TypeScript React-Redux application. We were motivatedto use React as the basis of our application as it provides an opin-ionated approach on how to build additional renderers, which isimportant for our approach to extensibility. Our template server isa cloud-based node.js server backed by PostgreSQL.As described in Sec. 6.1.1 our system is designed to be extensi-ble: support for each specification language is defined through anextension interface comprising metadata (such as a JSON Schemadescribing the syntax), a React component [105] that exposes therendering function of the language, and rewrite rule definitionsthat help users abstract specifications into templates. While Ivycurrently supports a relatively limited class of rewrite rules, fu-ture work could extend the approach with more expressive lan-guages for transforming structured data (such as in CDuce [103]and XDuce [107]).Our current implementation supports four languages—Vega,Vega-Lite, Atom, and a toy data table language—which serves as alimited demonstration of the validity of our extensible approach. As
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
JSON-mediated visualization grammars continue to gain popular-ity, additional languages will inevitably emerge to solve problemsunaddressed in prior efforts. Future languages could support morecomplex rendering schemes, focusing on particular domains suchas geospatial analytics [106], 3D visual analytics, pivot tables (per-haps simplifying the language of VizQL [24]), or even on the chartrecommendation language CompassQL [111]—which would enabletask-specific variations of Voyager [98].
A.3 Templates for Vega-Lite Gallery
As described in Sec. 5.1.1, we aimed to factor the Vega-Lite examplesinto templates in reasonable ways, a heuristic which was guidedby the minimization of complexity and the maximization of sim-plification. We illustrate these metrics in Fig. 11. The metrics inthis figure are computed following Equation 10. Most templatesexhibit a compression greater than 1, indicating that the templateis better than simply concatenating the examples together. Thosethat do worse tend to have particular affordances to the templateusable and also tend to only cover a single example, limiting theircompression. See https://osf.io/cture/ for further details.
A.4 Templates to Reproduce Chart Choosers
We describe here in greater detail our templatization of the GoogleSheets chart chooser (described in Sec. 5.1.2), as well as an additionalchart corpus provided by Russell [109]. The latter fell outside thenarrative in the main body of the paper, but we include it here as anexample of the chart making culture in one particular organization.Table 1 summarizes the resulting templates described below.
LOCCompression A S T C o m p r e ss i o n R ² : 0.69 51015 Count
CountImprovement over concatenation
Diverging Stacked Bar ChartDot Plot
Smaller (34)Partially smaller (4)Larger (6)Out of 43
Figure 11: The concatenation ratios across our reproductionof the Vega-Lite gallery between the compression of the Ab-stract Syntax Tree (a stand-in for complexity minimization)and Line of Code Compression (a stand-in simplification).Higher is better in both cases.
A.4.1 Google Sheets.
The Sheets chart chooser consists of 32 op-tions. We reproduce 29 of these through 16 templates, as summa-rized in Fig. 12. We skipped “3D Pie” charts because there is notyet a dominant grammar for browser-based non-VR 3D visual ana-lytics, although several recent works have put forward interestingapproaches [104, 110]. We skipped “Org. Charts” because they falloutside of our tabular data model, requiring a hierarchical one. Fi-nally, we skipped “Timelines” because we do not currently supportthe data manipulations required to support textual annotations asrequired by this chart form. Each of these deficiencies could be ad-dressed in future work, such as by extending the range of languagessupported.
A.4.2 Russell Survey.
This survey[109] of internal presentations atGoogle included approximately 1,300 charts, which were groupedinto 15 distinct visual forms. We reproduce 10 of these through 11templates. There are more templates than charts because, follow-ing Sheets, we split Russell’s “Map” into two templates, “CountryChoropleth” and “Scatter Map”. Of the 5 charts from this surveywe skipped, 3 involve a non-tabular data model, 1 requires domain-specific data (Lam et al.’s SessionViewer [108]), and 1 uses annota-tions. Among the 16 plus 11 templates described, 18 are distinct.While informative, this selection covers one particular analyticculture and one family of tool’s designs. For instance, Russell’sreview found SessionViewer [108], a system for understanding websearch usage behaviors, made up 2.1% of the review corpus. A re-view of a different corpus would likely yield a different selection ofcharts. Furthermore, this selection of charts is also a symptom ofsoftware availability. To wit: unit visualizations tend to be uncom-mon because few systems tend to support them [62].
3D PieOrg. ChartTimeline
Skipped Dual Y-axis
Combo
Histogram
Histogram
BigNumber
Scorecard
Box Plot
Candlestick
Scatter Map
ScatterMap
Choropleth
Choropleth
Line Chart Area ChartColumn ChartRadar WaterfallGaugeTable ScatterplotPie Chart Mosaic Plot
Waterfall
Radar GaugeTable Line Smooth line Area Stacked Area100% Stacked AreaStepped Area
Stacked Stepped Area
Column Stacked Column
Bar ChartStacked Bar 100% Stacked BarPie
Doughnut
Scatter Bubble
Tree map
Ivy Template Name
Template language
Sheets Chart Name
Template language
Vega (8)Vega-Lite (20)Data Table (1)Skip (3)
Figure 12: We created 16 Ivy templates that reconstruct thefunctionality of 29 of the 32 charts in the Google Sheetschart chooser, a . x compression. ntegrated Visualization Editing via Parameterized Declarative Templates CHI ’21, May 8–13, 2021, Yokohama, Japan Table 1: We now give additional detail regarding the overlap between our coverage of the Google Sheets Chart Chooser Optionsand the charts described in Russell’s survey. Options that are unsupported under the current set of grammars are marked asSkip . Each template was created by starting from a blank template, from using Polestar to approximate the right behavior,or by abstracting an example found online.
Russel Chart Sheets Chart Ivy Template Russell Sheets Language Creation Methodline graph Smooth Line Chart Line Chart ✓ ✓
Vega-lite Polestarline graph Line Chart Line Chart ✓ ✓
Vega-lite Polestarhistogram Histogram chart Histogram ✓ ✓
Vega-lite Polestartable Table chart Table ✓ ✓
Data table Blankpie Pie chart Pie chart ✓ ✓
Vega Examplepie Doughnut chart Pie chart ✓ ✓
Vega Examplestack histogram Column chart Column Chart ✓ ✓
Vega-lite Polestarstack histogram Stacked column chart Column Chart ✓ ✓
Vega-lite Polestarstack histogram 100% stacked column chart Column Chart ✓ ✓
Vega-lite Polestarstack histogram Bar chart Column Chart ✓ ✓
Vega-lite Polestarstack histogram Stacked bar chart Column Chart ✓ ✓
Vega-lite Polestarstack histogram 100% stacked bar chart Column Chart ✓ ✓
Vega-lite Polestarbox plot Candlestick chart Candle stick ✓ ✓
Vega Examplescatterplot Scatter chart Scatterplot ✓ ✓
Vega-lite Blankscatterplot Bubble chart Scatterplot ✓ ✓
Vega-lite Blankmap Geo chart Country Choropleth ✓ ✓
Vega Examplemap Geo chart with markers Scatter Map ✓ ✓
Vega Blanktimeline Timeline chart Skip ✓ ✓
N/A N/Apie 3D pie chart Skip ✓ ✓
N/A N/Aheatmap N/A Heatmap ✓ × Vega-lite Examplesunburst N/A Sunburst ✓ × Vega Examplearc/node graph N/A Skip ✓ × N/A N/ASessionView N/A Skip ✓ × N/A N/ASankey N/A Skip ✓ × N/A N/Aforce vector N/A Skip ✓ × N/A N/AN/A Organizational chart Skip × ✓ N/A N/AN/A Combo chart Dual Y-axis × ✓ Vega-lite ExampleN/A Waterfall chart Waterfall × ✓ Vega-lite ExampleN/A Radar chart Radar × ✓ Vega ExampleN/A Gauge chart Gauge × ✓ Vega ExampleN/A Scorecard chart BigNumber × ✓ Vega BlankN/A Tree map chart Mosaic Plot × ✓ Vega-lite ExampleN/A Area chart Area chart × ✓ Vega-lite PolestarN/A Stacked area chart Area chart × ✓ Vega-lite PolestarN/A 100% stacked area chart Area chart × ✓ Vega-lite PolestarN/A Stepped area chart Area chart × ✓ Vega-lite PolestarN/A Stacked stepped area chart Area chart × ✓ Vega-lite PolestarN/A 100% stacked stepped area chart Area chart × ✓ Vega-lite Polestar
HI ’21, May 8–13, 2021, Yokohama, Japan Andrew McNutt and Ravi Chugh
A.5 User Study Prompts
Here we provide the text of the tasks involved in the user study. Thefull study instrument can be found in the supplementary materials.These questions were divided into two sections,
Tutorial , whichinvolved substantial guidance, and
Independent , which were morefreeform. These tasks were selected because they were similar totasks that one might address in similar systems.
A.5.1 Tutorial Tasks. (1) In this task you will make a small multiple log-log scatter plot ofhappiness vs population for 2015 colored and faceted by region(such as by row or column) where tooltipping shows (amongother data) the name of the country. To do so you make useof the Polestar template. Start by finding the Polestar templatein the gallery, navigate to it. Now fill in the appropriate fieldsfor X and Y. To make the tooltip reveal useful information,place the Country field onto the detail target. Set the Regionto Column as well. Don’t forget to filter the appropriate year.What correlation can you see? Give your answer in plain text.(2) In this task we will make a SPLOM (scatter plot matrix) forour dataset. Start by creating a new view, and navigate to thePolestar template. Place the row and column cards on the x andy data targets respectively. Next, select 3 measures of interest(you decide!), place each of them in both of the “row” and“column” multi targets which are under the “meta columnssection”. Now click the lighting bolt next the Color field andselect 3 dimensions of interest (you decide!). Just as before itmight be helpful to place Country into Detail. What correlationscan you find? Give your answer in plain text.(3) Next you will make use of a particular template from the gallery.Specifically, you will make a radial scatterplot. This task is alittle different in that you will use a different dataset. Open thegallery in a new tab and find the template that will allow youto make a radial scatterplot. Once there select the penguinsdataset. What is the most interesting combination of variablesyou can find? Can you use fanout to effectively move throughthese options? These plots are a little big. Why don’t you try tonavigate to the code body and change their height and widthto be something a little more reasonable? Copy the code fromthe output into the below box.(4) Next you will try out making a template, specifically let’s make aheatmap. Load up the happiness dataset once again. Once againwe will start with Polestar. Start by placing the CountryTypeand GovernmentType variables on to the X and Y targets. Thenplace the happiness field onto the Color Field. Feel free to adjustthe heatmap as you like. You should probably change the marktype to make it more heatmap like! When you are ready, clickthe “Fork” button and select “Just output”. Click the suggestionfor CountryType that creates and configures a new field, thiswill create a new datatarget for this field while keeping thecurrent graphic in place. Select the gear to the right of this newfield and give it an informative field name, and select only theappropriate field data type. Do the same for GovernmentType.Next, let’s make our heatmap be better able to describe variousaggregates. Create a new List widget and add options for eachof the aggregation types (the ones that make the most sense areprobably count and distinct, but you are welcome to use others, see https://vega.github.io/vega-lite/docs/). Give the widget adescriptive name (no spaces though!) and replace the word“count” in the Body with your new name wrapped in brackets,like so “[YOUR NEW NAME]”. You should now be able to switchthrough various aggregations (or fan across them). Finally, giveyour template a name and description, and then publish it!
A.5.2 Independent Tasks. (1) How many rows are in this dataset? How many countries arerepresented in this dataset? How many countries are in eachregion? Please give you answers in plain text. (You can answerthe last question using the JSON output.)(2) Create a new template by adapting https://vega.github.io/vega-lite/examples/boxplot_minmax_2D_vertical.html to this datasetin a useful manner. The choice of columns is up to you. Tryadding height and width sliders. Try enabling switching be-tween types of box plot e.g. https://vega.github.io/vega-lite/examples/boxplot_2D_vertical.html). Copy the body of yourtemplate into the space below.(3) Please make charts that answer the following questions. Whatis the global trend in corruption? Do bigger countries tend tobe happier? Use code from the output for your answers.(4) Please combine the following examples drawn from the Vega-Lite gallery into an Ivy template: 1. https://vega.github.io/vega-lite/examples/bar_aggregate.html 2.https://vega.github.io/vega-lite/examples/bar_aggregate_sort_by_encoding.html. The par-ticular choice of columns and features is up to you. Hint: thebig difference is that one is sorted and the other is not! If youare having trouble with this task it may be helpful to check outthe documentation of the template language found on the webpage. Answer this question by giving the body of the templateyou’ve created.
APPENDIX REFERENCES [103] Véronique Benzaken, Giuseppe Castagna, and Alain Frisch. 2003. CDuce: AnXML-Centric General-Purpose Language.
ACM SIGPLAN Notices (2003). https://doi.org/10.1145/944746.944711[104] Peter WS Butcher, Nigel W. John, and Panagiotis D. Ritsos. 2019. VRIA-AFramework for Immersive Analytics on the Web. In
Extended Abstracts of the2019 CHI Conference on Human Factors in Computing Systems . 1–6. https://doi.org/10.1145/3290607[105] Facebook. 2020. React – A JavaScript library for building user interfaces. https://reactjs.org/[106] Shan He. 2020. Kepler.gl: Large-scale WebGL-powered Geospatial Data Vis.http://kepler.gl/[107] Haruo Hosoya and Benjamin C. Pierce. 2003. XDuce: A Statically Typed XMLProcessing Language.
ACM Transactions on Internet Technology (2003). https://doi.org/10.1145/767193.767195[108] Heidi Lam, Daniel Russell, Diane Tang, and Tamara Munzner. 2007. SessionViewer: Visual Exploratory Analysis of Web Session Logs. In . IEEE, 147–154. https://doi.org/10.1109/VAST.2007.4389008[109] Daniel M. Russell. 2016. Simple is Good: Observations of Visualization UseAmongst the Big Data Digerati. In
Proceedings of the International WorkingConference on Advanced Visual Interfaces . 7–12. https://doi.org/10.1145/2909132.2933287[110] Ronell Sicat, Jiabao Li, JunYoung Choi, Maxime Cordeil, Won-Ki Jeong, BenjaminBach, and Hanspeter Pfister. 2018. DXR: A Toolkit for Building Immersive DataVisualizations.
IEEE Transactions on Visualization and Computer Graphics
25, 1(2018), 715–725. https://doi.org/10.1109/TVCG.2018.2865152[111] Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, BillHowe, and Jeffrey Heer. 2016. Towards A General-Purpose Query Language forVisualization Recommendation. In