Data Visualization Practitioners' Perspectives on Chartjunk
DData Visualization Practitioners’ Perspectives on Chartjunk
Paul Parsons * Prakash Shukla † Purdue University A BSTRACT
Chartjunk is a popular yet contentious topic. Previous studies haveshown that extreme minimalism is not always best, and that visualembellishments can be useful depending on the context. While moreknowledge is being developed regarding the effects of embellish-ments on users, less attention has been given to the perspectivesof practitioners regarding how they design with embellishments.We conducted semi-structured interviews with 20 data visualizationpractitioners, investigating how they understand chartjunk and thefactors that influence how and when they make use of embellish-ments. Our investigation uncovers a broad and pluralistic under-standing of chartjunk among practitioners, and foregrounds a varietyof personal and situated factors that influence the use of chartjunkbeyond context. We highlight the personal nature of design practice,and discuss the need for more practice-led research to better under-stand the ways in which concepts like chartjunk are interpreted andused by practitioners.
Index Terms:
Human-centered computing—Visualization;
NTRODUCTION
Chartjunk has been a popular yet contentious concept for manydecades. Since its inception there has been considerable debate aboutthe value of visual embellishments, whether they have any properrole in visualization design, and under what conditions they may beappropriate to use. In the years after the term was coined by Tufte[40], only a small number of research studies investigated chartjunk(e.g., [4, 21, 39]). In contrast, the past decade has seen a renewed in-terest in the InfoVis research community [1, 5–7, 9, 16, 17, 24, 30, 32].The vast majority of this research has focused on how users perceivechartjunk, largely with respect to comprehension, memorability, andperformance measures such as speed and accuracy. While thesestudies have contributed valuable knowledge about the effects ofembellishments on users, there has been little attempt to study embel-lishments from the perspective of visualization designers. Studyingthe effects of a phenomenon on users provides at best a partial under-standing of its role in design, as there are many factors that influencethe use of concepts and principles in real-world settings.Designers have ways of knowing and thinking that are distinctfrom those of scientists and researchers—famously referred to byNigel Cross as designerly ways of knowing [10]. Studies of designpractice in other emergent design fields, such as interaction designand instructional design, indicate that designers confront the com-plexity of real-world situations in ways that are quite different fromthose of researchers [13, 37]. While a focus on design practice isgaining traction in InfoVis [3, 18, 25, 28, 29, 41] there is still notmuch investigation into how designers generate and use knowledgein practice. As chartjunk is one of the most well known and usedconcepts by practitioners [28], it presents a good opportunity topursue such an investigation. * e-mail: [email protected] † e-mail: [email protected] We conducted interviews with 20 DataVis practitioners, in whichthey were asked about their perspectives on the concept of chartjunkand the ways in which it influences their design practice. In doingso, we surface the concerns of practitioners, highlighting issuessurrounding the creation of embellishments and not only their effectson users . We contribute a practice-led perspective to the extantliterature, highlighting the personal and situated aspects of designpractice that are not considered in typical user studies.
HARTJUNK
Tufte is credited with coining the term chartjunk in his 1983 bookThe Visual Display of Quantitative Information [40]. He defined itas “ink that does not tell the viewer anything new” and “non-data-inkor redundant data-ink”. Tufte defined data-ink as “the non-erasablecore of a graphic, the non-redundant ink arranged in response tovariation in the numbers represented”, and the data–ink ratio as theratio of the data-ink over the total ink used in a graphic [40]. Tufte’s“theory of data graphics” proposed maximizing the data-ink ratio,which is another way of saying to reduce chartjunk—i.e., avoid non-data-ink. Despite Tufte’s extreme minimalism, he did admit a placefor considering “complexity, structure, density, and even beauty”in the design of graphics, seemingly opening opportunities wherenon-data-ink may be acceptable, although no clarity was providedon how and what this might look like.Despite Tufte’s highly influential work, many information andgraphic designers—most notably Nigel Holmes—continued to cre-ate graphics with embellishments, leading to a fierce debate amongtwo camps with opposing views [11]. Yet, as noted by Few [11],little of the debate was dispassionate and evidence-based.
Studies have investigated effects of chartjunk on users (often usingmore neutral labels such as embellishment or decoration), high-lighting some benefits of embellishments [1, 6, 7, 12, 16, 21]. Forinstance, Bateman et. al [1] found embellished charts to improve re-call (though there have been concerns about their methodology [11]).Borkin et. al [6,7] have found that visualizations with scenes, people,and pictograms can be more memorable than simple charts. Harozet al. [16] also investigated effects of pictographic representationson memory, speed, and engagement, finding that superfluous imagescan be distracting–but do not incur any significant user costs–andhave benefits for working memory and engagement.From a different perspective, Hullman et. al [19] have arguedthat adding desirable difficulties (e.g., embellishments) can augmentcomprehension and recall. Others have argued that going beyondminimalist chart design can improve engagement [20, 22]. Correlland Gleicher [9] have highlighted the reductionist view of the data-ink ratio as a guide for design, instead proposing less prescriptiveguidelines that can be applied using a designer’s best judgment.One takeaway from these studies is that a balance between ex-treme minimalism and embellishments should be sought in a con-textually appropriate manner. While this may seem like a simpleguideline, its application in practice is not well understood. In thiswork we attempt to uncover some ways practitioners think aboutchartjunk and its use in real-world design settings. a r X i v : . [ c s . H C ] S e p D Job Title Exp.(yrs) HighestDegree G
P1 Sr. DataVis Designer >
10 D FP2 Data Storyteller >
10 M MP3 DataVis Engineer 8-10 D MP4 DataVis/UX Designer >
10 M MP5 DataVis/UX Designer 2-4 M FP6 DataVis Designer 5-7 M FP7 DataVis Designer/Dev >
10 M MP8 DataVis Designer 8-10 B MP9 Graphics Editor 8-10 B FP10 DataVis Designer 5-7 B FP11 Sr. UX Design Lead 8-10 D MP12 DataVis Designer 8-10 M MP13 DataVis Designer 5-7 B MP14 Data Architect >
10 M MP15 Sr. UX/DataVis Designer 5-7 M FP16 Data Communicator 2-4 M MP17 DataVis Designer 5-7 M FP18 DataVis Journalist/Designer 5-7 M MP19 Sr. DataVis Dev >
10 D FP20 DataVis Designer >
10 M M
Table 1: Our 20 participants and their self-reported characteristics:job title, years of experience, highest degree (Bachelors-B, Masters-Mand Doctoral-D) and gender.
ETHOD
As part of a broader research effort to study DataVis design practice,we interviewed 20 DataVis practitioners and asked them about theirdesign process, including how they understand and make use ofspecific concepts and methods. Recruiting was done via socialmedia, the DataVis Societys Slack workspace, the InfoVis emaillist, and our personal networks. To mitigate sampling bias, we alsosearched widely online for practicing professionals and agencies,ultimately contacting more than 200 individuals and more than 30agencies.Interviews were semi-structured and were conducted remotelyvia videoconferencing. For this paper, we selected one sectionof the transcripts that focused on design methods and principles,in which we asked participants: (1) whether they were familiarwith the concept of chartjunk or visual embellishment (we specif-ically included “embellishment” in an attempt to not frame thediscussion too negatively); (2) if and how they relied on the ideaof chartjunk/embellishment in their design work; (3) whether theyhad any opinions about chartjunk/embellishment; and (4) whetherthey were familiar with the history of the concept and its discoursein the academic or practitioner spaces. This section accounted forroughly 10 minutes near the middle of the 60-75 minutes of eachinterview. The transcripts were inductively coded by the two re-searchers, following standard processes for thematic analysis [8].We went through several rounds of open coding independently, regu-larly meeting with the goal of establishing agreement on the codes.We subsequently conducted several rounds of searching for anddefining themes, eventually reaching consensus.
INDINGS
When asked about their perspectives on chartjunk, multiple partic-ipants mentioned there was somewhat of a corrective movementtaking place, away from Tufte’s extreme minimalism and toward anacceptance of more embellishment. For example, P15 stated “
I thinkthat there’s sort of this very minimalist, everything must be there fora reason, school of data visualization design. And I think hopefully that we’ve at least started to move past that ”. P18 similarly notedthat “
Tufte was really strong in opposing chart junk. And I thinknow there’s a movement going a little bit in the other direction say-ing that, well, adding icons to charts can make things clearer, forexample. So yeah, it’s bit like a pendulum, I think. And with Tufteit went in one direction and now it’s going a little bit to back in theother direction. ” P19, who has a PhD in visualization and is nowa practitioner, commented on both the practitioner and researchercommunities: “
Certainly I think it’s been a necessary corrective tothe way things were in the Vis community—both in the practitionerand I think in the research community—because the practitionerswere so influenced by Tufte and because the research communitywas so influenced by the Cleveland and McGill idea of being justfocused on the data. ” Regarding the ways in which participants interpret and definechartjunk, we found several significant conceptual and termino-logical issues. For instance, many participants noted the obviousnegative framing of the term, with P12 stating “
First of all, acknowl-edging that it’s called junk—who argues for junk? ” and P15 notingthat “
It sounds like a negative connotation because it is literallycalling anything extra trash. ”There was little consensus on the definition of chartjunk, with awide variety of interpretations. Aware of this, P16 stated “
I don’tknow if people have strong opinions about what is and isn’t chartjunk ”, and P6 noted “
I do think the definition is different [. . . ] I’mnot 100 percent sure that everyone knows, like thinks the same thingwhen they think of chart junk. ” P17 pointed to an “infographic”kind of interpretation, stating “
It might also be the definition for mepersonally—when I find chartjunk, that’s more for me over thesesort of infographic posters where there’s the idea that there’s a lotof graphic stuff. ” P13 described confusion about whether chartjunkrefers only to distortions or all ornamentation “
Now, if we’re talk-ing about distorting actual charts and the way that information isperceived, I definitely don’t do that. But if there’s some sort of orna-mentation around the chart to draw your eye in or keep you engaged[. . . ] then I think that’s a different purpose. ”Beyond the breadth of interpretation regarding what visual el-ements are considered as chartjunk, multiple participants talkedabout animation and interactivity as potential forms of chartjunk.For instance, P11 states “ that’s [chartjunk] just another side of thedata-ink-ratio conversation. But it goes beyond that, because I findpeople love to add little icons. You know, they’re constantly addinglittle icons to buttons and user interactions and, you know, do theysupport understanding? ” P13, when describing when embellish-ments might be desirable, gave an example focused on interaction“ if you moved your mouse along the screen and it, I dunno, changescolors, or when you click it radiates out [. . . ] maybe it’s helpful. ”P9, when describing coworkers who tend to desire chartjunk, de-scribed them in this way: “ they want stuff online that moves andhovers and does all this stuff, and we have to explain to them, wellyou have to hover—if the data isn’t there, someone has to click on itto see the data [. . . ] chart junk is always there, man. ”Finally, many participants noted the subjective nature of definingchartjunk. For instance, when describing the work of a well-knownpractitioner, P6 indicated that “ people wouldn’t call it chartjunk,people would call it beautiful. ” P9, when describing a previousproject involving annotations on a series of charts, stated “
I wouldn’tcall that chart junk at that point. I would just call it a call out—annotations. ” When asked about their view of chartjunk, P11 stated“ it depends on what you call chart junk. Like when I look at The NewYork Times data visualization, I don’t think there’s really any junkthere. They do have things that are sometimes whimsical, or ThePudding does this too, it’s always motivated. ” .2 Factors Influencing the Use of Chartjunk When describing the use of embellishments, most participants de-scribed a need for “balance”. For instance, according to P5, “
I try tostrike a balance between making something look nice and kind offun and cute versus just polluting it with a whole bunch of nonsense. ”Similarly, P4 mentioned “
So of course there’s this balance that youwant to also provide this kind of context and help texts and labelsand so on as well as possible, so that the users can use it ”. Manyparticipants also referred to this notion of context, indicating that itis a significant factor. For example, P4 stated “
I’m really relativisticabout this, that it depends on the case and the context [. . . ] I mean,I’m fine with data art or anything, but if somebody suggests that Ishould put that in some business dashboards, then I would kind ofquestion that this is not the right place for it. ” P20 similarly noted“
I have no problem with chart junk for certain audiences, certainapplications, certain types of media—go for it. ” P18 simply statedthat “ the context determines how far you can go with it. ” Although context was noted as important, there were multiple issuessurfaced beyond context. Participants described not using embellish-ments due to a lack of skill in creating them. For instance, P8 said “
Iwould say I may not be good at it because I just know how to writecode and visually represent the data set. But if I have to come upwith my own drawings or elements that are not part of the data, thenmaybe the challenge is a bit too big for me, to actually do those kindsof things. ” P18 also noted that “
Maybe I should also mention that Idon’t know how to draw. [. . . ] I’m just really bad at making thingsaround or outside the pure data visualization things. So I simplycannot produce chart junk, so to speak. I don’t know how to draw.So for me it always focuses on the data. ” P13 described making“junk” as a byproduct of attempting to create embellishments: “
It’s[embellishment] generally something that I try to avoid because Idon’t think that I do it well. So I think that you can embellish well ifI, if I had tools to add texture to things, or if I had the illustrationskills to add something that wasn’t a distraction but felt like it couldbe an included part of something, then great. But my efforts withthat tend to look more like junk than a part of the design. ”Multiple participants also described their use of embellishments inrelation to their own personal style, indicating influences beyond theimpersonal context of the situation. For example, when describinga recent project, P8 described a design decision as being due to “ astyle I personally liked. ” When asked about the context in whichembellishments might be used in their design process, P20 noted “
Ilean slightly towards chart junk because I will always, especiallyin interactive projects, I believe in eye candy 100 percent. Like I’mgoing to animate that chart and bring it to life. ” P18 describedtheir style as “ a bit minimalistic. So data-ink ratio and chart junkare definitely things I try to think about when I design. But usuallythey are just in the back of my head—I don’t have a list here thatI have to check in each step just to make sure that I don’t havetoo much chart junk in my work. It’s a bit more already integratedinto my style. ” P17 described their “stylistic” approach to usingembellishments: “
I wouldn’t say I’m a minimalist because the kindsof works that I do need to grab the reader’s eye [. . . ] in terms ofembellishments, I always try and keep it very close to the data. Soinstead of a color, I might make it a subtle gradient, I might giveit a slight drop shadow to emphasize it [. . . ] Things like goingbeyond the default in a more stylistic way, that’s kind of how I like toadd my embellishments. ” P19 described their style as not favoringembellishments: “
It’s still probably not my style to design somethingthat has a lot of embellishment to it. I’m generally a very ‘put thedata first’ kind of designer when it comes to working on datavis.I like having elements on the page, always going back to a datapoint in some way. I would rather introduce sort of visual variety by mapping something to multiple visual channels or having somethingdone in the encoding to just sort of introduce a little extra color ifit’s necessary or to space things out some more. ”Participants also often referenced a number of external constraintsthat influenced how and when they would use embellishments. Forinstance, multiple participants talked about using embellishmentsto maintain brand fidelity. When defining chartjunk, P14 stated“ chart junk speaks to two things [. . . ] composition, and [. . . ] thisnotion of brand fidelity, right? ” later describing the goal of designas to “ build a good composition and build it with allegiance to yourbranding requirements. ” P19, who works for a firm with a “ morestripped down ” style, described embellishment as “ not somethingthat comes up super often for us [. . . ] and you know, there’s no realreason for that other than it’s a branding thing. ” P14 described theirstyle as “ toward a clean minimalist approach ”, yet noting “ in orderto maintain some sort of fidelity to a brand, there are times whenthrowing a bunch of teddy bears on the screen makes sense. ”A group of participants also talked about having to consider thedesires of clients when thinking about chartjunk. P20 mentioned “
Iknow for a lot of my clients [. . . ] they may not want to do that [addembellishments] and sometimes you just want to show the data andhave it clean and simple. ” P4 described the challenge of pleasingclients who often want something that looks novel “
This can bea challenge with clients, because clients might expect some noveland special visualization. So then balancing that, can we now dosomething novel here, or can we just sell a well-thought bar chart tothe users. So how to balance between these different needs. ” While the vast majority of participants indicated that embellish-ments had a place as long as they fit the context and other personaland situational goals, participants would regularly reveal underly-ing commitments to a particular design philosophy when justifyingtheir views, often surfacing value-laden judgments about the properways in which visualizations should be designed and consumed.These typically manifested in terms of more cognitive or experien-tial concerns that were foregrounded as being important outcomes ofusers’ interactions with their work. For instance, P9 described beinggenerally against embellishment, stating that “
What does that [em-bellishment] help tell you? I’m not making these charts for someoneto be like, oh, that’s pretty cute. No, here’s the locations, and thenhere’s like the serious data—I’m not trying to be cutesy ”, revealinga rationalistic perspective on what gives value to a visualization andprioritizing “seriousness” over other potential metrics. Other par-ticipants foregrounded a cognitive focus while defending the use ofembellishments, typically noting that they can enhance comprehen-sion. For instance, when describing the value of embellishment, P5noted “ if you could make your data pop, then it increases people’sunderstanding of what they’re looking at. And that’s ultimately theidea behind data visualization—you want people to understand. ”Other participants revealed more of a focus on experiential issues,often pointing to concepts such as engagement, beauty, fun, andenjoyment. P6 mentioned the famous “monster” chart by NigelHolmes, stating that “
This chart I think is excellent [. . . ] it’s awe-some. I’ve seen many charts that with the same data, but if somebodywants to make a point for this chart, and they just show the datain a very simple line or bar chart, it’s so boring ”, later stating that“
I’d say that making a chart more beautiful, with extra elements thatdon’t distract from the actual methods, but are there more to attractpotential readers, they’re always helpful because it attracts read-ers or because it gives you a good feeling why you’re reading thisvisualization. ” P7, when describing work from a well-known practi-tioner with “ a lot of decorations going on ” noted that “ it engagesthe audience ” and that “ for those purposes they [embellishments]make sense. ” P10 also described using embellishments to convinceviewers of the potential for DataVis, stating that “
I just want peo-le to get interested, people that aren’t in datavis for them to getinterested in it, to see it as not only something extremely informative,but something that could potentially be beautiful. And I especiallywant students and younger kids, and especially kids that think thatcoding is boring or that it’s like matrix-y, that this is something thatcould be beautiful and fun [. . . ] So I don’t follow the purest data,like purest bar chart or anything like that. ” ISCUSSION
Our analysis shows that practitioners’ understanding of chartjunkis very broad. While almost all participants knew that Tufte coinedthe term, and many knew about the data-ink ratio, there was no dis-cernible consensus on the definition of chartjunk. Some participantsdescribed chartjunk as distortions of graphics, others as anything“extra”, while others included animation and interactivity in theirdefinitions. One possible conclusion about this finding is that betterdefinitions are needed so that everyone has a shared understanding.Or perhaps more language is needed to differentiate between differ-ent kinds of embellishments and their functions. Another possibleconclusion is that standardized training is needed to ensure concep-tual consistency among practitioners. While we do not discountthese conclusions altogether, as clear definitions and training arecertainly valuable, we argue that a more important takeaway is thatwe need to understand the unique ways in which designers generateand use knowledge in real-world settings, with a particular focus onthe personal and situated nature of design practice.One interesting finding is that some participants thought of inter-action as akin to “junk” or “embellishment”, as in something thatis superfluous or redundant. While interaction may sometimes beunnecessary and can have costs [23], it can also have significantvalue for comprehension, especially with complex data [36]. As wedid not explicitly ask our participants about interaction, it is unclearhow widespread these views are. However, it is still a topic thatcould benefit from more investigation. While some recent workhas looked at whether adding interaction to static visualizations isbeneficial [27], better design guidance would likely be valuable.
Multiple participants described embellishments as irrelevant forthem, not because they had particular opinions about their use, butbecause they thought of themselves as not being able to produce em-bellishments effectively. This finding surfaces questions about theways in which tools can and should support the activities of visual-ization designers. For instance, it is unclear whether embellishmentscan be supported by tools in any kind of standardized fashion, orwhether they need to be custom-made in tools like Adobe Illustrator.This may be relevant for discussions that have been taking placein the visualization community regarding challenges in evaluatingvisualization authoring systems [33, 35] and efforts to study tooluse and other practice-oriented issues [3, 18, 25, 28]. This findingalso surfaces questions about the ways in which practitioners viewtheir own design competence and the skills required to be a gooddesigner. While some kinds of embellishments may require artisticskill, others certainly do not. However, without better guidanceregarding the use of embellishments, some practitioners may simplysee their use as an art that is unattainable for them.
Our work shows that the ways in which practitioners use chartjunkin their practice are highly varied and pluralistic, being strongly in-fluenced by personal characteristics and goals of designers, their un-derlying philosophies, and the situated and pragmatic aspects of thedesign setting. Although findings from studies on chartjunk suggestwhen and how its use is appropriate (e.g., to increase memorabilityor engagement), these recommendations do not fully characterize the considerations that practitioners give towards its use. Manyparticipants were at least vaguely aware of studies on memorabilityand comprehension, and claimed to be influenced by their findings,although usually not in ways that could be clearly articulated. Inaddition to any influence these findings had on practitioners, therewere clearly other influences that were highly personal and situated.These included descriptions of personal style and preference, skill(or lack thereof) in creating embellishments, constraints from clientsand branding, and underlying philosophical commitments about thevalue and purpose of visualization.It is difficult to determine how influential research findings arefor these practitioners, especially because people have a tendency inretrospective analysis to make situated activity seem more “rational”than it really was [38]. When asked about the ongoing discourseon chartjunk in academic and practitioner spaces, our participantswere more likely to be familiar with discussions happening on socialmedia or in practitioner publications than in academic papers andconferences. Thus it is hard to know where the drivers of attitudechange lie—e.g., the “corrective” movement discussed previously—and whether the trends within practitioner spaces are more influentialthan findings coming from researchers. While multiple participantsdescribed a useful “correction” away from extreme minimalismtaking place, only P19, who has a PhD in visualization, referred toany specific work in the academic literature.InfoVis research has valued abstract and rationalistic ways ofdiscussing knowledge and practice, which is typical of disciplineswith positivist foundations [26]. As a result, much research aims toremove or ignore the messy, personal, situated aspects of artifactsand their use, which are the exact things that need to be considered ifwe are to understand visualization design practice [13,14,37]. Schol-arship on design practice in other fields indicates that designers tendnot to use knowledge that is too prescriptive; rather, designers appre-ciate individual concepts or high-level ideas that can inspire, can beused metaphorically as thinking tools, and are open to interpretationand pluralistic means of use [2, 37]. Designers tend to appropriateconcepts and methods, understanding them in their own terms andusing them in ways that fit the situation, which may be differentfrom the intentions of their creators [15]. Thus, our findings—thatchartjunk is not well defined, is interpreted quite broadly, and is influ-enced by personal and situated concerns–are in line with findings inother design disciplines. It is plausible that the broad interpretationsof chartjunk may account, at least partially, for its popularity andcontinued attention over many decades.
UMMARY AND F UTURE W ORK
Our work has shown that chartjunk is understood very broadly andused in a variety of ways by DataVis practitioners. Furthermore, wehave shown that the interpretation and use of chartjunk goes beyondsimply applying guidelines and research findings in practice settings.Rather, practitioners rely on a host of personal and situated matterswhen deciding on when and how to use embellishments.Our work points to a growing movement beyond cognitivism,highlighting a turn to experience as an important framing fordiscussing and participating in design practice. This movementseems to be a growing in the research community as well (e.g.,[20, 31, 34, 42]), and perhaps it opens up new ways for researchersto investigate the effects of embellishments on users.We hope our investigation surfaces a need for more practice-led research within the visualization community. If InfoVis canfollow the lead of other design disciplines in embracing practice-ledaccounts of design, a more holistic account of how and why conceptssuch as chartjunk are used in real-world settings can be developed. R EFERENCES [1] S. Bateman, R. L. Mandryk, C. Gutwin, A. Genest, D. McDine, andC. Brooks. Useful junk? The effects of visual embellishment onomprehension and memorability of charts. In
Proceedings of theSIGCHI Conference on Human Factors in Computing Systems , 2010.[2] O. Bertelsen. Design Artefacts: Towards a design-oriented epistemol-ogy.
Scandinavian Journal of Information Systems , 12(1):15, 2000.[3] A. Bigelow, S. Drucker, D. Fisher, and M. Meyer. Iterating betweentools to create and edit visualizations.
IEEE Transactions on Visualiza-tion and Computer Graphics , 23(1):481–490, 2016.[4] A. J. Blasio and A. M. Bisantz. A comparison of the effects of datainkratio on performance with dynamic displays in a monitoring task.
In-ternational Journal of Industrial Ergonomics , 30(2):89–101, 2002.[5] R. Borgo, A. Abdul-Rahman, F. Mohamed, P. W. Grant, I. Reppa,L. Floridi, and M. Chen. An Empirical Study on Using Visual Embel-lishments in Visualization.
IEEE Transactions on Visualization andComputer Graphics , 18(12):2759–2768, 2012.[6] M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh,D. Borkin, H. Pfister, and A. Oliva. Beyond Memorability: Visualiza-tion Recognition and Recall.
IEEE Transactions on Visualization andComputer Graphics , 22(1):519–528, 2016.[7] M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva,and H. Pfister. What Makes a Visualization Memorable?
IEEETransactions on Visualization and Computer Graphics , 19(12):2306–2315, 2013.[8] V. Braun and V. Clarke. Using thematic analysis in psychology.
Quali-tative Research in Psychology , 3(2):77101, Jan 2006.[9] M. Correll and M. Gleicher. Bad for data, good for the brain:Knowledge-first axioms for visualization design. In , 2014.[10] N. Cross. Designerly ways of knowing.
Design Studies , 3(4):7, 1982.[11] S. Few. The Chartjunk Debate A Close Examination of RecentFindings. 2011. Retrieved from http://perceptualedge.com/articles/visual_business_intelligence/the_chartjunk_debate.pdf on July 1, 2020.[12] D. J. Gillan and D. Sorensen. Minimalism and the Syntax of Graphs:II. Effects of Graph Backgrounds on Visual Search.
Proceedings of theHuman Factors and Ergonomics Society Annual Meeting , 53(17):1096–1100, 2009. doi: 10.1177/154193120905301711[13] E. Goodman, E. Stolterman, and R. Wakkary. Understanding interac-tion design practices. In
Proceedings of the 2011 SIGCHI Conferenceon Human Factors in Computing Systems (CHI ’11) , p. 1061. Vancou-ver, BC, Canada, 2011. doi: 10.1145/1978942.1979100[14] C. M. Gray, C. Dagli, M. Demiral-Uzan, F. Ergulec, V. Tan, A. A.Altuwaijri, K. Gyabak, M. Hilligoss, R. Kizilboga, K. Tomita, andet al. Judgment and instructional design: How ID practitioners work inpractice.
Performance Improvement Quarterly , 28(3):2549, 2015.[15] C. M. Gray, E. Stolterman, and M. A. Siegel. Reprioritizing therelationship between HCI research and practice: bubble-up and trickle-down effects. In
Proceedings of the 2014 conference on Designinginteractive systems - DIS ’14 , pp. 725–734. ACM Press, 2014.[16] S. Haroz, R. Kosara, and S. L. Franconeri. ISOTYPE Visualization:Working Memory, Performance, and Engagement with Pictographs.In
Proceedings of the 2015 SIGCHI Conference on Human Factors inComputing Systems (CHI ’15) , pp. 1191–1200, 2015.[17] S. Hill, B. Wray, and C. Sibona. Minimalism in Data Visualization:Perceptions of Beauty, Clarity, Effectiveness, and Simplicity. In
Pro-ceedings of the Conference on Information Systems Applied Research ,vol. 10, p. 13. Austin, TX, USA, 2017.[18] J. Hoffswell, W. Li, and Z. Liu. Techniques for flexible responsivevisualization design. In
Proceedings of the 2020 SIGCHI Conferenceon Human Factors in Computing Systems (CHI ’20) , pp. 1–13, 2020.[19] J. Hullman, E. Adar, and P. Shah. Benefitting InfoVis with Visual Dif-ficulties.
IEEE Transactions on Visualization and Computer Graphics ,17(12):2213–2222, 2011. doi: 10.1109/TVCG.2011.175[20] Y.-H. Hung and P. Parsons. Assessing User Engagement in InformationVisualization. In
Proceedings of the 2017 SIGCHI Conference onHuman Factors in Computing Systems (CHI ’17), Extended Abstracts ,pp. 1708–1717. ACM, 2017. doi: 10.1145/3027063.3053113[21] O. Inbar, N. Tractinsky, and J. Meyer. Minimalism in InformationVisualization Attitudes towards Maximizing the Data-Ink Ratio. In
Proceedings of the ECCE 2007 Conference , pp. 185–188, 2007.[22] C. Kostelnick. The visual rhetoric of data displays: The conundrum of clarity.
IEEE Transactions on Professional Communication , 51(1):116–130, 2008.[23] H. Lam. A framework of interaction costs in information visualiza-tion.
IEEE Transactions on Visualization and Computer Graphics ,14(6):1149–1156, 2008.[24] H. Li and N. Moacdieh. Is chart junk useful? An extended examinationof visual embellishment. In
Proceedings of the Human Factors andErgonomics Society Annual Meeting , vol. 58, pp. 1516–1520, 2014.[25] G. G. M´endez, U. Hinrichs, and M. A. Nacenta. Bottom-up vs. top-down: trade-offs in efficiency, understanding, freedom and creativitywith infovis tools. In
Proceedings of the 2017 SIGCHI conference onhuman factors in computing systems (CHI ’17) , pp. 841–852, 2017.[26] M. Meyer and J. Dykes. Criteria for Rigor in Visualization DesignStudy.
IEEE Transactions on Visualization and Computer Graphics ,26(1):87–97, 2020. doi: 10.1109/TVCG.2019.2934539[27] A. Mosca, A. Ottley, and R. Chang. Does interaction improve bayesianreasoning with visualization? In
IEEE VIS Workshop on Visualizationfor Communication (VisComm ’20) , 2020.[28] P. Parsons, A. Baigelenov, Y.-H. Hung, and C. Schrank. What DesignMethods do DataVis Practitioners Know and Use? In
Proceedings ofthe 2020 SIGCHI Conference on Human Factors in Computing Systems(CHI ’20), Extended Abstracts , pp. 1–8, 2020.[29] P. Parsons, C. M. Gray, A. Baigelenov, and I. Carr. Design judg-ment in data visualization practice. In
Proceedings of the 2020 IEEEVisualization Conference (VIS), short papers , 2020.[30] A. Pea, E. Ragan, and L. Harrison. Memorability of Enhanced Infor-mational Graphics: the effects of design relevance and chart type onrecall.
Interdisciplinary Journal of Signage and Wayfinding , 4(1), 2020.Number: 1. doi: 10.15763/issn.2470-9670.2020.v4.i1.a54[31] Z. Pousman, J. Stasko, and M. Mateas. Casual Information Visual-ization: Depictions of Data in Everyday Life.
IEEE Transactions onVisualization and Computer Graphics , 13(6):1145–1152, 2007.[32] A. Quispel and A. Maes. Would you prefer pie or cupcakes? Prefer-ences for data visualization designs of professionals and laypeople ingraphic design.
Journal of Visual Languages & Computing , 25(2):107–116, 2014.[33] D. Ren, B. Lee, M. Brehmer, and N. H. Riche. Reflecting on theevaluation of visualization authoring systems: Position paper. In , pp. 86–92. IEEE, 2018.[34] B. Saket, A. Endert, and J. Stasko. Beyond Usability and Performance:A Review of User Experience-focused Evaluations in Visualization.In , pp. 133–142, 2016.[35] A. Satyanarayan, B. Lee, D. Ren, J. Heer, J. Stasko, J. Thompson,M. Brehmer, and Z. Liu. Critical reflections on visualization authoringsystems.
IEEE Transactions on Visualization and Computer Graphics ,26(1):461–471, 2019.[36] K. Sedig and P. Parsons. Interaction design for complex cognitiveactivities with visual representations: A pattern-based approach.
AISTransactions on Human-Computer Interaction , 5(2):84–133, 2013.[37] E. Stolterman. The Nature of Design Practice and Implications forInteraction Design Research.
International Journal of Design , 2(1):55–65, 2008.[38] L. A. Suchman.
Plans and Situated Actions: The Problem of Human-Machine Communication . Cambridge University Press, 1987.[39] N. Tractinsky and J. Meyer. Chartjunk or Goldgraph? Effects of Presen-tation Objectives and Content Desirability on Information Presentation.
MIS Quarterly , 23(3):397–420, 1999. doi: 10.2307/249469[40] E. R. Tufte.
The Visual Display of Quantitative Information . GraphicsPress, Cheshire, CT, USA, 1st edition ed., 1983.[41] J. Walny, C. Frisson, M. West, D. Kosminsky, S. Knudsen, S. Carpen-dale, and W. Willett. Data changes everything: Challenges and oppor-tunities in data visualization design handoff.
IEEE Transactions onVisualization and Computer Graphics , 26(1):12–22, 2019.[42] Y. Wang, A. Segal, R. Klatzky, D. F. Keefe, P. Isenberg, J. Hurtienne,E. Hornecker, T. Dwyer, and S. Barrass. An emotional response tothe value of visualization.