Distributed Synchronous Visualization Design: Challenges and Strategies
Tatiana Losev, Sarah Storteboom, Sheelagh Carpendale, Søren Knudsen
DDistributed Synchronous Visualization Design:Challenges and Strategies
Tatiana Losev * Simon Fraser University
Sarah Storteboom † University of Calgary
Sheelagh Carpendale ‡ Simon Fraser University
Søren Knudsen § University of Calgary
CDABEF 58h50h70h6h48h51hTotal0.05 2.00 4.00 6.00 8.00Legend (Hours):Past Collaborations This Collaboration: Time Spent in Zoom Meetings by Team Member
March 23 March 30 April 6 April 13 April 20 April 27 May 4 May 11 May 25May 18 June 1
Member
March 16
Figure 1: Overview of time spent video conferencing from March 16 to June 4 with concentrated synchronous design activities visiblefrom late March to mid May; pivot points in the design process seen on April 2 when members C and D met for a prolonged designsession using dual cameras as discussed in Section 7. See Table 2 for team roles. A BSTRACT
We reflect on our experiences as designers of COVID-19 data visu-alizations working in a distributed synchronous design space duringthe pandemic. This is especially relevant as the pandemic posednew challenges to distributed collaboration amidst civic lockdownmeasures and an increased dependency on spatially distributed team-work across almost all sectors. Working from home being ‘the newnormal’, we explored potential solutions for collaborating and pro-totyping remotely from our own homes using the existing tools atour disposal. Since members of our cross-disciplinary team haddifferent technical skills, we used a range of synchronous remotedesign tools and methods. We aimed to preserve the richness of co-located collaboration such as face-to-face physical presence, bodygestures, facial expressions, and the making and sharing of physicalartifacts. While meeting over Zoom, we sketched on paper and useddigital collaboration tools, such as Miro and Google Docs. Usingan auto-ethnographic approach, we articulate our challenges andstrategies throughout the process, providing useful insights aboutsynchronous distributed collaboration.
NTRODUCTION
In this paper, we describe and discuss our experiences of formingand working in a distributed visualization design team. While priorwork has discussed visualization design processes [51], it tendedto focus on how the visualization community works during face-to-face (co-located) design activities and processes for includingusers or domain experts. Less attention has been paid to how teamcollaboration works within teams of visualization designers, bethey distributed or co-located. We discuss our experiences of thedifferences between co-located and distributed visualization designwork specific to our unique needs and experiences in light of anincreased reliance on spatially distributed teamwork across almostall sectors.Our work was set in motion as part of a provincial response to theCOVID-19 pandemic. We were a small team within a broader groupof public health researchers who were providing data to government * e-mail: tatiana [email protected] † e-mail: [email protected] ‡ e-mail: [email protected] § e-mail: [email protected] decision-makers during the pandemic. We formed our design team ofsix members (see Figure 1, members A-F) to support the local healthauthorities’ pandemic response in collaboration with our colleaguesat the Centre for Health Informatics at University of Calgary. As wewere a newly acquainted multi-disciplinary team, we found it morebeneficial to learn and to discuss the data together in a synchronousenvironment in the beginning of the project. We worked on thedesign of visualizations of provincial and national COVID-19 datafor a public-facing website showing data visualizations of case andpolicy data. The intention with this site was to inform city- andprovince-level leaders to assist them in making sense of the status ofthe public health crisis and the associated data. While our team hadvisualization design expertise, our colleagues provided a wider set ofskills and knowledge in fields such as public health, epidemiology,and data science. Due to the pandemic, we were unable to meet face-to-face. Instead and against our common work practice, all designteam members worked from their home. This situation intensifiedchallenges of distributed collaboration caused by distracting at-homework spaces and a sense of urgency amidst civic lockdown measures.Through an auto-ethnographic approach [9, 14, 40, 41, 43], wereflect on our experiences of distributed, primarily synchronousideation and prototyping as a way to identify some challenges in ourprocess of remote visualization design. Our main contribution is anarticulation of challenges and strategies for dealing with these whiledoing distributed visualization design centered on the potential fordoing activities, using technologies, and processes for communicat-ing early and often, in order to reduce friction in visualization designteams.We relied on remote design tools and methods to actively designin real-time, often for two hours per day. We aimed to preservethe richness and diversity of co-located collaboration; specifically,the facilitation of face-to-face presence with the ability to viewresponsive body gestures, facial expressions, and the making andsharing of physical and digital artifacts. ELATED W ORK
This work relates to literature on how to design visualizations; onhow to support collaboration through visualizations; and on how tosupport design in technology-mitigated collaboration.
Making use of design methodology is widely recognized as im-portant for creating useful and usable visualizations [42, 51]. Thisechoes the broader discussions about design [51], which span awide array of concerns and advice about concrete design activities. a r X i v : . [ c s . H C ] S e p ome examples that focus on co-located activities include the use ofsketching [10, 17, 48, 54]; prototyping with a range of media [53],and the use of cards to support ideation [27]. A considerable amountof this design work is collaborative in nature ranging from the shar-ing of work in progress in design critiques [65], through how peopledesign together such as with co-creating artifacts [2], to ideas abouthow to involve people that are affected by a design, for example,through human-centered design [44] and participatory design [2]. Inaddition, there are discussions about how to communicate designs,for example, in creating hand-off documents [35] — many of whichpeople colloquially refer to as design (studio) praxis. In visualiza-tion, people have considered how to design for the intended audiencesuch as specific experts or the general public (for example, designstudy methodology [37, 41, 51]), and how to design with these peo-ple (for example, user-centered visualization design [16, 31]). Alsothere are suggestions of potentially useful concrete design activities(for example: those based on constructive visualization [26], thoseconducted as speculative design workshops [8, 30], and the use ofpencil-and-paper based sketching of data visualizations [62]). Thereis also advice on how to structure design processes, for example, howto teach visualization design to computer science students using fivedesign sheet method [48]), and how to more clearly communicatevisualization designs (for example, in designers communicating vi-sualization designs to developers as part of a hand-off process [61]).These are all extremely informative when at least some of the activ-ities can happen co-located, however, they are less applicable forsynchronous distributed design activities. In synchronous distributeddesign activities, they serve more as goals than as methods. Theseideas will need to be re-interpreted in the context of synchronousdistributed design work. There is a rich body of literature on collaboration in related fieldssuch as human-computer interaction (HCI) and computer supportedcooperative work (CSCW). The CSCW matrix (see Table 1), whichseparates collaborative work along space time axes is useful forunderstanding this area [18]. Two main areas of the CSCW matrixhave been purposefully considered in visualization: co-located syn-chronous collaboration (Table 1, top-left quadrant) and distributedasynchronous collaboration (Table 1, bottom-right quadrant). Al-though these two modes of collaboration have primarily been studiedin isolation — perhaps due to their different technological basis —they share many challenges (see for example Isenberg et al. [28]).Research exploring co-located collaborative visualization includesthe use of tabletops for collaborative information access, for exam-ple, Scott et al. [49, 50], the consideration of tabletop displays forcollaborative browsing of hierarchical layouts of photographs [59],for analysis of scientific data [57], and for exploration of bookcollections [56]. More recently, people have considered large, high-resolution displays for supporting collaborative visual analytics (forexample, Langner et al. [32] and Knudsen & Hornbæk [29]).Research on visualization for communicating across both spaceand time (asynchronous distributed) have led to ideas about de-mocratizing visualization by making it accessible to all and to theincreasing inclusion of visualizations as a way to communicate datain news media. Sense.us [23] and ManyEyes [60] introduced thecollaborative possibilities for visualizations on the web. This typeof research situates visualizations in a broader social and societalcontext. Based on these kinds of systems, Heer & Agrawala [21] pro-vide design considerations for collaborative visualization on the webmore broadly. Later, work has shown that structuring the processesin collaborative asynchronous visual analysis can lead to increasedanalysis quality [63, 64], which might provide ideas for subsequentvisualization designs [25].While collaborative visualization-based analysis is distinct fromcollaborative visualization design, the CSCW matrix helps conceptu-
Table 1: CSCW matrix for consideration in visualization design.
Same time(synchronous) Different time(asynchronous)Same place(co-located) same place same time same place differenttimeDifferent place(distributed) different place sametime different place differ-ent timealizing the space of synchronous distributed collaboration in relationto other collaborative contexts. There are also relevant similaritiesbetween collaborative visualization-based analysis and visualizationdesign. For example, being able to point to a visualization or part ofone is both important when designing and using visualizations [22].Supporting collaborative use of visualization is an important researchdirection. However, supporting visualization design in synchronousdistributed settings has not yet been discussed.
While the visualization literature includes many discussion aboutdesign [3,5,16,20,31,37–39,41,42,51,61], the focus is on the designprocesses rather than the collaborative process. The collaborationsdiscussed tend to focus on how visualization researchers collaboratein long-term projects with domain experts. For example, while theterm “collaboration” (and related forms) is used 34 times in thedesign studies paper [51], it is only used a single time in the sectionthat discusses the “core phase” of the design study methodology.Discussions about collaborative design thinking seems to be missing.Similarly, in CSCW literature, while there are discussions aboutmany different types of work, the focus has been on distributedasynchronous and co-located synchronous. We are interested indistributed synchronous design activities.
There is CSCW literature about collaborative design processes (forsome examples see [1, 7, 15, 24, 45, 55, 66]). However, the focus isstill about collaborative design when co-location is part of the designprocess. For example, some have explored technology mitigatedcollaborations through tabletop display tools [7, 24], and throughinvestigating the impact of technology based feedback about thegroup creative design process [55]. One suggestion is to explorethe use of the crowd in design processes [66]. Visualization hasbeen used to provide feedback on speaking times and speaking turnsduring collaboration [4].There is some exploration of the space we are interested in – thespace of how to re-kindle the benefits of co-located collaboration ina technology-supported, synchronous distributed situation. Arias etal. [1] start by acknowledging the complex design problems oftenrequire group solutions and articulates needed design support forurban design problems. Both Fischer [15] and Obendorf et al. [45]consider the complexity of team-based design needs where teamsmust cope with difference in time, space, and knowledge and makea call for deeper exploration of the needs of these types of designteams. Our work, which describes our experiences of the challengesof collaborative synchronous, distributed technology-mediated de-sign, and a range of strategies for dealing with these challenges,contributes to this larger call for research.
YNCHRONOUS D ISTRIBUTED V ISUALIZATION D ESIGN
Amid the COVID-19 lockdown, we were faced with the urgentchallenge to design a useful COVID-19 visualization. We wereconfronted with the reality that our familiar co-located team-basedcollaboration design approaches could not be directly applied inur enforced distributed but synchronous realm. While this was achallenge in many ways, we managed to reach an effective designprocess. Through the use of a reflective auto-ethnographic researchapproach, we have obtained a deeper understanding of these chal-lenges. We first describe our research approach.
We conducted a team-based self-study for this project by combin-ing an auto-ethnographic research approach [12] with hermeneuticphenomenology [58]. Auto-ethnographic research is befitting be-cause its primary evidence base is the direct narrative of the peopleinvolved [12]. Hermeneutic phenomenology complements this asit studies the meanings of lived experience via texts and artifactsthrough iterative self-reflection, writing, and discussion [33, 58].Together, these approaches deepened our understanding of our ex-perience as a team of designers creating visualizations together ina synchronous distributed context. Though more commonly seenin the social sciences, we benefit from these qualitative researchapproaches. They enable us to focus on our unique experiences ofdesigning visualizations in a synchronous distributed setting throughthe discovery of themes that occurred in our non-linear collaborativepractice.
Our group of designers worked together, as a team, for the firsttime, though some team members (AB, AEF, ABC, BC, and BD —also shown in Figure 1) had worked together previously on separateprojects. The team dynamics and the social setting of the lockdownwere novel to the whole team. The domain specific data and theneeds of the project had yet to be learned. Thus, the team needed toget acquainted with one another as well as the data. We found thatspending more time together in a synchronous setting facilitated animmediate peer-to-peer learning, and improved our communication–– in our experience, this was the most suitable way for us to connect,to build personal relationships in our group, and to improve groupsynergy. Initially, team members A, E, and F discussed the projectfor about two weeks, followed by members A and B discussingpossible additional members to balance skills in data visualization,design, public health, and programming (see Table 2). Team memberA assembled the team and called the first team meeting.
Following an auto-ethnographic approach, the data is both a re-sult of our process and fueled it. Thus we discuss them as inter-twined. We looked closely into our process in the visualizationproject through iterative analysis of our own experiential data andproject artifacts [12, 13, 33]. In auto-ethnography we are both theparticipants and the researchers — through a self-reflexive account,we explore themes and patterns found in our experience as a group;we examine our experience as a multi-disciplinary team of designersworking towards deriving deeper meanings from collective experi-ence. Doing so, we questioned: how has distributed collaborationshaped our experience of synchronous design processes?
Our process was as follows: We continuously collected processdata and members of the team kept regular day-by-day written teamnotes; the team’s visualization sketches were collectively stored on aMiro board; our design brief on Google Docs and the Slack historytexts were gathered and reviewed; we created a visual timeline fromour Slack history; we formulated questions to guide our inquiry andreflection based on our collected data and previous personal experi-ences relevant to visualization design and health communications;we characterized, analyzed and reflected on our texts, discussed thetexts, and generated new texts; we held reflective discussions anddocumented this in our notes in Google Docs; themes that emergedin our text and from our reflective dialogue were grouped; throughour reflections, we interpreted our documented themes and continued
Table 2: Overview of our team.
Team member Role Expertise in useA PI Visualization, design, programming,management, communicationB PI Visualization, designC Design lead Visualization, designD Designer Public health, designE Intern Visualization, programmingF Intern Visualization, programmingto write these reflections on our account; we corroborated collec-tive written experiences with one another through further discussionto validate the findings. Lastly, we wrote this paper by detailingour experiences and findings by repeatedly going through the stepsabove.
March 13, 2020, two days before a local state of emergency was an-nounced, the office of the mayor requested help from the Centre forHealth Informatics to get a better sense of non-clinical interventionsof COVID-19 locally, nationally, and internationally. At that timethere was a generalized sense of fear and widespread sensationalismthat was broadcast via a multitude of media. A team of 37 membersof researchers in health and data sciences from the centre assembledon Slack to create a “COVID-19 Working Group” determined toresearch COVID-19 data in response to the urgent call for informa-tion. The hope was to help by contributing critical information to aidinformed decision-making about managing the pandemic. Workingunder an immense sense of urgency, the team collected cumulativeand daily case numbers, researched global COVID-19 policies, andworked on epidemiological disease models that informed the sta-tus of the pandemic on a local and national scale. This researchwas used by municipal and provincial policy-makers. Two teammembers promptly responded by using open COVID-19 data anda web charting library to, within a few days, assemble a websitethat tracked the changing local and provincial COVID-19 data. Thiswebsite became known within the team as the “COVID-19 Tracker”[11]. During the first 8 weeks, the site garnered 15,000 page views.While delighted about the speed of this action, the need for amore carefully designed response was noted and a different teammember assembled a smaller design team. It is this smaller groupthat is our design team and it is our actions in this team that wefocus on. Our initial conversations in design team were via a specifi-cally formed Slack channel. Our design team’s dedicated channeldecreased the amount of notifications to members of the COVID-19Working Group and helped focus our discussions. Additionally, ourdesign team started to meet using video conferencing. We discussedCOVID-19 design issues to better understand the impact of designon a broader cross-section of people. Our design team delved intothe intricacies of the available public COVID-19 data with the goalof designing data visualizations that would support a broad cross-section of the population, which the working group had identified asimportant: provincial and municipal decision makers, public healthofficers, as well as the general public. We met frequently — of-ten several hours a day and still met more than once a week withthe COVID-19 Working Group to align with the project needs anddirection in response to the status of the pandemic.
ESCRIBING O UR D ESIGN P ROCESS E XPERIENCES
In this section we describe how we experienced the activities thatwe engaged in as the design team. In keeping with our auto-ethnographic and phenomenological methodologies, rigor in thisreport means staying true to the reality of our experiences in ourteam via detailed descriptions and iterative reflections on our textsnd artifacts [13, 33]. In starting our distributed design process, weconsulted with the literature. However, we discovered only limitedadvice on how to organize collaborative synchronous distributedvisualization design processes. While we considered our readingsabout visualization design and distributed design in CSCW, welargely relied on our own, largely face-to-face, experience of priordesign processes in visualization design and beyond. Here, we de-scribe how we experienced synchronous distributed visualizationdesign. By articulating challenges and strategies, we discuss the fac-tors that arose in our experiences that may prove useful to considerwhen doing synchronous distributed visualization design.
The design team met to discuss and to sketch together for two hoursa day, five days a week and attended half hour meetings with thebroader team a few times per week. We mostly used Zoom formeetings, Slack for short asynchronous communications, GoogleDocs for written notes and records, and Miro Board for collabo-rative design.Daily 2-hour meetings over Zoom brought the team together andallowed for developing an awareness of each other and established asocial dynamic and rapport among the team members. The meetingswere a time to convene and establish project expectations, sketch,and design together. Most of us had device cameras positioned toshow our face during the Zoom meetings. The team also participatedin the larger Zoom meetings with the COVID-19 Working Group togain feedback on our visualization ideas and sketches, and to hearof new developments or requests from senior leadership. A Slackchannel was the main hub to set up meeting times, to share readingand video materials about COVID-19 data and visualization design,and to inform each other of online events such as webinars. We alsoused the Slack channel to post our design ideas. Our visualizationdesign team made use of a collaborative digital whiteboard (Miro).This provided digital space for the group to post sketches, PDF’s,and virtual sticky notes during design meetings. Meeting notes and adesign brief were created and stored in Google Docs and were usedconcurrently during our team meetings with one team member takingnotes. After five weeks of daily Zoom meetings, our team reducedmeeting times over a collective sense that meetings needed to bemore directed and convergent — the ideation phase was wrappingup and the team was keen to implement the design. To reach a shared understanding of the project expectations, includ-ing timelines and target audiences, we collaboratively authored adesign brief. This document directed some of our discussions as weconsidered the purpose of the site, our audience, and their familiaritywith the data. The ideation process began with a distributed face-to-face critique of the COVID-19 visualizations that were alreadyavailable on our site and across all of the provincial sites in Canada.Screenshots of the various visualizations were compiled on a Miroboard along with suggestions for improvements. This in turn led toideas for improvements to the COVID Tracker website. The activityof critiquing visual elements of other visualizations was a beneficiallearning exercise that focused our sketching sessions and informedour design choices. This activity enabled an engaged explorationof the data, of the end-user audience, and the purpose and messag-ing of the visualizations. Importantly, this activity presented thecomplexity of the data and enabled us to identify further questionsand consultations necessary to validate our findings. While we wereinterested in helping people understand relationships between oth-erwise disparate data sets, such as case numbers and policies, we https://zoom.com/ https://slack.com/ https://docs.google.com/ https://miro.com/ recognized that separating certain aspects of the data was critical soas to not suggest causation where correlations might exist. For exam-ple, it became clear that juxtaposing policy data with case numberscould be misconstrued as a causal relationship. The goal of our synchronous online meetings during the ideationphase was to generate many ideas and sketches, while gaining anunderstanding of COVID-19 data. We spent a lot of time consideringand exploring the data; understanding testing rates, positive casenumbers, hospitalization cases, and disease transmission along withpolicies and correlations. We formed questions through repeateddiscussions, which, in consultation with members of the COVID-19Working Group, provided a rich method for developing an in-depthunderstanding of the data and issues of interest.Sketching was a valuable activity that helped us to think throughconcepts, envision a story, and share ideas. Sketching enabledour team to see the data and gain a shared sense of our individualperspectives. During the meetings, and while apart, we sketchedon paper and tablets. The sketches were the main artifacts that weeach created and showed to each other either through presenting ourphysical sketch to the device camera or posting it onto our Miroboard. The sketches served as the foundation for our discussions.The design meeting notes and artifacts were stored, categorizedby date, and accessible to the team. The cache of sketches alongwith inspiration clippings and meeting notes proved very useful. Wewere able to refer to previously posted resources and sketches duringdesign meetings, enhancing our ability to recall our previous workand build on it. However, We found the lag time when posting ourpaper sketches onto our virtual whiteboard to be challenging. Wedealt with this by holding sketches up to the camera, but they werenot easily referenced until they were scanned and added to the board.
Illustrator versions of our design were produced by the lead designerso we could view a pixel-perfect design. After several iterations, theIllustrator file was handed off to three team members who imple-mented the design using D3 [6]. Several iterations of the softwareprototype were critiqued during collaborative design team sessionsand subsequently tweaked. The design phase for the data visual-ization continued through implementation as updating data sourcespresents new challenges to be solved. When prototypes were pol-ished and reflected live data sources, they were presented in a Zoommeeting to the COVID-19 Working Group for feedback.
MERGING F ACTORS IN O UR V IRTUAL V ISUALIZATION D ESIGN
Our purpose was to think critically about how to communicate thecomplexity of the pandemic and the data while ensuring that ourvisualisation would not be misinforming. Our process, however,started by identifying the missing factors in our distributed designsituation. The familiar lab environment facilitated serendipity, nat-ural discussion, and a tangible sense of togetherness that allowedfor ideas to spontaneously emerge. In order to support the processof ideation and data discovery in our distributed environment, wewished to collaborate via real-time sketching and discussion akinto a collocated design environment such as a lab. We searched foruseful tools and materials to enable the team members to communi-cate ideas about interactions for the design and to share them, as wenormally would, “in-person”.We note that being distributed forced us to be upfront about theprocess. This in turn supported later reflection because our dis-tributed work had been logged through the various tools used in thedesign process. This design experience was distinct as a result ofdistributed collaboration under the time sensitive demands of a pub-lic health emergency. Furthermore, remoteness posed a perceivedisk of misunderstanding and miscommunication, more so given aflux of ever-changing public health data that decision makers wererelying upon. This design experience elucidated several factors forre-consideration. Notably, these factors were initially experiencedas challenges but sometimes, through working with these challenges,we also noted potential strategies and opportunities.In the following, we describe eleven factors that emerged froman auto-ethnographic exploration of our distributed visualizationdesign process. While the design factors encompass a wide arrayof considerations, we recognize that there are more opportunitiesand strategies possible through re-purposing current software andhardware tools.
First and unsurprisingly, the pandemic lockdown led to challenges.These are important to recognise as they heavily influenced ourability to carry out work.
C1. Negotiating Time and Resources
Coordinating time and access to physical space for some team mem-bers was a challenge amidst the social lockdown. This challengeis important because the pandemic brought forward some socialconstraints to distributed work from home such as interruptions ineach member’s home environment, lack of physical working space,and faulty hardware that played a role in how the team managedto design together remotely.
For example, some collaborators wereforced to leave meetings in order to take care of their children and attimes their children showed themselves to the camera during designmeetings. There were instances when team members had to movetheir device to a different room during meetings because their smallshared living spaces were prioritized based on homeschooling needsof children or for other family members who were working fromhome. We contended with web cams that did not function, so someteam members were not visible during camera-to-camera meetings.Purchasing web cameras at the time was difficult due to a high de-mand in the market with increased remote work. As a team, weresolved to continue meetings during these circumstances. Audiowas muted for a moment when children interrupted our online meet-ings and we waited until family disruption ended. We persisted withmeetings though some members were not visible, interacting onlyvia audio. Additionally, this work provided an opportunity to “bewith” other people and to “contribute” in a way that was a catharticexercise for several team members in the midst of this pandemic.
OMMUNICATION B REAKDOWNS
This group of challenges focus on how our adaptation to distributedcollaborations caused several types of communication challenges —some of which we learned how to mitigate.
C2. Establishing Team Cohesion
The sense of uncertainty that arose steeply during the initial COVID-19 lockdown, along with our own perceived challenges of remotecollaboration, posed risks of miscommunication and possible diffi-culties in establishing a spirit of team trust and cohesion.
This wasa challenge we were particularly aware of, because, while most of uswere in the same geographical region, we designed synchronouslyfrom our homes. Our intention was to recreate the familiar “face-to-face” co-located communication flow even though we were workingin a distributed virtual setting. We chose frequent synchronous vir-tual collaboration as a way to 1) align our ideas and learn 2) mitigatea sense of uncertainty during a pandemic with frequent feedback 3)establish team cohesion and rapport. This virtual space was a newreality for many collaborators. Seeing one another brought a senseof togetherness and co-presence that was conducive to successfuland natural synchronous teamwork. Notably, it was possible forour virtual interaction to achieve a similar communication workflow with the team’s facial expressions and gestures visible through ourdevice cameras. Human communication and connections are aninterplay of nuanced cultural mannerisms that may not be visibleor as obvious through a webcam. Nonetheless, seeing each otherthrough webcams and computer screens established a collectiveawareness and a sense of attendance within the team though weweren’t co-located. However, eye-to-eye contact differed from beingco-located. We tended to look at each other’s faces on the screen andnot directly into our cameras, often seeming as though eye contactwas averted during conversations. We found that group etiquettenaturally developed in our online work space such as muting themicrophone during excessive background noise, putting up a handto speak, and waving goodbye into the camera. A couple of theteam members chose to keep their cameras off during all onlinemeetings but remained audible, which created a barrier to gaining anunderstanding of their affect and their level of engagement becausegestures, facial expressions, and eye contact were not visible. Wefound that members who had their cameras turned on often domi-nated and contributed more to the discussion and invested more timethroughout the project.
C3. Understanding Team Members’ Progress
The challenge was to determine the progress of individuals on theteam so that we could move forward with project goals. It was impor-tant to understand where team members were in their work to ensurethat we were completing our goals.
The team was rapidly assembled;some members were volunteers. They were unfamiliar with eachother’s skills and were unsure if it was okay to ask others given thecircumstances. Some work needed to be done asynchronously and,from time to time, team members were not able to follow throughwith tasks. For example, we decided to assign tasks such as to scopenationwide provincial sites for visualizations of COVID-19 dataand compile the findings in an Excel file, while another memberwas reviewing other information outside of our team meetings. Itwas also valuable to prepare some information before our designteam meetings to bolster our discussions about visualization design.To ensure that everyone was aware of the status of each task, teammember A would check on the progress via Slack, which was visibleto the whole team. This method was useful to ensure transparencyand clear expectations for individual task completion within ourdesign process. Some members, notably team members C and Doften self-assigned tasks while other members relied upon A for taskassignment and review.
C4. Diversity of Software and Sketching Expertise
There was considerable variation in levels of skill when sketchingideas or visualizing data, meaning that some of the team mem-bers struggled more than others with design activities and requiredguidance.
As a solution to some of these differences, the lead visu-alization designer mentored willing team members who were lessexperienced in designing data visualizations. For example, we heldoptional group sketching sessions via Zoom as a way to learn andto practice sketching apart from our team meetings. To bypass soft-ware knowledge gaps, we photographed physical paper sketchesand shared them on the Miro board because everyone was able touse their device cameras comfortably. However, photographing oursketches stifled the natural flow of sketching and sharing duringmeetings, so we reverted to sketching and presenting our sketchesthrough the camera. This method led to more productive discussionsand more sketching to take place during the meetings. We also sawa benefit in the virtual environment because sketches could be easilyseen by everyone at the same time provided the camera was set upin such a way that the sketch was well lit and in focus. We wouldoften scan sketches using a phone app or transfer them to the digitalwhiteboard, where we could continue to review the sketches. Afterreviewing the sketches and discussing their features we decided onhe best sketch to prototype. We also found screen sharing to be abenefit during the polishing stage of a working prototype. Whiledesigners might want to adjust some of the positioning and style ofelements in a working prototype, they may not have a developmentspace set up or the skills to make those adjustments. The program-ming team members utilized screen sharing to collaboratively makethose adjustments in real time with the designers.
C5. Understanding and Sharing Data
Learning about the project needs and the health data was founda-tional to our visualization design, so we spent considerable timereviewing and discussing policy data, testing data, health care sys-tem capacity, social determinants of health, studying COVID-19visualizations across Canadian jurisdictions, and in regular con-sultations with the COVID-19 Working Group in a remote setting.
As COVID-19 data was being visualized broadly across the world,we knew how the data was typically shown, so we considered theclarity and lack of clarity in existing visualizations. Therefore, welooked for new ways to visualize the same data that would not per-petuate the same issues. Information gathering was crucial to ourdesign process because it enabled us to identify relationships andassumptions within the data as we sketched data visualizations inthe early design stage. Notably, this process provided a fruitfuldistributed design environment with a serendipitous sense of datadiscovery. This collaborative online knowledge seeking provideda generative online learning space. Additionally, this activity al-lowed us to deepen our collective understanding of the data andcultivated team rapport. This is an area where we identified a clearbenefit to working in a distributed environment. We were able towork independently and then quickly share our findings when iden-tifying something interesting. For example, we considered criticaldiscussions of visualization, such as the understanding that positivecases were indicative of testing capacity. We followed this insightby brainstorming ways to visualize this relationship in a more clearmanner as part of a larger visualization product.
C6. Sharing and Acquiring Knowledge
It was challenging to share and introduce knowledge and back-ground skills to other team members. Likewise, it was challenging toacquire new understanding from other team members.
Distributedmodes of collaboration introduce more friction in sharing knowl-edge. It is more difficult to find and share resources and point tospecific parts of a resource as it is prone to error and requires time.This is particularly relevant in a visualization context. In contrast,in co-located situations pointing, body language, and using artifactssuch as laptops to communicate “see this” is easy and affords alow-friction possibility for understanding whether you have caughtpeoples’ attention. In extension, these situations allow the receivingside to see exactly what is meant and allow for clarifying questionsto a specific aspect. For example, we discussed story-telling aspectsof our design. However, discovering the depth and subtlety of thedata and how it fits with storytelling concepts appeared overwhelm-ing. Especially as we intended to deepen a shared understanding ofthis data through visualizations and storytelling, and further, from aninteraction perspective, to use scrollytelling to show the complexityof COVID-19 data. We considered literature on using storytelling el-ements in visualization [47] to prompt meaningful discussions aboutthe data. For example, we referred to the Martini Glass structure [52]as a potential way to scaffold the complexity of the data and to createcohesion between COVID-19 charts. However, while all team mem-bers attempted to grasp these concepts, some team members had asense of superficially understanding these ideas and found it difficultto use them when thinking about designs. Despite these challenges,working remotely also poses benefits for knowledge exchange. Byworking in a distributed team, resources will typically be sharedthrough a medium that allows people to return to them, which is particularly beneficial when acquiring new knowledge.
C7. Understanding Design Ideas
Working in a distributed design team adds friction to the processof understanding other collaborator’s perspective throughout thedesign process [19]. In our case, it took more time to establish under-standing when sharing diverse ideas in the process of interpretingdomain-specific data and creating data visualizations.
Typically, aco-located physical space that facilitates the sharing of ideas throughface-to-face team activities is used during ideation and iterationphases of the design process. For example, we would compile oursketches on the wall of a meeting room for all members to viewand discuss. We adapted this activity to our online synchronoussetting by posting sketches to an online collaborative whiteboard inMiro while simultaneously meeting via Zoom. Despite the learningcurve of adopting the software, we found this strategy allowed usto share and discuss our ideas productively. Working in the virtualwhiteboard environment opened up new opportunities that would notbe as feasible when working with the physical counterpart. Endlessspace offered in the virtual whiteboard allowed the team a great dealof flexibility in how the content was laid out. Items on the boardcould be arranged linearly, and grouped and regrouped as the designprocess unfolded. The team was able to quickly reference previ-ously shared content, which aided in clarifying how assumptionsand misunderstanding had arisen. The ability to use virtual pointingand to jump to another’s pointing location played a beneficial rolein the virtual work space. While pointing is possible in the physicalenvironment, this can at times lack accuracy or require people tophysically move closer to the item they are pointing at, which istime-consuming and may occlude it. There is also the limitation thatonly so many people can be close to a small clipping or sketchbookpage. The virtual space permits all collaborators to have an optimalview of where the speaker is pointing and an equal opportunity topoint at things. Iteration was an integral part of the design process,and for this it was often useful to markup existing sketches and de-sign outputs. The digital environment offered the ability to quicklyduplicate and markup as many copies of something as needed with-out compromising the integrity of the original. This was especiallyenabling when one team member wanted to iterate on another teammember’s sketch because doing so was immediate and essentiallyrisk-free.
TRATEGIES FOR D ISTRIBUTED D ESIGN
The intensity of the pandemic situation coupled with the team’sgeneral willingness to experiment led us to explore alternative uses ofhardware and software. Next, we discuss current potential strategiesand potential opportunities that lie ahead.
S1. Simulating a Co-located Design Space for Sketching
It is difficult to gain a full view of group sketching in a distributedsetting because online meetings are generally limited to one viewper participant. In an attempt to create a real-time collaborativesketching experience, two team members tested a setup of two devicecameras per person during a Zoom meeting.
The setup includedsigning into the meeting twice with two separate devices; one de-vice was a PC camera aimed at the face of each participant and asecond device was a phone camera that was directed on their paperand pen. Though it was awkward to find a suitable angle and stabi-lize the phone, facial expressions, gestures and gaze were capturedalong with a view of real time sketching. This method allowedthe conversation to be held concurrently with a view of sketchingpractice as it was unfolding within the discussion. This remotesketching activity was seamless in a virtual distributed design setupthat simulated a co-located design environment. It supported conver-sational and visual communication, however, it was cumbersome torecreate this physical setup and it was not attempted after a single igure 2: An excerpt of our Miro board that captures a design session for lo-fi prototyping of visualization interaction. During the session allparticipants were able to duplicate and arrange assets to mimic paper prototyping. trial. There is considerable potential for developing better and easierways to set up this type of distributed collaboration. For instance,this approach could bypass knowledge gaps or inaccessibility tousing digital sketching tools such as a Wacom tablet. Likewise, weimagine specialised software might provide support for this purpose.
S2. Screen-Sharing to Collaborate in Software Applications
Different team members can use their software fluency and bringdifferent skills to the table. With the goal of including everyonethroughout the design process, we used screen-sharing via Zoom tocollaborate in specialized environments such as designing in AdobeIllustrator or editing code.
What made this method particularly use-ful was when there was skill cross-over in the team because meetingparticipants could control the mouse during a screen-sharing session.For example, a team member shared their design on the screen ofan open Adobe Illustrator workspace during the Zoom meeting. An-other team member controlled the mouse from a separate locationon their shared Adobe Illustrator workspace. This process allowedfor real-time manipulation of visual components of our design andto share skills between multiple team members. Even in cases wherecollaborators were not manipulating the software remotely, we foundsharing specialized environments to be useful as it mimicked casualco-located collaboration. For example, designers might often gatheraround the work space of the person implementing a design to tweakposition, padding, and other style details. We simulated this experi-ence with screen-sharing and being able to grant remote access tothe mouse meant that team members could point out these detailswith accuracy.
S3. Using Hand Gestures for Discussing Interaction
Animating designs using video or animation software is difficult forteam members that lack such technical skills. Not everyone in thegroup knew how to digitally animate a design, which excluded someteam members.
We discovered that the most accessible way for eachteam member to show their ideas was by talking camera-to-camerasimilar to a face-to-face co-located setting. The software knowledgegap was resolved by relying on hand gestures and moving staticdesign artifacts with our hands while remaining visible in the camera.Everyone in the team with access to a web camera could speak totheir ideas and show how they imagined the design would move withintuitive hand motions, facial expressions, sounds, pointing to, ormoving their own sketches or cut outs while explaining their ideain front of the camera. This emphasized a fuller presence within aremote collaborative experience similar to that of a lab. We reflected on our choice of communicating about interactions and comparedit to our previous visualization design experiences. Based on this,we think we would have pointed to sketches in a co-located modeof collaboration instead of moving our hands in mid-air in front ofthe camera. We see the hand gestures as adding an extra barrier tocommunicating about interactions.
S4. Lo-fi Prototyping for Visualization Interaction
It was challenging to find optimal tools to enable all of us to testinteractions in a digital format given our variable programmingexpertise. We aimed to re-create the use of paper prototyping ina digital format.
To set this up, we used a digital whiteboard inMiro as a “table top” space and exported assets from Illustratorto use as “paper” clippings. Teleconferencing via Zoom enabledgesturing and speaking into the camera, while all team memberscould simultaneously copy, resize, and rearrange assets in Miro.Team members were able to collaboratively create UI mock-ups(Figure 2) and animate them by clicking and dragging. We foundthis method to be quick and inclusive to all members, regardlessof skill. Additionally, we found the ability to duplicate elementsand groups of elements without disrupting the integrity of previousmodels provided a benefit over traditional paper prototyping.
ISCUSSION
As a team of information visualization designers we described andstudied our experience using an auto-ethnographic approach in orderto deepen our understanding of the socio-cultural and technologicalfacets that affected how we experienced visualization design meth-ods in our project. Iterative reflection and thematic organization ofour experiences presented us with overarching themes of challengesand strategies. During our pursuit to create a vibrant online collabo-ration in the early stages of visualization design, we were constantlyreminded of situations that needed to be addressed, and that wefelt were essential to maintain our co-located design process eventhough we were no longer able to meet in person. In doing so, wediscovered themes that emerged from the identified challenges andstrategies that arose through our continued reflective process, leadingus to try new ways to conduct distributed synchronous collaboration.The themes and strategies we extracted from our account may offeran example to the members of the visualization community whomay identify with similar experiences that they too have had. Thismay serve to amplify an experiential evidence base to inform futurevisualization research and methodology in this domain.e found that tools for designing, creating, and using visualiza-tions such as Adobe Illustrator, RAW graphs [36], Tableau, DataIllustrator [34], and Charticulator [46] did not offer the supportwe were looking for in our synchronous collaboration. We morefrequently relied on external web applications to collaborate bothindependently and together during our design sessions. We foundthe combined use of multiple applications such as teleconferencing(in our case, Zoom), virtual whiteboards (in our case, Miro), andcollaborative word processors (in our case, Google Docs) providedconsiderable flexibility. However, constantly switching betweenapplications became cumbersome at times, particularly when co-ordinating these between several team members people during ateleconferencing session. Improved orchestration and integrationbetween these applications, as well as more fine-tuned support forcollaborative online sketching would be interesting directions toinvestigate and welcome improvements in such contexts.In many design processes there are often activities that requirethe use of web tools or computing environments and for these wefelt the shift to remote work did not hinder our performance of theseactivities. Research, data understanding, collaborative writing, fi-nessing of high fidelity designs and prototypes are activities thatoften require the use of a computer. In a co-located scenario, weoften have team members gather in a conference room with personallaptops that they must attach to an external display to properly sharewith the group, or gather at the shoulder of one team member whilethey informally demo something on their desktop. These types ofin-person collaborative moments can be cumbersome. However, wefound they happened more rapidly and comfortably over telecon-ferencing by utilizing built in tools such as screen sharing, or incombination with external collaborative software.We struggled more to adapt in the ideation and early iterationphases because our process relied on brainstorming, sketching, andrapid paper prototyping. We tried to mimic real-time collaborativesketching by setting up a dual screen conference, or by holdingsketches up to the webcam. To some extent, we were able to sustainspontaneity of brainstorming and idea sharing similar to those of co-located collaboration by continuing to use mostly physical sketchingmaterials like pen and paper. The transfer of physical sketches to thevirtual whiteboard was cumbersome and we see this as an area forpotential improvement. However, once the sketches are on a digitalwhiteboard they continue to be accessible for all future discussionsand are easily duplicated and iterated on much more easily than theirphysical counterparts. While we discussed access to equipment anddistraction as being primarily a pandemic related challenge, we alsosee this as being potentially more widely problematic. Improvementswe could make for our sketching sessions might involve high-qualitywebcams and sketching tablets for all team members. However, thecost and learning curve associated with adopting such technologyis high considering that low-fi technology such as pen and paperwork very efficiently and with more versatility. Likewise, havingdedicated space and time to do one’s work is essential for focus, andoffloading the responsibility to the individual to carve it out of theirdomestic space is understandably challenging for remote workers.Where we saw the largest challenge for remote work was commu-nication and social dynamics. Working in a co-located environmentfacilitates serendipity, natural discussion, and a tangible sense oftogetherness that allows for ideas to emerge spontaneously. Wefound some team members engaged in prolonged teleconferencingsessions to experience the sense of community and to concurrentlyshare ideas as more of a “hangout” rather than a meeting. However,other team members who neither could nor chose to utilize videoand did not engage in the “hangouts” slowly drifted away from theproject likely because they felt excluded. In a co-located environ-ment, it is quite clear that an individual is committed to workingbecause they have physically arrived at work, and likewise one canpick up on how busy or idle a co-worker might be when they are co- located. Being disconnected to these physical cues, in conjunctionwith voluntary roles in our particular project, made assigning tasksor setting expectations challenging. We discovered that social dy-namics that seamlessly sort themselves out in a co-located scenariorequire a lot more facilitation and management effort. We imaginethat this may require an additional role or skill set added to a team, orbetter integration of a person’s status (“available”, “busy”, “away”)into the collaborative environments. Messaging applications suchas Slack provide status information, but in our experience, they lacknuance and integration with realistic highly synchronous workflows.There is no shortage of collaborative software that aims to increaseproductivity and facilitate focus sessions, but few that fulfill thesense of community and spontaneous collaboration that happens inco-located work environments.Finally, it is worth noting that the ability to reflect on our experi-ence of synchronous distributed design is largely due to the fact thatwe were distributed. We were forced to put every piece of inspirationand every sketch into Miro. All of our notes and communicationsare documented on Slack and in Google docs. Having this detailedrepository at every stage allowed us to reflect and gain insight intoour process and learn from our experience.
ONCLUSION
In this paper, we reflected on our experiences doing distributed vi-sualization design in a remote synchronous design space during thepandemic. We based our reflections on our work on designing aCOVID-19 visualization as part of a larger team effort on supportingcity- and provincial-level decision-making. We discussed how thepandemic posed new challenges to remote collaboration amidst civiclockdown measures and imposed an increased dependency on spa-tially distributed teamwork across almost all sectors. As a responseto the barriers of working from home being “the new normal”, weused various synchronous remote design tools and methods withan aim to preserve the richness of co-located collaboration suchas face-to-face physical presence with the ability to view real-timebody gestures, facial expressions, and the making and sharing ofphysical artifacts. Based on these, we articulated issues in teamcomposition and communication, both of which affected our visual-ization design process. We discussed the challenges of working ina distributed visualization design team, such as creating a sense ofa shared work environment, which we liken to the idea of a designstudio, and how to share data and potential ideas for visualizationdata. Finally, we offered potential solutions and benefits to workingremotely. Our discovery of the challenges and strategies through-out the process provide useful insights about enabling a more fluiddistributed collaboration.While descriptions and discussions of visualization design tendedto assume co-located design teams, designers clearly need to carryout some work independently and in different locations. Prior workin other fields have shed light on this. However, there is importantwork to be done in considering these contexts for visualization designprocesses. As a step towards developing a better understanding ofdistributed, synchronous visualization design, this paper provides anexperiential understanding and conceptualization of this area. A CKNOWLEDGMENTS
The authors would like to thank Jackie Yu, Neil Chulpongsatorn andthe Centre for Health Informatics, Cumming School of Medicine atthe University of Calgary for their contributions to the process. Thisresearch was supported in part by the Natural Sciences and Engi-neering Research Council of Canada (NSERC) and the EuropeanUnion’s Horizon 2020 research and innovation programme underthe Marie Sklodowska-Curie grant agreement No. 753816.
EFERENCES [1] E. Arias, H. Eden, G. Fischer, A. Gorman, and E. Scharff. Tran-scending the individual human mind—creating shared understandingthrough collaborative design.
ACM Transactions on Computer-HumanInteraction (TOCHI) , 7(1):84–113, 2000.[2] L. Bannon and P. Ehn. Design matters in participatory design. In
Routledge International Handbook of Participatory Design . Routledge,2012.[3] R. Beecham, J. Dykes, C. Rooney, and W. Wong. Design ExpositionDiscussion Documents for Rich Design Discourse in Applied Visual-ization.
IEEE Transactions on Visualization and Computer Graphics ,pp. 1–1, 2020. doi: 10.1109/TVCG.2020.2979433[4] T. Bergstrom and K. Karahalios. Conversation clock: Visualizingaudio patterns in co-located groups. In , pp. 78–78.IEEE, 2007.[5] A. Bigelow, S. Drucker, D. Fisher, and M. Meyer. Reflections on HowDesigners Design with Data. In
Proceedings of the 2014 InternationalWorking Conference on Advanced Visual Interfaces , AVI ’14, pp. 17–24.ACM, New York, NY, USA, 2014. doi: 10.1145/2598153.2598175[6] M. Bostock, V. Ogievetsky, and J. Heer. D³ data-driven docu-ments.
IEEE Transactions on Visualization and Computer Graphics ,17(12):2301–2309, 2011.[7] T. Bratteteig and I. Wagner. Spaces for participatory creativity.
CoDe-sign , 8(2-3):105–126, 2012.[8] N. Bressa, K. Wannamaker, H. Korsgaard, W. Willett, and J. Vermeulen.Sketching and Ideation Activities for Situated Visualization Design. In
Proceedings of the 2019 on Designing Interactive Systems Conference ,DIS ’19, pp. 173–185. Association for Computing Machinery, SanDiego, CA, USA, June 2019. doi: 10.1145/3322276.3322326[9] R. V. Bullough and S. Pinnegar. Guidelines for Quality in Autobi-ographical Forms of Self-Study Research.
Educational Researcher ,30(3):13–21, Apr. 2001. doi: 10.3102/0013189X030003013[10] B. Buxton.
Sketching User Experiences: Getting the Design Right andthe Right Design . Morgan Kaufmann Publishers Inc., San Francisco,CA, USA, 2007.[11] Centre for Health Informatics, Cumming School of Medicine, Uni-versity of Calgary. The COVID-19 Response: How is Canada do-ing and what should we be doing next? 2020. Available at: (accessed on September 2, 2020).[12] H. Chang. Individual and collaborative autoethnography as method.
Handbook of autoethnography , pp. 107–122, 2013.[13] J. W. Creswell and C. N. Poth.
Qualitative inquiry and research design:Choosing among five approaches . Sage publications, 2016.[14] M. Duncan. Autoethnography: Critical Appreciation of an EmergingArt.
International Journal of Qualitative Methods , 3(4):28–39, Dec.2004. doi: 10.1177/160940690400300403[15] G. Fischer. Social creativity: turning barriers into opportunities forcollaborative design. In
Proceedings of the eighth conference on Par-ticipatory design: Artful integration: interweaving media, materialsand practices-Volume 1 , pp. 152–161, 2004.[16] S. Goodwin, J. Dykes, S. Jones, I. Dillingham, G. Dove, A. Duffy,A. Kachkaev, A. Slingsby, and J. Wood. Creative User-Centered Vi-sualization Design for Energy Analysts and Modelers.
IEEE Trans-actions on Visualization and Computer Graphics , 19(12):2516–2525,Dec. 2013. doi: 10.1109/TVCG.2013.145[17] S. Greenberg, S. Carpendale, N. Marquardt, and B. Buxton.
SketchingUser Experiences: The Workbook . Elsevier, 2011.[18] J. Grudin. Computer-supported cooperative work: History and focus.
Computer , 27(5):19–26, May 1994. doi: 10.1109/2.291294[19] C. Gutwin and S. Greenberg. Design for individuals, design for groups:Tradeoffs between power and workspace awareness. In
Proceedingsof the 1998 ACM Conference on Computer Supported CooperativeWork , CSCW ’98, pp. 207–216. Association for Computing Machinery,Seattle, Washington, USA, Nov. 1998. doi: 10.1145/289444.289495[20] K. W. Hall, A. J. Bradley, U. Hinrichs, S. Huron, J. Wood, C. Collins,and S. Carpendale. Design by Immersion: A Transdisciplinary Ap-proach to Problem-Driven Visualizations.
IEEE Transactions on Visu-alization and Computer Graphics , 26(1):109–118, Jan. 2020. doi: 10. 1109/TVCG.2019.2934790[21] J. Heer and M. Agrawala. Design Considerations for CollaborativeVisual Analytics.
Information Visualization , 7(1):49–62, Mar. 2008.doi: 10.1057/palgrave.ivs.9500167[22] J. Heer, F. van Ham, S. Carpendale, C. Weaver, and P. Isenberg. Cre-ation and Collaboration: Engaging New Audiences for InformationVisualization. In A. Kerren, J. T. Stasko, J.-D. Fekete, and C. North,eds.,
Information Visualization: Human-Centered Issues and Perspec-tives , Lecture Notes in Computer Science, pp. 92–133. Springer, Berlin,Heidelberg, 2008. doi: 10.1007/978-3-540-70956-5 5[23] J. Heer, F. B. Vi´egas, and M. Wattenberg. Voyagers and voyeurs: Sup-porting asynchronous collaborative information visualization. In
Pro-ceedings of the SIGCHI Conference on Human Factors in ComputingSystems , CHI ’07, pp. 1029–1038. Association for Computing Ma-chinery, San Jose, California, USA, Apr. 2007. doi: 10.1145/1240624.1240781[24] O. Hilliges, L. Terrenghi, S. Boring, D. Kim, H. Richter, and A. Butz.Designing for collaborative creative problem solving. In
Proceedingsof the 6th ACM SIGCHI conference on Creativity & cognition , pp.137–146, 2007.[25] J. Hullman, N. Diakopoulos, E. Momeni, and E. Adar. Content, Con-text, and Critique: Commenting on a Data Visualization Blog. In
Proceedings of the 18th ACM Conference on Computer SupportedCooperative Work & Social Computing , CSCW ’15, pp. 1170–1175.Association for Computing Machinery, Vancouver, BC, Canada, Feb.2015. doi: 10.1145/2675133.2675207[26] S. Huron, S. Carpendale, J. Boy, and J.-D. Fekete. Using VisKit:A Manual for Running a Constructive Visualization Workshop. In
Pedagogy of Data Visualization Workshop at IEEE VIS 2016 , Oct.2016.[27] IDEO. Method cards: 51 ways to inspire design.
Palo Alto, CA, USA ,2003. Available at: (accessed on August 7, 2020).[28] P. Isenberg, N. Elmqvist, J. Scholtz, D. Cernea, K.-L. Ma, and H. Ha-gen. Collaborative visualization: Definition, challenges, and researchagenda.
Information Visualization , 10(4):310–326, Oct. 2011. doi: 10.1177/1473871611412817[29] S. Knudsen and K. Hornbæk. PADE: Supporting Collaborative Vi-sual Analysis of Patient Administrative Systems Data with a LargeTouch Display System. In , pp. 9–16, Oct. 2019. doi: 10.1109/VAHC47919.2019.8945039[30] S. Knudsen, M. R. Jakobsen, and K. Hornbæk. An exploratory studyof how abundant display space may support data analysis. In
ProcNordiCHI , pp. 558–567. ACM, 2012.[31] L. C. Koh, A. Slingsby, J. Dykes, and T. S. Kam. Developing andApplying a User-Centered Model for the Design and Implementation ofInformation Visualization Tools. In , pp. 90–95, July 2011. doi: 10.1109/IV.2011.32[32] R. Langner, U. Kister, and R. Dachselt. Multiple Coordinated Viewsat Large Displays for Multiple Users: Empirical Findings on UserBehavior, Movements, and Distances.
IEEE Transactions on Visual-ization and Computer Graphics , 25(1):608–618, Jan. 2019. doi: 10.1109/TVCG.2018.2865235[33] S. M. Laverty. Hermeneutic phenomenology and phenomenology: Acomparison of historical and methodological considerations.
Interna-tional journal of qualitative methods , 2(3):21–35, 2003.[34] Z. Liu, J. Thompson, A. Wilson, M. Dontcheva, J. Delorey, S. Grigg,B. Kerr, and J. Stasko. Data illustrator: Augmenting vector design toolswith lazy data binding for expressive visualization authoring. In
Pro-ceedings of the 2018 CHI Conference on Human Factors in ComputingSystems , CHI ’18, p. 1–13. Association for Computing Machinery, NewYork, NY, USA, 2018. doi: 10.1145/3173574.3173697[35] N. Maudet, G. Leiva, M. Beaudouin-Lafon, and W. Mackay. DesignBreakdowns: Designer-Developer Gaps in Representing and Interpret-ing Interactive Systems. In
Proceedings of the 2017 ACM Conferenceon Computer Supported Cooperative Work and Social Computing ,CSCW ’17, pp. 630–641. Association for Computing Machinery, Port-land, Oregon, USA, Feb. 2017. doi: 10.1145/2998181.299819036] M. Mauri, T. Elli, G. Caviglia, G. Uboldi, and M. Azzi. Rawgraphs:A visualisation platform to create open outputs. In
Proceedings of the12th Biannual Conference on Italian SIGCHI Chapter , CHItaly ’17.Association for Computing Machinery, New York, NY, USA, 2017.doi: 10.1145/3125571.3125585[37] N. McCurdy, J. Dykes, and M. Meyer. Action Design Research andVisualization Design. In
Proceedings of the Sixth Workshop on BeyondTime and Errors on Novel Evaluation Methods for Visualization , BE-LIV ’16, pp. 10–18. Association for Computing Machinery, Baltimore,MD, USA, Oct. 2016. doi: 10.1145/2993901.2993916[38] S. McKenna, A. Lex, and M. Meyer. Worksheets for Guiding Novicesthrough the Visualization Design Process. arXiv:1709.05723 [cs] , Sept.2017.[39] S. McKenna, D. Mazur, J. Agutter, and M. Meyer. Design ActivityFramework for Visualization Design.
IEEE Transactions on Visualiza-tion and Computer Graphics , 20(12):2191–2200, Dec. 2014. doi: 10.1109/TVCG.2014.2346331[40] M. Meyer and J. Dykes. Reflection on Reflection in Applied Visualiza-tion Research.
IEEE Computer Graphics and Applications , 38(6):9–16,Nov. 2018. doi: 10.1109/MCG.2018.2874523[41] M. Meyer and J. Dykes. Criteria for Rigor in Visualization DesignStudy.
IEEE Transactions on Visualization and Computer Graphics ,26(1):87–97, Jan. 2020. doi: 10.1109/TVCG.2019.2934539[42] T. Munzner.
Visualization Analysis and Design . AK Peters/CRC Press,2014.[43] C. Neustaedter and P. Sengers. Autobiographical design in hci research:designing and learning through use-it-yourself. In
Proceedings of theDesigning Interactive Systems Conference , pp. 514–523, 2012.[44] D. A. Norman and S. W. Draper.
User Centered System Design; NewPerspectives on Human-Computer Interaction . L. Erlbaum AssociatesInc., USA, 1986.[45] H. Obendorf, M. Janneck, and M. Finck. Inter-contextual distributedparticipatory design.
Scandinavian Journal of Information Systems ,21(1):2, 2009.[46] D. Ren, B. Lee, and M. Brehmer. Charticulator: Interactive construc-tion of bespoke chart layouts.
IEEE Transactions on Visualization andComputer Graphics , 25(1):789–799, 2019.[47] N. H. Riche, C. Hurter, N. Diakopoulos, and S. Carpendale.
Data-driven storytelling . CRC Press, 2018.[48] J. C. Roberts, C. Headleand, and P. D. Ritsos. Sketching DesignsUsing the Five Design-Sheet Methodology.
IEEE Transactions onVisualization and Computer Graphics , 22(1):419–428, Jan. 2016. doi:10.1109/TVCG.2015.2467271[49] S. D. Scott, M. S. T. Carpendale, and K. M. Inkpen. Territorialityin Collaborative Tabletop Workspaces. In
Proc CSCW , pp. 294–303.ACM, 2004. doi: 10.1145/1031607.1031655[50] S. D. Scott, K. D. Grant, and R. L. Mandryk. System Guidelines forCo-located, Collaborative Work on a Tabletop Display. In K. Kuutti,E. H. Karsten, G. Fitzpatrick, P. Dourish, and K. Schmidt, eds.,
ECSCW2003 , pp. 159–178. Springer Netherlands, Dordrecht, 2003. doi: 10.1007/978-94-010-0068-0 9[51] M. Sedlmair, M. Meyer, and T. Munzner. Design Study Methodology:Reflections from the Trenches and the Stacks.
IEEE Transactions onVisualization and Computer Graphics , 18(12):2431–2440, Dec. 2012.doi: 10.1109/TVCG.2012.213[52] E. Segel and J. Heer. Narrative Visualization: Telling Stories withData.
IEEE Transactions on Visualization and Computer Graphics ,16(6):1139–1148, Nov. 2010. doi: 10.1109/TVCG.2010.179[53] C. Snyder.
Paper Prototyping . Morgan Kaufmann, 1st ed., 2003.[54] M. Sturdee and J. Lindley. Sketching & Drawing as Future Inquiry inHCI. In
Proceedings of the Halfway to the Future Symposium 2019 ,pp. 1–10. ACM, Nottingham United Kingdom, Nov. 2019. doi: 10.1145/3363384.3363402[55] S. Tausch, F. Steinberger, and H. Hußmann. Thinking like disney:Supporting the disney method using ambient feedback based on groupperformance. In
IFIP Conference on Human-Computer Interaction ,pp. 614–621. Springer, 2015.[56] A. Thudt, U. Hinrichs, and S. Carpendale. The bohemian bookshelf:Supporting serendipitous book discoveries through information visu-alization. In
Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI ’12 , p. 1461. ACM Press,Austin, Texas, USA, 2012. doi: 10.1145/2207676.2208607[57] M. Tobiasz, P. Isenberg, and S. Carpendale. Lark: Coordinating Co-located Collaboration with Information Visualization.
IEEE Transac-tions on Visualization and Computer Graphics , 15(6):1065–1072, Nov.2009. doi: 10.1109/TVCG.2009.162[58] M. Van Manen.
Researching lived experience: Human science for anaction sensitive pedagogy . Routledge, 2016.[59] F. Vernier, N. Lesh, and C. Shen. Visualization techniques for circulartabletop interfaces. In
Proceedings of the Working Conference onAdvanced Visual Interfaces , AVI ’02, pp. 257–265. Association forComputing Machinery, Trento, Italy, May 2002. doi: 10.1145/1556262.1556305[60] F. B. Viegas, M. Wattenberg, F. van Ham, J. Kriss, and M. McKeon.ManyEyes: A Site for Visualization at Internet Scale.
IEEE Transac-tions on Visualization and Computer Graphics , 13(6):1121–1128, Nov.2007. doi: 10.1109/TVCG.2007.70577[61] J. Walny, C. Frisson, M. West, D. Kosminsky, S. Knudsen, S. Carpen-dale, and W. Willett. Data Changes Everything: Challenges and Op-portunities in Data Visualization Design Handoff.
IEEE Transactionson Visualization and Computer Graphics , 26(1):12–22, Jan. 2020. doi:10.1109/TVCG.2019.2934538[62] J. Walny, S. Huron, and S. Carpendale. An Exploratory Study ofData Sketching for Visual Representation.
Computer Graphics Forum ,34(3):231–240, 2015. doi: 10.1111/cgf.12635[63] W. Willett, J. Heer, and M. Agrawala. Strategies for crowdsourcingsocial data analysis. In
Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems , CHI ’12, pp. 227–236. Associ-ation for Computing Machinery, Austin, Texas, USA, May 2012. doi:10.1145/2207676.2207709[64] W. Willett, J. Heer, J. Hellerstein, and M. Agrawala. CommentSpace:Structured support for collaborative visual analysis. In
Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems , CHI’11, pp. 3131–3140. Association for Computing Machinery, Vancouver,BC, Canada, May 2011. doi: 10.1145/1978942.1979407[65] T. V. Wolf, J. A. Rode, J. Sussman, and W. A. Kellogg. Dispelling”design” as the black art of CHI. In
Proceedings of the SIGCHI Con-ference on Human Factors in Computing Systems - CHI ’06 , p. 521.ACM Press, Montreal, Quebec, Canada, 2006. doi: 10.1145/1124772.1124853[66] L. Yu and J. V. Nickerson. Cooks or cobblers? crowd creativitythrough combination. In