How do Visualization Designers Think? Design Cognition as a Core Aspect of Visualization Psychology
HHow do Visualization Designers Think? Design Cognition as a CoreAspect of Visualization Psychology
Paul Parsons * Purdue University A BSTRACT
There are numerous opportunities for engaging in research at theintersection of psychology and visualization. While most oppor-tunities taken up by the VIS community will likely focus on thepsychology of users, there are also opportunities for studying thepsychology of designers. In this position paper, I argue the impor-tance of studying design cognition as a necessary component of aholistic program of research on visualization psychology. I providea brief overview of research on design cognition in other disciplines,and discuss opportunities for VIS to build an analogous researchprogram. Doing so can lead to a stronger integration of research anddesign practice, can provide a better understanding of how to educateand train future designers, and will likely surface both challengesand opportunities for future research.
NTRODUCTION
Psychology is undoubtedly important for visualization research andpractice. Numerous scholars have pointed to the need for betterintegration of the two fields, especially as an expansion beyond per-ception to focus on higher-level cognitive processes and structures.However, virtually all calls for more of this kind of research focus entirely on users and not on designers . Although more research onuser psychology is indeed necessary, it can provide only a partialview of the ways in which visualizations are created and used. Mostdisciplines with a robust relationship between research and prac-tice have developed research programs investigating how designersthink while engaged in their design practice, commonly referredto as design cognition . These include both longstanding fields likearchitecture [21] and emergent fields like instructional design [27],graphic design [33], and user experience design [16]. These disci-plines have all recognized the value of studying how designers thinkand know as a valid and useful complement to studying the psycho-logical aspects of the use of artifacts. The integration of these twoperspectives can lead to a more holistic view of how visualizationsare created and used.Design research in VIS has largely been model driven, whereattempts are made to codify aspects of design in abstract forms. Pop-ular examples include process models (e.g., Design Study Method-ology [30], Design Activity Framework [23]) and decision mod-els (e.g., Nested Blocks [25], Nested Blocks and Guidelines [24]).While the latter models focus on decisions that designers can orshould make in design situations, they do not engage with decisionmaking at the level of cognitive processes or structures. Thus thereis a gap in our understanding of how and why design decisions aremade, and the psychological factors that influence their outcomes.In this position paper, I argue that there is a need to focus on design cognition in VIS research. I provide a brief overview ofresearch on design cognition in other disciplines, and discuss oppor-tunities for VIS to build an analogous research program. Doing socan lead to a stronger integration of research and design practice, * e-mail: [email protected] can provide a better understanding of how to educate and train futuredesigners, and will likely surface both challenges and opportunitiesfor future research. ESIGN C OGNITION
How do designers formulate and solve design problems? What kindsof cognitive processes do they rely on while doing so? These are thetypes of questions asked by researchers studying design cognition.Rather than foregrounding the methods, tools, or outcomes of design-ers, studies in design cognition investigate how and why designersthink the ways they do while designing. Design cognition has beenstudied across a wide variety of domains, including engineering [2],architecture [21], computer science [5], instructional design [27],and graphic design [33]. Across these disciplines, many aspects ofcognition in design have been investigated, including, among oth-ers, episodic memory [19], fixation [28], chunking [22], bias [13],abductive reasoning [7], analogical reasoning [34], metacognitivemonitoring and control [3], and recall [9].Much of the research on design cognition is concerned with howdesigners navigate the complexity and uncertainty of real-world de-sign situations [4,31,32]. A number of core strategies have been iden-tified through empirical investigation, including conjecture-basedproblem formulation, problem-solution co-evolution, analogical rea-soning, mental simulation, and xated solution generation [3, 15].Many of the cognitive processes that are relevant for studying the useof visualizations are also important for understanding their design.For instance, studies have shown that designers rely on chunking toideate effectively [22], employ abductive reasoning during conceptselection [14], are influenced by color in ways that bias their think-ing while sketching [13], and struggle with fixation while generatingideas [8]. Given the considerable evidence of such issues impact-ing design across numerous disciplines, it is very likely that theseare also important for understanding how and why visualizationdesigners design the ways they do.Although it may be tempting to simply borrow these findings fromthese other disciplines and apply them to VIS, research has shownthat significant differences exist in design cognition across domains,even though there are similarities [35]. For instance, Akin [1] notessignificant differences in design cognition among engineers andarchitects, and Purcell and Gero [28] identifies differences betweenmechanical engineers and product designers. In a review paperexamining design across numerous domains, Visser [35] affirmsboth these differences and similarities, and speculates that thesedifferences may have implications for the kinds of knowledge thatdesigners rely on.Following this work, it is reasonable to assume that design cog-nition in VIS will share similarities with these other fields, yet willalso have its own unique characteristics. For instance, designers inother fields do not deal with issues regarding data, mapping abstractinformation onto visual forms, and interactivity in the ways that VISdesigners must. Research on design cognition has significantly influ-enced theory, practice, and education in numerous design fields [18],and could similarly do so for VIS. However, the particular facets ofVIS that make it different from engineering or graphic design, forinstance, must be carefully examined as a part of such an effort. a r X i v : . [ c s . H C ] S e p .1 Design and Applied Research One possible reason why VIS has not seen a focus on design cog-nition, and design practice more broadly, is a common assump-tion that design is simply an application of knowledge generatedfrom scientific research. This view suggests that if designers knowenough—if they understand the principles and concepts that comefrom research—then they can apply them in their work. For instance,if enough studies are done on visual encoding and perception, ormemorability and embellishment, or bias and chart types, designersjust have to know the results and determine how to apply them incontext.This view was prominent in multiple design fields decades ago,but has since largely been abandoned by design scholars, as it doesnot accurately reflect the true nature of design practice [10, 20].While scientific knowledge certainly plays a role in design, it is notsufficient for good design [32]. Rather, designers rely on a hostof personal and situated factors, along with more formal types ofknowledge, to engage appropriately with the complexity of designpractice [26]. Buchanan [4] articulates how widespread this assump-tion has been, noting that “each of the sciences that have come intocontact with design has tended to regard design as an ‘applied’ ver-sion of its own knowledge”, emphasizing the mistake of viewingdesign as simply a “practical demonstration” of scientific findings.Thus, even if a robust program of research at the intersection ofpsychology and visualization is developed, if its scope is limitedto users only—and especially if design is viewed as an applicationof research findings—we will fail to understand how to influencedesign practice effectively.
PPORTUNITIES AND R ESEARCH Q UESTIONS
Because the study of design cognition has a rich history in otherdisciplines, yet is still nascent in VIS, research questions can betranslated from fields like architecture, instructional design, andothers, and posed in a VIS context. For instance, much research hasfocused on the differences between novice and expert designers, andresearch questions can be generated to investigate these differences,including: How do novices and experts differ with respect to framingproblems involving complex datasets and use cases? Are expertscapable of more flexible ways of recognizing and framing problems?How do novices and experts differ with respect to chunking whilegenerating ideas for chart types? Do experts engage in more ad-vanced chunking strategies that allow for nuanced application ofprinciples regarding visual encodings or embellishments?Another line of research can select individual concepts or topicsto study. For instance, with respect to creativity, the ways in whichcognitive strategies are used during ideation, and how designers mayget fixated on certain things, can be examined. Given an example ofa well-known visualization for a particular kind of data and context,do designers become fixated on that particular type of solution,unable to see viable alternatives? And what kinds of supports canbe given to mitigate that fixation?Another line of research may characterize the similarities and dif-ferences among VIS and other fields with respect to design cognition.For instance, do VIS designers rely on abductive reasoning in similarways to instructional designers? Is mental simulation different forVIS designers and UX designers? And what kinds of metacognitivestrategies do VIS designers rely on that may be similar or differentto software engineers?A different approach can start with the more universal processesof design cognition, using those to investigate differences in novicesand experts or specific cognitive processes and structures. In hisseminal work, Cross [11, 12] articulated two fundamental processesof design cognition: problem formulation and solution generation.These two processes are fairly abstract, yet are essential aspects ofdesign and are thus mostly universal. Problem formulation refers toprocesses in which designers make sense of ill-defined situations and determine the ‘problem’ and its implications. Solution generationrefers to processes in which designers move from a problem to asatisfactory solution. This could be a reasonable starting point fordesign cognition research in VIS. Designers could be recruited andgiven a problem brief comprising a dataset, target users, and setof tasks that users need to accomplish. Their verbalizations couldbe captured and analyzed, with specific attention paid to knownindicators of these cognitive processes. For instance, it is generallyrecognized that during problem formulation designers attend tospecific aspects of the problem space by naming them, articulatingrelevant concerns as part of the problem frame, and articulating acoherent narrative that helps guide subsequent design decisions [29].The possibilities noted here are only a small sample, and it iscertain that this list is far from exhaustive. There are likely dozensof research questions that can be posed based on prior studies onother areas. However, the ideas mentioned here are simply a startingpoint to begin a discussion on engaging in this kind of research.
Previous research has heavily relied on “protocol studies” to investi-gate the nature of design cognition [11, 15]. This method, which isalready well-known to VIS researchers doing human-subjects stud-ies, involves asking designers to ‘think-aloud’ while doing a designactivity. These studies generate verbal protocols that can be tran-scribed and analyzed with the goal of uncovering aspects of thinkingand reasoning. This kind of approach can be taken with individualdesigners who work alone on design problems, or with teams ofdesigners working together. Team-based protocols can be used toelicit socio-cognitive facets of collaborative design cognition.Similar to other aspects of VIS research, designers can be studiedin both controlled settings, such as a lab or workshop, and in lesscontrolled settings, such in their everyday design settings. Studiesin controlled settings are beneficial as they allow common designtasks to be given to participants, and allow for the control of vari-ables, including time spent, access to resources, and so on. Althoughempirical lab studies are commonly employed in design research(e.g., [6, 17]), they differ from realistic practitioner contexts in anumber of ways. For instance, lab studies may exclude factors thatshape design work in commercial settings, including the effectsof organizational culture, project timescales, project managementand workload. Lab studies may also present participants with rela-tively simple problems over short time periods, which are not oftenrepresentative of real-world design tasks. As is often the case inexperimental research, there is the risk of reducing both ecologi-cal and external validity [6]. For these reasons, it is beneficial toconduct studies in both controlled “lab” settings and “in the wild”of real-world practice. Studies can employ a range of methods, in-cluding protocol analysis, semi-structured interviews, diary studies,contextual observations, and co-design workshops.
UMMARY
There are numerous opportunities for engaging in research at theintersection of psychology and visualization. While most oppor-tunities taken up by the VIS community will likely focus on thepsychology of users, there are also opportunities for studying thepsychology of designers. In this position paper, I have argued theimportance of studying design cognition as a necessary componentof a holistic program of research on visualizaiton psychology. Per-haps the most obvious implication for this kind of research is in theeducation and training of future visualization designers. However,there are also implications for research and design practice. Un-derstanding design cognition can help generate research topics andquestions for the research community, and can stimulate the creationof design methods, concepts, and other types of design knowledgethat can be used in practice.
EFERENCES [1] O. Akin. Variants in Design Cognition. In C. Eastman, W. Michael Mc-Cracken, and W. C. Newstetter, eds.,
Design Knowing and Learning:Cognition in Design Education , pp. 105–124. Elsevier, 2001.[2] C. J. Atman, R. S. Adams, M. E. Cardella, J. Turns, S. Mosborg, andJ. Saleem. Engineering design processes: A comparison of students andexpert practitioners.
Journal of Engineering Education , 96(4):359–379,2007.[3] L. J. Ball and B. T. Christensen. Advancing an understanding of designcognition and design metacognition: Progress and prospects.
DesignStudies , 65:35–59, 2019. doi: 10.1016/j.destud.2019.10.003[4] R. Buchanan. Wicked Problems in Design Thinking.
Design Issues ,8(2):5–21, 1992.[5] J. M. Carroll. Scenarios and design cognition. In
Proceedings IEEEJoint International Conference on Requirements Engineering , pp. 3–5,2002. doi: 10.1109/ICRE.2002.1048498[6] P. J. Cash, B. J. Hicks, and S. J. Culley. A comparison of designeractivity using core design situations in the laboratory and practice.
Design Studies , 34(5):575–611, 2013. doi: 10.1016/j.destud.2013.03.002[7] C. L. Cramer-Petersen, B. T. Christensen, and S. Ahmed-Kristensen.Empirically analysing design reasoning patterns: Abductive-deductivereasoning patterns dominate design idea generation.
Design Studies ,60:39–70, 2019. doi: 10.1016/j.destud.2018.10.001[8] N. Crilly. Creativity and fixation in the real world: A literature reviewof case study research.
Design Studies , p. S0142694X1930047X, 2019.doi: 10.1016/j.destud.2019.07.002[9] D. P. Crismond.
Investigate-and-Redesign Tasks as a Context for Learn-ing and Doing Science and Technology: A Study of Naive, Novice andExpert High School and Adult Designers Doing Product Comparisonsand Redesign Tasks . PhD thesis, Harvard University, 1997.[10] N. Cross. Designerly ways of knowing.
Design Studies , 3(4):7, 1982.[11] N. Cross. Design cognition: results from protocol and other empiricalstudies of design activity. In C. Eastman, W. Michael McCracken, andW. C. Newstetter, eds.,
Design Knowing and Learning: Cognition inDesign Education , pp. 79–103. Elsevier, 2001.[12] N. Cross. Designerly Ways of Knowing: Design Discipline VersusDesign Science.
Design Issues , 17(3):49–55, 2001. doi: 10.1162/074793601750357196[13] A. Damle and P. J. Smith. Biasing cognitive processes during design:the effects of color.
Design Studies , 30(5):521–540, 2009. doi: 10.1016/j.destud.2009.01.001[14] A. Dong, D. Lovallo, and R. Mounarath. The effect of abductivereasoning on concept selection decisions.
Design Studies , 37:37–58,2015. doi: 10.1016/j.destud.2014.12.004[15] C. M. Eastman. New Directions in Design Cognition: Studies of Rep-resentation and Recall. In
Design Knowing and Learning: Cognitionin Design Education , pp. 147–198. 2001.[16] C. M. Gray. “Its more of a mindset than a method”: UX practitionersconception of design methods. In
Proceedings of the 2016 CHI Confer-ence on Human Factors in Computing Systems - CHI 16 , p. 40444055,2016. doi: 10.1145/2858036.2858410[17] N. V. Hernandez, J. J. Shah, and S. M. Smith. Understanding designideation mechanisms through multilevel aligned empirical studies.
De-sign Studies , 31(4):382–410, July 2010. doi: 10.1016/j.destud.2010.04.001[18] H. Jiang and C.-C. Yen. Protocol Analysis in Design Research: areview. In
Proceedings of the International Association of Societies ofDesign Research Conference (IASDR 2009) , 2009.[19] B. Lawson. The context of mind.
Designing in context , pp. 133–148,2001.[20] B. Lawson.
How Designers Think . Routledge, 2006.[21] B. R. Lawson. Cognitive Strategies in Architectural Design.
Er-gonomics , 22(1):59–68, 1979. doi: 10.1080/00140137908924589[22] X. Mao, O. Galil, Q. Parrish, and C. Sen. Evidence of cognitivechunking in freehand sketching during design ideation.
Design Studies ,67:1–26, Mar. 2020. doi: 10.1016/j.destud.2019.11.009[23] S. McKenna, D. Mazur, J. Agutter, and M. Meyer. Design ActivityFramework for Visualization Design.
IEEE Transactions on Visual- ization and Computer Graphics , 20(12):2191–2200, 2014. doi: 10.1109/TVCG.2014.2346331[24] M. Meyer, M. Sedlmair, P. S. Quinan, and T. Munzner. The nestedblocks and guidelines model.
Information Visualization , 14(3):234–249, 2015. doi: 10.1177/1473871613510429[25] T. Munzner. A nested model for visualization design and validation. In
IEEE Transactions on Visualization and Computer Graphics , vol. 15,pp. 921–928, 2009. doi: 10.1109/TVCG.2009.111[26] P. Parsons, C. M. Gray, A. Baigelenov, and I. Carr. Design judg-ment in data visualization practice. In
Proceedings of the 2020 IEEEVisualization Conference (VIS), short papers , 2020.[27] R. S. Perez and C. D. Emery. Designer thinking: How novices andexperts think about instructional design.
Performance ImprovementQuarterly , 8(3):80–95, 1995.[28] A. Purcell and J. S. Gero. Design and other types of fixation.
DesignStudies , 17(4):363–383, 1996. doi: 10.1016/S0142-694X(96)00023-3[29] D. A. Schn.
The Reflective Practitioner: How Professionals Think InAction . Basic Books, 1983.[30] M. Sedlmair, M. Meyer, and T. Munzner. Design Study Methodology:Reflections from the Trenches and the Stacks.
IEEE Transactions onVisualization and Computer Graphics , 18(12):2431–2440, 2012. doi:10.1109/TVCG.2012.213[31] H. A. Simon.
The Sciences of the Artificial . MIT Press, 1969.[32] E. Stolterman. The Nature of Design Practice and Implications forInteraction Design Research.
International Journal of Design , 2(1):55–65, 2008. doi: 10.1016/j.phymed.2007.09.005[33] C. Stones and T. Cassidy. Seeing and discovering: how do studentdesigners reinterpret sketches and digital marks during graphic designideation?
Design Studies , 31(5):439–460, Sept. 2010. doi: 10.1016/j.destud.2010.05.003[34] W. Visser. Two functions of analogical reasoning in design: a cognitive-psychology approach.
Design Studies , 17(4):417–434, 1996. doi: 10.1016/S0142-694X(96)00020-8[35] W. Visser. Design: one, but in different forms.