Surveying the Landscape of Ethics-Focused Design Methods
Shruthi Sai Chivukula, Ziqing Li, Anne C. Pivonka, Jingning Chen, Colin M. Gray
SSurveying the Landscape of Ethics-Focused Design Methods
SHRUTHI SAI CHIVUKULA,
Purdue University, USA
ZIQING LI,
Purdue University, USA
ANNE C. PIVONKA,
Purdue University, USA
JINGNING CHEN,
Purdue University, USA
COLIN M. GRAY,
Purdue University, USA
Over the past decade, HCI researchers, design researchers, and practitioners have increasingly addressed ethics-focused issues througha range of theoretical, methodological and pragmatic contributions to the field. While many forms of design knowledge have beenproposed and described, we focus explicitly on knowledge that has been codified as “methods,” which we define as any supports foreveryday work practices of designers. In this paper, we identify, analyze, and map a collection of 63 existing ethics-focused methodsintentionally designed for ethical impact. We present a content analysis, providing a descriptive record of how they operationalizeethics, their intended audience or context of use, their “core” or “script,” and the means by which these methods are formulated,articulated, and languaged. Building on these results, we provide an initial definition of ethics-focused methods identifying potentialopportunities for the development of future methods to support design practice and research.CCS Concepts: •
Human-centered computing → Interaction design process and methods ; •
Social and professional topics → Codes of ethics .Additional Key Words and Phrases: design methods, ethics, values, design practice
Draft: February 11, 2021
There is a growing interest in socially responsible design, evidenced by the efforts of practitioners and third-sectororganizations alike in building awareness and support for ethically-centered design practices [8, 17, 61, 67]. Theseefforts have frequently built upon academic discourses such as ethics [83, 130, 141], values [56, 61, 64], moral philosophy[29, 61], and critically-oriented models of participation [26, 97, 130]. While substantial efforts have been made to describethe value-centered or ethics-focused methods landscape from a scholarly and empirical perspective (e.g, value-sensitivedesign [61, 63], values at play [57], ethical standards or codes [78], policies [1, 73]), a specific landscape of ethics-focusedmethods that are intended to pragmatically support the actions of designers and technologists in their everyday workis less well defined.In prior work, design and HCI researchers have defined and engaged with methods as cognitive and pragmaticsupports [80, 91, 136], a means of encouraging creative production [95], an enabler of dialogue and communicationduring design activity [122], and a way of bridging multiple disciplinary ways of knowing to inform effective practice [89].However, design methods by themselves do not contain any action or inherently prescriptive or binding directives [81],but rather are tools which enable design activity through the knowledge they contain [79, 91], under the control of thedesigner who activates this knowledge in situated and pragmatic ways to support their design activity [79, 105, 136, 137].Thus, we seek to investigate not only the means of supporting design practices in a broad sense, but also seek to describethe relationship between knowledge bound up in methods to the activation of that knowledge to create the potentialfor socially responsible design practices. The investigation of knowledge in design methods points towards questions, a r X i v : . [ c s . H C ] F e b hivukula, et al. such as: How is a method structured? What are the constituent elements of a method? and What is the language usedto describe a method? In parallel, the analysis of methods through an ethics-focused lens reveal yet more questions,such as: How do methods enable designers to identify and act upon potential social impacts? Can methods guaranteeethical outcomes? and How are ethical concerns inscribed into the language of methods? We do not seek to answerthese questions in full, yet reveal this landscape of questions to demonstrate the potential broader impact of this work.In this paper, we identify, analyze, and describe a set of existing ethics-focused methods designed to support designresearch and practice for a range of audiences. Building on a content analysis of 63 collected methods, we describehow these methods operationalize ethics, are framed for particular audience(s), and are built to convey specific typesof knowledge and sensitizing concepts. Across this collection of methods, we have deconstructed the language andspecifications from the method source to identify the intended audience(s), format of guidance, interaction qualities,utilization of existing knowledge or concepts, implementation opportunities within design processes, the “core” or “script”of the method, and the ways in which the method builds upon or refers to existing ethical frameworks. These aspects ofethics-focused methods aid us in characterizing a current landscape of ethical support for practitioners, elucidatingopportunities for the adaptation of existing methods and the creation of future ethics-focused or value-centered methodsfor supporting design research and practice.The contribution of this paper is three-fold: 1) We identify existing ethics-focused or value-centered methods inorder to map the space of current ethical support for designers; 2) We deconstruct the language and specificationsof these methods to describe the framing used for the intended audience(s) and describe the means by which ethicsis operationalized, facilitating more detailed inquiry into how methods are constructed and how they might furthersupport ethical awareness and action; and 3) We identify opportunities, synergies, and gaps in ethics-focused methods,providing a roadmap for the creation and adaptation of methods that are resonant with the needs of practitioners. The notion of “design knowledge” has been extensively researched in the design and HCI literature, broadly definingwhat constitutes design knowledge (e.g., patterns of reflection [43], ontologies [145]), levels of instigation of designknowledge in design activity [76, 99], and different types of design knowledge [104, 105]. For the purpose of this paper,we explore “methods” as a particular form of design knowledge that enables “the creation of design states” [99] thatsupport and advance a designer’s capability [105]. We draw on the definition given by Stolterman and colleagues [136]of design methods as “tools, techniques, and approaches that support design activity in [a] way that is appreciatedby practicing interaction designers,” and Gray’s [81] definition that describes design methods as “tool[s] that allowdesigners to support thinking, reflecting and acting upon design activities.” Within this framing, we wish to furtherdescribe how methods-focused knowledge allows researchers to better understand design practices, including theidentification of areas where there is stronger and weaker support. Prior research on the use of methods by practitionershas shown evidence that methods are largely selected and used based on emergent aspects of the design context, wherepractitioners leverage knowledge enabled through the use of tools either for thinking or generating artifacts [136]. Inthis sense, methods are primarily activated through a “mindset” rather than a precisely defined way to conduct designactivity, and the performance of any given method or combination of methods is dependent on how a designer choosesto appropriate methods to support their design work [79]. urveying the Landscape of Ethics-Focused Design Methods We view methods as a form of design knowledge that does not function alone, but is rather activated through thedesigner’s activity and judgment, reflecting on the design knowledge contained within the methods [76, 81, 137]. Thisknowledge can be abstracted further to describe the repertoire of an individual designer, which includes both stores ofexisting design precedent [125] and larger assemblages of tool knowledge that Gray et al. [80] have previously referredto as a designer’s conceptual repertoire . Thus, in building and elaborating the inherent structures of existing designmethods, we are able to point towards a conceptual repertoire that is implicit in both design knowledge and use. Thisnotion of a conceptual repertoire also builds on previous work in the HCI and design communities that has interrogatedboth prescribed and performative accounts of design practices, including both the exploration and performance ofmethods by practitioners from Goodman and colleagues [75, 76] and Reeves [120], and the differentiation betweencodification and performance proposed by Gray [81]. In particular, we focus on the notion of method “cores” [79] todescribe an inscribed potential for design moves contained within methods, driven by a synergistic overlap of context,script (embedded instructions), and the lived experience of the designer.We also build upon prior work that has defined and curated a range of methods to support design activity, with suchwork having a stated goal of describing methods in ways that are simple enough for designers to adapt, apply andcombine different methods in various ways [44]. Löwgren and Stolterman [105] have built upon this notion of methodreuse, stating that methods should be accessible, flexible, and adaptable for designers to apply either independentlyor alongside the designer’s current “toolbox” in different contexts. Finally, Stolterman [135] has claimed that anyknowledge introduced into design practice should bear a “rationality resonance,” whereby the content of methodsshould resonate with the complexity of practice. Following this guidance and framing, there have been numerousattempts to create and curate design method collections, including: method for creativity and innovation to improve therange of design production [30, 113]; a collection of UX evaluation methods [143]; a classification of methods, includingtraditional, adapted, innovative methods, and methods for interpretation and analysis [89]; a popular collection of UXresearch and design methods for design students and practitioners titled
Universal Methods of Design [90]; a design kitfor Human-Centered Design practice by IDEO [6]; a collection of product design methods and approaches known asthe
Delft Design Guide [140]; an overview of strategies and methods for design innovation titled
Design. Think. Make.Break. Repeat [139]; and a set of generative approaches for design research [124]. While the list of curated collections ofmethods is already substantial, and still growing, we intend to build upon these collections with an explicit focus onethically-centered practice, building upon existing language to describe methods while also proposing new vocabularyto conceptualize, categorize, and propose links within and among methods.
HCI, Science and Technology Studies (STS), and design researchers have previously explored ethical practice acrossmultiple dimensions, including: theoretical accounts [64, 130], methodological descriptions [57, 61, 107], identification ofpragmatic and practice-led work [17, 40, 82, 83, 129, 134], and philosophical accounts [50, 142]. When focusing on priorresearch contributions relating to methodology, we have identified numerous frameworks that propose methodologicalmeans for designers to engage in value discovery and implementation. Common and well known methodologies includeValue Sensitive Design (VSD; [61, 63]) and Values at Play [57]. Other researchers have proposed strategies for designersor technologists to advocate for values in practice contexts, including organizationally-focused approaches such asShilton’s “Value Levers” [129] or van Wynsberghe’s “Ethicist as Designer” [141]. It is claimed that these strategies andmethodologies can “open new conversations about social values and encourage consensus around those values as designcriteria” [129]; identify new ways to expose and reflect upon designers’ responsibility or attitudes towards value-based hivukula, et al. decisions [84, 85]; propose suggestions for critical and reflective technical practice [21]; foreground tools for valuecomprehension in particular contexts [61]; provide practitioners with ethical codes for computing work [1]; framepolicies for ethical responsibility for organizations [73]; and offer requirements for ethics curriculum for computing andengineering education [77, 92]. This range of prior work illustrates the efforts of the HCI, STS, and design communitiestowards identifying opportunities for supporting ethically-focused work practices. However, as an additional point ofcomplication, portions of this prior work has been critiqued regarding its lack of resonance in authentic work settings,or due to the lack of translation of these practices from academia to practice [83, 107, 130]. In this paper, we focusour efforts on surveying the landscape of ethics literature through the framing of design methods, with the goal ofgathering and characterizing the existing landscape of ethics-focused and value-conscious design frameworks. To map the landscape of existing ethics-focused methods, we collected a total of 63 methods and conducted a contentanalysis [93, 118] to describe the knowledge contained in these methods. This content analysis included the characteri-zation of these methods on various levels and dimensions that will be detailed below as a part of our analysis approach.The research questions addressed through this paper are as follows:(1) What design methods have an ethical focus, and how is ethics operationalized in these methods?(2) Who are the intended audience(s) for these methods?(3) How are these methods formulated, articulated, and languaged?
Our approach in using content analysis [118] involved a process of constant reflexivity and researcher alignment,given the complexity of the method and the ill-defined nature of our topic of interest. All researchers involved in thisprocess have taken both design and qualitative research methods coursework, and/or were involved in previous researchprojects that used content analysis or similar qualitative or critical analysis methods. Additionally, all researchers hadprior experience engaging with conventional design methods through classroom projects and/or professional designwork. These experiences enabled our research team to collect and characterize these methods, as we collectively broughtknowledge of a broad spectrum of ethics-focused knowledge and design expertise. We reflexively engaged in openand axial coding as a key part of our content analysis process, employing strategies such as coder comments to trackand build consensus, note taking and memoing to create robust coding schemes at every stage, peer debriefing of eachothers’ codes to improve the rigor of the analysis process, and regular conversations with the research team to ensurealignment with generated coding schema at each stage.
Through a series of structured web searches to locate ethics-focused methods, we collected a list of 89 methods/ tools/approaches to begin our collection. The searches began by considering the Value Sensitive Design (VSD) methods[61], which enabled us to characterize the nature and purpose of ethics-focused methods, leading to our web searcheson Google, Google Scholar, and the ACM Digital Library. The following keywords were used for the search queries:“ethics focused methods,” “ethical tools in design,” “ethics methods,” “HCI ethics and values methods,” and other relatedcombinations of these terms. The methods we located were considered to be part of our initial collection if they hada clear ethical valence or had a stated intention to produce ethical or socially-responsible outputs. No specific year urveying the Landscape of Ethics-Focused Design Methods ranges were used as filters; nevertheless, we sought to identify as many methods fitting our criteria as possible withinboth traditional academic literature and from practitioner sources, given the lack of clear and consistent language tosearch using a more traditional “systematic review” approach. We more fully define our inclusion and exclusion criteriabelow. The precise scope of “ethical outputs” was not defined until the end of the analysis, as our goal was to identifymethods that broadly had social or human considerations during the design process that related to an ethical valence.We found methods published only between 2008 and 2020, but our search strategy may have missed methods publishedin earlier years, or different terms might have been used to describe such methods. All methods were collected in aspreadsheet with descriptors such as the title of the method, published year, author names, and source files (documentsor web links). The source files aided us in accessing the method’s description, which was our primary unit of analysis. For the purpose of our analysis, we sought to include any design method thatwas created with the intent of supporting value-centered, ethically-focused, or socially responsible decision makingpractices, as indicated by the method description. We recognized that some methods functioned as methodologies , andother methods contained multiple sub-methods; in these cases, we sought to identify the smallest method unit foranalysis to increase precision. Through our reflexive data collection and analysis process, we also identified severalexclusion criteria to narrow our focus. First, we excluded any methods that were computationally, algorithmic, andUI-focused. For example, by computational or algorithmic focused, we refer to toolkits that offer a Python packageto computationally test for biases, and algorithms that are intended to mitigate bias in datasets and models, such asthe AI 360 Fairness Kit[2] that was created for supporting software developer work. Second, we excluded UI-focusedpackages such as the IF Data Patterns Catalogue [4], which includes a set of interface choices suggested for handlinguser data. Third, we excluded codes of ethics [78, 147] and technology or legal policies [155] as past work has evaluatedthe role of these codes in professional practice [37, 100], and the focus of these tools is generally on professionalpractice and not specific to design decision making. Fourth, we excluded methods intended to improve accessibility (e.g.,recommendations for optimizing screen reading) and inclusivity (e.g., general broadening of participation in digitaltechnologies), since this is already a well-defined area of technology practice and scholarship. Finally, we excluded anyentries that were not clearly expressed as a design method; as an example of the latter type, Stark [133] proposed atranslational means of involving artists for “work to produce a sense of defamiliarization and critical distance fromcontemporary digital technologies in their audiences.” Using these inclusion and exclusion criteria, we identified a list of83 ethics-focused or value-centered methods, tools, approaches, conceptual vocabulary, methodologies, or frameworks.At this stage, we recognized that we had a heterogeneous collection of theoretical frameworks, concepts, methodologies,approaches, methods, which led to a further classification effort as described in the following section. We went througha reflexive process to define various potential classification approaches to define a final set of actionable methods.
Table 1. Classification of Collected Ethics-Focused or Value-Centred methods
Types of Framing ExamplesMethods
Detailed and described in Figure 1 and 2
Theoretical Commitments
Feminist HCI [27, 138], dark patterns [35, 86, 106, 108], Data Feminism [49]and others [15, 42, 48, 58, 84, 114, 119].
Conceptual Frames
Speculative Design [52], Critical Design [25], Reflective Design [126], andothers [51, 94, 101]. hivukula, et al. Building on our collection of 83 artifacts, we sorted them into three maincategories based on their potential function in design activity. As shown listed in Table 1, these functions include:prescriptive methods, theoretical commitments, and conceptual frames.
Methods:
These provide guidance on a practical level, indicating to the designer how they might apply, operationalize,or activate ethics and values in technology design work. For this paper, we identified a list of 63 ethics-focused orvalue-centered methods prescribed for design action, which will be referred to simply as “methods” throughout theremainder of the paper. We will further elaborate how these methods serve as the main contribution of this paper, andwe focus on this set to answer our research questions.
Theoretical Commitments:
These provide guidance to designers on a theoretical level by characterizing thedesigner’s ethical commitments (e.g., Data Feminism [49] and Ethical by Design: A Manifesto [114]), listing qualitiesrequired for building ethical outcomes (e.g., Feminist Interaction Design Qualities [27] and dark patterns [35]), describingexisting designs that are manipulative or value-centered (e.g., Asshole designer properties [84] and Nodder’s Seven Sins[119]), or suggesting organizational structural changes to include ethicists to incorporate ethical reflection into theproduct (e.g., Ethicist as Designer [141]). Theoretical commitments do not tell the designer precisely how to engage insome of these practices or point towards actionable ways of implementing the concepts; these commitments are not yetprocedural in form or defined for the designers in a way that directly activates their principles in concrete contexts, butrather suggests the required perspectives and language that might be considered when building a prescriptive method.For example, Bardzell’s Feminist HCI commitment [27] lists qualities that “characterizes feminist interaction” such aspluralism, participation, advocacy, ecology, embodiment, and self-disclosure; these qualities could be used to create oneor more methods for
Feminist Interaction Design by defining steps or other means by which designers could apply thesequalities in their design work. Other examples that fall under theoretical commitments include dark patterns strategies[17, 86, 106, 108], Design Justice [42], and In-Action Ethics [58].Between prescriptive methods and theoretical commitments lie methodologies which include a theoretical framingor umbrella of relevant and appropriate practices that can be applied in a design situation, often without the suggestionof specific tools and techniques. As one example of a methodology, Values at Play [57] suggests that the user “discover,analyze, and integrate values” specifically to game design; however, this methodology can be applied across any designsituation within these stages. Other examples that are included as methodologies in our collection include Value Levers[129], Research through Design Fiction [32], and VSD [61].
Conceptual Frames:
These provide guidance at an epistemological level, providing a more expansive set of proposedpractices and knowledge which point towards broader approaches to building knowledge. For instance, a Critical Designapproach [25] focuses on non-affirmative design practices, recognizing the knowledge that is built in the process ofcreating design artifacts. Other examples that fall under conceptual frames include Adversarial design [51], Agonisticdesign [31], Critical Design [25], Postcolonial computing [94], Reflective Design [126], Speculative design [52], and theQueering of HCI [101].
Using the method descriptions as our unit of analysis, we describe our data analysis procedures in three broad steps, asguided by Neuendorf’s content analysis approach [118]: 1) familiarizing ourselves with the data set; 2) creating andvalidating the coding scheme; and 3) performing open and axial coding. urveying the Landscape of Ethics-Focused Design Methods Table 2. Codebook of method characteristics.
Characteristic DescriptionPrimary Audience
Intended users of the method.
Discipline/Domain
Ex: Engineering, software development, design, etc.
Primary Medium
Tangible form in which the method has to be used or structured. Axialcodes include: worksheet, template, cards, document/ guidebook, physicalmanipulatives, videos, idea/ practice, and game . Type of Guidance
Ontological description and knowledge proposed as a part of the method’sdescription. Axial codes include: steps, guidelines, framework, lens/ perspec-tive, reflective questions, examples, heuristics, epistemology, methodology, andcase study.
Input
Elements the method operates on, which is inputted by the user(s) ofthe method. Axial codes include: user information, design artifacts/ ser-vices, users/stakeholders, values, framing constraints, problem frame, scenar-ios/context, and research material.
Mechanics
Actions expected from the users while using this method. Axial codesinclude: altering, storytelling, filtering, creating, mapping, and evaluating.
Output
Tangible results after using this method. Axial codes include: values, con-cepts, research outcomes, evaluation, users/stakeholders, opportunities, proce-dural information, and research outcomes.
Outcome Expected
Expectations from the user(s) and the ways in which the output might bemanipulated by the user(s).
Established Method(s) used
Established design methods/ methodologies that are referenced or used aspart of using or building the method.
Sensitizing Concepts
Established theoretical concepts that are used in this method and the theorythat has given the method’s vocabulary.
Context of Use
Environmental or logistical aspects of using the method. Axial codes include:group types ( team, individual ) and ecology types ( industry,instructionalcontext ). Design Process Steps
Design phase in which the method is prescribed to be used or can be used.Axial codes include: a priori phases from Univeral Methods of Design [90].
Ethical Framework(s)
Ethics theor(ies) inscribed into the method as a part of the “outcome ex-pected.” Axial codes include: deontological, consequentialist, virtue, pragma-tist, and care ethics . Attributes
Characteristics of the method mentioned in the description.
Core
The central mechanic of using this method that remains relatively stableduring adaptation and use. Axial codes include: posture types ( elicitingvalues, critically engaging, defamiliarizing ) and actions ( consensus building,evaluating, framing, generating ). hivukula, et al. We began our analysis process by performing a close reading of several methodsto familiarize ourselves with the organization of knowledge and language used to characterize each method. Tworesearchers individually identified preliminary codes for nine methods (including a diversity of topics, audience, andgoals), including potential descriptors that aided our team in characterizing the content of the methods, pointing towardsan initial coding scheme. The focus of this content analysis was to describe the method and analyze its characteristicsbased on a clear reading of its text, and not the instigation, creation or evaluation of the method in the context ofpractice.As a part of the initial coding scheme, the researchers listed candidate descriptors that ranged from the form of themethod, its potential application in design processes, expected outcomes, intended audience, attributes, and means ofinteraction with the method. After multiple rounds of iteration and discussion among the research team, a preliminarycodebook of descriptors was created as detailed in Table 2. The more robust axial codes underneath the broaderdescriptors were determined later in the analysis process. During this stage, we also began to identify open codes[34, 123] to describe potential methods “cores,” alongside researcher-inferred assumptions about the potential primaryaudiences that may use the method. By “core” of the method, we refer to the central mechanic of using this method thatremains relatively stable during adaptation and use [79, 81].
During the second stage of analysis, we focused on validating theoverarching descriptors from the initial coding scheme by revisiting the same nine methods coded in the previousround. We used a linked set of spreadsheets to conduct and document the content analysis of all the collected methods.This approach (facilitated by the tool, AirTable ) aided us in clearly building the audit trail of our coding process andrelating these codes to previous coding work, increasing the validity and robustness of our codebook. At this stage, webegan with an iterative process of open and axial coding under each of the main descriptors as described in our finalstage of analysis. For each descriptor, we reflexively moved through stages of open coding, identification of preliminarydefinitions in a codebook, and extended conversation among members of the research team. Through deliberation overmultiple weeks and rounds of coding and revisions to the codebook, we identified a final codebook for each descriptorset. All codebook elements, and the use of these elements in the coding process, were evaluated by pair coding and allapplication of descriptors was discussed until full agreement was reached. With the high-level descriptors (Table 2) finalized and the researchers alignedin their understanding, we conducted open coding of the full set of methods using the codebook. Once this initial codingwas completed, we used these open codes to identify axial codes within each descriptor, using the process describedabove. The final round of analysis included summative, top-down coding using the final descriptors and sets of axialcodes, including the type of guidance, primary medium, input, mechanics, output, core; all axial codes are listed inthe description column of Table 2 in italics. The role of axial coding varied for each descriptor, and is detailed in thefindings section below.
In this section, we report on the findings of our content analysis, divided by research question. The three mainsubsections include: 1) The method’s operationalization of ethics, where we describe the core of the methods and ethicalframeworks that are activated; 2) The intended audience for these methods; and 3) The formulation, articulation, and https://airtable.com 8 urveying the Landscape of Ethics-Focused Design Methods conceptual language used to describe these ethics-focused methods. A summary of the method descriptors is providedin Figures 1 and 2. In this subsection, we identify how ethics or values were operationalized in these methods. We describe this op-erationalization through two properties: 1) the core of the method and 2) the ethical framework(s) activated in themethod.
By method core , we refer to what we inferred as the central conceit or essence of the mechanicsof the method. Each method’s core was identified from two groups: 1)
Postures : eliciting values (n=32), criticallyengaging (n=10), defamiliarising (n=21), and 2)
Actions : consensus building (n=2), evaluating (n=26), framing (n=20)and generating (n=15). We propose “postures” to be very specific to ethics-focused or value-centered methods, whereasthe “actions” apply across any design method. Cores involving postures target attitudes towards a certain action, eitherto identify an existing or generated list of values as a conceptual frame (eliciting values) , engage in critical perspectivesor theories drawn from critical theory as an epistemological argument (critically engaging) , or take part in alternativeforms of looking at existing concepts or forms of thinking (defamiliarising) . Cores involving actions encourage users toalign their decision making with other stakeholders (consensus building) , assess and validate the decision (evaluating) ,map the design space for using the method (framing) , and produce design artifacts (generating) . For example, the
Inclusive Design Toolkit [10] had a core of “eliciting values,” since it proposes to evaluate a design artifact using values ofinclusivity or accessibility. In
Judgment Call the Game [24], the method core focuses on “defamiliarizing,” through whichdesigners can “evaluate” a design scenario through reviews and ratings from the perspective of alternative users in thesituation. Another example is
Security Fictions [110], which asks users of the method to “defamiliarize” themselves tothink differently about security issues as they “generate” concepts to solve security threats. As these examples illustrate,the two groups of cores represent how ethics is operationalized based on the postures leading to those actions.
We identified how each method related to established ethical frameworks as a wayto illustrate and provide us a vocabulary for how ethics were operationalized; drawing from multiple key texts in thephilosophy literature, including Becker [28], Kant [96], Aristotle [23], and Gert [66]. We used the following frameworksto categorize the methods: deontological ethics (n=11), consequentialist ethics (n=39), virtue ethics (n=8), pragmatistethics (n=40), and care ethics (n= 1). Table 3 summarizes the ethical framework(s) and associated methods. Methodsthat used a deontological ethical framework manifest certain values through the method as a “duty” of the user toimplement through their decision making. For example,
Ethical Contract [68] requested that the users divide the ethicalresponsibilities in the project planning stage and physically sign the document (provided by the method) as a means offoregrounding their duty in accepting ethical responsibility. Methods that used a consequentialist ethical frameworkfocus primarily on evaluating or considering consequences of the decision made. For example,
Re-shape [127] requestedthat students reflect on their actions towards physical movement data collection and its impact on the communitythrough which they were navigating.
Re-shape was also the only method that relied upon a care ethics framework toforeground a reciprocity of care towards the community through data. Methods using a virtue ethics framework focusedits impact on the designer themself, seeking to build their ethical awareness. For example,
HuValue [98] focused onbuilding students’ sensitivity towards human values by engaging with its knowledge in their design activities. Methodsusing a pragmatist ethical framework considered the designer’s judgment, situationality, and context of decision makingas a landscape through which to consider ethical complexity. For example,
Moral Value Map [71] asked the designer hivukula, et al. Method Name Medium Context Design Phase Core Input Mechanics Output T e a m I n d i v i d u a l I n d u s t r y I n s t r u c t i o n a l C r i t i c a ll y E n g a g i n g E l i c i t i n g V a l u e s D e f a m i l i a r i s i n g E v a l u a t i n g F r a m i n g G e n e r a t i n g C o n s e n s u s B u i l d i n g
360 Review a Users/ Stakeholders
Mapping
Opportunities
Adversary Personas
User Information
Storytelling
User Information
Blackmirror Brainstorming
Scenarios/Context
Storytelling,Creating
Evaluation Results
Co-evolve Technology... c Problem Frame
Creating
Opportunities
Data Ethics Canvas
Scenarios/Context
Altering,Evaluating,Mapping
Opportunities,Procedural Information
De-scription b Design Artifact
Altering,Mapping
Values
Design Fiction Memos
Scenarios/Context,Data
Storytelling
Values,Opportunities
Design for Social Acc...
Problem Frame
Creating,Mapping,Evaluating
Concepts
Design with Intent
Constraints
Creating,Evaluating,Filtering
Concepts
Dichotomy Mapping a Design Artifact
Evaluating
Evaluation Results,Opportunities
Diverse Voices c Research Material
Storytelling
Evaluation Results
Eliciting Values Refl...
Scenarios/Context
Storytelling,Creating,Evaluating
Values
Envisioning Cards c Design Artifact
Storytelling,Filtering
Concepts,Values
Ethical Contract b Design Artifact
Altering,Storytelling
Values
Ethical Disclaimer b Scenarios/Context
Altering,Mapping
Opportunities,Values
Ethicography
Research Material
Mapping
Concepts
Ethics Canvas
Design Artifact
Altering,Storytelling,Mapping
Users/Stakeholders,Values,Procedural Information
Ethnographically Inform... c Constraints
Creating
Research Outcomes
GenderMag
Design Artifact
Storytelling,Filtering,Evaluating
Evaluation Results,User Information,Opportunities
Hippocratic Oath a Values
Creating,Storytelling
Values,Procedural Information
HuValue
Problem Frame
Filtering,Mapping,Storytelling,Evaluating
Concepts,Evaluation Results,Values
Idea Generation...
User Information
Mapping,Storytelling,Creating
Concepts
Inclusive Design Toolkit
Design Artifact
Evaluating,Mapping
Opportunities,Evaluation Results
Inverted Behavior Model a Design Artifact
Evaluating
Opportunities,Evaluation Results
Judgment Call the Game
Design Artifact
Storytelling,Altering,Filtering
Users/Stakeholders,User Information,Evaluation
Layers of Effect a Design Artifact
Evaluating,Mapping
Opportunities
Make It Critical
Scenarios/Context
Filtering,Mapping,Creating
Concepts
Making an Ethical Decision
Users/Stakeholders
Filtering,Mapping,Evaluating
Evaluation Results
Maslow Mirrored a Design Artifact,Users/Stakeholders
Evaluating
Evaluation Results,Opportunities
Metaphor Cards c Scenarios/Context
Storytelling,Creating
Concepts,Opportunities
Microsoft Inclusive Design...
Constraints
Creating
Concepts,Opportunities
Medium Legend:
Cards WorksheetDocument/Guidebook Physical ManipulativeTemplate Video
Design Phase Legend: Planning, Scoping, & Definition Evaluation, Refinement, & Production Exploration, Synthesis, & Design Implications Launch & Monitor Concept Generation & Early Prototype Generation a Part of the Design Ethically Toolkit. b Part of the Ethics for Designers Toolkit. c Part of the Value Sensitive Design Toolkit.
Fig. 1. Methods (part 1) classified and organized by medium, context, design phase, core, input, mechanic(s), and output(s). urveying the Landscape of Ethics-Focused Design Methods Method Name Medium Context Design Phase Core Input Mechanics Output T e a m I n d i v i d u a l I n d u s t r y I n s t r u c t i o n a l C r i t i c a ll y E n g a g i n g E l i c i t i n g V a l u e s D e f a m i l i a r i s i n g E v a l u a t i n g F r a m i n g G e n e r a t i n g C o n s e n s u s B u i l d i n g Model for Informed ... c [...] [...] Values
Monitoring Checklist a Design Artifact
Evaluating
Evaluation Results,Opportunities
Moral Agent b Constraints
Filtering,Creating,Storytelling
Concepts
Moral and Legal Decks
Design Artifact
Evaluating,Filtering
Evaluation Results,Values
Moral Value Map b Design Artifact
Filtering,Altering
Values
Motivation Matrix a User Information
Storytelling
User Information,Opportunities
Multi-lifespan Co-design c Problem Frame
Creating
Research Outcomes
Mutli-lifespan Timeline c Design Artifact
Mapping
Opportunities
Normative Design Sche... b Problem Frame
Altering,Evaluating
Concepts,Opportunities
Re-Shape
Data
Mapping,Evaluating
Evaluation Results
Scalable Assessments... c Design Artifact
Storytelling,Evaluating
Values,Evaluation Results
Scenario Co-Creation
Scenarios/Context
Storytelling,Filtering
Opportunities,User Information
Security Cards c Design Artifact
Filtering,Mapping
Opportunities
Security Fictions
Problem Frame
Storytelling
Concepts
Speculative Enactments
Scenarios/Context
Storytelling
Research Outcomes
Spotify Design Investig...
Design Artifact
Altering,Evaluating
Opportunities
Stakeholder Analysis c Users/Stakeholders
Creating,Mapping
Users/ Stakeholders
Stakeholder Tokens c Constraints
Storytelling,Mapping
Users/ Stakeholders,Values
Tarot Cards of Tech
Design Artifact
Filtering,Evaluating
Opportunities
The Ethical Design Score...
Design Artifact
Altering
Evaluation Results
The Ethics and Inclusion...
Design Artifact
Filtering,Evaluating
Procedural Information
The Oracle for Transfem... [...]
Creating,Filtering
Concepts
Timelines
Design Artifact
Storytelling,Filtering
Values,Opportunities
Value Dams and Flows c – Design Artifact
Evaluating
Values,Evaluation Results
Value Source Analysis c Problem Frame
Mapping
Values
Value-oriented Coding c Research Material
Mapping
Research Outcomes,Values
Value-oriented Mock-up... c Constraints
Creating
Concepts,Values
Value-oriented Semi-... c Problem Frame
Storytelling
Research Outcomes
White Hat Design Patterns
Constraints
Evaluating,Creating
Concepts
Fig. 2. Methods (part 2) classified and organized by medium, context, design phase, core, input, mechanic(s), and output(s). to choose relevant human values relevant to “your design” and combine these values with the “context of use” at thedesigner’s discretion, encouraging flexible use of the method. hivukula, et al. Table 3. Ethical frameworks by method
Ethical Theory Methods Using this Theory. . . Methods Relying upon thisFramework
Deontological Ethics . . . foreground particular values through themethod as a “duty” of the practitioner to im-plement or address in their decision making. [10–12, 60, 67, 68, 70, 88, 98, 103,131, 132]Consequential Ethics . . . focus on evaluating or considering conse-quences of design decisions. [3, 10, 14, 16, 18–20, 24, 36, 47,53, 55, 60, 69, 71, 88, 102, 109–111, 115, 117, 121, 127, 144, 148–150, 153, 154, 156–158, 160–163]Virtue Ethics . . . address the designer themself, providingguidance to increase their ethical awareness. [19, 24, 33, 54, 59, 98, 128, 159]Pragmatist Ethics . . . consider the designer’s judgment, situation-ality, and the context of decision making as itencourages the designer to decide which ethicallens best addresses the situation at hand. [3, 11, 13, 14, 16, 20, 22, 33, 36,39, 45, 54, 55, 59, 62, 65, 68–71,88, 102, 103, 110–112, 115, 116,127, 144, 146, 150, 151, 154, 160,162, 163]Care Ethics . . . foreground a reciprocity of care towards oth-ers or a community. [127]
In this subsection, we describe the implied audience and context for these methods across the following descriptors:1) the primary intended audience of the methods; 2) the defined context of use of these methods; and 3) the publishedformat as a means of disseminating the methods.
Primary audience.
We relied upon the method developer’s identification of the intended audience from the methoddescription to infer the primary audience of the method. The stated audience types included: educators (n=3), academicresearchers (n=15), students (n=6), industry practitioners (n=54), or anyone (n=1). Of these methods, 13 did not explicitlystate the primary audience (leaving the researchers to infer the audience), while the remaining methods mentioned theirintended audience in the method description. For methods that intended to encourage conversations and collaborationsamong industry practitioners, sub-audiences primarily targeted design professionals (n=28), technology professionals(n=17), industry researchers (n=12), policy makers (n=2), and managers (n=1). The majority of methods had an intendedaudience from only one category rather a combination of multiple stakeholders.
Context of use.
We describe the context in which the method is intended to be used within two categories. First, wesought to identify whether a team and/or individual was the ideal group size for the method to be used. Methods thatappeared to be designed for a group as the intended audience were the most common (n=35), while individuals were aminority (n=4); methods identified for use by either a group or an individual were coded under both categories (n=23).Second, we sought to describe whether industry and/or instructional settings were the primary ecological setting formethods to be used. Methods were coded to represent the anticipated ecological setting, including industry work forpractitioners (n=45), an instructional setting for students and educators (n=3), or research in an academic context (n=4).A minority of methods (n=9) anticipated use both in industry and academia.
Published format.
We identified the published format for each method to identify its dissemination strategy oravailability, thus revealing assumptions regarding the intended audience or the type of knowledge building the method urveying the Landscape of Ethics-Focused Design Methods represented. The publication formats we identified included academic papers (n=28), websites (n=20), blog posts (n=16),books (n=4), and unpublished (n=1; [132]). It is interesting to note that the majority of the methods we analyzed werepublished as academic papers, while the primary intended audience of these methods were industry practitioners. Thelimited availability of these papers behind a paywall perhaps brings into question the accessibility of these methods forthe intended audience. In this subsection, we describe and characterize the collection of methods in three ways: 1)
Formulation of the methodsas scripted to define input-mechanics-output, existing frameworks/structure and design process implementation ofthese methods; 2)
Articulation of these methods to the audience to describe type of guidance and medium of thesemethods; and 3)
Language that formed core of these methods. This section answers our research question
We coded the “script” of these methods which illustrate the structure and interactionwith these methods. We describe the formulation of the methods through major categories: 1) input required, mechanics of interaction and output generated from the method; 2) existing frameworks or methods used to build the method; and3) practical implementation of the method in a design process . We present different axial codes, definitions and examplesin the paragraphs below.
Input–Mechanics–Output.
The input–mechanics–output sequence describes how the methods are formulated,pointing towards potential patterns of performance. We identified ten salient inputs , six action-oriented mechanics ,and eight tangible outputs across the collection of methods. We will describe each element of the sequence separatelybelow using a variety of methods as supporting examples. We have observed that despite a similar input, the change inmechanic has the potential to result in different outputs, giving us an opportunity to explore the interactions amongthese three elements. We provide more details regarding the patterns of interaction in the discussion section.
Input.
We coded the required materials or knowledge the method developer wants the users to input as a meansto proceed with the method. The identified inputs include design artifact/service/business (n=23), research material(n=3), problem frame (n=8), constraints (n=9), users/stakeholders (n=4), user information (n=3), scenarios/context(n=10), values (n=1), data (n=2), and one method without any required input. Each method may include a wide range ofinputs, but we chose to exclusively code only the most salient input required for each method to provide a more preciseset of entry points. Methods with the input of design artifact/service/business require the user to select an existingdesign material, product, or business service. For example, Multi Lifespan Timeline [153] requires the user to provide atechnological design artifact as an input in order to map how this artifact would exist in a social context at differenttimelines beyond the product lifecycle. Methods with research materials as an input require users to bring materialsconstructed for research purposes, such as interview protocols, co-design materials, tech policy documents, and others.For example, Scenario Co-Creation Cards [22] require an interview protocol along with the cards for conductingvalue-eliciting interviews with a culturally-diverse population. Other examples in this category include the ValueSensitive Action-Reflection Model [154], which requires the input of co-design materials, and Diverse Voices [7], whichrequires a tech policy document for expert panel discussions. Some methods required users to input their problemframe in order to construct, define, and approach a design space through the lens of that method. We differentiatethis input from constraints based on their level of definition; whereas constraints are expected to be precise, problemframes are frequently more open-ended.
Constraints include stakeholder requirements, a project brief, time constraints,limited resources, or other explicit requirements that must be met. Methods that require users/stakeholders or user hivukula, et al. information as inputs encourage the description of stakeholder needs in relation to method-guided decision making. Users/stakeholders indicates who the design is created or evaluated on behalf of, or who needs to be considered indecision making, whereas user information describes user needs, user actions, and user values (e.g., a persona or userstory). Methods with scenarios/context as an input require a design situation or product scenario that the team hasencountered, or a fictional situation that the user envisions. For example, Data Ethics Canvas [3] requires the user toformulate a scenario for which they need to plan data collection, storage, or opportunities for analysis. Methods with values as an input require the user to formulate a list of personal, social, team, company, and/or project values to frametheir decision making. Methods with data as input, for example
Re-Shape [128], require the users to provide data toteach data ethics and use the resulting data as a starting point for analysis and reflection. In a rare example,
The Oraclefor Transfeminist Technologies [14] did not require any input to use the method, because the first step of the using themethod involves filtering cards to create a problem space to generate futuristic concepts.
Mechanics.
We coded the action(s) expected from the users while using the method as its mechanic . The mechanics weidentified include: altering (n=10), creating (n=19), mapping (n=21), storytelling (n=23), filtering (n=17), and evaluating(n=23). Depending on the type of guidance provided by the method, there could be more than one type of mechanicfor each method, hence these were non-exclusively coded. Methods with altering as a mechanic expect users to edit agiven worksheet as they follow the prescribed steps/guidelines in the method. For example, the Ethics Canvas [121]provides a template for the users to collaboratively edit, move and add Post-It notes in appropriate sections to fill outthe template. Methods using creating as a mechanic encouraged a more conceptual and divergent approach wherebyusers produce artifacts through brainstorming, sketching, prototyping, and developing as they interact with the method,instead of providing existing artifacts for users to alter. Methods using mapping expect users to draw connections orassociations between method elements and artifacts created through the method. For example, Ethicography [39] is themethod of value discovery that requires researchers to physically draw links that visualize the conversation changeand growth through a design discussion; Value Source Analysis [33] requires users to identify disagreements amongstakeholders in order to conceptually map values for “other environments.” Methods engaging in storytelling involve anact of role-playing, narrating stories, performing activities, or playing games as the users interact with the method. Forexample, in Judgement Call the Game [24], users role-play as stakeholders and write a fictional review framed up bya combination of the rating card, stakeholder card, and ethical principle card that the player draws. Methods whichrequire filtering expect users to select scenarios, stakeholders, or draw cards, selecting salient options from a list ofpossibilities either provided by the method or created through the method. Methods with mechanics of evaluating request users to assess the components provided by the method or artifacts produced through the method. For example,the Moral and Legal IT Deck [18] provides a wide range of critical questions, legal principles, and ethical principles fordesigners to follow and thereby evaluate ethically-related risks of the proposed new technology.
Output.
We coded the methods based on the tangible outcome that would be produced when using the method asthe output . The identified outputs include concepts (n=17), opportunities (n=23), evaluation results (n=16), values (n=20),users/stakeholders (n=4), user information (n=6), procedural information (n=4), and research outcomes (n=6). Themajority of the outputs align with the list of inputs, although there is a clear change in their function. Depending on themethod, it is possible that there is more than one possible or likely output for each methods, hence we non-exclusivelycoded for this descriptor. Methods with concepts as an output result in new ideas, sketches, artifacts, and/or inspirationfor future work. Methods resulting in opportunities aid the user in locating ethical risks, recognizing future designpossibilities through the method. For example, Ethical Disclaimer [69] is meant to allow the users to “discuss for whichof the unethical situations you will take responsibility.” Methods with evaluation results as outputs allow the user to urveying the Landscape of Ethics-Focused Design Methods assess their design through quantitative scores, ethical scores, design requirements, reflections, or other evaluationmetrics. For instance, to illustrate the range of evaluation results, Ethical Design Scorecards [55] provide an “ethicalscore” for the users to indicate the potential ethical valence of their design decision, and Re-Shape [128] allows computerscience students to evaluate their own decision making in the form of a reflection of their responsibilities towards data.Methods with values as an output define a new mindset or outlook on what elements or abstract principles are mostimportant in the users’ design process. Methods with outputs such as users/stakeholders and user information have asimilar definition as when they are used as inputs; however, as outputs the information about users is realized andproduced through the methods. An output of procedural information encompasses relevant and possible next steps anda future plan of action for decision making. Finally, methods with research outcomes include artifacts produced usingthe method that are possible sources of future research or analysis, such as design research artifacts (e.g., user stories inspeculative enactments [53]) or co-design materials (e.g., in the Value sensitive action-reflection model [154]). Patterns of Input->Mechanics->Output.
Based on our analysis, we have recorded the number of occurrences ofeach possible combination of input, mechanics, and output. As part of this approach, we mapped the interactions orpatterns from “input” of the method as it was exclusively coded and present the most salient and frequently occurringinteraction patterns. These interaction patterns aid us in describing the most typical ways in which the methods functionas specification, and these patterns also elucidate possible opportunities for new or altered methods beyond theseexisting interaction patterns. We identified six common interaction patterns and provide their descriptions as follows: • Design Artifacts->Evaluating->Values/Evaluation Results : This interaction pattern was found in methods that guideusers to evaluate existing design artifacts , resulting in a range of evaluation results . Such methods are intendedto address existing product deficiencies, reveal ethical dilemmas of the system, and discover new values to beembedded in the design. Methods using this pattern include: Value dams and flows [112], Scalable assessments ofinformation dimensions [115], Moral and Legal Decks [18], Inclusive Design Toolkit [10], and GenderMag [36]. • Design Artifacts->Mapping->Opportunities : This interaction pattern was found frequently in methods that helpusers recombine, envision, and derive new design opportunities from existing artifacts by mapping out methodelements or design space. Methods using this pattern include: Security Cards [47], Mutli-lifespan timeline [153],and the Inclusive Design Toolkit [10]. • Design Artifacts->Storytelling->Values : This interaction pattern is identified in methods which expect the userto interact with their design artifacts through storytelling or by playing games in order to explore, elicit, andengage with values in designed artifacts. Methods using this pattern include: Scalable assessments of informationdimensions [115], Ethics Canvas [121], Ethical Contract [68], and Envisioning cards [60]. • Constraints->Creating->Concepts : This interaction pattern is found in methods with design constraints such as adesign prompt, business timeline, or resource constraints, which results in creating original or iterated concepts .Methods using this pattern include: White Hat UX Patterns [54], Value-oriented mock-up, prototype, or fielddeployment [154], Value Sensitive Action-Reflection Model [154], Moral Agent [70], and Design with Intent[102]. • Problem Frame->Creating->Concepts : This interaction pattern is used in methods that assist participants in generating concepts within a given or defined problem frame. Methods using this pattern include: Design forSocial Accessibility Method Cards [131]. hivukula, et al. • Scenario/Context->Creating->Concepts : This interaction pattern is seen in methods which results in design conceptscreated within a defined, assumed or fictional scenario/context. Methods using this pattern include: Value Sketch[146], Metaphor Cards [103], and Make It Critical [132].
Existing frameworks/methods used.
We sought to identify any existing design methods or frameworks thatmethods were built on, relied upon, or referenced. These frameworks are not translated into the method for the user,but rather they require the user to directly interact with these frameworks in order to successfully implement theethics-focused method in their work. We identified two kinds of existing frameworks that were used: 1) establisheddesign methods; and 2) other standalone methods. Not all methods used existing frameworks, with only 25 out ofthe 63 methods representing this behavior.
Established design methods that were referenced (as listed in the UniversalMethods of Design [90]) included: personas [36, 111], cognitive walkthrough [88], stakeholder map [152], scenarios[22, 117, 149], experience mapping [132], cultural probes [154], qualitative research interviews (used as requiredtechnique in [22, 45, 115]), and ethnography (used as a basic methodology for Ethnographically Informed [116]). Thesemethods build upon the familiarity of existing methods or approaches, facilitating the use of these ethics-focusedmethods with little prior preparation and expert knowledge.
Other standalone methods include less common methods,and frequently ethics-focused methods, with their own mechanics that are used in one of the methods we analyzed.Examples include: Ethics Canvas (another ethics focused method [121], which was used to build Data Ethics Canvas[3]); Design Heuristics (a card deck [5] which was used in the cognitive walkthrough approach in the Idea Generationthrough Empathy method [88]); Ethical Disclaimer (another ethics-focused method which was used as an input in EthicalContract [68]); Linkography (used as a baseline framework [74] in Ethicography [39]); and a combination of ValueScenarios [117], Envisioning Cards [60], and Value Sketch [146] (used to build and follow steps in the Value-SensitiveAction-Reflection Model [154]). These methods require user(s)’ existing knowledge in completing the method, includingknowledge about the functioning of connecting methods or other related methods for using the ethics-focused method.
Design Process Implementation.
We coded each method’s suggested use or implementation across existingnotions of design process stages (Table 4). We used an a priori list of five design phases as suggested in UniversalMethods of Design [90] as an existing acknowledged mapping of design methods and process. We have chosen thesefive phases as they allow our analysis to build upon connections to implementation of the ethics-focused methods andthe established mappings identified in [90]. Methods identified within
Phase 1 included activities as planning, scoping,and definition, “where project parameters are explored and defined” (n=9). Methods in
Phase 2 included activitiessuch as “exploration, synthesis, and design implications which are characterized by immersive research and designethnography leading to design implications” (n=40). Methods in
Phase 3 included activities as “concept generation andearly prototype iteration, often involving generative and participatory design activities” (n=27). Methods in
Phase 4 included activities as “evaluation, refinement, and production based on iterative testing and feedback” (n=34). Finally,methods in
Phase 5 included activities as “launch[ing] and monitor[ing] the quality assurance testing of design toensure readiness for market and public use, and ongoing review and analysis” (n=7).Based on our analysis, we found the majority of the methods were designed for Phases 2, 3, and 4, with only rareexamples in Phases 1 and 5. For methods coded as Phase 2, the focus was primarily on identifying design implicationswhich aided the user in framing the problem space in more ethical ways; in contrast, methods coded as Phase 4encouraged the user to build upon a generated design concept in more ethically-centered ways. As examples of Phase2-focused methods, card decks were used in Adversary Personas [111] to list potential adversaries in a particular design urveying the Landscape of Ethics-Focused Design Methods Table 4. Methods and Design Process Implementation
Design Process Phase [90] MethodsPhase 1 (planning, scoping, and definition) [3, 68, 69, 112, 141, 148, 156, 157, 159, 163]
Phase 2 (exploration, synthesis, and design implica-tions) [3, 7, 13, 18, 22, 39, 45, 47, 53, 59, 60, 65, 67, 70, 72, 98,102, 103, 109–112, 115–117, 127, 132, 144, 148, 150, 152–154, 156, 157, 159, 160, 163]
Phase 3 (concept generation and early prototype itera-tion) [3, 13, 14, 33, 36, 47, 54, 55, 60, 70, 88, 98, 102, 103, 109,110, 131, 132, 146, 148, 149, 153, 154, 156–158]
Phase 4 (evaluation, refinement, and production) [10, 11, 13, 16, 18–20, 24, 33, 36, 45, 53–55, 71, 72, 102,110, 112, 115, 116, 121, 127, 144, 148, 149, 153, 156–158,160–162]
Phase 5 (launching and monitoring) [36, 53, 127, 148, 156, 157, 162]situation, while Envisioning Cards [60] were used to expand potential issues in the “ immediate context of use, ” with thegoal of envisioning the potential long-term impact of technology. The actions supported through these methods arelikely to occur before concept generation, with the goal of framing the problem space by providing new ways of viewingthe context. In Phase 3 and Phase 4, methods enable the production of ethically-focused designs and the evaluation ofcreated or existing designs, respectively. For example, the Design for Social Accessibility Method Cards [131] provideusers with “concrete and real-life scenarios” in Phase 3 to “to generate accessible designs and appropriately engage deafand hard-of-hearing users to incorporate social considerations.” In Phase 4, the majority of methods provided waysfor the user to evaluate their decisions or design outcomes, using guidance to refine their decisions. For example, theEthics and Inclusion Framework [19] aids the user in calculating the “degree of inclusion of your product or service” orfacilitates “assessment of potential negative outcomes” for intended or unintended stakeholders.
In this section, we describe the methods based on the way they are articulated to their respectiveaudiences. We describe the ways methods are communicated to these audience(s) through two properties: 1) the type ofguidance; and 2) the medium through which the method is communicated.
Type of guidance
We coded the descriptors that communicate the scaffolding or means of support to engage withthe method in a way that is accessible to users as type of guidance and the tangible form in which the guidance wasprovided as medium . The type of guidance frames how the method is structured and conveyed to the user as instructionalsupport or scaffolding [46] in the following ways: steps (n=30), guidelines (n=23), framework (n=19), lens/perspectives(n=14), reflective questions (n=10), examples (n=22), heuristics (n=4), and case study (n=9). The method descriptionsprovided in the cited material frequently consisted of more than one kind of guidance, given the structure of the method,resulting in non-exclusive coding. If the method had multiple components or sub-components, we coded for all kindsof guidance provided, including any sub-structures of the method.
Steps are prescribed instructions to be followedby the user in order to interacting with the method in the provided order, whereas guidelines do not insist on beingfollowed in a specified order. For example, Diverse Voices [7] provides “main steps” to be followed by the user startingwith “Select a tech policy document” and additional guidelines under each step to describe how and what kinds of techpolicies documents to be selected. A framework is a defined structure provided by the method developer in the form ofa table, illustration, or schema.
Lens/Perspectives are possible attitudes or perspectives provided to focus the thoughtprocesses of the method user, while heuristics are techniques that can be implemented non-deterministically in order to hivukula, et al. guide the user of the method. For example, White Hat UX Patterns [54] lists a set of heuristics “to ensure ethical design” outcomes, guided by heuristics such as: “Use data to improve the human experience,’ ’ and “Advertising without tracking.”Reflective questions are posed as questions for the user to critically think through the “input” as intended by the “core”of the method. Examples are real world scenarios and/or visually illustrated guidance provided along with other typesof guidance, while case studies represent a real world design context through which the method is described rather thana standalone description of the method. Many of the methods proposed as part of the VSD methodology [61] includedcase study-focused guidance that was represented as bound within a specific design decision, taking on characteristicsof a case study . Medium.
We coded the descriptors that describe the tangible form of the methods as its primary medium , whichaided in communicating the above listed types of guidance to the user. The medium also inscribes how the method canbe interacted with by the user in digital or physical format. The different medium types include: worksheets (n=17),templates (n=17), cards (n= 15), document/guidebook (n= 33), physical manipulatives (n=3), videos (n=1), and games(n=2).
Worksheets are documents where the user is asked to add specified information as they are working with themethod, whereas a template is a document that is expected to be used as a baseline reference in order to interact withusing other components of the method. For example, HuValue [98] has a template with different value groups sectionedas a part of a circle which is intended to be used as a base with which to “filter” cards of user’s choice under the valuegroups. Other physical media include cards or physical manipulatives, while digital media also include videos, andsome methods could be presented in a combination of digital and physical forms. Design with Intent [102] presentsdifferent “lens” through which design artifacts can be evaluated in the form of deck of physical color-coded cards . A document/guidebook could include a digital or physical standalone booklet that contains a method description andtype of guidance for the user to refer as they are using the method during their design activity. Two methods—MoralAgent [70] and Judgement Call the Game [24]—were designed to encourage interactions in the form of a board game,consisting of a combination of several of the media described above. As another example of hybrid media, Moral Agent[70] consisted of a card deck to draw from, worksheets to write on, and two documents/guidebooks to filter values anddescribe the rules of the game.
We coded the descriptors that frame the language of these methods as sensitizingconcepts, drawing on a term by the same name that is often used to identify structure and conceptual foundations ingrounded theory research [38]. These concepts provide a conceptual and methodological vocabulary that the methoddevelopers use to define the expected purpose or core of the method. Depending on the method focus, this vocabularyranges from established social constructs (e.g., culture, gender); published policies (e.g., GDPR, EU Draft e-PrivacyRegulation 2017); defined methodologies (e.g., critical design, VSD, speculative design, design fiction, co-design); knownethical or privacy concerns (e.g., user behavior change, cyber-security, data privacy, data ethics); defined human values(e.g., privacy, security, spirituality); commonly used interaction design concepts or methods (e.g., form, function, empathy,scenarios); and applied ethical concepts (e.g., justice, human rights, common good, utility). For example, GenderMag[9, 36] relies upon the social category of “gender” as a sensitizing concept to frame the designer’s construction ofpersonas; this social construct shapes the method’s purpose in engaging software developers in a more inclusive formof building technological artifacts and links the use of the method to broader social and academic conceptions of gender.The Moral and Legal Decks [18] cards used vocabulary from “ relevant rights, principles, definitions and responsibilitieswithin the: EU General Data Protection Regulation 2016; EU Draft e-Privacy Regulation 2017; EU Network and Information urveying the Landscape of Ethics-Focused Design Methods Security Directive 2016; Cybercrime Convention 2001; and Attacks Against Information Systems Directive 2013 ” to designthe content and guidelines provided through the method, thereby grounding design activity in legal definitions ofprivacy and data protection.These sensitizing concepts had different functions in different methods, with some concepts being used acrossmultiple methods in different ways to encourage or foreground specific mechanics, design judgments, or framingsof design activity. For example, design fiction was defined as the means by which the designer should create “designconcepts” in the Security Fictions method [110], whereas design fiction was used as an ideology with which the usercould evaluate design concepts in Judgment Call the Game [24]. Design fiction was also treated as an opportunityspace through which to explore and elicit values in relation to privacy in future technological artifacts in ElicitingValues Reflections method [149]. Overall, we identified more than 80 sensitizing concepts across the set of methodswe analyzed, and few sensitizing concepts appeared to be used consistently. Several key sensitizing concepts whichdid appear in multiple methods include: GDPR [18, 55], co-design [53, 146, 153, 154], VSD (all methods within theVSD methodology), speculative design [53, 110, 132, 149], human values [22, 33, 39, 45, 98, 146, 154] and design fiction[24, 53, 110]. This analysis demonstrates that most methods include their own distinct vocabulary which is generallynot shared or standardized across multiple methods, illustrating both the variety in the existing ethics-focused methodsand the lack of consistency across methods.
In this paper, we have identified a range of descriptors of ethics-focused methods that allow us to identify existingmechanisms for ethical support, along with opportunities for the development, adaptation, and dissemination of newmethods. In this section, we outline how these findings point towards gaps in our current knowledge of ethics-focusedmethods, identifying spaces for new method development, and also revealing ways to interrogate the performance andperformativity of these methods in complex organizational contexts. We conclude this section with a more philosophicaldiscussion regarding the ontological dimensions of methods, pointing towards methods use and performance as anormatively-infused way of engaging with design practice.
A synthesis of the results from the content analysis facilitates additional focus on complexities that exist in termsof dissemination of the methods’ knowledge; codification of these methods with respect to design process; and the performance of these methods in everyday practice. While our analytic approach does not allow us to resolve questionsor issues relating to dissemination challenges, we are able to observe a disconnect between the published formats ofmost of these method sources and their intended audience. According to our results on the published format, 44.4%of the methods were published in the form of academic papers, and the majority of these methods were created fordesign practitioners or educators. However, this audience is frequently unable to access materials published in formalacademic venues due to paywall restrictions. The lack of the access and awareness of these methods from the perspectiveof practitioners likely results in reduced adoption of the methods, even without accounting for other translationalbarriers observed by HCI researchers [41, 87]. While our analytic approach does not allow us to resolve questions aboutdissemination challenges, our critique depends on the quantifiable results of comparing the method’s intended audienceand published formats, which shows that most methods are behind a paywall. Additionally, as shown in Table 4, it isevident that the majority of methods we analyzed were research-focused (Phases 2 and 4), generatively-focused (Phase3 and 4), or evaluation-focused (Phase 4). This finding demonstrates the potential for exploring or creating method for hivukula, et al. Phase 1 (project planning focused) and phase 5 (monitoring focused) work. Creating methods focused on Phase 1 and 5also have the potential to be more impactful in creating a space for ethically-focused work, inscribing the problemspace with ethical concerns (Phase 1) and continuously evaluating the work in a social context (Phase 5). One of therare example of methods involved in Phase 1 of the process includes the Ethical Contract method [68], which enables adiscussion among all the stakeholders on the project to clearly discuss and divide their “ethical responsibilities” prior toconcept generation. This method shows potential in discussing, communicating, and formalizing ethical responsibilitiesacross multiple stakeholders in everyday practice. As we provide the above critique, we also highlight limitations in ourcoding approach. Our analysis was solely dependent on the available method descriptions text, and we worked to avoidhigh levels of inference. We also rely on the qualitative results as we provide the gaps in the existing methods that aremore inclined towards Phases 2, 3, and 4, proposing that method developers and publishers considering using Table 2as a framework to build, refine, or standardize their manuals.Our findings also point towards the intended performance of these methods in everyday practice, revealing underlyingbeliefs about practitioners, practice, and available resources. Methods we evaluated include mechanisms to aid in:addressing power dynamics and solving complexities due to organizational rules (e.g., Data Ethics Canvas [3]); bringinga balance between stakeholder requirements and designer intentions (e.g., A Value-Sensitive Action-Reflection Model[154]); facilitating realization of designers’ ethical responsibilities and extending application of these responsibilitiesbeyond instructional settings (e.g., Re-Shape [127]); guiding through self-provocation to evaluate the impacts of createtechnology (e.g., The Tarot of Tech [16]); bridging knowledge for practitioners from different disciplines (e.g., IdeaGeneration through Empathy method [88]); monitoring impacts of shipped products (e.g., Design Ethically-MonitoringChecklist [162]); and providing ethical or critical knowledge, concepts, and vocabulary to be applied to support decisionmaking (e.g. values through HuValue [98] and gender-inclusivity through GenderMag [36]). Our synthesis also revealsseveral inscribed assumptions regarding the performance of these methods. First, the knowledge of these methodsare presented using types of guidance that are intended to encourage certain patterns of performance. For instance,heuristics are used in the White Hat UX patterns [54] method, with the underlying assumption that these heuristicscan be applied without the impedance of existing business forces or other forms of complexity beyond the designerthemself. Second, the methods reveal beliefs that designers already have the vocabulary and capacity to express theirsocial responsibility and have the capability to take responsibility for positive social impact (e.g., as required to list inEthical Disclaimer [69]). Third, the methods reveal beliefs that practitioners are able to evaluate their decisions basedon concepts such as utility, human rights, and justice, which often resist quantification. These qualities of the intendedperformance of methods’ knowledge in practice brings to the foreground the resonance of these methods with theconstraints and complexity of practice settings, and the capacity and existing knowledge of the practitioners. Buildingon Stolterman’s [135] concept of rationality resonance , which highlights the relationship between suggested vs. existing practice, we can further question the barriers to adoption of these methods in practice settings, identifying spaceswhere existing assumed knowledge of practitioners is incomplete; spaces where the agency and power of designers isnot available to the extent that methods might assume; and spaces where ethical complexity across multiple stakeholderpositions is unaccounted for [83]. While we cannot resolve this issue of performance in the context of this paper, wedo propose that future work could address the role of methods as an emergent “new rationality” that could promoteethical practices, while also guarding against methods descriptions and codification that lacks resonance with theethical design complexity present in everyday work practices. urveying the Landscape of Ethics-Focused Design Methods Prior design theory literature defines methods as a source of design knowledge that enables or supports design activity,acting as a toolset to support the designer’s judgment and action throughout their design work. Based on the sensitizingconcepts and core of these methods, we have identified a range of underlying assumptions regarding the use ofthese methods to discover knowledge, identify new possibilities, and locate hidden assumptions. These attitudes orstances towards engagement with knowledge range from uncovering ethical components of a design situation orproblem through in situ speculation (e.g., Speculative Enactments [53]); considering ethical evaluation through actsof iterating and futuring (e.g., The Oracle for Transfeminist Technologies [14]); fostering innovation through uniquevalue propositions by uncovering neglected or negatively impacted user groups in a use scenario (e.g., The Ethicsand Inclusion Framework [19]); influencing designers’ thinking to design for sustainable and non-deceptive behaviorchange (e.g., Design with Intent [102]); helping to foreground and map the underlying intentions and world-view of adesigner (e.g., Description [67]); and providing a means for designers to operationalize ethics in their design process.The primary aim of the knowledge contained in these methods was ethical impact—influenced by ethical theories,values, and frameworks. We chose to define the collection as containing “ethics-focused methods” to differentiate thesemethods from conventional design methods, conceptual frames, or theoretical commitments. In this way, the resultingcollection of 63 ethics-focused methods include prescriptive forms that are actionable and potentially performative onthe part of designers. Thus, the function of the method revealed through this embedded knowledge allows designers toconvert ethics-focused discovery into design outcomes. We have identified several means by which this translationmight occur, including: converting a prescribed value into a concern by describing how it is present in the designcontext (e.g., Moral Value Map [71]); maximizing the ethical valence of a design situation by considering many ethicaltheories together (e.g., Normative Design Scheme [72]); bringing legal, moral and ethical policies and values together(e.g., Moral and Legal IT Deck [18]); introducing user-centric concepts into different disciplines (e.g., Idea Generationthrough Empathy method [88]); broadening the scope of human values such as privacy (e.g., Privacy Futures throughDesign Workbooks [149]); quantifying ethical decisions for users to improve their decision making (e.g., The EthicalDesign Scorecards [55]); and introducing critical and feminist constructs such as gender to expand the horizons oftechnology design (e.g., Gender Mag [36]). This embedded knowledge aids the designer in translating complex ethicalconcepts into normatively-informed work practices, bridging the liminal space between awareness and action.
Our findings include the identification of various descriptors embedded in existing ethics-focused methods, and thediscussion reveals even more complexities and underlying assumptions about the practical use of these methods thatrelate to their specification, dissemination, and performance. Building on this work, there is substantial potential forfuture research through investigation of issues relating to method adaption, evolution, and the resonance of thesemethods with everyday work practices. Work in this area may lead to the identification of productive areas for thecreation of future ethics-focused methods, while also pointing towards barriers to adoption of existing methods inpractitioner discourses and work practices.While we have identified a large set of ethics-focused methods, we do not claim that our collection is decidedlycomplete. Due to the wide range of languaging approaches and dissemination strategies used in the set of methods wewere able to identify, collect, and analyze, we anticipate that there are likely other existing and emergent sources thatcould enhance our collection. Thus, the implications and contribution of this work does not rest on the collection being hivukula, et al. objectively “complete”; rather, the primary contribution resides in the ways we have analyzed and attached descriptorsto these methods, revealing opportunities for new methods, refinement of existing methods, and means of standardizinglanguage among methods to increase portability and adaptation in everyday design work.Further investigation into the practical use and popular awareness of these methods may provide opportunities formethod developers to consider alternative dissemination strategies. Future work may productively focus on studyingthe creation of these methods, revealing the considerations method developers take into account, and the ways in whichthey constrain or include aspects of ethical design complexity into the methods they create.Building on our findings, we are able to identify opportunities for the creation of new ethics-focused methods, andadditional practices that may result in more resonant forms of methods dissemination, design process implementation,iteration and adaptation, and translational opportunities among design practitioners, educators, and researchers. First,we underscore the need to disseminate and distribute methods to the intended audience in more accessible andpublic formats, including the potential creation of channels to sharing material between researcher and practitionercommunities. Second, we describe a substantial gap in the provision of design methods that support phases 1 and 5,including ethical engagement with the framing of a problem space by considering ethical responsibilities across allstakeholders (Phase 1) and iteratively evaluating a product in a social context after launching in the market (Phase 5).The creation of new methods to address these spaces, or the intentional adaptation of existing methods to support theseforms of inquiry could bring substantial value to design conversations in these areas of practice. Third, methods couldbe further evaluated in terms of their fit and portability, encouraging increased iterative use of a range of ethics-focusedmethods across a range of design activities. The input->mechanic->output patterns reveal rich opportunities for thisinterplay among methods, yet the languaging and media of the methods we have evaluated show distinct differences inapproach that make this ad hoc assemblages of methods difficult or unlikely under the pressures of everyday practice.Fourth, we identify potential opportunities to build prescriptive methods that rely upon theoretical commitments(e.g., theoretical commitments listed in Table 1), activating those concepts through new ethics-focused methods tosupport design practice. Finally, we propose the creation or articulation of value-focused design frameworks for acombination of stakeholders to build ethical alignment and engage members of a multi-disciplinary team, bringingresonance across multiple practice contexts. Successfully engaging with these implications and provocations for futurework may substantively impact the ethical awareness of design and technology practitioners, while also pointingtowards needs and gaps in design education practices, where a core set of methods is often learned. In this paper, we present a content analysis of 63 ethics-focused methods intended for use in design and technologypractice. We map the current landscape of ethical support and tools by characterizing the collection of these methodsalong multiple dimensions, including the ways in which they operationalize ethics, their intended primary audience andcontext of use, the core of the methods, their interaction qualities, and the ways in which these methods are articulated,formulated, and languaged. We provide these methods as an initial collection, alongside a set of descriptors that markthe existing landscape of ethical support for researchers, practitioners, and educators. We propose a definition forethics-focused methods and identify means of making these methods more resonant with HCI and design practice,articulating multiple areas of future work to support method development and design practice.
ACKNOWLEDGMENTS
This work is funded in part by the National Science Foundation under Grant No. 1909714. urveying the Landscape of Ethics-Focused Design Methods REFERENCES
G. , Star, S. , Turner, W. , and Gasser, L. , eds,Social Science, Technical Systems and Cooperative Work: Beyond the Great Divide, Erlbaum (1997).[22] Taghreed Alshehri, Reuben Kirkham, and Patrick Olivier. 2020. Scenario Co-Creation Cards: A Culturally Sensitive Tool for Eliciting Values.In
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20) . Association for ComputingMachinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376608[23] Aristotle. 2000.
Aristotle: Nicomachean Ethics . Cambridge University Press.[24] Stephanie Ballard, Karen M Chappell, and Kristen Kennedy. 2019. Judgment Call the Game: Using Value Sensitive Design and Design Fiction toSurface Ethical Concerns Related to Technology. In
Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS’19) . ACM, New York, NY, USA, 421–433. https://doi.org/10.1145/3322276.3323697[25] Jeffrey Bardzell and Shaowen Bardzell. 2013. What is critical about critical design?. In
Proceedings of the SIGCHI conference on human factors incomputing systems . 3297–3306.[26] Jeffrey Bardzell, Shaowen Bardzell, and Mark A Blythe. 2018.
Critical Theory and Interaction Design . MIT Press.[27] Shaowen Bardzell. 2010. Feminist HCI: Taking Stock and Outlining an Agenda for Design. In
Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10) . ACM, New York, NY, USA, 1301–1310. https://doi.org/10.1145/1753326.1753521[28] Lawrence C Becker and Charlotte B Becker. 2001.
Encyclopedia of ethics (2nd ed. ed.). Routledge, New York [N.Y.].[29] Elettra Bietti. 2020. From ethics washing to ethics bashing: a view on tech ethics from within moral philosophy. In
Proceedings of the 2020 Conferenceon Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* ’20) . Association for Computing Machinery, New York, NY, USA, 210–219.https://doi.org/10.1145/3351095.3372860[30] Michael Mose Biskjaer, Peter Dalsgaard, and Kim Halskov. 2010. Creativity methods in interaction design. In
Proceedings of the 1st DESIRE NetworkConference on Creativity and Innovation in Design (Aarhus, Denmark) (DESIRE ’10) . Desire Network, Lancaster, GBR, 12–21.[31] Erling Björgvinsson, Pelle Ehn, and Per-Anders Hillgren. 2012. Agonistic participatory design: working with marginalised social movements.
CoDesign
8, 2-3 (June 2012), 127–144. https://doi.org/10.1080/15710882.2012.672577[32] Mark Blythe. 2014. Research through design fiction: narrative in real and imaginary abstracts. In
Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14) . Association for Computing Machinery, New York, NY, USA, 703–712.https://doi.org/10.1145/2556288.2557098[33] Alan Borning, Batya Friedman, Janet Davis, and Peyina Lin. 2005. Informing Public Deliberation: Value Sensitive Design of Indicators for aLarge-Scale Urban Simulation. In
ECSCW 2005 . Springer Netherlands, 449–468. https://doi.org/10.1007/1-4020-4023-7_23[34] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.
Qualitative research in psychology
3, 2 (Jan. 2006), 77–101.https://doi.org/10.1191/1478088706qp063oa[35] Harry Brignull, Marc Miquel, Jeremy Rosenberg, and James Offer. 2015. Dark Patterns-User Interfaces Designed to Trick People.23 hivukula, et al. [36] Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016.GenderMag: A Method for Evaluating Software’s Gender Inclusiveness.
Interact. Comput.
28, 6 (Nov. 2016), 760–787. https://doi.org/10.1093/iwc/iwv046[37] Peter Buwert and Edinburgh Napier University. 2018. Examining the Professional Codes of Design Organisations. https://doi.org/10.21606/drs.2018.493[38] K Charmaz. 2008. Constructionism and the grounded theory method.
Handbook of constructionist research (2008).[39] Shruthi Sai Chivukula, Colin M Gray, and Jason A Brier. 2019. Analyzing Value Discovery in Design Decisions Through Ethicography. In
Proceedingsof the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19) . ACM, New York, NY, USA, 77:1–77:12.https://doi.org/10.1145/3290605.3300307[40] Shruthi Sai Chivukula, Chris Rhys Watkins, Rhea Manocha, Jingle Chen, and Colin M Gray. 2020. Dimensions of UX Practice that Shape EthicalAwareness. In
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20) . Association forComputing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376459[41] Lucas Colusso, Ridley Jones, Sean Munson, and Gary Hsieh. 2019. A Translational Science Model for HCI. In
Proceedings of the 2019 CHI Conferenceon Human Factors in Computing Systems (CHI ’19) . ACM Press, New York, NY. https://doi.org/10.1145/3290605.3300231[42] Sasha Costanza-Chock. 2020.
Design Justice: Community-Led Practices to Build the Worlds We Need . MIT Press.[43] Nigel Cross. 1984.
Developments in design methodology . John Wiley & Sons.[44] Robert Curedale. 2012.
Design methods 1: 200 ways to apply design thinking . Design Community College Incorporated.[45] Alexei Czeskis, Ivayla Dermendjieva, Hussein Yapit, Alan Borning, Batya Friedman, Brian Gill, and Tadayoshi Kohno. 2010. Parenting fromthe pocket: value tensions and technical directions for secure and private parent-teen mobile safety. In
Proceedings of the Sixth Symposiumon Usable Privacy and Security (Redmond, Washington, USA) (SOUPS ’10) . Association for Computing Machinery, New York, NY, USA, 1–15.https://doi.org/10.1145/1837110.1837130[46] Vanessa Paz Dennen. 2004. Cognitive apprenticeship in educational practice: Research on scaffolding, modeling, mentoring, and coaching asinstructional strategies.
Handbook of research on educational communications and technology
2, 2004 (2004), 813–828. http://faculty.weber.edu/eamsel/Classes/Projects%20and%20Research%20(4800)/Teaching%20and%20Learning/Dennen%20(2004).pdf[47] Tamara Denning, Batya Friedman, , and Tadayoshi Kohno. 2013. The Security Cards: A Security Threat Brainstorming Kit. http://securitycards.cs.washington.edu/. Accessed: 2020-5-12.[48] Michael Dieter and David Gauthier. 2019. On the Politics of Chrono-Design: Capture, Time and the Interface.
Theory, Culture & Society
36, 2(March 2019), 61–87. https://doi.org/10.1177/0263276418819053[49] Catherine D’Ignazio and Lauren F Klein. 2020.
Data Feminism . MIT Press.[50] Clive Dilnot. 2005.
Ethics? design?
Archeworks.[51] Carl DiSalvo. 2015.
Adversarial Design . MIT Press.[52] Anthony Dunne and Fiona Raby. 2013.
Speculative Everything: Design, Fiction, and Social Dreaming . MIT Press.[53] Chris Elsden, David Chatting, Abigail C Durrant, Andrew Garbett, Bettina Nissen, John Vines, and David S Kirk. 2017. On Speculative Enactments.In
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17) . Association for ComputingMachinery, New York, NY, USA, 5386–5399. https://doi.org/10.1145/3025453.3025503[54] Trine Falbe, Kim Andersen, and Martin Michael Frederiksen. 2017.
White Hat UX . Smashing Media AG.[55] Trine Falbe, Martin Michael Frederiksen, and Kim Andersen. [n.d.]. The Ethical Design Scorecards. https://ethicaldesignhandbook.com/. Accessed:2020-5-14.[56] Mary Flanagan and Helen Nissenbaum. 2014.
Values at Play in Digital Games . MIT Press, Cambridge, MA. https://market.android.com/details?id=book-iIYRBAAAQBAJ[57] Mary Flanagan and Helen Nissenbaum. 2014.
Values at Play in Digital Games . MIT Press. https://doi.org/10.1177/1461444816631742[58] Christopher Frauenberger, Marjo Rauhala, and Geraldine Fitzpatrick. 2017. In-Action Ethics.
Interact. Comput.
29, 2 (March 2017), 220–236.https://doi.org/10.1093/iwc/iww024[59] Batya Friedman, Edward Felten, and Lynette I Millett. 2000. Informed consent online: A conceptual model and design principles.
University ofWashington Computer Science & Engineering Technical Report 00–12–2
Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12) . Association for Computing Machinery, New York,NY, USA, 1145–1148. https://doi.org/10.1145/2207676.2208562[61] Batya Friedman and David G Hendry. 2019.
Value Sensitive Design: Shaping Technology with Moral Imagination . MIT Press.[62] Batya Friedman, Peter Kahn, Jennifer Hagman, Rachel Severson, and Brian Gill. 2006. The Watcher and the Watched: Social Judgments AboutPrivacy in a Public Place. , 235–272 pages. https://doi.org/10.1207/s15327051hci2102_3[63] Batya Friedman, Peter H Kahn, and Alan Borning. 2008. Value sensitive design and information systems.
The handbook of information and computerethics (2008), 69–101.[64] Batya Friedman and Peter H Kahn, Jr. 2003. Human values, ethics, and design.
The human-computer interaction handbook (2003), 1177–1201.[65] Batya Friedman, Ian Smith, Peter H. Kahn, Sunny Consolvo, and Jaina Selawski. 2006. Development of a Privacy Addendum for Open SourceLicenses: Value Sensitive Design in Industry. In
UbiComp 2006: Ubiquitous Computing . Springer Berlin Heidelberg, 194–211. https://doi.org/10.24 urveying the Landscape of Ethics-Focused Design Methods
Monist
International Journalof Market Research
59, 6 (Nov. 2017), 703–705. https://doi.org/10.2501/IJMR-2017-050[74] Gabriela Goldschmidt. 2014.
Linkography: Unfolding the Design Process . MIT Press.[75] Elizabeth Goodman. 2013.
Delivering Design: Performance and Materiality in Professional Interaction Design . Ph.D. Dissertation. UC Berkeley.https://escholarship.org/uc/item/31w6q6x3[76] Elizabeth Goodman, Erik Stolterman, and Ron Wakkary. 2011. Understanding Interaction Design Practices. In
Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11) . ACM, New York, NY, USA, 1061–1070. https://doi.org/10.1145/1978942.1979100[77] Don Gotterbarn. 1998. Reconstructing the ACM code of ethics and teaching computer ethics.
ACM SIGCSE Bulletin
30, 4 (Dec. 1998), 9–11.https://doi.org/10.1145/306286.306293[78] D W Gotterbarn, Bo Brinkman, Catherine Flick, Michael S Kirkpatrick, Keith Miller, Kate Vazansky, and Marty J Wolf. 2018. ACM Code of Ethicsand Professional Conduct. (2018).[79] Colin M Gray. 2016. “It’s More of a Mindset Than a Method”: UX Practitioners’ Conception of Design Methods. In
Proceedings of the 2016CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16) . ACM, New York, NY, USA, 4044–4055. https://doi.org/10.1145/2858036.2858410[80] Colin M Gray. 2016. What is the Content of ”Design Thinking”? Design Heuristics as Conceptual Repertoire.
International Journal of EngineeringEducation
Proceedings of the Design Research Society (2016), 27–30.[82] Colin M Gray and Elizabeth Boling. 2016. Inscribing ethics and values in designs for learning: a problematic.
Educ. Technol. Res. Dev.
64, 5 (Oct.2016), 969–1001. https://doi.org/10.1007/s11423-016-9478-x[83] Colin M Gray and Shruthi Sai Chivukula. 2019. Ethical Mediation in UX Practice. In
Proceedings of the 2019 CHI Conference on Human Factors inComputing Systems (Glasgow, Scotland Uk) (CHI ’19) . ACM, New York, NY, USA, 178:1–178:11. https://doi.org/10.1145/3290605.3300408[84] Colin M Gray, Shruthi Sai Chivukula, and Ahreum Lee. 2020. What Kind of Work Do “Asshole Designers” Create? Describing Properties of EthicalConcern on Reddit. In
Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20) . Association forComputing Machinery, New York, NY, USA, 61–73. https://doi.org/10.1145/3357236.3395486[85] Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The Dark (Patterns) Side of UX Design. In
Proceedings of the2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18) . Association for Computing Machinery, New York,NY, USA, 1–14. https://doi.org/10.1145/3173574.3174108[86] Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The Dark (Patterns) Side of UX Design. In
Proceedings of the2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18) . Association for Computing Machinery, New York,NY, USA, 1–14. https://doi.org/10.1145/3173574.3174108[87] Colin M Gray, Erik Stolterman, and Martin A Siegel. 2014. Reprioritizing the relationship between HCI research and practice: bubble-up andtrickle-down effects. In
Proceedings of the 2014 Conference on Designing Interactive Systems (Vancouver, BC, Canada) (DIS ’14) . ACM, New York,New York, USA, 725–734. https://doi.org/10.1145/2598510.2598595[88] Colin M Gray, Seda Yilmaz, Shanna R Daly, Colleen M Seifert, and Richard Gonzalez. 2015. Idea Generation Through Empathy: Reimagining the‘Cognitive Walkthrough’. (2015).[89] Bruce Hanington. 2003. Methods in the Making: A Perspective on the State of Human Research in Design.
Design Issues
19, 4 (Oct. 2003), 9–18.https://doi.org/10.1162/074793603322545019[90] Bruce Hanington and Bella Martin. 2012.
Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and DesignEffective Solutions . Rockport Publishers.[91] S Harrison, M Back, and D Tatar. 2006. It’s Just a Method!: a pedagogical experiment in interdisciplinary design. In
Proceedings of the 6th conferenceon Designing Interactive systems . ACM, 261–270. https://doi.org/10.1145/1142405.1142445[92] Justin L Hess and Grant Fore. 2018. A Systematic Literature Review of US Engineering Ethics Interventions.
Science and engineering ethics
24, 2(April 2018), 551–583. https://doi.org/10.1007/s11948-017-9910-6[93] Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis.
Qual. Health Res.
15, 9 (Nov. 2005), 1277–1288.https://doi.org/10.1177/1049732305276687[94] Lilly Irani, Janet Vertesi, Paul Dourish, Kavita Philip, and Rebecca E Grinter. 2010. Postcolonial computing: a lens on design and development.In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . Association for Computing Machinery, New York, NY, USA,25 hivukula, et al.
Design Methods . John Wiley & Sons.[96] Immanuel Kant. 1785. Groundwork of the Metaphysics of Morals.[97] Os Keyes, Jevan Hutson, and Meredith Durbin. 2019. A Mulching Proposal: Analysing and Improving an Algorithmic System for Turning theElderly into High-Nutrient Slurry. In
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19) . ACM, New York, NY, USA, alt06:1–alt06:11. https://doi.org/10.1145/3290607.3310433[98] Shadi Kheirandish, Mathias Funk, Stephan Wensveen, Maarten Verkerk, and Matthias Rauterberg. 2019. HuValue: a tool to support design studentsin considering human values in their design.
Int. J. Technol. Des. Educ. (May 2019). https://doi.org/10.1007/s10798-019-09527-3[99] S Kolarić, J Beck, and E Stolterman. 2020. On the Hierarchical Levels of Design Knowledge.
Proceedings of the Design Society: DESIGN Conference
J. Bus.Ethics
48, 4 (Dec. 2003), 365–379. https://doi.org/10.1023/B:BUSI.0000005747.37500.c8[101] A Light. 2011. HCI as heterodoxy: Technologies of identity and the queering of interaction with computers.
Interact. Comput.
23, 5 (Sept. 2011),430–438. https://doi.org/10.1016/j.intcom.2011.02.002[102] Dan Lockton. 2010.
Design with Intent: 101 Patterns for Influencing Behaviour Through Design . Equifine.[103] Nick Logler, Daisy Yoo, and Batya Friedman. 2018. Metaphor Cards: A How-to-Guide for Making and Using a Generative Metaphorical DesignToolkit. In
Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18) . Association for Computing Machinery,New York, NY, USA, 1373–1386. https://doi.org/10.1145/3196709.3196811[104] Jonas Löwgren. 2013. Annotated portfolios and other forms of intermediate-level knowledge.
Interactions
20, 1 (Jan. 2013), 30–34. https://doi.org/10.1145/2405716.2405725[105] Jonas Löwgren and Erik Stolterman. 1999. Methods & Tools: Design Methodology and Design Practice. 6, 1 (Jan. 1999), 13–20. https://doi.org/10.1145/291224.291233[106] Jamie Luguri and Lior Strahilevitz. 2019. Shining a Light on Dark Patterns. (Aug. 2019). https://doi.org/10.2139/ssrn.3431205[107] Noëmi Manders-Huits and Michael Zimmer. 2009. Values and pragmatic action: The challenges of introducing ethical intelligence in technicaldesign communities.
International Review of Information Ethics
10, 2 (2009), 37–45.[108] Arunesh Mathur, Gunes Acar, Michael J Friedman, Elena Lucherini, Jonathan Mayer, Marshini Chetty, and Arvind Narayanan. 2019. DarkPatterns at Scale: Findings from a Crawl of 11K Shopping Websites.
Proc. ACM Hum.-Comput. Interact.
3, CSCW (Nov. 2019), 1–32. https://doi.org/10.1145/3359183[109] Joshua Mauldin. 2018. Black Mirror brainstorms — a product design exercise. https://uxdesign.cc/black-mirror-brainstorms-f919ccf5938c. Accessed:2020-5-14.[110] Nick Merrill. 2020. Security Fictions: Bridging Speculative Design and Computer Security. In
Proceedings of the 2020 ACM Designing InteractiveSystems Conference (Eindhoven, Netherlands) (DIS ’20) . Association for Computing Machinery, New York, NY, USA, 1727–1735. https://doi.org/10.1145/3357236.3395451[111] Nick Merrill and Joanne Ma. [n.d.]. Adversary Personas. https://daylight.berkeley.edu/adversary-personas/. Accessed: 2020-5-14.[112] Jessica K Miller, Batya Friedman, Gavin Jancke, and Brian Gill. 2007. Value tensions in design: the value sensitive design, development, andappropriation of a corporation’s groupware system. In
Proceedings of the 2007 international ACM conference on Supporting group work (SanibelIsland, Florida, USA) (GROUP ’07) . Association for Computing Machinery, New York, NY, USA, 281–290. https://doi.org/10.1145/1316624.1316668[113] Michael Mose Biskjaer, Peter Dalsgaard, and Kim Halskov. 2017. Understanding creativity methods in design. In
Proceedings of the 2017 conferenceon designing interactive systems . 839–851.[114] Maurice Mulvenna, Jennifer Boger, and Raymond Bond. 2017. Ethical by Design: A Manifesto. In
Proceedings of the European Conference onCognitive Ergonomics 2017 (Umeå, Sweden) (ECCE 2017) . ACM, New York, NY, USA, 51–54. https://doi.org/10.1145/3121283.3121300[115] Sean A Munson, Daniel Avrahami, Sunny Consolvo, James Fogarty, Batya Friedman, and Ian Smith. 2011. Attitudes toward online availabilityof US public records. In
Proceedings of the 12th Annual International Digital Government Research Conference: Digital Government Innovation inChallenging Times (College Park, Maryland, USA) (dg.o ’11) . Association for Computing Machinery, New York, NY, USA, 2–9. https://doi.org/10.1145/2037556.2037558[116] Lisa P Nathan. 2012. Sustainable information practice: An ethnographic investigation.
J. Am. Soc. Inf. Sci. Technol.
63, 11 (Nov. 2012), 2254–2268.https://doi.org/10.1002/asi.22726[117] Lisa P Nathan, Predrag V Klasnja, and Batya Friedman. 2007. Value scenarios: a technique for envisioning systemic effects of new technologies. In
CHI ’07 Extended Abstracts on Human Factors in Computing Systems (San Jose, CA, USA) (CHI EA ’07) . Association for Computing Machinery, NewYork, NY, USA, 2585–2590. https://doi.org/10.1145/1240866.1241046[118] Kimberly A Neuendorf. 2016.
The Content Analysis Guidebook . SAGE.[119] Chris Nodder. 2013.
Evil by Design: Interaction Design to Lead Us into Temptation . John Wiley & Sons, Indianapolis, IN. 303 pages.[120] Stuart Reeves. 2019. How UX Practitioners Produce Findings in Usability Testing.
ACM transactions on computer-human interaction: a publicationof the Association for Computing Machinery
Developments in Design Methodology (1984), 317–327.26 urveying the Landscape of Ethics-Focused Design Methods [123] Johnny Saldana. 2015.
The Coding Manual for Qualitative Researchers . SAGE. https://market.android.com/details?id=book-ZhxiCgAAQBAJ[124] Elizabeth Sanders and Pieter Jan Stappers. 2012.
Convivial Toolbox: Generative Research for the Front End of Design . BIS.[125] Donald A Schön. 1990. The design process. In
Varieties of thinking: Essays from Harvard’s philosophy of education research center , V A Howard (Ed.).Routledge, 111–141.[126] Phoebe Sengers, Kirsten Boehner, Shay David, and Joseph ’jofish’ Kaye. 2005. Reflective Design. In
Proceedings of the 4th Decennial Conference onCritical Computing: Between Sense and Sensibility (Aarhus, Denmark) (CC ’05) . ACM, New York, NY, USA, 49–58. https://doi.org/10.1145/1094562.1094569[127] Ben Rydal Shapiro, Amanda Meng, Cody O’Donnell, Charlotte Lou, Edwin Zhao, Bianca Dankwa, and Andrew Hostetler. 2020. Re-Shape: A Methodto Teach Data Ethics for Data Science Education. In
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI,USA) (CHI ’20)
Sci. Technol. Human Values
38, 3 (May 2013), 374–397. https://doi.org/10.1177/0162243912436985[130] Katie Shilton. 2018. Values and Ethics in Human-Computer Interaction.
Foundations and Trends® Human–Computer Interaction
12, 2 (2018),107–171. https://doi.org/10.1561/1100000073[131] Kristen Shinohara, Nayeri Jacobo, Wanda Pratt, and Jacob O Wobbrock. 2020. Design for Social Accessibility Method Cards: Engaging Usersand Reflecting on Social Scenarios for Accessible Design.
ACM Trans. Access. Comput.
Article 17 (Jan. 2020). https://doi.org/blication:ACMTransactionsonAccessibleComputingJanuary2020ArticleNo.:17https://doi.org/10.1145/3369903[132] Michael Stallings. [n.d.]. Make it (Critical). https://crit.luddy.indiana.edu/design/. Accessed: 2020-5-14.[133] Luke Stark and Kate Crawford. 2019. The Work of Art in the Age of Artificial Intelligence: What Artists Can Teach Us About the Ethics of DataPractice.
17, 3/4 (Sept. 2019), 442–455. https://doi.org/10.24908/ss.v17i3/4.10821[134] Marc Steen. 2015. Upon Opening the Black Box and Finding It Full: Exploring the Ethics in Design Practices.
Sci. Technol. Human Values
40, 3 (May2015), 389–420. https://doi.org/10.1177/0162243914547645[135] E Stolterman. 2008. The nature of design practice and implications for interaction design research.
International Journal of Design
2, 1 (Jan. 2008),55–65. https://doi.org/10.1016/j.phymed.2007.09.005[136] Erik Stolterman, Jamie McAtee, David Royer, and Selvan Thandapani. 2008. Designerly tools. In
Undisciplined! Design Research Society Conference2008 (Sheffield, UK). Sheffield Hallam University, Sheffield, UK, 116:1–14.[137] Erik Stolterman and James Pierce. 2012. Design Tools in Practice: Studying the Designer-Tool Relationship in Interaction Design. In
Proceedings ofthe Designing Interactive Systems Conference on - DIS ’12 . ACM Press, New York, New York, USA, 25–28. https://doi.org/10.1145/2317956.2317961[138] Jason Chew Kit Tham. 2019. Feminist design thinking: a norm-creative approach to communication design. In
Proceedings of the 37th ACMInternational Conference on the Design of Communication (Portland, Oregon) (SIGDOC ’19) . Association for Computing Machinery, New York, NY,USA, 1–3. https://doi.org/10.1145/3328020.3353919[139] Martin Tomitsch, Cara Wrigley, Madeleine Borthwick, Naseem Ahmadpour, Jessica Frawley, A Baki Kocaballi, Claudia Nunez-Pacheco, and KarlaStraker. 2018.
Design. Think. Make. Break. Repeat. A handbook of methods . BIS Publishers, The Netherlands.[140] Annemiek Van Boeijen, Jaap Daalhuizen, Roos van der Schoor, and Jelle Zijlstra. 2014.
Delft design guide: Design strategies and methods . BISPublishers.[141] Aimee van Wynsberghe and Scott Robbins. 2014. Ethicist as designer: a pragmatic approach to ethics in the lab.
Sci. Eng. Ethics
20, 4 (Dec. 2014),947–961. https://doi.org/10.1007/s11948-013-9498-4[142] Peter-Paul Verbeek. 2005.
What Things Do: Philosophical Reflections on Technology, Agency, and Design . Penn State Press.[143] Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-Vainio-Mattila. 2010. Userexperience evaluation methods: current state and development needs. In
Proceedings of the 6th Nordic conference on human-computer interaction:Extending boundaries . 521–530.[144] Kari Edison Watkins, Alan Borning, G Scott Rutherford, Brian Ferris, and Brian Gill. 2013. Attitudes of bus operators towards real-time transitinformation tools. , 961–980 pages. https://doi.org/10.1007/s11116-013-9450-0[145] Anne-Marie Willis. 2006. Ontological Designing.
Design Philosophy Papers
4, 2 (June 2006), 69–92. https://doi.org/10.2752/144871306X13966268131514[146] Jill Palzkill Woelfer, Amy Iverson, David G Hendry, Batya Friedman, and Brian T Gill. 2011. Improving the safety of homeless young peoplewith mobile phones: values, form and function. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC,Canada) (CHI ’11) . Association for Computing Machinery, New York, NY, USA, 1707–1716. https://doi.org/10.1145/1978942.1979191[147] Marty J Wolf, Don Gotterbarn, and Michael S Kirkpatrick. 2019. ACM Code of Ethics: Looking Back and Forging Ahead. In
Proceedings ofthe 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE ’19) . ACM, New York, NY, USA, 801–802.https://doi.org/10.1145/3287324.3287519[148] Richmond Y Wong. 2021. Using Design Fiction Memos to Analyze UX Professionals’ Values Work Practices: A Case Study Bridging Ethnographic andDesign Futuring Methods. In
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . https://doi.org/10.1145/3411764.344570927 hivukula, et al. [149] Richmond Y Wong, Deirdre K Mulligan, Ellen Van Wyk, James Pierce, and John Chuang. 2017. Eliciting Values Reflections by Engaging PrivacyFutures Using Design Workbooks.
Proc. ACM Hum. -Comput. Interact.
1, CSCW (Dec. 2017), 111:1–111:26. https://doi.org/10.1145/3134746[150] Richmond Y Wong and Tonya Nguyen. 2021. Timelines: A World-Building Activity for Values Advocacy. In
Proceedings of the 2021 CHI Conferenceon Human Factors in Computing Systems . https://doi.org/10.1145/3411764.3445447[151] Daisy Yoo. 2017. Stakeholder Tokens: A Constructive Method for Value Sensitive Design Stakeholder Analysis. In
Proceedings of the 2017 ACMConference Companion Publication on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS ’17 Companion) . Association for ComputingMachinery, New York, NY, USA, 280–284. https://doi.org/10.1145/3064857.3079161[152] Daisy Yoo. 2018. Stakeholder Tokens: a constructive method for value sensitive design stakeholder analysis.
Ethics Inf. Technol. (2018), 1–5.[153] Daisy Yoo, Katie Derthick, Shaghayegh Ghassemian, Jean Hakizimana, Brian Gill, and Batya Friedman. 2016. Multi-lifespan Design Thinking: TwoMethods and a Case Study with the Rwandan Diaspora. In
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (SanJose, California, USA) (CHI ’16) . Association for Computing Machinery, New York, NY, USA, 4423–4434. https://doi.org/10.1145/2858036.2858366[154] Daisy Yoo, Alina Huldtgren, Jill Palzkill Woelfer, David G Hendry, and Batya Friedman. 2013. A value sensitive action-reflection model: evolving aco-design space with stakeholder and designer prompts. In
Proceedings of the SIGCHI conference on human factors in computing systems . 419–428.[155] Meg Young, Lassana Magassa, and Batya Friedman. 2019. Toward inclusive tech policy design: a method for underrepresented voices to strengthentech policy documents.