Mahmoud Dinar

Empirical Studies of Designer Thinking: Past, Present, and Future

Understanding how designers think is core to advancing design methods, tools, and outcomes. Engineering researchers have effectively turned to cognitive science approaches to studying the engineering design process. Empirical methods used for studying designer thinking have included verbal protocols, case studies, and controlled experiments. Studies have looked at the role of design methods, strategies, tools, environment, experience, and group dynamics. Early empirical studies were casual and exploratory with loosely defined objectives and informal analysis methods. Current studies have become more formal, factor controlled, aiming at hypothesis testing, using statistical design of experiments (DOE) and analysis methods such as analysis of variations (ANOVA). Popular pursuits include comparison of experts and novices, identifying and overcoming fixation, role of analogies, effectiveness of ideation methods, and other various tools. This paper first reviews a snapshot of the different approaches to study designers and their processes. Once the current basis is established, the paper explores directions for future or expanded research in this rich and critical area of designer thinking. A variety of data may be collected, and related to both the process and the outcome (designs). But there are still no standards for designing, collecting and analyzing data, partly due to the lack of cognitive models and theories of designer thinking. Data analysis is tedious and the rate of discoveries has been slow. Future studies may need to develop computer based data collection and automated analyses, which may facilitate collection of massive amounts of data with the potential of rapid advancement of the rate of discoveries and development of designer thinking cognitive models. The purpose of this paper is to provide a roadmap to the vast literature for the benefit of new researchers, and also a retrospective for the community. [DOI: 10.1115/1.4029025]

Empirical Studies of Design Thinking: Past, Present, Future

Empirical methods used for studying design thinking have included verbal protocols, case studies, and controlled experiments. Studies have looked at the role of design methods, strategies, tools, environment, experience, and group dynamics. Early empirical studies were casual and exploratory with loosely defined objectives and informal analysis methods. Current studies have become more formal, factor controlled, aiming at hypothesis testing, using statistical DOE and analysis methods such as ANOVA. Popular pursuits include comparison of experts and novices, identifying and overcoming fixation, role of analogies, effectiveness of ideation methods, and other various tools. A variety of data may be collected, related to both the process and the outcome (designs).There are still no standards for designing, collecting and analyzing data, partly due to the lack of cognitive models and theories of design thinking. Data analysis is tedious and the rate of discoveries has been slow. Future studies may need to develop computer based data collection and automated analyses, which may facilitate collection of massive amounts of data with the potential of rapid advancement of the rate of discoveries and development of cognitive models of design thinking.Copyright

Towards a Formal Representation Model of Problem Formulation in Design

Studies on design, show that problem formulation plays a major role in creative design. We plan to construct an interactive computer system that aids problem formulation. In the current stage, to improve our understanding of problem formulation, we have conducted exploratory protocol studies of novice designers and collected data from an expert designer in the form of a depositional interview. A formal representation of the design problem is needed to improve our empirical investigation. We propose a preliminary framework for such a model and we call it a problem map. It provides a basis for comparing how different designers perceive a problem. Our study is based on the design of a model aircraft for the AIAA student design competition. This preliminary analysis shows the evolution of the problem and the solution spaces in the elaboration of the problem maps through time. The problem maps also show a richer representation of attended attributes and relations for the expert and more attributes left in vacuum for the novices.© 2011 ASME

Problem Map: An Ontological Framework for a Computational Study of Problem Formulation in Engineering Design

Studies of design cognition often face two challenges. One is a lack of formal cognitive models of design processes that have the appropriate granularity: fine enough to distinguish differences among individuals and coarse enough to detect patterns of similar actions. The other is the inadequacies in automating the recourse-intensive analyses of data collected from large samples of designers. To overcome these barriers, we have developed the problem map (P-maps) ontological framework. It can be used to explain design thinking through changes in state models that are represented in terms of requirements, functions, artifacts, behaviors, and issues. The different ways these entities can be combined, in addition to disjunctive relations and hierarchies, support detailed modeling and analysis of design problem formulation. A node‐link representation of P-maps enables one to visualize how a designer formulates a problem or to compare how different designers formulate the same problem. Descriptive statistics and time series of entities provide more detailed comparisons. Answer set programming (ASP), a predicate logic formalism, is used to formalize and trace strategies that designers adopt. Data mining techniques (association rule and sequence mining) are used to search for patterns among large number of designers. Potential uses of P-maps are computer-assisted collection of large data sets for design research, development of a test for the problem formulation skill, and a tutoring system. [DOI: 10.1115/1.4030076]

A Computational Aid for Problem Formulation in Early Conceptual Design

Conceptual design is a high-level cognitive activity that draws upon distinctive human mental abilities. An early and fundamental part of the design process is problem formulation, in which designers determine the structure of the problem space they will later search. Although many tools have been developed to aid the later stages of design, few tools exist that aid designers in the early stages. In this paper, we describe Problem Formulator, an interactive environment that focuses on this stage of the design process. This tool has representations and operations that let designers create, visualize, explore, and reflect on their formulations. Although this process remains entirely under the user’s control, these capabilities make the system well positioned to aid the early stages of conceptual design. [DOI: 10.1115/1.4024714]

The Structure of Creative Design: What Problem Maps Can Tell Us About Problem Formulation and Creative Designers

Problem formulation is an important part of the design process that has been largely underexplored. Similarly, the relationship between how designers formulate problems and creative outcome is not well understood. To shed light on what the process of problem formulation can tell us about creativity in design, we use the problem map model ‐ a flexible, domainindependent ontology for modeling the design formulation process ‐ to analyze protocols from eight expert designers. In this paper, we discuss the effectiveness of using problem maps for coding design protocols and what the problem map model can tell us about the protocols of designers. In this exploratory study, we use the problem map model to code and analyze the problem formulation stage of the design process.

Beyond Function-Behavior-Structure

Our research is investigating the relationship between design problem formulation and creative outcome. Our research is investigating the relationship between design problem formulation and creative outcome. Towards that goal we have conducted experiments with designers engaged in problem formulation. In order to analyze such empirical data, a formal representation is needed. One popular model is Function-Behavior-Structure (FBS) and its several variants. Our problem map (P-map) model shares many features with FBS but also has important differences. We introduce a hierarchical representation not only in each of the F, B, S domains but in additional domains (requirements and issues). We also identify generic inter and intra-domain relationships between these entities, leading to a more expressive and flexible model that is domain independent and well suited for representing problem formulations of designers with different expertise levels and creativity. We have used the model for coding protocol data in a formal predicate logic language (Answer Set Prolog).

Evaluation of Empirical Design Studies and Metrics

Engineering design is a complex multifaceted and knowledge-intensive process. No single theory or model can capture all aspects of such an activity. Various empirical methods have been used by researchers to study particular aspects of design thinking and cognition, design processes, design artefacts, and design strategies. Research methods include think-aloud protocol analysis and its many variants, case studies, controlled experiments of design cognition, and fMRI. The field has gradually progressed from subjective to objective analyses, requiring well-defined metrics since design of experiments (DOE) involves controlling or blocking particular variables. DOE also requires setting experiment variables at particular levels, which means that each variable needs to be characterized and quantified. Without such quantification, statistical analyses cannot be carried out. This chapter focuses on quantifiable characteristics of designers, targeted users, artefacts, and processes.

Patterns of Creative Design: Predicting Ideation From Problem Formulation

The main objective of our research is to understand the role of problem formulation in creative design ideation. To that end, we have used the web-based testbed of the Problem Map (P-map) computational framework which represents designers’ problem formulation in terms of a series of state models, where each state consists of six types of entities in addition to relations within and between different entity types. We gave two design problems to twenty five graduate students in an advanced product design course. We collected their problem formulation data in the P-map testbed and their ideation data through concept sketches. We conducted correlation analysis between variables extracted from the P-maps, and the ideation metrics. We also built regression models for each of the ideation metrics as the dependent variable, and the P-map variables as the independent variables. We used the data from the first problem to predict the ideation scores for the second problem. The predicted results were compared to the actual outcome reported by an independent panel of judges. Models of variety, average and max quality had more accurate predictions while average novelty, average and max quality had statistically more reliable models.Copyright

Enhancing Design Problem Formulation Skills for Engineering Design Students

Problem formulation is an essential design skill for which assessment methods have been less commonly developed. In order to evaluate the progress of a group of graduate students in mechanical engineering design in regard with the problem formulation skill, they were asked to work on three design problems using the Problem Formulator web tool during their course work. Changes in a set of measures elicited from this data were examined in addition to sketches, simulations, and working prototypes. Inventories of requirements and issues, as well as concepts derived from morphological charts were created to assess designers’ skills and outcomes.Copyright