Noe Vargas-Hernandez

Metrics for measuring ideation effectiveness

Abstract Systematic methods for idea generation in engineering design have come about from a variety of sources. Do these methods really aid ideation? Some empirical studies have been conducted by researchers to answer this question. These studies include highly controlled lab experiments by cognitive psychologists, as well as experiments in simulated design environments carried out by engineering design theorists. A key factor in design and analysis of empirical studies is characterization and measurement of ideation effectiveness. This paper describes four objective measures of ideation effectiveness. The theoretical basis of each is discussed and procedures for application of each are outlined and illustrated with case studies.

Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments

A wide range of formal methods have been devised and used for idea generation in conceptual design. Experimental evidence is needed to support claims regarding the effectiveness of these methods in promoting idea generation in engineering design. Towards that goal this paper presents a set of effectiveness metrics experimental methods, data collection and analysis techniques. Statistically based Design of Experiments (DOE) principles were used in developing the guidelines. Four classes of operating variables were considered to characterize the design problem and the environment. The effectiveness metrics proposed are based on outcome and consists of the quantity, quality, novelty, and variety of ideas generated. Two experimental approaches have been developed. In the Direct Method, the influence of the type of design problem and various parameters related to the procedure of an idea generation method is measured by using the method in its entirety. In the Indirect Method, each idea generation method is decomposed into key components and its overall effectiveness is predicted by experimentally studying the effectiveness of its components and their mutual interactions.

EMPIRICAL STUDIES OF DESIGN IDEATION: ALIGNMENT OF DESIGN EXPERIMENTS WITH LAB EXPERIMENTS

Although various Idea Generation (IG) methods exist for conceptual design, the ideation process is still hardly understood. There is a need for a Design Ideation Model that explains the variables and processes occurring during IG. Cognitive Science provides models and theories, but these are usually derived from simple tasks or problems. On the other hand, Design Research simulates real world design better, but experimentation at the design level is time consuming and is difficult to isolate due to interactions of the variables involved. This paper introduces an approach for the alignment of experiments at the design level with lab experiments in cognitive psychology. Two key concepts that make this alignment possible are: ideation components (mechanisms believed to promote IG) recognized in Design Research and Cognitive Science, and uniform measures. The long-term objective of this research is the creation of a Design Ideation Model; this will require the testing and modeling of several of these ideation components. This paper presents results from Design and Lab Experiments for a selected component: incubation. Results are discussed and their significance explained in the context of the Design Ideation Model. This study found that, based on the correlation at both the Lab and Design Experiments, incubation had a positive impact on Design Ideation. Further, the alignment approach followed proved to be appropriate for the individual modeling of ideation components.Copyright

Empirical studies of creative cognition in idea generation

Contents: Series Foreword. Preface. Introduction. Part I: Cognitive Processes of Creative Thinking. S.M. Smith, D.R. Gerkens, J.J. Shah, N. Vargas-Hernandez, Empirical Studies of Creative Cognition in Idea Generation. E.L. Santanen, Opening the Black Box of Creativity: Causal Effects in Creative Solution Generation. J. Goldenberg, D. Nir, E. Maoz, Structuring Creativity: Creative Templates in Negotiation. Part II: Team and Group Dynamics of Brainstorming. P.B. Paulus, T. Nakui, V.L. Putman, Group Brainstorming and Teamwork: Some Rules for the Road to Innovation. H-S. Choi, L.L. Thompson, Membership Change in Groups: Implications for Group Creativity. A.C. Edmondson, J.P. Mogelof, Explaining Psychological Safety in Innovation Teams: Organizational Culture, Team Dynamics, or Personality? M.A. West, C.A. Sacramento, D. Fay, Creativity and Innovation Implementation in Work Groups: The Paradoxical Role of Demands. B.A. Nijstad, E.F. Rietzschel, W. Stroebe, Four Principles of Group Creativity. Part III: Organizational Influences of Creativity and Innovation. D. Ancona, H. Bresman, Begging, Borrowing, and Building on Ideas From the Outside to Create Pulsed Innovation Inside Teams. A.B. Hargadon, Bridging Old Worlds and Building New Ones: Towards a Microsociology of Creativity. C.M. Ford, Creative Associations and Entrepreneurial Opportunities.

Empirical Studies of Design Thinking: Past, Present, Future

Empirical methods used for studying design thinking have included verbal protocols, case studies, and controlled experiments. Studies have looked at the role of design methods, strategies, tools, environment, experience, and group dynamics. Early empirical studies were casual and exploratory with loosely defined objectives and informal analysis methods. Current studies have become more formal, factor controlled, aiming at hypothesis testing, using statistical DOE and analysis methods such as ANOVA. Popular pursuits include comparison of experts and novices, identifying and overcoming fixation, role of analogies, effectiveness of ideation methods, and other various tools. A variety of data may be collected, related to both the process and the outcome (designs).There are still no standards for designing, collecting and analyzing data, partly due to the lack of cognitive models and theories of design thinking. Data analysis is tedious and the rate of discoveries has been slow. Future studies may need to develop computer based data collection and automated analyses, which may facilitate collection of massive amounts of data with the potential of rapid advancement of the rate of discoveries and development of cognitive models of design thinking.Copyright

Multilevel aligned empirical studies of ideation: Final results

The overall objective of the study is to gain an insight into design ideation. Towards that goal we are empirically evaluating the effectiveness of design ideation methods. Key components of ideation methods have been identified and effectiveness metrics have been developed. This paper presents experimental results conducted on six ideation components (Provocative Stimuli, Suspend Judgment, Flexible Representation, Frame of Reference Shifting, Incubation, and Provocative Stimuli). These experiments were conducted simultaneously at the Design (Engineering) and Lab (Cognitive Psychology) levels; a previously developed experimental procedure considered the alignment of experiments at these two levels. The understanding of ideation components was improved (some are stronger, some are easier to manipulate, interactions are complex, etc.). Data collected revealed that some ideation components have similar effects and could be grouped into higher-level (ideation) principles according to their effects. A distinction was made between process and outcome metrics and it was found that outcome metrics were harder to improve than process metrics. A correlation was also found between quality and quantity, this supports the widely accepted belief that generating more ideas increases the chances of obtaining higher quality ideas.Copyright

Design principles for the conceptual design of low-cost prosthetic knee

The objective of this paper is to document design lessons learned in the development of low-cost lower-limb prosthetic devices. Four design principles are introduced based on the authors′ experience redesigning a low-cost prosthetic knee mechanism. These design principles are valuable sources to complete the conceptual design of the knee mechanism in the absence of a unique or exact methodology that comprises all the aspects concerning engineering design.The paper defines the motivation to improve life quality by restoring knee functionality in terms of mobility and stability. Subsequent sections explore the state of the art, design constraints, gait analysis, and 3-D modeling using CAD. The model is validated using kinematic and structural simulation in NX 7.5™ covering basic geometric parameters and motion assumptions. The four conceptual design principles are: Hybrid Design Model, Simultaneous Modeling, Multidisciplinary Pulses, and Built-in and Post-design Optimization. Each principle is presented to exemplify its contribution in the knee mechanism redesign. Finally, the design principles are intended to assist the designer with empirical guidelines in the development of prosthetic devices.© 2013 ASME