Yoram Reich

Evaluating machine learning models for engineering problems

The use of machine learning (ML), and in particular, artificial neural networks (ANN), in engineering applications has increased dramatically over the last years. However, by and large, the development of such applications or their report lack proper evaluation. Deficient evaluation practice was observed in the general neural networks community and again in engineering applications through a survey we conducted of articles published in AI in Engineering and elsewhere. This status hinders understanding and prevents progress. This article goal is to remedy this situation. First, several evaluation methods are discussed with their relative qualities. Second, these qualities are illustrated by using the methods to evaluate ANN performance in two engineering problems. Third, a systematic evaluation procedure for ML is discussed. This procedure will lead to better evaluation of studies, and consequently to improved research and practice in the area of ML in engineering applications.

Varieties and issues of participation and design

Abstract Participatory design is the antithesis of traditional design in which designers are expected to exhibit their expertise. The right to participate in design is often ignored and even when it is accepted, many obstacles including perceived pragmatic/economic deficiencies and organizational concerns, impede participation. This paper critizes the foundations of traditional design and elaborates some features of participation in various design disciplines particularly in the context of architectural design and urban planning. An approach to participation founded on widening communication channels among participants is presented. Finally, the potential applications of computer tools for supporting participation are discussed.

A critical review of General Design Theory

This study is a critical review of General Design Theory (GDT), a mathematical theory of design. It reviews the assumptions (axioms) and predictions (theorems) of GDT with respect to design and illustrates them with simple examples. The scope of GDT with respect to design, the guidelines it provides for building computer-aided design (CAD) systems, and the possibility of implementing these guidelines are examined. GDT assumptions are too restrictive to apply directly to design, and several potential avenues for modifying the theory to attempt to broaden its scope are discussed. Nevertheless, these modifications may not lead to proving strong predictions about design. Treating GDT as a model, rather than as an accurate reflection of design, allows treating the guidelines as hypotheses to be tested empirically. The article discusses these guidelines and some experimental implementations that embody some of them.

The formation and use of abstract concepts in design

Publisher Summary This chapter explains the formation and use of abstract concepts in design. The principal focus of this chapter is on acquiring expert design knowledge. One promising solution uses inductive methods for supervised and unsupervised concept learning. The chapter discusses the relationships between these learning strategies and design. Design is a complex task. The chapter focuses on the synthesis aspect of design, namely, the assembly of candidate designs from an infinite set of alternatives. The synthesis is an ill-structured process with respect to all three characteristics of such problems. The only process that can reduce the complexity of synthesis is the acquisition and reorganization of knowledge. The chapter presents BRIDGER, which is a concept formation system that partially automates the acquisition of design knowledge for a class of design domains in which the structure of the artifact is fixed. Ideally, learning should render synthesis more efficient by allowing a design system to construct designs appropriate to the specification predicates.

Equations aren’t enough: informal modeling in design

Arguing that design is a social process, we expand the meaning of modeling and analysis to include all activities facilitating continual refinement and criticism of the design requirements, process, and solutions. We do not assume any a priori methods for modeling or analysis; rather, we provide a framework and an approach to study designers and give them whatever modeling and analysis capabilities they choose. Our approach is the basis for a support tool, /i-dim, currently under development. 1 The Objective of Modeling and Analysis Design as a social process involving designers, customers, and other participants consists of creating and refining a shared meaning of requirements and potential solutions through continual negotiations, discussions, clarifications, and evaluations. This shared meaning, crystalized as the design artifact and made persistent as shared memory forms the basis of accumulated experience upon which subsequent designs draw. Therefore, design requires support for the following activities: negotiating to establish shared meaning, maintaining and refining the components of the shared meaning, and maintaining and accessing prior information constituting fragments of shared memory. All these requirements are facilitated through iterative modeling and analysis (MA) activities of various forms. If the information about these MA activities is maintained property, the development of shared meanings can be incremental. Therefore, MA activities can rely on previous experience, instead of being rc-invcnied each time, and pitfalls typically encountered in MA can be avoided. In the process of reaching this shared meaning, both modeling and analysis take place, albeit often in an informal and inchoate fashion. For instance, when two designers interact, their exchange involves a particular aspect of the design that is modeled in their discussion. A question posed by one designer constitutes modeling and the response an analysis. Often, the focus of the discussion or negotiation drifts marking the use of several models which, while possibly loosely connected, are nevertheless invaluable for the negotiation. Therefore, to benefit from past models arising in collaborative processes, the information derived from previous negotiations between designers needs to be maintained. Access to information from previous, analogically related, design situations is a basic requirement for improving design. In fact, the very act of accessing and applying previous information implies a model of past information and requires models and analyses of the present. To illustrate, if designers create a quexy to retrieve pans from a database for satisfying a specific function, they model the functionality required using a relatively small set of parameters related to, and perhaps derived from, past models. If the query retrieves useful pans, the analysis was successful and the modeling appropriate. If the query fails, knowledge about the failure constitutes valuable information as well. Consequently; it is necessary that not only successes but also that failures be MA activities manifest in negotiation and information retrieval are by and large informal, as opposed to formal modeling via models cast in mathematical form as traditionally conceived of in engineering.

Designing the process design process

Abstract We suggest that designing design processes is an ill-posed problem which must be tackled with great care and in an evolutionary fashion. We argue it is an important activity, however, as companies today use a small percentage of the intellectual capital they own when designing, suggesting there is room for significant improvement. We discuss who in industry and academia are currently involved with designing design processes. Based on empirical studies we and others have carried out, we have based our approach to study and support design processes on managing the information they generate and use. We are learning how to carry out studies more effectively with industrial partners, what features we need for managing information to study and improve design processes. We are even learning some general observations about the effect of different behavior of the group on its success at designing.

From DSM-Based Planning to Design Process Simulation: A Review of Process Scheme Logic Verification Issues

Planning product development processes (PDP), and particularly new product development (NPD) processes, is complex and challenging. The plan should reflect the product-related knowledge, including the influences of performing changes in one product component on the need to rework the design of other components. Given the complexity, dynamics, and uncertainties of design processes (DPs), the plan evaluation requires simulation tools. The design structure matrix (DSM) is a known method for DP planning. However, the DSM itself does not express all the relevant information required for defining process logic. Many logic interpretations are applicable in different business cases; yet, a consistent method of transforming a DSM-based plan to a logically correct concurrent process model in the case of iterative activities is lacking. A gap was identified between the literature concerning activities sequencing based on DSM and the process modeling literature concerning process verification. This survey systematically classifies the approaches used in DSM-based process planning, and discusses their strengths and limitations with problems related to process modeling logic verification of iterative processes. Demonstration of the logic differences emphasizes the need for simulation-based decision making according to the specific process attributes.

Building Agility for Developing Agile Design Information Systems

Abstract:Agile manufacturing relies heavily on the quality of information that organizations have and on their ability to organize and reuse it. Constant inflow of information and knowledge is the fuel of agile manufacturing. In the process of forming virtual enterprises, these new organizations have to be equipped with information systems that integrate their present legacy technology and improve upon it. To support the quick formation of virtual organizations, one must have the ability to develop such systems quickly. Over the past few years we have evolved, through collaborative projects with industry, an approach composed of methods and an information infrastructure calledn-dim that improves the ability of becoming agile manufacturers of information systems, by responding quickly to information needs of new and evolving organizations. Following an analysis of the requirements of information systems for agile design, we discuss this approach; describe some of the infrastructure features; and present several examples of simple applications that illustrate them. We summarize by discussing the advantages and limitations of our approach.

Progressive sharing of modules among product variants

Abstract Recent market transition from mass production to mass customization forces manufacturers to design products that meet individual requirements. In order to address the high cost of this practice, manufacturers develop product families with a common platform, whose variants are designed to meet different customer demands. Parallel to this transition, the dynamics of the market forces designers to develop products composed of modules that are standardized as much as possible across products, thus can be more resilient than complete designs in a changing world. Starting from an original set of different components, our method designs a modular common platform and additional modules, shared by subsets of the designs, from which variants are composed. We applied the method to the layout design of a set of products. Consequently, the geometric aspect of the product family optimization is emphasized, but functional aspects related to the product features and to customer needs are also addressed due to their manifestation in the layout. The design search space is explored using shape grammar rules that alter component geometry and therefore, functionality. The search for optimal design is performed using simulated annealing. Given different objective formulations or parameter settings, the method can be used to explore the solution space. A simple example problem demonstrates the feasibility of the method.

Decomposing the problem of constrained surface fitting in reverse engineering

Abstract This paper presents a practical solution for surface fitting problems with prioritized geometry constraints in reverse engineering. The approach allows prioritizing constraints and uses them for decomposing the problem into a set of sequentially solved, manageable sub-problems. The result of each solution step is trade-off between satisfying the set of constraints and fitting of the surfaces to the measured points. The overall solution process trades off solution quality with complexity of the problem. Solution quality is checked against pre-defined tolerances assigned to the geometry constraints. Results on a benchmark problem demonstrate the suitability of the approach to solving large problems with hundreds of surfaces and constraints.