Sunith Bandaru

Towards automating the discovery of certain innovative design principles through a clustering-based optimization technique

In this article, a methodology is proposed for automatically extracting innovative design principles which make a system or process (subject to conflicting objectives) optimal using its Pareto-optimal dataset. Such ‘higher knowledge’ would not only help designers to execute the system better, but also enable them to predict how changes in one variable would affect other variables if the system has to retain its optimal behaviour. This in turn would help solve other similar systems with different parameter settings easily without the need to perform a fresh optimization task. The proposed methodology uses a clustering-based optimization technique and is capable of discovering hidden functional relationships between the variables, objective and constraint functions and any other function that the designer wishes to include as a ‘basis function’. A number of engineering design problems are considered for which the mathematical structure of these explicit relationships exists and has been revealed by a previous study. A comparison with the multivariate adaptive regression splines (MARS) approach reveals the practicality of the proposed approach due to its ability to find meaningful design principles. The success of this procedure for automated innovization is highly encouraging and indicates its suitability for further development in tackling more complex design scenarios.

An integrated approach to automated innovization for discovering useful design principles: Case studies from engineering

Computational optimization methods are most often used to find a single or multiple optimal or near-optimal solutions to the underlying optimization problem describing the problem at hand. In this paper, we elevate the use of optimization to a higher level in arriving at useful problem knowledge associated with the optimal or near-optimal solutions to a problem. In the proposed innovization process, first a set of trade-off optimal or near-optimal solutions are found using an evolutionary algorithm. Thereafter, the trade-off solutions are analyzed to decipher useful relationships among problem entities automatically so as to provide a better understanding of the problem to a designer or a practitioner. We provide an integrated algorithm for the innovization process and demonstrate the usefulness of the procedure to three real-world engineering design problems. New and innovative design principles obtained in each case should clearly motivate engineers and practitioners for its further application to more complex problems and its further development as a more efficient data analysis procedure.

Automated discovery of vital knowledge from Pareto-optimal solutions: First results from engineering design

Real world multi-objective optimization problems are often solved with the only intention of selecting a single trade-off solution by taking up a decision-making task. The computational effort and time spent on obtaining the entire Pareto front is thus not justifiable. The Pareto solutions as a whole contain within them a lot more information than that is used. Extracting this knowledge would not only give designers a better understanding of the system, but also bring worth to the resources spent. The obtained knowledge acts as governing principles which can help solve other similar systems easily. We propose a genetic algorithm based unsupervised approach for learning these principles from the Pareto-optimal dataset of the base problem. The methodology is capable of discovering analytical relationships of a certain type between different problem entities.

Data mining methods for knowledge discovery in multi-objective optimization

Four methods are developed for data mining discrete multi-objective optimization datasets.Two of the methods are unsupervised, one is supervised and the other is hybrid.Knowledge is represented as patterns in one method, and as rules in other methods.Methods are applied to three real-world production system optimization problems.Extracted knowledge is compared across methods and provides new insights. The first part of this paper served as a comprehensive survey of data mining methods that have been used to extract knowledge from solutions generated during multi-objective optimization. The current paper addresses three major shortcomings of existing methods, namely, lack of interactiveness in the objective space, inability to handle discrete variables and inability to generate explicit knowledge. Four data mining methods are developed that can discover knowledge in the decision space and visualize it in the objective space. These methods are (i) sequential pattern mining, (ii) clustering-based classification trees, (iii) hybrid learning, and (iv) flexible pattern mining. Each method uses a unique learning strategy to generate explicit knowledge in the form of patterns, decision rules and unsupervised rules. The methods are also capable of taking the decision makers preferences into account to generate knowledge unique to preferred regions of the objective space. Three realistic production systems involving different types of discrete variables are chosen as application studies. A multi-objective optimization problem is formulated for each system and solved using NSGA-II to generate the optimization datasets. Next, all four methods are applied to each dataset. In each application, the methods discover similar knowledge for specified regions of the objective space. Overall, the unsupervised rules generated by flexible pattern mining are found to be the most consistent, whereas the supervised rules from classification trees are the most sensitive to user-preferences.

Automated innovization for simultaneous discovery of multiple rules in bi-objective problems

The trade-off solutions of a multi-objective optimization problem, as a whole, often hold crucial information in the form of rules. These rules, if predominantly present in most trade-off solutions, can be considered as the characteristic features of the Pareto-optimal front. Knowledge of such features, in addition to providing better insights to the problem, enables the designer to handcraft solutions for other optimization tasks which are structurally similar to it; thus eliminating the need to actually optimize. Innovization is the process of extracting these so called design rules. This paper proposes to move a step closer towards the complete automation of the innovization process through a niched clustering based optimization technique. The focus is on obtaining multiple design rules in a single knowledge discovery step using the niching strategy.

A parameterless-niching-assisted bi-objective approach to multimodal optimization

Evolutionary algorithms are becoming increasingly popular for multimodal and multi-objective optimization. Their population based nature allows them to be modified in a way so as to locate and preserve multiple optimal solutions (referred to as Pareto-optimal solutions in multi-objective optimization). These modifications are called niching methods, particularly in the context of multimodal optimization. In evolutionary multiobjective optimization, the concept of dominance and diversity preservation inherently causes niching. This paper proposes an approach to multimodal optimization which combines this power of dominance with traditional variable-space niching. The approach is implemented within the NSGA-II framework and its performance is studied on 20 benchmark problems. The simplicity of the approach and the absence of any special niching parameters are the hallmarks of this study.

Data mining methods for knowledge discovery in multi-objective optimization: Part B - New developments and applications

Abstract The first part of this paper served as a comprehensive survey of data mining methods that have been used to extract knowledge from solutions generated during multi-objective optimization. The current paper addresses three major shortcomings of existing methods, namely, lack of interactiveness in the objective space, inability to handle discrete variables and inability to generate explicit knowledge. Four data mining methods are developed that can discover knowledge in the decision space and visualize it in the objective space. These methods are (i) sequential pattern mining, (ii) clustering-based classification trees, (iii) hybrid learning, and (iv) flexible pattern mining. Each method uses a unique learning strategy to generate explicit knowledge in the form of patterns, decision rules and unsupervised rules. The methods are also capable of taking the decision maker’s preferences into account to generate knowledge unique to preferred regions of the objective space. Three realistic production systems involving different types of discrete variables are chosen as application studies. A multi-objective optimization problem is formulated for each system and solved using NSGA-II to generate the optimization datasets. Next, all four methods are applied to each dataset. In each application, the methods discover similar knowledge for specified regions of the objective space. Overall, the unsupervised rules generated by flexible pattern mining are found to be the most consistent, whereas the supervised rules from classification trees are the most sensitive to user-preferences.

Higher and lower-level knowledge discovery from Pareto-optimal sets

Innovization (innovation through optimization) is a relatively new concept in the field of multi-objective engineering design optimization. It involves the use of Pareto-optimal solutions of a problem to unveil hidden mathematical relationships between variables, objectives and constraint functions. The obtained relationships can be thought of as essential properties that make a feasible solution Pareto-optimal. This paper proposes two major extensions to innovization, namely higher-level innovization and lower-level innovization. While the former deals with the discovery of common features among solutions from different Pareto-optimal fronts, the latter concerns features commonly occurring among solutions that belong to a specified (or preferred) part of the Pareto-optimal front. The knowledge of such lower-level information is extremely beneficial to a decision maker, since it focuses on a preferred set of designs. On the other hand, higher-level innovization reveals interesting knowledge about the general problem structure. Neither of these crucial aspects concerning multi-objective designs has been addressed before, to the authors’ knowledge. We develop methodologies for handling both levels of innovization by extending the authors’ earlier automated innovization algorithm and apply them to two well-known engineering design problems. Results demonstrate that the proposed methodologies are generic and are ready to be applied to other engineering design problems.

Higher-level innovization: A case study from Friction Stir Welding process optimization

The task of finding crucial design interdependencies in the form of mathematical relationships (empirical or otherwise) in an engineering design problem using the Pareto-optimal front is referred to as innovization. Past studies on the subject have limited themselves to a single front. In this paper we introduce the higher-level innovization task through an application of a manufacturing process simulation for the Friction Stir Welding (FSW) process where commonalities among two different Pareto-optimal fronts are analyzed. Multiple design rules are simultaneously deciphered from each front separately and compared. Important design aspects of the FSW problem are revealed in the process. The overall study aims at showing how some design principles can considerably ease the task of optimizing future enhancements to the design.

Generalized higher-level automated innovization with application to inventory management

This paper generalizes the automated innovization framework using genetic programming in the context of higher-level innovization. Automated innovization is an unsupervised machine learning technique that can automatically extract significant mathematical relationships from Pareto-optimal solution sets. These resulting relationships describe the conditions for Pareto-optimality for the multi-objective problem under consideration and can be used by scientists and practitioners as thumb rules to understand the problem better and to innovate new problem solving techniques; hence the name innovization (innovation through optimization). Higher-level innovization involves performing automated innovization on multiple Pareto-optimal solution sets obtained by varying one or more problem parameters. The automated innovization framework was recently updated using genetic programming. We extend this generalization to perform higher-level automated innovization and demonstrate the methodology on a standard two-bar bi-objective truss design problem. The procedure is then applied to a classic case of inventory management with multi-objective optimization performed at both system and process levels. The applicability of automated innovization to this area should motivate its use in other avenues of operational research.