Alan J. Barton

Visualizing high dimensional objective spaces for multi-objective optimization: A virtual reality approach

This paper presents an approach for constructing visual representations of high dimensional objective spaces using virtual reality. These spaces arise from the solution of multi-objective optimization problems with more than 3 objective functions which lead to high dimensional Pareto fronts which are difficult to use. This approach is preliminarily investigated using both theoretically derived high dimensional Pareto fronts for a test problem (DTLZ2) and practically obtained objective spaces for the 4 dimensional knapsack problem via multi-objective evolutionary algorithms like HLGA, NSGA, and VEGA. The expected characteristics of the high dimensional fronts in terms of relative sizes, sequencing, embedding and asymmetry were systematically observed in the constructed virtual reality spaces.

Exploring medical data using visual spaces with genetic programming and implicit functional mappings

Two medical data sets (Breast cancer and Colon cancer) are investigated within a visual data mining paradigm through the unsupervised construction of virtual reality spaces using genetic programming and classical optimization (for comparison purposes). The desired visual spaces are such that a modified genetic programming approach was proposed in order to generate programs representing vector functions. The extension leads to populations that are composed of forests, instead of single expression trees. No particular kind of genetic programming algorithm is required due to the generic nature of the approach taken in the paper. The results (visual spaces) show that the relationships between the data objects and their classes can be appreciated in all of the obtained spaces regardless of the mapping error. In addition, the spaces obtained with genetic programming resulted in lower mapping errors than a classical optimizer and produced relatively simple equations. Further, the set of obtained equations can be statistically analyzed in terms of the original attributes in order to further the understanding of the derivation of the new nonlinear features that are constructed. Thus, explicit mappings provided by genetic programming can be used for feature selection and generation in data mining where scalar and/or vector functions are involved.

Virtual Reality Spaces for Visual Data Mining with Multiobjective Evolutionary Optimization: Implicit and Explicit Function Representations Mixing Unsupervised and Supervised Properties

Multi-objective optimization is used for the computation of virtual reality spaces for visual data mining and knowledge discovery. Two methods for computing new spaces are discussed: implicit and explicit function representations. In the first, the images of the objects are computed directly, and in the second, universal function approximators (neural networks) are obtained. The pros and cons of each approach are discussed, as well as their complementary character. The NSGA-II algorithm is used for computing spaces requested to minimize two objectives: a similarity structure loss measure (Sammons error) and classification error (mean cross-validation error on a k-nn classifier). Two examples using solutions along approximations to the Pareto front are presented: Alzheimers disease gene expressions and geophysical fields for prospecting underground caves. This approach is a general non-linear feature generation and can be used in problems not necessarily oriented to the construction of visual data representations.

Relevant attribute discovery in high dimensional data: application to breast cancer gene expressions

In many domains, the data objects are described in terms of a large number of features. The pipelined data mining approach introduced in [1] using two clustering algorithms in combination with rough sets and extended with genetic programming, is investigated with the purpose of discovering important subsets of attributes in high dimensional data. Their classification ability is described in terms of both collections of rules and analytic functions obtained by genetic programming (gene expression programming). The Leader and several k-means algorithms are used as procedures for attribute set simplification of the information systems later presented to rough sets algorithms. Visual data mining techniques including virtual reality were used for inspecting results. The data mining process is setup using high throughput distributed computing techniques. This approach was applied to Breast Cancer microarray data and it led to subsets of genes with high discrimination power with respect to the decision classes

Gene discovery in leukemia revisited: a computational intelligence perspective

One of the difficulties of using Artificial Neural Networks (ANNs) to estimate atmospheric temperature is the large number of potential input variables available. In this study, four different feature extraction methods were used to reduce the input vector to train four networks to estimate temperature at different atmospheric levels. The four techniques used were: genetic algorithms (GA), coefficient of determination (CoD), mutual information (MI) and simple neural analysis (SNA). The results demonstrate that of the four methods used for this data set, mutual information and simple neural analysis can generate networks that have a smaller input parameter set, while still maintaining a high degree of accuracy.

Virtual reality high dimensional objective spaces for multi-objective optimization: An improved representation

This paper presents an approach for constructing improved visual representations of high dimensional objective spaces using virtual reality. These spaces arise from the solution of multi-objective optimization problems with more than 3 objective functions which lead to high dimensional Pareto fronts. The 3-D representations of m-dimensional Pareto fronts, or their approximations, are constructed via similarity structure mappings between the original objective spaces and the 3-D space. Alpha shapes are introduced for the representation and compared with previous approaches based on convex hulls. In addition, the mappings minimizing a measure of the amount of dissimilarity loss are obtained via genetic programming. This approach is preliminarily investigated using both theoretically derived high dimensional Pareto fronts for a test problem (DTLZ2) and practically obtained objective spaces for the 4 dimensional knapsack problem via multi-objective evolutionary algorithms like HLGA, NSGA, and VEGA. The improved representation captures more accurately the real nature of the m-dimensional objective spaces and the quality of the mappings obtained with genetic programming is equivalent to those computed with classical optimization algorithms.

Virtual reality visual data mining with nonlinear discriminant neural networks: application to leukemia and Alzheimer gene expression data

A hybrid stochastic-deterministic approach for solving NDA problems on very high dimensional biological data is investigated. It is based on networks trained with a combination of simulated annealing and conjugate gradient within a broad scale, high throughput computing data mining environment. High quality networks from the point of view of both discrimination and generalization capabilities are discovered. The NDA mappings generated by these networks, together with unsupervised representations of the data, lead to a deeper understanding of complex high dimensional data like leukemia and Alzheimer gene expression microarray experiments.

Data and knowledge visualization with virtual reality spaces, neural networks and rough sets: Application to cancer and geophysical prospecting data

Visual data mining with virtual reality spaces is used for the representation of data and symbolic knowledge. High quality structure-preserving and maximally discriminative visual representations can be obtained using a combination of neural networks (SAMANN and NDA) and rough sets techniques, so that a proper subsequent analysis can be made. The approach is illustrated with two types of data: for gene expression cancer data, an improvement in classification performance with respect to the original spaces was obtained; for geophysical prospecting data for cave detection, a cavity was successfully predicted.

Multi-objective Evolutionary Optimization of Neural Networks for Virtual Reality Visual Data Mining: Application to Hydrochemistry

A method for the construction of virtual reality spaces for visual data mining using multi-objective optimization with genetic algorithms on neural networks is presented. Two neural network layers (output and last hidden) are used for the construction of simultaneous solutions for: a supervised classification of data patterns and the computation of two unsupervised similarity structure preservation measures between the original data matrix and its image in the new space. A set of spaces is constructed from selected solutions along the Pareto front which enables the understanding of the internal properties of the data based on visual inspection of non-dominating spaces with different properties. This strategy represents a conceptual improvement over spaces computed by single-objective optimization. The presented approach is domain independent and is illustrated with an application to the study of hydrochemical properties of ice and water samples from the Arctic.

2007 Special Issue: Multi-objective evolutionary optimization for constructing neural networks for virtual reality visual data mining: Application to geophysical prospecting

A method for the construction of virtual reality spaces for visual data mining using multi-objective optimization with genetic algorithms on nonlinear discriminant (NDA) neural networks is presented. Two neural network layers (the output and the last hidden) are used for the construction of simultaneous solutions for: (i) a supervised classification of data patterns and (ii) an unsupervised similarity structure preservation between the original data matrix and its image in the new space. A set of spaces are constructed from selected solutions along the Pareto front. This strategy represents a conceptual improvement over spaces computed by single-objective optimization. In addition, genetic programming (in particular gene expression programming) is used for finding analytic representations of the complex mappings generating the spaces (a composition of NDA and orthogonal principal components). The presented approach is domain independent and is illustrated via application to the geophysical prospecting of caves.