Georgios C. Anagnostopoulos

A branch-and-bound algorithm for the early/tardy machine scheduling problem with a common due-date and sequence-dependent setup time

The single-machine early/tardy (E/T) scheduling problem is addressed in this research. The objective of this problem is to minimize the total amount of earliness and tardiness. Earliness and tardiness, are weighted equally and the due date is common and large (unrestricted) for all jobs. Machine setup time is included and is considered sequence-dependent. When sequence-dependent setup times are included, the complexity of the problem increases significantly and the problem becomes NP-hard. In the literature, only mixed integer programming formulation is available to optimally solve the problem at hand. In this paper, a branch-and-bound algorithm (B&B) is developed to obtain optimal solutions for the problem. As it will be shown, the B&B solved problems three times larger than what has been reported in the literature.

Multiclass Cancer Classification Using Semisupervised Ellipsoid ARTMAP and Particle Swarm Optimization with Gene Expression Data

It is crucial for cancer diagnosis and treatment to accurately identify the site of origin of a tumor. With the emergence and rapid advancement of DNA microarray technologies, constructing gene expression profiles for different cancer types has already become a promising means for cancer classification. In addition to research on binary classification such as normal versus tumor samples, which attracts numerous efforts from a variety of disciplines, the discrimination of multiple tumor types is also important. Meanwhile, the selection of genes which are relevant to a certain cancer type not only improves the performance of the classifiers, but also provides molecular insights for treatment and drug development. Here, we use Semisupervised Ellipsoid ARTMAP (ssEAM) for multiclass cancer discrimination and particle swarm optimization for informative gene selection. ssEAM is a neural network architecture rooted in Adaptive Resonance Theory and suitable for classification tasks. ssEAM features fast, stable, and finite learning and creates hyperellipsoidal clusters, inducing complex nonlinear decision boundaries. PSO is an evolutionary algorithm-based technique for global optimization. A discrete binary version of PSO is employed to indicate whether genes are chosen or not. The effectiveness of ssEAM/PSO for multiclass cancer diagnosis is demonstrated by testing it on three publicly available multiple-class cancer data sets. ssEAM/PSO achieves competitive performance on all these data sets, with results comparable to or better than those obtained by other classifiers.

A Scalable and Efficient Outlier Detection Strategy for Categorical Data

Outlier detection has received significant attention in many applications, such as detecting credit card fraud or network intrusions. Most existing research focuses on numerical datasets, and cannot directly apply to categorical sets where there is little sense in calculating distances among data points. Furthermore, a number of outlier detection methods require quadratic time with respect to the dataset size and usually multiple dataset scans. These characteristics are undesirable for large datasets, potentially scattered over multiple distributed sites. In this paper, we introduce Attribute Value Frequency (A VF), a fast and scalable outlier detection strategy for categorical data. A VF scales linearly with the number of data points and attributes, and relies on a single data scan. AVF is compared with a list of representative outlier detection approaches that have not been contrasted against each other. Our proposed solution is experimentally shown to be significantly faster, and as effective in discovering outliers.

Ellipsoid ART and ARTMAP for incremental clustering and classification

We introduce ellipsoid-ART (EA) and ellipsoid-ARTMAP (EAM) as a generalization of hypersphere ART (HA) and hypersphere-ARTMAP (HAM) respectively. As was the case with HA/HAM, these novel architectures are based on ideas rooted in fuzzy-ART (FA) and fuzzy-ARTMAP (FAM). While FA/FAM aggregate input data using hyper-rectangles, EA/EAM utilize hyper-ellipsoids for the same purpose. Due to their learning rules, EA and EAM share virtually all properties and characteristics of their FA/FAM counterparts. Preliminary experimentation implies that EA and EAM are to be viewed as good alternatives to FA and FAM for data clustering and classification tasks respectively.

Hypersphere ART and ARTMAP for unsupervised and supervised, incremental learning

A novel adaptive resonance theory (ART) neural network architecture is being proposed. The new model, called Hypersphere ART (H-ART) is based on the same principals as Fuzzy-ART and, thus, inherits most of its qualities for unsupervised learning. Among these properties is fast, stable, incremental learning on the training set and good generalization on the testing set. While H-ART is intended for clustering tasks, its extension, H-ARTMAP is playing the role of Fuzzy-ARTMAPs counterpart for the supervised learning of real-valued, multi-dimensional mappings. Also in this paper, some experimental results are presented involving the comparison of H-ARTMAP and Fuzzy-ARTMAP in simple, illustrative classification problems. The results are indicating comparable performances in error rate but also a good potential for substantial superiority of H-ARTMAP in terms of nodes (categories) utilized. The latter effect can be attributed to H-ARTs more efficient internal knowledge representation.

Ellipsoid ART and ARTMAP for incremental unsupervised and supervised learning

We introduce Ellipsoid-ART, EA and Ellipsoid-ARTMAP, EAM as a generalization of Hyper-sphere ART and Hypersphere-ARTMAP respectively. Our novel archetectures are based on ideas rooted on Fuzzy-ART, FA and Fuzzy-ARTMAP, FAM. While FA/FAM summarize input data using hyper-rectangles, EA/EAM utilize hyper-ellipsoids for the same purpose. Due to their learning rules, EA and EAM share virtually all properties and characteristics of their FA/FAM counterparts. Preliminary experimentation implies that EA and EAM are to be viewed as good alternatives to FA and FAM for data clustering and classification tasks. Extensive pseudo-code is provided in the appendices for computationally efficient implementations of EA/EAM training and performance phases.

Category regions as new geometrical concepts in Fuzzy-ART and Fuzzy-ARTMAP

In this paper we introduce novel geometric concepts, namely category regions, in the original framework of Fuzzy-ART (FA) and Fuzzy-ARTMAP (FAM). The definitions of these regions are based on geometric interpretations of the vigilance test and the F2 layer competition of committed nodes with uncommitted ones, that we call commitment test. It turns out that not only these regions have the same geometrical shape (polytope structure), but they also share a lot of common and interesting properties that are demonstrated in this paper. One of these properties is the shrinking of the volume that each one of these polytope structures occupies, as training progresses, which alludes to the stability of learning in FA and FAM, a well-known result. Furthermore, properties of learning of FA and FAM are also proven utilizing the geometrical structure and properties that these regions possess; some of these properties were proven before using counterintuitive, algebraic manipulations and are now demonstrated again via intuitive geometrical arguments. One of the results that is worth mentioning as having practical ramifications is the one which states that for certain areas of the vigilance-choice parameter space (rho,a), the training and performance (testing) phases of FA and FAM do not depend on the particular choices of the vigilance parameter. Finally, it is worth noting that, although the idea of the category regions has been developed under the premises of FA and FAM, category regions are also meaningful for later developed ART neural network structures, such as ARTEMAP, ARTMAP-IC, Boosted ARTMAP, Micro-ARTMAP, Ellipsoid-ART/ARTMAP, among others.

Cross-validation in Fuzzy ARTMAP for large databases

In this paper we are examining the issue of overtraining in Fuzzy ARTMAP. Over-training in Fuzzy ARTMAP manifests itself in two different ways: (a) it degrades the generalization performance of Fuzzy ARTMAP as training progresses; and (b) it creates unnecessarily large Fuzzy ARTMAP neural network architectures. In this work, we are demonstrating that overtraining happens in Fuzzy ARTMAP and we propose an old remedy for its cure: cross-validation. In our experiments, we compare the performance of Fuzzy ARTMAP that is trained (i) until the completion of training, (ii) for one epoch, and (iii) until its performance on a validation set is maximized. The experiments were performed on artificial and real databases. The conclusion derived from those experiments is that cross-validation is a useful procedure in Fuzzy ARTMAP, because it produces smaller Fuzzy ARTMAP architectures with improved generalization performance. The trade-off is that cross-validation introduces additional computational complexity in the training phase of Fuzzy ARTMAP.

An Adaptive Multiobjective Approach to Evolving ART Architectures

In this paper, we present the evolution of adaptive resonance theory (ART) neural network architectures (classifiers) using a multiobjective optimization approach. In particular, we propose the use of a multiobjective evolutionary approach to simultaneously evolve the weights and the topology of three well-known ART architectures; fuzzy ARTMAP (FAM), ellipsoidal ARTMAP (EAM), and Gaussian ARTMAP (GAM). We refer to the resulting architectures as MO-GFAM, MO-GEAM, and MO-GGAM, and collectively as MO-GART. The major advantage of MO-GART is that it produces a number of solutions for the classification problem at hand that have different levels of merit [accuracy on unseen data (generalization) and size (number of categories created)]. MO-GART is shown to be more elegant (does not require user intervention to define the network parameters), more effective (of better accuracy and smaller size), and more efficient (faster to produce the solution networks) than other ART neural network architectures that have appeared in the literature. Furthermore, MO-GART is shown to be competitive with other popular classifiers, such as classification and regression tree (CART) and support vector machines (SVMs).

Exemplar-based pattern recognition via semi-supervised learning

The focus of this paper is semi-supervised learning in the context of pattern recognition. Semi-supervised learning (SSL) refers to the semi-supervised construction of clusters during the training phase of exemplar-based classifiers. Using artificially generated data sets we present experimental results of classifiers that follow the SSL paradigm and we show that, especially for difficult pattern recognition problems featuring high class overlap, for exemplar-based classifiers implementing SSL i) the generalization performance improves, while ii) the number of necessary exemplars decreases significantly, when compared to the original versions of the classifiers.