Serkan Kiranyaz

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

In this paper, we propose a novel technique for the automatic design of Artificial Neural Networks (ANNs) by evolving to the optimal network configuration(s) within an architecture space. It is entirely based on a multi-dimensional Particle Swarm Optimization (MD PSO) technique, which re-forms the native structure of swarm particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. Therefore, in a multidimensional search space where the optimum dimension is unknown, swarm particles can seek both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. With the proper encoding of the network configurations and parameters into particles, MD PSO can then seek the positional optimum in the error space and the dimensional optimum in the architecture space. The optimum dimension converged at the end of a MD PSO process corresponds to a unique ANN configuration where the network parameters (connections, weights and biases) can then be resolved from the positional optimum reached on that dimension. In addition to this, the proposed technique generates a ranked list of network configurations, from the best to the worst. This is indeed a crucial piece of information, indicating what potential configurations can be alternatives to the best one, and which configurations should not be used at all for a particular problem. In this study, the architecture space is defined over feed-forward, fully-connected ANNs so as to use the conventional techniques such as back-propagation and some other evolutionary methods in this field. The proposed technique is applied over the most challenging synthetic problems to test its optimality on evolving networks and over the benchmark problems to test its generalization capability as well as to make comparative evaluations with the several competing techniques. The experimental results show that the MD PSO evolves to optimum or near-optimum networks in general and has a superior generalization capability. Furthermore, the MD PSO naturally favors a low-dimension solution when it exhibits a competitive performance with a high dimension counterpart and such a native tendency eventually yields the evolution process to the compact network configurations in the architecture space rather than the complex ones, as long as the optimality prevails.

A Generic and Robust System for Automated Patient-Specific Classification of ECG Signals

This paper presents a generic and patient-specific classification system designed for robust and accurate detection of ECG heartbeat patterns. The proposed feature extraction process utilizes morphological wavelet transform features, which are projected onto a lower dimensional feature space using principal component analysis, and temporal features from the ECG data. For the pattern recognition unit, feedforward and fully connected artificial neural networks, which are optimally designed for each patient by the proposed multidimensional particle swarm optimization technique, are employed. By using relatively small common and patient-specific training data, the proposed classification system can adapt to significant interpatient variations in ECG patterns by training the optimal network structure, and thus, achieves higher accuracy over larger datasets. The classification experiments over a benchmark database demonstrate that the proposed system achieves such average accuracies and sensitivities better than most of the current state-of-the-art algorithms for detection of ventricular ectopic beats (VEBs) and supra-VEBs (SVEBs). Over the entire database, the average accuracy-sensitivity performances of the proposed system for VEB and SVEB detections are 98.3%-84.6% and 97.4%-63.5%, respectively. Finally, due to its parameter-invariant nature, the proposed system is highly generic, and thus, applicable to any ECG dataset.

Fractional Particle Swarm Optimization in Multidimensional Search Space

In this paper, we propose two novel techniques, which successfully address several major problems in the field of particle swarm optimization (PSO) and promise a significant breakthrough over complex multimodal optimization problems at high dimensions. The first one, which is the so-called multidimensional (MD) PSO, re-forms the native structure of swarm particles in such a way that they can make interdimensional passes with a dedicated dimensional PSO process. Therefore, in an MD search space, where the optimum dimension is unknown, swarm particles can seek both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. Nevertheless, MD PSO is still susceptible to premature convergences due to lack of divergence. Among many PSO variants in the literature, none yields a robust solution, particularly over multimodal complex problems at high dimensions. To address this problem, we propose the fractional global best formation (FGBF) technique, which basically collects all the best dimensional components and fractionally creates an artificial global best (aGB) particle that has the potential to be a better ¿guide¿ than the PSOs native gbest particle. This way, the potential diversity that is present among the dimensions of swarm particles can be efficiently used within the aGB particle. We investigated both individual and mutual applications of the proposed techniques over the following two well-known domains: 1) nonlinear function minimization and 2) data clustering. An extensive set of experiments shows that in both application domains, MD PSO with FGBF exhibits an impressive speed gain and converges to the global optima at the true dimension regardless of the search space dimension, swarm size, and the complexity of the problem.

A generic audio classification and segmentation approach for multimedia indexing and retrieval

We focus the attention on the area of generic and automatic audio classification and segmentation for audio-based multimedia indexing and retrieval applications. In particular, we present a fuzzy approach toward hierarchic audio classification and global segmentation framework based on automatic audio analysis providing robust, bi-modal, efficient and parameter invariant classification over global audio segments. The input audio is split into segments, which are classified as speech, music, fuzzy or silent. The proposed method minimizes critical errors of misclassification by fuzzy region modeling, thus increasing the efficiency of both pure and fuzzy classification. The experimental results show that the critical errors are minimized and the proposed framework significantly increases the efficiency and the accuracy of audio-based retrieval especially in large multimedia databases.

Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks

Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring.

Integrating Color Features in Polarimetric SAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) data are used extensively for terrain classification applying SAR features from various target decompositions and certain textural features. However, one source of information has so far been neglected from PolSAR classification: Color. It is a common practice to visualize PolSAR data by color coding methods and thus, it is possible to extract powerful color features from such pseudocolor images so as to provide additional data for a superior terrain classification. In this paper, we first review previous attempts for PolSAR classifications using various feature combinations and then we introduce and perform in-depth investigation of the application of color features over the Pauli color-coded images besides SAR and texture features. We run an extensive set of comparative evaluations using 24 different feature set combinations over three images of the Flevoland- and the San Francisco Bay region from the RADARSAT-2 and the AIRSAR systems operating in C- and L-bands, respectively. We then consider support vector machines and random forests classifier topologies to test and evaluate the role of color features over the classification performance. The classification results show that the additional color features introduce a new level of discrimination and provide noteworthy improvement in classification performance (compared with the traditionally employed PolSAR and texture features) within the application of land use and land cover classification.

The 2nd competition on counter measures to 2D face spoofing attacks

As a crucial security problem, anti-spoofing in biometrics, and particularly for the face modality, has achieved great progress in the recent years. Still, new threats arrive inform of better, more realistic and more sophisticated spoofing attacks. The objective of the 2nd Competition on Counter Measures to 2D Face Spoofing Attacks is to challenge researchers to create counter measures effectively detecting a variety of attacks. The submitted propositions are evaluated on the Replay-Attack database and the achieved results are presented in this paper.

Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition

For many engineering problems we require optimization processes with dynamic adaptation as we aim to establish the dimension of the search space where the optimum solution resides and develop robust techniques to avoid the local optima usually associated with multimodal problems. This book explores multidimensional particle swarm optimization, a technique developed by the authors that addresses these requirements in a well-defined algorithmic approach. After an introduction to the key optimization techniques, the authors introduce their unified framework and demonstrate its advantages in challenging application domains, focusing on the state of the art of multidimensional extensions such as global convergence in particle swarm optimization, dynamic data clustering, evolutionary neural networks, biomedical applications and personalized ECG classification, content-based image classification and retrieval, and evolutionary feature synthesis. The content is characterized by strong practical considerations, and the book is supported with fully documented source code for all applications presented, as well as many sample datasets. The book will be of benefit to researchers and practitioners working in the areas of machine intelligence, signal processing, pattern recognition, and data mining, or using principles from these areas in their application domains. It may also be used as a reference text for graduate courses on swarm optimization, data clustering and classification, content-based multimedia search, and biomedical signal processing applications.

Multi-dimensional particle swarm optimization in dynamic environments

The particle swarm optimization (PSO) was introduced as a population based stochastic search and optimization process for static environments; however, many real problems are dynamic, meaning that the environment and the characteristics of the global optimum can change over time. Thanks to its stochastic and population based nature, PSO can avoid being trapped in local optima and find the global optimum. However, this is never guaranteed and as the complexity of the problem rises, it becomes more probable that the PSO algorithm gets trapped into a local optimum due to premature convergence. In this paper, we propose novel techniques, which successfully address several major problems in the field of particle swarm optimization (PSO) and promise efficient and robust solutions for multi-dimensional and dynamic problems. The first one, so-called multi-dimensional (MD) PSO, re-forms the native structure of swarm particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. Therefore, in a multi-dimensional search space where the optimum dimension is unknown, swarm particles can seek for both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. To address the premature convergence problem, we then propose fractional global best formation (FGBF) technique, which basically collects all the best dimensional components and fractionally creates an artificial global-best particle (aGB) that has the potential to be a better ldquoguiderdquo than the PSOs native gbest particle. To establish follow-up of (current) local optima, we then introduce a novel multi-swarm algorithm, which enables each swarm to converge to a different optimum and use FGBF technique distinctively. We then propose a multi-dimensional extension of the moving peaks benchmark (MPB), which is a publicly available for testing optimization algorithms in a multi-modal dynamic environment. In this extended benchmark an extensive set of experiments show that MD PSO using FGBF technique with multi-swarms exhibits an impressive performance and tracks the global maximum peak with the minimum error.

Hierarchical Cellular Tree: An Efficient Indexing Scheme for Content-Based Retrieval on Multimedia Databases

One of the challenges in the development of a content-based multimedia indexing and retrieval application is to achieve an efficient indexing scheme. The developers and users who are accustomed to making queries to retrieve a particular multimedia item from a large scale database can be frustrated by the long query times. Conventional indexing structures cannot usually cope with the requirements of a multimedia database, such as dynamic indexing or the presence of high-dimensional audiovisual features. Such structures do not scale well with the ever increasing size of multimedia databases whilst inducing corruption and resulting in an over-crowded indexing structure. This paper addresses such problems and presents a novel indexing technique, hierarchical cellular tree (HCT), which is designed to bring an effective solution especially for indexing large multimedia databases. Furthermore it provides an enhanced browsing capability, which enables user to make a guided tour within the database. A pre-emptive cell-search mechanism is introduced in order to prevent corruption, which may occur due to erroneous item insertions. Among the hierarchical levels that are built in a bottom-up fashion, similar items are collected into appropriate cellular structures at some level. Cells are subject to mitosis operations when the dissimilarity exceeds a required level. By mitosis operations, cells are kept focused and compact and yet, they can grow into any dimension as long as the compactness is maintained. The proposed indexing scheme is then used along with a recently introduced query method, the progressive query, in order to achieve the ultimate goal, from the user point of view that is retrieval of the most relevant items in the earliest possible time regardless of the database size. Experimental results show that the speed of retrievals is significantly improved and the indexing structure shows no sign of degradations when the database size is increased. Furthermore, HCT indexing body can conveniently be used for efficient browsing and navigation operations among the multimedia database items