Donald C. Wunsch

Artificial Neural Networks for Control of a Grid-Connected Rectifier/Inverter Under Disturbance, Dynamic and Power Converter Switching Conditions

Three-phase grid-connected converters are widely used in renewable and electric power system applications. Traditionally, grid-connected converters are controlled with standard decoupled d-q vector control mechanisms. However, recent studies indicate that such mechanisms show limitations in their applicability to dynamic systems. This paper investigates how to mitigate such restrictions using a neural network to control a grid-connected rectifier/inverter. The neural network implements a dynamic programming algorithm and is trained by using backpropagation through time. To enhance performance and stability under disturbance, additional strategies are adopted, including the use of integrals of error signals to the network inputs and the introduction of grid disturbance voltage to the outputs of a well-trained network. The performance of the neural-network controller is studied under typical vector control conditions and compared against conventional vector control methods, which demonstrates that the neural vector control strategy proposed in this paper is effective. Even in dynamic and power converter switching environments, the neural vector controller shows strong ability to trace rapidly changing reference commands, tolerate system disturbances, and satisfy control requirements for a faulted power system.

Training Recurrent Neural Networks With the Levenberg–Marquardt Algorithm for Optimal Control of a Grid-Connected Converter

This paper investigates how to train a recurrent neural network (RNN) using the Levenberg-Marquardt (LM) algorithm as well as how to implement optimal control of a grid-connected converter (GCC) using an RNN. To successfully and efficiently train an RNN using the LM algorithm, a new forward accumulation through time (FATT) algorithm is proposed to calculate the Jacobian matrix required by the LM algorithm. This paper explores how to incorporate FATT into the LM algorithm. The results show that the combination of the LM and FATT algorithms trains RNNs better than the conventional backpropagation through time algorithm. This paper presents an analytical study on the optimal control of GCCs, including theoretically ideal optimal and suboptimal controllers. To overcome the inapplicability of the optimal GCC controller under practical conditions, a new RNN controller with an improved input structure is proposed to approximate the ideal optimal controller. The performance of an ideal optimal controller and a well-trained RNN controller was compared in close to real-life power converter switching environments, demonstrating that the proposed RNN controller can achieve close to ideal optimal control performance even under low sampling rate conditions. The excellent performance of the proposed RNN controller under challenging and distorted system conditions further indicates the feasibility of using an RNN to approximate optimal control in practical applications.

Adaptive Scaling of Cluster Boundaries for Large-Scale Social Media Data Clustering

The large scale and complex nature of social media data raises the need to scale clustering techniques to big data and make them capable of automatically identifying data clusters with few empirical settings. In this paper, we present our investigation and three algorithms based on the fuzzy adaptive resonance theory (Fuzzy ART) that have linear computational complexity, use a single parameter, i.e., the vigilance parameter to identify data clusters, and are robust to modest parameter settings. The contribution of this paper lies in two aspects. First, we theoretically demonstrate how complement coding, commonly known as a normalization method, changes the clustering mechanism of Fuzzy ART, and discover the vigilance region (VR) that essentially determines how a cluster in the Fuzzy ART system recognizes similar patterns in the feature space. The VR gives an intrinsic interpretation of the clustering mechanism and limitations of Fuzzy ART. Second, we introduce the idea of allowing different clusters in the Fuzzy ART system to have different vigilance levels in order to meet the diverse nature of the pattern distribution of social media data. To this end, we propose three vigilance adaptation methods, namely, the activation maximization (AM) rule, the confliction minimization (CM) rule, and the hybrid integration (HI) rule. With an initial vigilance value, the resulting clustering algorithms, namely, the AM-ART, CM-ART, and HI-ART, can automatically adapt the vigilance values of all clusters during the learning epochs in order to produce better cluster boundaries. Experiments on four social media data sets show that AM-ART, CM-ART, and HI-ART are more robust than Fuzzy ART to the initial vigilance value, and they usually achieve better or comparable performance and much faster speed than the state-of-the-art clustering algorithms that also do not require a predefined number of clusters.

Clustering Data of Mixed Categorical and Numerical Type With Unsupervised Feature Learning

Mixed-type categorical and numerical data are a challenge in many applications. This general area of mixed-type data is among the frontier areas, where computational intelligence approaches are often brittle compared with the capabilities of living creatures. In this paper, unsupervised feature learning (UFL) is applied to the mixed-type data to achieve a sparse representation, which makes it easier for clustering algorithms to separate the data. Unlike other UFL methods that work with homogeneous data, such as image and video data, the presented UFL works with the mixed-type data using fuzzy adaptive resonance theory (ART). UFL with fuzzy ART (UFLA) obtains a better clustering result by removing the differences in treating categorical and numeric features. The advantages of doing this are demonstrated with several real-world data sets with ground truth, including heart disease, teaching assistant evaluation, and credit approval. The approach is also demonstrated on noisy, mixed-type petroleum industry data. UFLA is compared with several alternative methods. To the best of our knowledge, this is the first time UFL has been extended to accomplish the fusion of mixed data types.

Particle Swarm Optimization in an adaptive resonance framework

A Particle Swarm Optimization (PSO) technique, in conjunction with Fuzzy Adaptive Resonance Theory (ART), was implemented to adapt vigilance values to appropriately compensate for a disparity in data sparsity. Gaining the ability to optimize a vigilance threshold over each cluster as it is created is useful because not all conceivable clusters have the same sparsity from the cluster centroid. Instead of selecting a single vigilance threshold, a metric must be selected for the PSO to optimize on. This trades one design decision for another. The performance gain, however, motivates the tradeoff in certain applications.

Big data - a 21st century science Maginot Line? No-boundary thinking: shifting from the big data paradigm

Whether your interests lie in scientific arenas, the corporate world, or in government, you have certainly heard the praises of big data: Big data will give you new insights, allow you to become more efficient, and/or will solve your problems. While big data has had some outstanding successes, many are now beginning to see that it is not the Silver Bullet that it has been touted to be. Here our main concern is the overall impact of big data; the current manifestation of big data is constructing a Maginot Line in science in the 21st century. Big data is not “lots of data” as a phenomena anymore; The big data paradigm is putting the spirit of the Maginot Line into lots of data. Big data overall is disconnecting researchers and science challenges. We propose No-Boundary Thinking (NBT), applying no-boundary thinking in problem defining to address science challenges.

Unsupervised Feature Learning Classification With Radial Basis Function Extreme Learning Machine Using Graphic Processors

Ever-increasing size and complexity of data sets create challenges and potential tradeoffs of accuracy and speed in learning algorithms. This paper offers progress on both fronts. It presents a mechanism to train the unsupervised learning features learned from only one layer to improve performance in both speed and accuracy. The features are learned by an unsupervised feature learning (UFL) algorithm. Then, those features are trained by a fast radial basis function (RBF) extreme learning machine (ELM). By exploiting the massive parallel computing attribute of modern graphics processing unit, a customized compute unified device architecture (CUDA) kernel is developed to further speed up the computing of the RBF kernel in the ELM. Results tested on Canadian Institute for Advanced Research and Mixed National Institute of Standards and Technology data sets confirm the UFL RBF ELM achieves high accuracy, and the CUDA implementation is up to 20 times faster than CPU and the naive parallel approach.

Sorting the phenotypic heterogeneity of autism spectrum disorders: A hierarchical clustering model

Autism spectrum disorder (ASD) is characterized by notable phenotypic heterogeneity, which is often viewed as an obstacle to the study of its etiology, diagnosis, treatment, and prognosis. Heterogeneity in ASD is multidimensional and complex including variability in phenotype as well as clinical, physiologic, and pathologic parameters. We apply a hierarchical clustering model suited to dealing with datasets of mixed data types to stratify children with ASD into more homogeneous subgroups in line with the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 model. The results of this cluster analysis will provide a better understanding the complex issue of ASD phenotypic heterogeneity and identify subgroups useful for further ASD genetic studies. Our goal is to provide insight into viable phenotypic and genotypic markers that would guide further cluster analysis of ASD genetic data. We suggest that analyzing the clusters in a hierarchical structure is a well-suited and meaningful model to unravel the complex heterogeneity of this disorder.

Ensemble statistical and subspace clustering model for analysis of autism spectrum disorder phenotypes

Heterogeneity in Autism Spectrum Disorder (ASD) is complex including variability in behavioral phenotype as well as clinical, physiologic, and pathologic parameters. The fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) now diagnoses ASD using a 2-dimensional model based social communication deficits and fixated interests and repetitive behaviors. Sorting out heterogeneity is crucial for study of etiology, diagnosis, treatment and prognosis. In this paper, we present an ensemble model for analyzing ASD phenotypes using several machine learning techniques and a k-dimensional subspace clustering algorithm. Our ensemble also incorporates statistical methods at several stages of analysis. We apply this model to a sample of 208 probands drawn from the Simon Simplex Collection Missouri Site patients. The results provide useful evidence that is helpful in elucidating the phenotype complexity within ASD. Our model can be extended to other disorders that exhibit a diverse range of heterogeneity.

Multi-prototype local density-based hierarchical clustering

In this paper, novel hierarchical clustering algorithms, Growing Fuzzy ART (GFA) and Self-Resonant Growing Fuzzy ART (SRGFA), based on connecting prototypes, are presented. The prototypes are generated by vector quantization algorithms: K-means, Self-Organizing Maps, and Fuzzy ART. The Euclidean distance is used to train the first two algorithms in order to allocate the centroids and neurons, respectively. The latter uses fuzzy set operations to check resonance and learn the categories. For each method, a subset of the data set is associated with each prototype; this subset consists of all patterns that, according to a similarity measure, are within a certain threshold from a given prototype. In the case of K-means and Self-Organizing Map, the region is a hypersphere, and in the case of Fuzzy ART, it is a hyperbox. In order to relax the similarity constraint and create larger subsets of data for each prototype, the values of the Euclidean norm and the vigilance parameter are continuously increased and decreased, respectively, according to a step size. Prototypes that have patterns in common are linked together in the process. The data sets final partition is selected as the clustering state in which the algorithm spent most of its time. Synthetic and real world data sets are used to depict the experimental results. External validity indices are used as figures of merit to evaluate the quality of the final partitions.