Leslie S. Smith

Feature subset selection in large dimensionality domains

Searching for an optimal feature subset from a high dimensional feature space is known to be an NP-complete problem. We present a hybrid algorithm, SAGA, for this task. SAGA combines the ability to avoid being trapped in a local minimum of simulated annealing with the very high rate of convergence of the crossover operator of genetic algorithms, the strong local search ability of greedy algorithms and the high computational efficiency of generalized regression neural networks. We compare the performance over time of SAGA and well-known algorithms on synthetic and real datasets. The results show that SAGA outperforms existing algorithms.

The principal components of natural images

A neural net was used to analyse samples of natural images and text. For the natural images, components resemble derivatives of Gaussian operators, similar to those found in visual cortex and inferred from psychophysics. While the results from natural images do not depend on scale, those from text images are highly scale dependent. Convolution of one of the text components with an original image shows that it is sensitive to inter-word gaps.

A tool for synthesizing spike trains with realistic interference

Spike detection and spike sorting techniques are often difficult to assess because of the lack of ground truth data (i.e., spike timings for each neuron). This is particularly important for in vitro recordings where the signal to noise ratio is poor (as is the case for multi-electrode arrays at the bottom of a cell culture dish). We present an analysis of the transmission of intracellular signals from neurons to an extracellular electrode, and a set of MATLAB functions based on this analysis. These produce realistic signals from neighboring neurons as well as interference from more distant neurons, and Gaussian noise. They thus generate realistic but controllable synthetic signals (for which the ground truth is known) for assessing spike detection and spike sorting techniques. They can also be used to generate realistic (non-Gaussian) background noise. We use signals generated in this way to compare two automated spike-sorting techniques. The software is available freely on the web.

A biologically supported error-correcting learning rule

We show that a form of synaptic plasticity recently discovered in slices of the rat visual cortex (Artola et al. 1990) can support an error-correcting learning rule. The rule increases weights when both pre- and postsynaptic units are highly active, and decreases them when pre-synaptic activity is high and postsynaptic activation is less than the threshold for weight increment but greater than a lower threshold. We show that this rule corrects false positive outputs in feedforward associative memory, that in an appropriate opponent-unit architecture it corrects misses, and that it performs better than the optimal Hebbian learning rule reported by Willshaw and Dayan (1990).

A neural network-based framework for the reconstruction of incomplete data sets

The treatment of incomplete data is an important step in the pre-processing of data. We propose a novel nonparametric algorithm Generalized regression neural network Ensemble for Multiple Imputation (GEMI). We also developed a single imputation (SI) version of this approach-GESI. We compare our algorithms with 25 popular missing data imputation algorithms on 98 real-world and synthetic datasets for various percentage of missing values. The effectiveness of the algorithms is evaluated in terms of (i) the accuracy of output classification: three classifiers (a generalized regression neural network, a multilayer perceptron and a logistic regression technique) are separately trained and tested on the dataset imputed with each imputation algorithm, (ii) interval analysis with missing observations and (iii) point estimation accuracy of the missing value imputation. GEMI outperformed GESI and all the conventional imputation algorithms in terms of all three criteria considered.

A novel neural network ensemble architecture for time series forecasting

We propose a novel homogeneous neural network ensemble approach called Generalized Regression Neural Network (GEFTS-GRNN) Ensemble for Forecasting Time Series, which is a concatenation of existing machine learning algorithms. GEFTS uses a dynamic nonlinear weighting system wherein the outputs from several base-level GRNNs are combined using a combiner GRNN to produce the final output. We compare GEFTS with the 11 most used algorithms on 30 real datasets. The proposed algorithm appears to be more powerful than existing ones. Unlike conventional algorithms, GEFTS is effective in forecasting time series with seasonal patterns.

Smoothing and thresholding in neuronal spike detection

We discuss spike detection for noisy neuronal data. Robust spike detection techniques are especially important for probes which have fixed electrode sites that cannot be independently manipulated to isolate signals from specific neurons. Low signal-to-noise ratio (SNR) and similarity of spectral characteristic between the target signal and background noise are obstacles to spike detection. We propose a new technique based on cumulative energy.

Neuromorphic Systems: Engineering Silicon from Neurobiology

From the Publisher: Neuromorphic systems are implementations in silicon of sensory and neural systems whose architecture and design are based on neurobiology. This growing area proffers exciting possibilities, such as sensory systems that can compete with human senses and pattern recognition systems that can run in real time. The area is at the intersection of neurophysiology, computer science and electrical engineering. This book brings together recent developments in Europe and the US, so that researchers in both academia and industry can find out about the state of the art. As well as elementary material on what neuromorphic systems are and why they are growing in importance, the book contains details of current work. Them are articles on aspects of implementing sensory neuromorphic systems, as well as articles on neuromorphic hardware.

A New Spike Detection Algorithm for Extracellular Neural Recordings

Signals from extracellular electrodes in neural systems record voltages resulting from activity in many neurons. Detecting action potentials (spikes) in a small number of specific (target) neurons is difficult because many neurons, both near and more distant, contribute to the signal at the electrode. We consider some nearby neurons as target neurons (providing a signal) and all the other contributions to the signal as noise. A new algorithm for spike detection has been developed: this applies a cepstrum of bispectrum (CoB) estimated inverse filter to provide blind equalization. This technique is based on higher order statistics, and seeks to find a sequence of event times or delta sequence. We show that the CoB-based technique can achieve a 98% hit rate on an extracellular signal containing three spike trains at up to 0 dB SNR. Threshold setting for this technique is discussed, and we show the application of the technique to some real signals. We compare performance with four established techniques and report that the CoB-based algorithm performs best.

Robust sound onset detection using leaky integrate-and-fire neurons with depressing synapses

A biologically inspired technique for detecting onsets in sound is presented. Outputs from a cochlea-like filter are spike coded, in a way similar to the auditory nerve (AN). These AN-like spikes are presented to a leaky integrate-and-fire neuron through a depressing synapse. Onsets are detected with essentially zero latency relative to these AN spikes. Onset detection results for a tone burst, musical sounds and the DARPA/NIST TIMIT speech corpus are presented.