Les E. Atlas

Improving Generalization with Active Learning

Active learning differs from “learning from examples” in that the learning algorithm assumes at least some control over what part of the input domain it receives information about. In some situations, active learning is provably more powerful than learning from examples alone, giving better generalization for a fixed number of training examples.In this article, we consider the problem of learning a binary concept in the absence of noise. We describe a formalism for active concept learning calledselective sampling and show how it may be approximately implemented by a neural network. In selective sampling, a learner receives distribution information from the environment and queries an oracle on parts of the domain it considers “useful.” We test our implementation, called anSG-network, on three domains and observe significant improvement in generalization.Active learning differs from “learning from examples” in that the learning algorithm assumes at least some control over what part of the input domain it receives information about. In some situations, active learning is provably more powerful than learning from examples alone, giving better generalization for a fixed number of training examples.In this article, we consider the problem of learning a binary concept in the absence of noise. We describe a formalism for active concept learning called selective sampling and show how it may be approximately implemented by a neural network. In selective sampling, a learner receives distribution information from the environment and queries an oracle on parts of the domain it considers “useful.” We test our implementation, called an SG-network, on three domains and observe significant improvement in generalization.

Electric load forecasting using an artificial neural network

An artificial neural network (ANN) approach is presented for electric load forecasting. The ANN is used to learn the relationship among past, current and future temperatures and loads. In order to provide the forecasted load, the ANN interpolates among the load and temperature data in a training data set. The average absolute errors of the 1 h and 24 h-ahead forecasts in tests on actual utility data are shown to be 1.40% and 2.06%, respectively. This compares with an average error of 4.22% for 24 h ahead forecasts with a currently used forecasting technique applied to the same data. >

The use of cone-shaped kernels for generalized time-frequency representations of nonstationary signals

Generalized time-frequency representations (GTFRs) which use cone-shaped kernels for nonstationary signal analysis are presented. The cone-shaped kernels are formulated for the GTFRs to produce good resolution simultaneously in time and frequency. Specifically, for a GTFR with a cone-shaped kernel, finite time support is maintained in the time dimension along with an enhanced spectrum in the frequency dimension, and the cross-terms are smoothed out. Experimental results on simulated data and real speech show the advantages of the GTFRs with the cone-shaped kernels through comparisons to the spectrogram and the pseudo-Wigner distribution. >

Construction of positive time-frequency distributions

A general method for constructing nonnegative definite, joint time-frequency distributions (TFDs) satisfying the marginals of time |s(t)|/sup 2/ and frequency |S(f)|/sup 2/ is presented. As nonnegative-definite distributions with the correct marginals, these TFDs are members of the Cohen-Posch class. Several examples illustrating properties of these TFDs are presented for both synthetic and real signals. >

A performance comparison of trained multilayer perceptrons and trained classification trees

The important differences between multilayer perceptrons and classification trees are considered. A number of empirical tests on three real-world problems in power-system load forecasting, power-system security prediction, and speaker-independent vowel recognition are presented. The load-forecasting problem, which is partially a regression problem, uses past trends to predict the critical needs of future power generation. The power-security problem uses the classifier as an interpolator of previously known states of the system. The vowel-recognition problem is representative of the difficulties in automatic speech recognition caused by variability across speakers and phonetic context. In all cases even with various sizes of training sets, the multilayer perceptron performed as well as or better than the trained classification trees. It is therefore concluded that there is not enough theoretical basis to demonstrate clear-cut superiority of one technique over the other. >

Scalable and progressive audio codec

A source coding technique for variable, bandwidth-constrained channels such as the Internet must do two things: offer high quality at low data rates, and adapt gracefully to changes in available bandwidth. Here we propose an audio coding algorithm that is superior on both counts. It is inherently scalable, meaning that channel conditions can be matched without the need for additional computation. Moreover, it is compact: in subjective tests our algorithm, coded at 32 kb/s/channel, outperformed MPEG-1 Layer 3 (MP3) coded at 56 kb/s/channel (both at 44.1 kHz). We achieve this simultaneous increase in compression and scalability through use of a two-dimensional transform that concentrates relevant information into a small number of coefficients.

Optimizing time-frequency kernels for classification

In many pattern recognition applications, features are traditionally extracted from standard time-frequency representations (TFRs). This assumes that the implicit smoothing of, say, a spectrogram is appropriate for the classification task. Making such assumptions may degrade classification performance. In general, ana time-frequency classification technique that uses a singular quadratic TFR (e.g., the spectrogram) as a source of features will never surpass the performance of the same technique using a regular quadratic TFR (e,g., Rihaczek or Wigner-Ville). Any TFR that is not regular is said to be singular. Use of a singular quadratic TFR implicitly discards information without explicitly determining if it is germane to the classification task. We propose smoothing regular quadratic TFRs to retain only that information that is essential for classification. We call the resulting quadratic TFRs class-dependent TFRs. This approach makes no a priori assumptions about the amount and type of time-frequency smoothing required for classification. The performance of our approach is demonstrated on simulated and real data. The simulated study indicates that the performance can approach the Bayes optimal classifier. The real-world pilot studies involved helicopter fault diagnosis and radar transmitter identification.

Bilinear time-frequency representations: new insights and properties

An analysis of the interference terms of Cohen-class bilinear time-frequency representations (TFR) of multicomponent signals is presented. Constraints for achieving new interference properties are derived. Imposing these interference time and interference frequency concentration constraints on a TFR guarantees that the TFR will be zero everywhere the signal s(t) is zero, and the TFR will contain only those frequencies that occur in the signal. Thus, these new constraints guarantee strong finite support in a TFR. When these interference concentration properties are combined with interference attenuation, tradeoffs between finite support, the marginals, and the interference properties are shown to be unavoidable. The useful class of product kernels are considered and generalized further to allow TFR with potentially superior interference properties. The interference frequency concentration and attenuation properties allow TFR with spectrogram-like interference suppression, but without the spectrograms inherent time/frequency resolution tradeoff. Other useful combinations of properties are discussed and analyzed, and properties and tradeoffs are illustrated by examples. >

Coherent envelope detection for modulation filtering of speech

Modulation filtering, which has been previously described as several related approaches to achieve modification of speech temporal dynamics, is shown to be less effective than intended. In particular, past Hilbert envelope approaches generate distortion which spreads across frequency sub-bands and modulation rejection is far from the amount intended. The source of this distortion is analyzed and a solution, based upon coherent envelope detection in each sub-band is proposed. This coherent approach is shown to be substantially more effective than conventional incoherent approaches on speech samples.

Hidden Markov models for monitoring machining tool-wear

As summarized by Atlas, Bernard, and Narayanan (1996), the sensing of acoustic vibrations can remotely estimate the state of wear at the tool edge. This form of monitoring offers the potential to characterize, in real time, the efficiency of metal removal processes such as drilling and milling. For example, information about sudden increases in tool wear, if manifest as a change in acoustic vibration, could be valuable to a machine operator. The nature of this monitoring problem has some similarities to automatic speech recognition. For example, there is significant tool-to-tool variation in details of vibration and lifetime. Also, the easy adaptability of monitoring systems across manufacturing processes is important. In this work we model the evolution of vibration signals with the same technique which has shown to be successful in speech recognition: hidden Markov models (HMMs). We focus on the monitoring of milling processes at three different time scales and show the how HMMs can give accurate wear prediction.