Luca Cazzanti

Local similarity discriminant analysis

We propose a local, generative model for similarity-based classification. The method is applicable to the case that only pairwise similarities between samples are available. The classifier models the local class-conditional distribution using a maximum entropy estimate and empirical moment constraints. The resulting exponential class conditional-distributions are combined with class prior probabilities and misclassification costs to form the local similarity discriminant analysis (local SDA) classifier. We compare the performance of local SDA to a non-local version, to the local nearest centroid classifier, the nearest centroid classifier, k-NN, and to the recently-developed potential support vector machine (PSVM). Results show that local SDA is competitive with k-NN and the computationally-demanding PSVM while offering the advantages of a generative classifier.

Generative models for similarity-based classification

A maximum-entropy approach to generative similarity-based classifiers model is proposed. First, a descriptive set of similarity statistics is assumed to be sufficient for classification. Then the class-conditional distributions of these descriptive statistics are estimated as the maximum-entropy distributions subject to empirical moment constraints. The resulting exponential class-conditional distributions are used in a maximum a posteriori decision rule, forming the similarity discriminant analysis (SDA) classifier. Simulated and real data experiments compare performance to the k-nearest neighbor classifier, the nearest-centroid classifier, and the potential support vector machine (PSVM).

Information-theoretic and Set-theoretic Similarity

We introduce a definition of similarity based on Tverskys set-theoretic linear contrast model and on information-theoretic principles. The similarity measures the residual entropy with respect to a random object. This residual entropy similarity strongly captures context, which we conjecture is important for similarity-based statistical learning. Properties of the similarity definition are established and examples illustrate its characteristics. We show that a previously-defined information-theoretic similarity is also set-theoretic, and compare it to the residual entropy similarity. The similarity between random objects is also treated

Regularizing the Local Similarity Discriminant Analysis Classifier

We investigate parameter-based and distribution-based approaches to regularizing the generative, similarity-based classifier called local similarity discriminant analysis classifier (local SDA). We argue that regularizing distributions rather than parameters can both increase the model flexibility and decrease estimation variance while retaining the conceptual underpinnings of the local SDA classifier. Experiments with four benchmark similarity-based classification datasets show that the proposed regularization significantly improves classification performance compared to the local SDA classifier, and the distribution-based approach improves performance more consistently than the parameter-based approaches. Also, regularized local SDA can perform significantly better than similarity-based SVM classifiers, particularly on sparse and highly nonmetric similarities.

Underwater acoustic single- and multi-user differential frequency hopping communications

Differential frequency hopping (DFH) is a fast frequency hopping, digital signaling technology that achieves the desirable performance features of non-interfering spread spectrum operation, spectral re-use, fading mitigation, and interference resistance. Therefore, DFH coding provides the critical capability for multiple users to seamlessly communicate in the bandwidth-limited acoustic channel. In previous work, DFH coding has been shown to be superior to other coding schemes in additive Gaussian white noise and Rayleigh-fading environments when considering the joint constraints of multiple user access, detectability mitigation, and the presence of jamming. In this paper, we describe the auto-synchronizing single-user DFH decoder we have developed for a single hydrophone receiver. We present the performance of this decoder on multi-user simulated data and on multi-user data collected at sea during the Rescheduled Acoustic Communications Experiment (RACE08).We use the Sonar Simulation Toolset (SST) to produce the simulated data for soft through hard bottom compositions to provide a range of multipath severity to gain insight into DFH performance across environments. Based on these initial results, the DFH waveform shows considerable promise for computationally minimal, high reliability communications among uncoordinated users in an underwater acoustic channel.

Bayesian and pairwise local similarity discriminant analysis

We investigate three extensions to the generative similarity-based classifier called local similarity discriminant analysis (local SDA): a Bayesian approach to estimating the pmfs based on the assumption that similarities are multinomially distributed and on the Dirichlet prior distribution; a pairwise-similarity formulation of local SDA that accounts for all local pairwise similarities to estimate the pmfs; a combined Bayesian pairwise-similarity approach. We discuss how the proposed extensions afford more modeling flexibility than standard local SDA and less cumbersome model training than previously-published local SDA regularization strategies. Experiments with five benchmark similarity-based classification datasets show that the increased modeling flexibility and lighter computational burden of the proposed extensions are coupled with the good classification performance of the local SDA classification paradigm.

Using speech technology to enhance isotope ID and classification

Scientific advances are often made when researchers identify mathematical or physical commonalities between different fields and are able to apply mature techniques or algorithms developed in one field to another field which shares some of the same challenges. The authors of this paper have identified similarities between the unsolved problems faced in gamma-spectroscopy for automated radioisotope identification and the challenges of the much larger body of research in speech processing. In this paper we describe such commonalities and use them as a motivation for a preliminary investigation of the applicability of speech processing methods to gamma-ray spectra. This approach enables the development of proof-of-concept isotope classifiers, whose performance is presented for both simulated and field-collected gamma-ray spectra.

Maximum Entropy Generative Models for Similarity-based Learning

A generative model for similarity-based classification is proposed using maximum entropy estimation. First, a descriptive set of similarity statistics is assumed to be sufficient for classification. Then the class conditional distributions of these descriptive statistics are estimated as the maximum entropy distributions subject to empirical moment constraints. The resulting exponential class conditional distributions are used in a maximum a posteriori decision rule, forming the similarity discriminant analysis (SDA) classifier. The relationship between SDA and the quadratic discriminant analysis classifier is discussed. An example SDA classifier is given that uses the class centroids as the descriptive statistics. Compared to the nearest-centroid classifier, which is also based only on the class centroids, simulation and experimental results show SDA consistently improves performance.

Improved multipath robustness of DFH modulation in the underwater acoustic channel

We characterize performance improvement of differential frequency hopping modulation under two techniques that mitigate the multipath effects of the underwater acoustic channel: blind adaptive equalization and beamforming. We report results on data collected at-sea during the RACE08, SPACE08 and WHOI09 experiments, and show that the bit-error rate improves with the application of these two techniques. Future improvements may combine joint single-element, blind equalization and beamforming (multi-element) approaches to leverage their respective strengths.

Multi-task regularization of generative similarity models

We investigate a multi-task approach to similarity discriminant analysis, where we propose treating the estimation of the different class-conditional distributions of the pairwise similarities as multiple tasks. We show that regularizing these estimates together using a leastsquares regularization weighted by a task-relatedness matrix can reduce the resulting maximum a posteriori classification errors. Results are given for benchmark data sets spanning a range of applications. In addition, we present a new application of similarity-based learning to analyzing the rhetoric of multiple insurgent groups in Iraq. We show how to produce the necessary task relatedness information from standard given training data, as well as how to derive task-relatedness information if given side information about the class relatedness.