Defeng Wang

Structured large margin machines: sensitive to data distributions

AbstractnThis paper proposes a new large margin classifier—the structured large margin machine (SLMM)—that is sensitive to the structure of the data distribution. The SLMM approach incorporates the merits of “structured” learning models, such as radial basis function networks and Gaussian mixture models, with the advantages of “unstructured” large margin learning schemes, such as support vector machines and maxi-min margin machines. We derive the SLMM model from the concepts of “structured degree” and “homospace”, based on an analysis of existing structured and unstructured learning models. Then, by using Ward’s agglomerative hierarchical clustering on input data (or data mappings in the kernel space) to extract the underlying data structure, we formulate SLMM training as a sequential second order cone programming. Many promising features of the SLMM approach are illustrated, including its accuracy, scalability, extensibility, and noise tolerance. We also demonstrate the theoretical importance of the SLMM model by showing that it generalizes existing approaches, such as SVMs and M4s, provides novel insight into learning models, and lays a foundation for conceiving other “structured” classifiers.n

Segmentation of human skull in MRI using statistical shape information from CT data.

To automatically segment the skull from the MRI data using a model‐based three‐dimensional segmentation scheme.

Localized Generalization Error of Gaussian-based Classifiers and Visualization of Decision Boundaries

In pattern classification problem, one trains a classifier to recognize future unseen samples using a training dataset. Practically, one should not expect the trained classifier could correctly recognize samples dissimilar to the training dataset. Therefore, finding the generalization capability of a classifier for those unseen samples may not help in improving the classifiers accuracy. The localized generalization error model was proposed to bound above the generalization mean square error for those unseen samples similar to the training dataset only. This error model is derived based on the stochastic sensitivity measure(ST-SM)of the classifiers. We present the ST-SMS for various Gaussian based classifiers: radial basis function neural networks and support vector machine in this paper. At the end of this work, we compare the decision boundaries visualization using the training samples yielding the largest sensitivity measures and the one using support vectors in the input space.

Support vector clustering for brain activation detection

In this paper, we propose a new approach to detect activated time series in functional MRI using support vector clustering (SVC). We extract Fourier coefficients as the features of fMRI time series and cluster these features by SVC. In SVC, these features are mapped from their original feature space to a very high dimensional kernel space. By finding a compact sphere that encloses the mapped features in the kernel space, one achieves a set of cluster boundaries in the feature space. The SVC is an effective and robust fMRI activation detection method because of its advantages in (1) better discovery of real data structure since there is no cluster shape restriction, (2) high quality detection results without explicitly specifying the number of clusters, (3) the stronger robustness due to the mechanism in outlier elimination. Experimental results on simulated and real fMRI data demonstrate the effectiveness of SVC.

Gene selection through sensitivity analysis of support vector machines

We present a novel approach to gene selection for microarry data through the sensitivity analysis of support vector machines (SVMs). A new measurement (sensitivity) is defined to quantify the saliencies of individual features (genes) by analyzing the discriminative function in SVMs. Our feature selection strategy is first to select the features with higher sensitivities but meanwhile keep the remaining ones, and then refine the selected subset by tentatively substituting some part with fragments of the previously rejected features. The accuracy of our method is validated experimentally on the benchmark microarray datasets.

Structured Large Margin Machine Ensemble

Large margin classifiers have been widely applied in solving supervised learning problems. One representative model in large margin learning is the support vector machine (SVM). SVM is an unstructured classifier since the data structure information is underutilized and the decision hyperplane calculation relies exclusively on the support vectors. To incorporate the data covariance information into the large margin learning, structured large margin machine (SLMM) is recently proposed and show better performance than classical SVM in some applications. Instead of utilizing the data structures straightly like SLMM, SVM ensemble (SVMe) improves the generalization ability of SVM in another way by combining the outputs of a series of SVMs. Inspired by SVMe, we are going to explore the ensemble counterpart for SLMM, i.e., SLMMe, and validate the effectiveness of multiple SLMM system. Experimental results on benchmark datasets demonstrate that SLMMe improves SLMM by reducing its variance, and SLMMe outperforms SVMe in most cases in terms of both classification accuracy and variance.