Sibel Yaman

An Integrative and Discriminative Technique for Spoken Utterance Classification

Traditional methods of spoken utterance classification (SUC) adopt two independently trained phases. In the first phase, an automatic speech recognition (ASR) module returns the most likely sentence for the observed acoustic signal. In the second phase, a semantic classifier transforms the resulting sentence into the most likely semantic class. Since the two phases are isolated from each other, such traditional SUC systems are suboptimal. In this paper, we present a novel integrative and discriminative learning technique for SUC to alleviate this problem, and thereby, reduce the semantic classification error rate (CER). Our approach revolves around the effective use of the N-best lists generated by the ASR module to reduce semantic classification errors. The N-best list sentences are first rescored using all the available knowledge sources. Then, the sentence that is most likely to helps reduce the CER are extracted from the N-best lists as well as those sentences that are most likely to increase the CER. These sentences are used to discriminatively train the language and semantic-classifier models to minimize the overall semantic CER. Our experiments resulted in a reduction of CER from its initial value of 4.92% to 4.04% in the standard ATIS task.

Language Recognition Based on Score Distribution Feature Vectors and Discriminative Classifier Fusion

We present the GT-IIR language recognition system submitted to the 2005 NIST Language Recognition Evaluation. Different from conventional frame-based feature extraction, our system adopts a collection of broad output scores from different language recognition systems to form utterance-level score distribution feature vectors over all competing languages, and build vector-based spoken language recognizers by fusing two distinct verifiers, one based on a simple linear discriminant function (LDF) and the other on a complex artificial neural network (ANN), to make final language recognition decisions. The diverse error patterns exhibited in individual LDF and ANN systems facilitate smaller overall verification errors in the combined system than those obtained in separate systems

A low-complexity video encoder with decoder motion estimator

We investigate the use of video compression methods that require a simple encoder and a complex decoder for applications such as video surveillance, smart spaces, and sensor networks. In our proposed method, the encoder tries to identify the locations whose content cannot be predicted at the decoder, and codes such areas at higher fidelity. Typically, high-motion macro-blocks represent such significant state regions. A shape-adaptive (SA) SPIHT (set partitioning in hierarchical trees) encoder is then used to code these regions efficiently. On the decoder side, we perform motion extrapolation using the previously decoded frames to construct an estimate for the current frame. This estimate then serves as the side information at the decoder. Receiving the SA-SPIHT coded and the motion extrapolated frame, the decoder fuses the information in these two frames to produce a frame that is of higher quality than the component images. Experimental results show that the proposed codec outperforms H.264 intra mode by 3 dB.

A Flexible Classifier Design Framework Based on Multiobjective Programming

We propose a multiobjective programming (MOP) framework for finding compromise solutions that are satisfactory for each of multiple competing performance criteria in a pattern classification task. The fundamental idea for our formulation of classifier learning, which we refer to as iterative constrained optimization (ICO), evolves around improving one objective while allowing the rest to degrade. This is achieved by the optimization of individual objectives with proper constraints on the remaining competing objectives. The constraint bounds are adjusted based on the objective functions obtained in the most recent iteration. An aggregated utility function is used to evaluate the acceptability of local changes in competing criteria, i.e., changes from one iteration to the next. Although many MOP approaches developed so far are formal and extensible to large number of competing objectives, their capabilities are examined only with two or three objectives. This is mainly because practical problems become significantly harder to manage when the number of objectives gets larger. We, however, illustrate the proposed framework in the context of automatic language identification (LID) of 12 languages and three dialects. This LID task requires the simultaneous minimization of the false-acceptance and false-rejection rates for each of the 15 languages/dialects, and, hence, is an MOP problem with a total of 30 competing objectives. In our experiments, we observed that the ICO-trained classifiers result in not only reduced error rates but also a good balance among the many competing objectives when compared to those classifiers that minimize an overall objective. We interpret our experimental findings as evidence for ICO offering a greater degree of freedom for classifier design.

A Comparison of Single- and Multi-Objective Programming Approaches to Problems with Multiple Design Objectives

In this paper, we propose and compare single- and multi-objective programming (MOP) approaches to the language model (LM) adaptation that require the optimization of a number of competing objectives. In LM adaptation, an adapted LM is found so that it is as close as possible to two independently trained LMs. The LM adaptation approach developed in this paper is based on reformulating the training objective of a maximum a posteriori (MAP) method as an MOP problem. We extract the individual at least partially conflicting objective functions, which yields a problem with four objectives for a bigram LM: The first two objectives are concerned with the best fit to the adaptation data while the remaining two objectives are concerned with the best prior information obtained from a general domain corpus. Solving this problem in an iterative manner such that each objective is optimized one after another with constraints on the rest, we obtain a target LM that is a log-linear interpolation of the component LMs. The LM weights are found such that all the (at least partially conflicting) objectives are optimized simultaneously. We compare the performance of the SOP- and MOP-based solutions. Our experimental results demonstrate that the ICO method achieves a better balance among the design objectives. Furthermore, the ICO method gives an improved system performance.

A Discriminative Training Framework using N-Best Speech Recognition Transcriptions and Scores for Spoken Utterance Classification

In this paper, we propose a novel discriminative training approach to spoken utterance classification (SUC). The ultimate objective of the SUC task, originally developed to map a spoken speech utterance into the most appropriate semantic class, is to minimize the classification error rate (CER). Conventionally, a two-phase approach is adapted, in which the first phase is the ASR transcription phase, and the second phase is the semantic classification phase. In the proposed framework, the classification error rate is approximated as differentiable functions of the language and classifier model parameters. Furthermore, in order to exploit all the available information from the first phase, class-specific discriminant functions are defined based on score functions derived from the N-best lists. Our experimental results on the standard ATIS database indicate a notable reduction in CER from the earlier best result on the identical task. The proposed framework achieved a reduction of CER from 4.92% to 4.04%.

An Iterative Constrained Optimization Approach to Classifier Design

In this paper, we propose an iterative constrained optimization (ICO) approach to classifier design. When a set of conflicting objectives needs to be simultaneously satisfied, it is often not easy to combine all the utilities in a single overall objective function for optimization. We instead formulate the problem with conflicting objectives as a single-objective optimization scenario while embedding other competing objectives in constraints so that the original problem can be solved by adopting conventional constrained nonlinear optimization techniques. The bounds needed to constrain each objective are determined based on the objective function values obtained in the previous iterate. The so-formed individual constrained optimization problems are solved until a stable solution is obtained. We illustrate the utility of our framework in the context of designing classifiers for text categorization and automatic language identification. The results of our experiments demonstrate that our approach achieves a significant improvement in one objective with only slight degradation of the other conflicting objective

A Multi-Objective Programming Approach to Compromising Classification Performance Metrics

In this paper, we propose an MOP approach for finding the best compromise solution among more than two competing performance criteria. Our formulation for classifier learning, which we refer to as iterative constrained optimization (ICO), involves an iterative process of the optimization of individual objectives with proper constraints on the remaining competing objectives. The fundamental idea is improving one objective while the rest are allowed to degrade. One of the main components of ICO is the supervision mechanism based on the local changes on a selected utility function for controlling the changes in the individual objectives. The utility is an aggregated preference chosen to make a joint decision when evaluating the appropriateness of local changes in competing criteria, i.e. changes from one iteration to the next. Another important component is the adjustment of constraint bounds based on the objective functions attained in the previous iteration using a development set. Many MOP approaches developed so far are formal and extensible to large number of competing objectives. However, their utilities are illustrated using a few objectives. We illustrate the utility of the proposed framework in the context of automatic language identification of 12 languages and 3 dialects, i.e., with a total of 30 objectives. In our experiments, we observed that the ICO-trained classifiers give not only reduced error rates but also a better balance among the many competing objectives.