Pascual Martínez-Gómez

Online adaptation strategies for statistical machine translation in post-editing scenarios

One of the most promising approaches to machine translation consists in formulating the problem by means of a pattern recognition approach. By doing so, there are some tasks in which online adaptation is needed in order to adapt the system to changing scenarios. In the present work, we perform an exhaustive comparison of four online learning algorithms when combined with two adaptation strategies for the task of online adaptation in statistical machine translation. Two of these algorithms are already well-known in the pattern recognition community, such as the perceptron and passive-aggressive algorithms, but here they are thoroughly analyzed for their applicability in the statistical machine translation task. In addition, we also compare them with two novel methods, i.e., Bayesian predictive adaptation and discriminative ridge regression. In statistical machine translation, the most successful approach is based on a log-linear approximation to a posteriori distribution. According to experimental results, adapting the scaling factors of this log-linear combination of models using discriminative ridge regression or Bayesian predictive adaptation yields the best performance.

Recognition of understanding level and language skill using measurements of reading behavior

The reading act is an intimate and elusive process that is important to understand. Psycholinguists have long studied the effects of task, personal or document characteristics on reading behavior. An essential factor in the success of those studies lies in the capability of analyzing eye-movements. These studies aim to recognize causal effects on patterns of eye-movements, by contriving variations in task, personal or document characteristics. In this work, we follow the opposite direction. We present a formal framework to recognize readers level of understanding and language skill given measurements of reading behavior via eye-gaze data. We show significant error reductions to recognize these attributes and provide a detailed study of the most discriminative features.

Higher-order logical inference with compositional semantics

We present a higher-order inference system based on a formal compositional semantics and the wide-coverage CCG parser. We develop an improved method to bridge between the parser and semantic composition. The system is evaluated on the FraCaS test suite. In contrast to the widely held view that higher-order logic is unsuitable for efficient logical inferences, the results show that a system based on a reasonably-sized semantic lexicon and a manageable number of non-first-order axioms enables efficient logical inferences, including those concerned with generalized quantifiers and intensional operators, and outperforms the state-of-the-art firstorder inference system.

Recognition of translator expertise using sequences of fixations and keystrokes

Professional human translation is necessary to meet high quality standards in industry and governmental agencies. Translators engage in multiple activities during their task, and there is a need to model their behavior, with the objective to understand and optimize the translation process. In recent years, user interfaces enabled us to record user events such as eye-movements or keystrokes. Although there have been insightful descriptive analysis of the translation process, there are multiple advantages in enabling quantitative inference. We present methods to classify sequences of fixations and keystrokes into activities and model translation sessions with the objective to recognize translator expertise. We show significant error reductions in the task of recognizing certified translators and their years of experience, and analyze the characterizing patterns.

On multimodal interactive machine translation using speech recognition

Interactive machine translation (IMT) is an increasingly popular paradigm for semi-automated machine translation, where a human expert is integrated into the core of an automatic machine translation system. The human expert interacts with the IMT system by partially correcting the errors of the systems output. Then, the system proposes a new solution. This process is repeated until the output meets the desired quality. In this scenario, the interaction is typically performed using the keyboard and the mouse. However, speech is also a very interesting input modality since the user does not need to abandon the keyboard to interact with it. In this work, we present a new approach to perform speech interaction in a way that translation and speech inputs are tightly fused. This integration is performed early in the speech recognition step. Thus, the information from the translation models allows the speech recognition system to recover from errors that otherwise would be impossible to amend. In addition, this technique allows to use currently available speech recognition technology. The proposed system achieves an important boost in performance with respect to previous approaches.

Online learning via dynamic reranking for computer assisted translation

New techniques for online adaptation in computer assisted translation are explored and compared to previously existing approaches. Under the online adaptation paradigm, the translation system needs to adapt itself to real-world changing scenarios, where training and tuning may only take place once, when the system is set-up for the first time. For this purpose, post-edit information, as described by a given quality measure, is used as valuable feedback within a dynamic reranking algorithm. Two possible approaches are presented and evaluated. The first one relies on the well-known perceptron algorithm, whereas the second one is a novel approach using the Ridge regression in order to compute the optimum scaling factors within a state-of-the-art SMT system. Experimental results show that such algorithms are able to improve translation quality by learning from the errors produced by the system on a sentence-by-sentence basis.

Image registration for text-gaze alignment

Applications using eye-tracking devices need a higher accuracy in recognition when the task reaches a certain complexity. Thus, more sophisticated methods to correct eye-tracking measurement errors are necessary to lower the penetration barrier of eye-trackers in unconstrained tasks. We propose to take advantage of the content or the structure of textual information displayed on the screen to build informed error-correction algorithms that generalize well. The idea is to use feature-based image registration techniques to perform a linear transformation of gaze coordinates to find a good alignment with text printed on the screen. In order to estimate the parameters of the linear transformation, three optimization strategies are proposed to avoid the problem of local minima, namely Monte Carlo, multi-resolution and multi-blur optimization. Experimental results show that a more precise alignment of gaze data with words on the screen can be achieved by using these methods, allowing a more reliable use of eye-trackers in complex and unconstrained tasks.

ccg2lambda: A Compositional Semantics System

We demonstrate a simple and easy-to-use system to produce logical semantic representations of sentences. Our software operates by composing semantic formulas bottom-up given a CCG parse tree. It uses flexible semantic templates to specify semantic patterns. Templates for English and Japanese accompany our software, and they are easy to understand, use and extend to cover other linguistic phenomena or languages. We also provide scripts to use our semantic representations in a textual entailment task, and a visualization tool to display semantically augmented CCG trees in HTML.

Synthesizing image representations of linguistic and topological features for predicting areas of attention

Depending on the reading objective or task, text portions with certain linguistic features require more user attention to maximize the level of understanding. The goal is to build a predictor of these text areas. Our strategy consists in synthesizing image representations of linguistic features, that allows us to use natural language processing techniques while preserving the topology of the text. Eye-tracking technology allows us to precisely observe the identity of fixated words on a screen and their fixation duration. Then, we estimate the scaling factors of a linear combination of image representations of linguistic features that best explain certain gaze evidence, which leads us to a quantification of the influence of linguistic features in reading behavior. Finally, we can compute saliency maps that contain a prediction of the most interesting or cognitive demanding areas along the text. We achieve an important prediction accuracy of the text areas that require more attention for users to maximize their understanding in certain reading tasks, suggesting that linguistic features are good signals for prediction.

Dynamic-Programming–Based Method for Fixation-to-Word Mapping

Eye movements made when reading text are considered to be important clues for estimating both understanding and interest. To analyze gaze data captured by the eye tracker with respect to a text, we need a noise-robust mapping between a fixation point and a word in the text. In this paper, we propose a dynamic-programming–based method for effective fixation-to-word mappings that can reduce the vertical displacement in gaze location. The golden dataset is created using FixFix, our web-based manual annotation tool. We first divide the gaze data into a number of sequential reading segments, then attempt to find the best segment-to-line alignment. To determine the best alignment, we select candidates for each segment, and calculate the cost based on the length characteristics of both the segment and document lines. We compare our method with the naive mapping method, and show that it is capable of producing more accurate fixation-to-word mappings.