Branimir Todorovic

Named entity recognition and classification using context Hidden Markov Model

Named entity (NE) recognition is a core technology for understanding low level semantics of texts. In this paper we report our preliminary results for Named Entity Recognition on MUC 7 corpus by combining the supervised machine learning system in the form of probabilistic generative Hidden Markov Model (HMM) for named entity classes PERSON, ORGANIZATION and LOCATION, and grammar based component for DATE, TIME, MONEY and PERCENT. We have implemented two variations of the basic Hidden Markov Model, where the second one is our version of HMM which uses the context of surrounding words to determine the NE class of the current word, leading to more accurate and faster NE recognition.

Extended Kalman Filter Trained Recurrent Radial Basis Function Network in Nonlinear System Identification

We consider the recurrent radial basis function network as a model of nonlinear dynamic system. On-line parameter and structure adaptation is unified under the framework of extended Kalman filter. The ability of adaptive system to deal with high observation noise, and the generalization ability of the resulting RRBF network are demonstrated in nonlinear system identification.

Coreference Resolution Using Decision Trees

Coreference resolution is the process of determining whether two expressions in natural language refer to the same entity in the world. We adopt machine learning approach using decision tree to a coreference resolution of general noun phrases in unrestricted text based on well defined features. We also use approximate matching algorithms for a string match feature and databases of American last names and male and female first names for gender agreement and alias feature. For the evaluation we use MUC-6 coreference corpora. We show that pessimistic error pruning method gives better generalization in a coreference resolution task than that reported in W.M. Soon et al. (2001) when weights of positive and negative examples are properly chosen

Confidence based learning of a two-model committee for sequence labeling

The paper presents the use of a two structural model committee, where the output of the first model together with its confidence is set as the input of the second model. The confidence for the given context of predictions in the sequence is extracted from the alternative hypotheses generated from the first model. We present experiments on the shallow parsing, comparing the performance of the proposed method to the separate models.

A primal sub-gradient method for structured classification with the averaged sum loss

Abstract We present a primal sub-gradient method for structured SVM optimization defined with the averaged sum of hinge losses inside each example. Compared with the mini-batch version of the Pegasos algorithm for the structured case, which deals with a single structure from each of multiple examples, our algorithm considers multiple structures from a single example in one update. This approach should increase the amount of information learned from the example. We show that the proposed version with the averaged sum loss has at least the same guarantees in terms of the prediction loss as the stochastic version. Experiments are conducted on two sequence labeling problems, shallow parsing and part-of-speech tagging, and also include a comparison with other popular sequential structured learning algorithms.

Gradient computation in linear-chain conditional random fields using the entropy message passing algorithm

The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of enhancing memory efficiency when applied to long observation sequences. Unlike the traditional algorithm based on the forward-backward recursions, the memory complexity of our algorithm does not depend on the sequence length. The experiments on real data show that it can be useful for the problems which deal with long sequences.

Computation of cross-moments using message passing over factor graphs

This paper considers the problem of cross-moments computation for functions which decompose according to cycle-free factor graphs. Two algorithms are derived, both based on message passing computation of a corresponding moment-generating function (

Context Hidden Markov Model for Named Entity Recognition

MGF

Sequential Bayesian Estimation of Recurrent Neural Networks

). The first one is realized as message passing algorithm over a polynomial semiring and represents a computation of the

Recurrent Neural Networks Training Using Derivative Free Nonlinear Bayesian Filters

MGF