João Luís Garcia Rosa

Evaluating word embeddings and a revised corpus for part-of-speech tagging in Portuguese

BackgroundPart-of-speech tagging is an important preprocessing step in many natural language processing applications. Despite much work already carried out in this field, there is still room for improvement, especially in Portuguese. We experiment here with an architecture based on neural networks and word embeddings, and that has achieved promising results in English.MethodsWe tested our classifier in different corpora: a new revision of the Mac-Morpho corpus, in which we merged some tags and performed corrections and two previous versions of it. We evaluate the impact of using different types of word embeddings and explicit features as input.ResultsWe compare our tagger’s performance with other systems and achieve state-of-the-art results in the new corpus. We show how different methods for generating word embeddings and additional features differ in accuracy.ConclusionsThe work reported here contributes with a new revision of the Mac-Morpho corpus and a state-of-the-art new tagger available for use out-of-the-box.

A two-step convolutional neural network approach for semantic role labeling

Semantic role labeling (SRL) is a well known task in Natural Language Processing, consisting of identifying and labeling verbal arguments. It has been widely studied in English, but scarcely explored in other languages. In this paper, we employ a two-step convolutional neural architecture to label semantic arguments in Brazilian Portuguese texts, and avoid the use of external NLP tools. We achieve an F1 score of 62.2, which, although considerably lower than the state-of-the-art for English, seems promising considering the available resources. Also, dividing the process into two easier subtasks makes it more feasible to further improve performance through semi-supervised learning. Our system is available online and ready to be used out of the box to label new texts.

A biologically inspired connectionist system for natural language processing

Nowadays artificial neural network models often lack many physiological properties of the nervous cell. Current learning algorithms are more oriented to computational performance than to biological credibility. The aim of this paper is to propose an artificial neural network system, called Bio-/spl theta/R, including architecture and algorithm, to take care of a natural language processing problem, the thematic relationship, in a biologically inspired connectionist approach. Instead of feedforward or simple recurrent network, it is presented as a bi-directional architecture. Instead of the well-known biologically implausible backpropagation algorithm, a neurophysiologically motivated one is employed to account for linguistic thematic role assignment in natural language sentences. In addition, several features concerning biological plausibility are also included.

An Artificial Neural Network Model Based on Neuroscience: Looking Closely at the Brain

Classical connectionist models [3, 8, 11] are based upon a simple description of the neuron taking into account the presence of pre-synaptic cells and their synaptic potentials, the activation threshold, and the propagation of an action potential. Certainly, this is an impoverished explanation of human brain characteristics [1, 9, 12]. In this paper, a mechanism to generate a biologically plausible artificial neural network model is presented [10], which is taken to be closer to some of the human brain features. In such a mechanism, the classical framework is redesigned in order to encompass not only the “traditional” features but also labels that model the binding affinities between transmitters and receptors. This is accomplished by a restricted data set, which explains the neural network behavior. In addition to feedforward networks, the present model also contemplates recurrence in its architecture, which allows the system to have re-entrant connections [2].

Cognitive clustering algorithm for efficient cybersecurity applications

Cyber security is an important issue in todays global computer networks. Advanced clustering methods are relevant for efficient data mining over the web. KIII is a biologically plausible neural network model. In its multi-layer architecture there are excitatory and inhibitory neurons, which present lateral, feedforward, and delayed feedback connections between layers in a massive way. KIII has been successfully employed in classification and pattern recognition tasks. In this work we develop a methodology to use KIII for community detection. It is shown that clustering methods that employ KIII related to cybersecurity achieve better results, despite the amount of data available by such application.

Semantic Role Labeling for Brazilian Portuguese: A Benchmark

One of the main research challenges in Semantic Role Labeling (SRL) is the development of systems for languages other than English. For Brazilian Portuguese, a corpus with appropriate manually-annotated data, PropBank.Br, has recently become available. Proposals for implementing SRL systems using this corpus have already been made, but no standard way of comparing their results is available. We present a benchmark for comparing SRL systems for Brazilian Portuguese, based on the CoNLL Shared Tasks on SRL for English. Training and test data sets, evaluation metrics and a baseline system are provided as part of this benchmark. These resources have been used to implement a supervised SRL system which outperforms the baseline (17 points better in F 1 measure). Most importantly, the benchmark proved to be useful for evaluating the performance of SRL systems for Brazilian Portuguese.

Linguistic Relations Encoding in a Symbolic-Connectionist Hybrid Natural Language Processor

In recent years, the Natural Language Processing scene has witnessed the steady growth of interest in connectionist modeling. The main appeal of such an approach is that one does not have to determine the grammar rules in advance: the learning abilities displayed by such systems take care of input regularities. Better and faster learning can be obtained through the implementation of a symbolic-connectionist hybrid system. Such system combines the advantages of symbolic approaches, by introducing symbolic rules as network connection weights, with the advantages of connectionism. In a hybrid system called HTRP, words within a sentence are represented by means of semantic features. The features for the verbs are arranged along certain semantic dimensions, and are mutually exclusive within each dimension. One may infer that this happens because of the semantic features encoded in the network inputs.

Application and development of biologically plausible neural networks in a multiagent artificial life system

The present article introduces the system BioAnt, which is a computational simulation of a small colony of ants (up to 99 members) in which every ant relies on a biologically more plausible artificial neural networks as control mechanism for guidance. The environment, in which the ants are placed, is three-dimensional, consisting of the anthill, sugar, water, earth elevations, walls and predators. The ants’ foraging behavior was successfully implemented as well as some basic defense mechanisms. Typical sensors and actuators of ants were modeled and the efficiency of the connectionist approach has been validated by the comparison with a simple symbolical approach. Apart from several surprising results on technical details, which are reported, the present approach clearly demonstrates the feasibility of such an implementation with connectionist and biologically more plausible principles, offering promising perspectives as a basis for further artificial life systems.

A Connectionist Thematic Grid Predictor for Pre-parsed Natural Language Sentences

Inspired on psycholinguistics and neuroscience, a symbolic-connectionist hybrid system called i¾?- Pred (Thematic Pred ictor for natural language) is proposed, designed to reveal the thematic grid assigned to a sentence. Through a symbolic module, which includes anaphor resolution and relative clause processing, a parsing of the input sentence is performed, generating logical formulae based on events and thematic roles for Portuguese language sentences. Previously, a morphological analysis is carried out. The parsing displays, for grammatical sentences, the existing readings and their thematic grids. In order to disambiguate among possible interpretations, there is a connectionist module, comprising, as input, a featural representation of the words (based on verb/noun WordNetclassification and on classical semantic microfeature representation), and, as output, the thematic grid assigned to the sentence. i¾?- Pred employs biologically inspired training algorithm and architecture, adopting a psycholinguistic view of thematic theory.

Motor imagery classification for brain-computer interfaces through a chaotic neural network

In this paper, we propose to enhance the detection of control states in online brain-computer interfaces (BCI) with the use of the biologically inspired K-set neural network. This neural network was initially built to model brain waves of small sets of neurons in the brain and later showed a great capability of encoding complex and noisy data into oscillation patterns. We apply the K-set network to classification of motor imagery, a type of mental state very useful for BCI applications. Experimental results show that the network can work efficiently in this task and thus provide better control for BCI applications.