Priscila M. V. Lima

Multilingual part-of-speech tagging with weightless neural networks

Training part-of-speech taggers (POS-taggers) requires iterative time-consuming convergence-dependable steps, which involve either expectation maximization or weight balancing processes, depending on whether the tagger uses stochastic or neural approaches, respectively. Due to the complexity of these steps, multilingual part-of-speech tagging can be an intractable task, where as the number of languages increases so does the time demanded by these steps. WiSARD (Wilkie, Stonham and Aleksanders Recognition Device), a weightless artificial neural network architecture that proved to be both robust and efficient in classification tasks, has been previously used in order to turn the training phase faster. WiSARD is a RAM-based system that requires only one memory writing operation to train each sentence. Additionally, the mechanism is capable of learning new tagged sentences during the classification phase, on an incremental basis. Nevertheless, parameters such as RAM size, context window, and probability bit mapping, make the multilingual part-of-speech tagging task hard. This article proposes mWANN-Tagger (multilingual Weightless Artificial Neural Network tagger), a WiSARD POS-tagger. This tagger is proposed due to its one-pass learning capability. It allows language-specific parameter configurations to be thoroughly searched in quite an agile fashion. Experimental evaluation indicates that mWANN-Tagger either outperforms or matches state-of-art methods in accuracy with very low standard deviation, i.e., lower than 0.25%. Experimental results also suggest that the vast majority of the languages can benefit from this architecture.

A weightless neural network-based approach for stream data clustering

One of the major data mining tasks is to cluster similar data, because of its usefulness, providing means of summarizing large ammounts of raw data into handy information. Clustering data streams is particularly challenging, because of the constraints imposed when dealing with this kind of input. Here we report our work, in which it was investigated the use of WiSARD discriminators as primary data synthesizing units. An analysis of StreamWiSARD, a new sliding-window stream data clustering system, the benefits and the drawbacks of its use and a comparison to other approaches are all presented.

Optimized RFV analysis

Purpose – In the classic recency‐frequency‐monetary value (RFV or RFM) approach to market segmentation, customers are grouped together into an arbitrary number of segments according to data on their most recent day of purchase (R), the number of buying orders placed (F) and the total monetary value of their purchases (V). The purpose of this paper is to show how to select the order in which the RFV dimensions are applied to data and choose the number of segments and the time frame used in such a way as to maximize the results of direct marketing campaigns.Design/methodology/approach – A “genetically” optimized RFV model is built from data collected from a real world direct marketing campaign. The results produced when it is used are compared with the results yielded without the use of any forecasting method at all and with the support of a widely used basic RFV model.Findings – Not only does the new model provide better results, but it is also easy to build and allows for the introduction of new dimension...

Financial credit analysis via a clustering weightless neural classifier

Credit analysis is a real-world classification problem where it is quite common to find datasets with a large amount of noisy data. State-of-the-art classifiers that employ error minimisation techniques, on the other hand, require a long time to converge, in order to achieve robustness. This paper explores ClusWiSARD, a clustering customisation of the WiSARD weightless neural network model, applied to two different credit analysis real-world problems. Experimental evidence shows that ClusWiSARD is very competitive with Support Vector Machine (SVM) w.r.t. accuracy, with the advantage of being capable of online learning. ClusWiSARD outperforms SVM in training time, by two orders of magnitude, and is slightly faster in test time.

A distributed dynamics for webgraph decontamination

In order to increase the visibility of a target pageT, web spammers create hyperlink structures called web bubbles, or link farms. As countermeasure, special mobile agents, called web marshals, are deployed in the detection and disassembling of link farms. Interestingly, the process of minimizing the number of web marshals and the number of hops needed to dismantle a web bubble is analogous to the graph decontamination problem. A novel distributed algorithm for graph decontamination, which can be used to define the behavior of web marshals, is introduced in this work. The new algorithm is asynchronous and topology independent. Moreover, it presents equal or better performance and needs smaller numbers of web marshals when compared to recent related works targeting only circulant graphs, a typical structure of link farms.

Mapping and combining combinatorial problems into energy landscapes via pseudo-boolean constraints

This paper introduces a novel approach to the specification of hard combinatorial problems as pseudo-Boolean constraints. It is shown (i) how this set of constraints defines an energy landscape representing the space state of solutions of the target problem, and (ii) how easy is to combine different problems into new ones mostly via the union of the corresponding constraints. Graph colouring and Traveling Salesperson Problem (TSP) were chosen as the basic problems from which new combinations were investigated. Higher-order Hopfield networks of stochastic neurons were adopted as search engines in order to solve the mapped problems.

The Influence of Order on a Large Bag of Words

Text classification has been mostly performed through implicit semantic correlation techniques, such as latent semantic analysis. This approach however, has proved insufficient for situations where there are short texts to be classified into one or more from many classes. That is the case of the classification of statements of purpose of Brazilian companies, according to the around one thousand and eight hundred categories of the government administration detailment of National Classification of Economical Activities (CNAE), CNAE-Subclasses. The impact of the order of words in a text is evaluated by comparing the performance of three classifiers based on the weightless artificial neural model, WISARD. Results point to the need of combining semantic with syntactic information in order to improve the classifiers performance.

A dynamic binding mechanism for retrieving and unifying complex predicate-logic knowledge

We show how to encode, retrieve and process complex structures equivalent to First-Order Logic (FOL) formulae, with Artificial Neural Networks (ANNs) designed for energy-minimization. The solution constitutes a binding mechanism that uses a neural Working Memory (WM) and a long-term synaptic memory (LTM) that can store both procedural and declarative FOL-like Knowledge-Base (KB). Complex structures stored in LTM are retrieved into the WM only upon need, where they are further processed. The power of our binding mechanism is demonstrated on unification problems: as neurons are dynamically allocated from a pool, most generally unified structures emerge at equilibrium. The networks size is O(n·k), where n is the size of the retrieved FOL structures and k is the size of the KB. The mechanism is fault-tolerant, as no fatal failures occur when random units fail. The paradigm can be used in a variety of applications, such as language processing, reasoning and planning.

A universal multilingual weightless neural network tagger via quantitative linguistics

In the last decade, given the availability of corpora in several distinct languages, research on multilingual part-of-speech tagging started to grow. Amongst the novelties there is mWANN-Tagger (multilingual weightless artificial neural network tagger), a weightless neural part-of-speech tagger capable of being used for mostly-suffix-oriented languages. The tagger was subjected to corpora in eight languages of quite distinct natures and had a remarkable accuracy with very low sample deviation in every one of them, indicating the robustness of weightless neural systems for part-of-speech tagging tasks. However, mWANN-Tagger needed to be tuned for every new corpus, since each one required a different parameter configuration. For mWANN-Tagger to be truly multilingual, it should be usable for any new language with no need of parameter tuning. This article proposes a study that aims to find a relation between the lexical diversity of a language and the parameter configuration that would produce the best performing mWANN-Tagger instance. Preliminary analyses suggested that a single parameter configuration may be applied to the eight aforementioned languages. The mWANN-Tagger instance produced by this configuration was as accurate as the language-dependent ones obtained through tuning. Afterwards, the weightless neural tagger was further subjected to new corpora in languages that range from very isolating to polysynthetic ones. The best performing instances of mWANN-Tagger are again the ones produced by the universal parameter configuration. Hence, mWANN-Tagger can be applied to new corpora with no need of parameter tuning, making it a universal multilingual part-of-speech tagger. Further experiments with Universal Dependencies treebanks reveal that mWANN-Tagger may be extended and that it has potential to outperform most state-of-the-art part-of-speech taggers if better word representations are provided.

Spatio-temporal Pattern Classification with KernelCanvas and WiSARD

This work proposes a new method, KernelCanvas, that is adequate to the Weightless Neural Model known as WiSARD for generating a fixed length binary input from spatio-temporal patterns. The method, based on kernel distances, is simple to implement and scales linearly to the number of kernels. Five different datasets were used to evaluate its performance in comparison with more widely employed approaches. One dataset was related to human movements, two to handwritten characters, one to speaker recognition and the last one to speech recognition. The KernelCanvas combined with WiSARD classifier approach frequently achieved the highest scores, sometimes losing only for the much slower K-Nearest Neighbors approach. In comparison with other results in the literature, our model has performed better or very close to them in all datasets.