Daša Munková

Data Pre-processing Evaluation for Text Mining: Transaction/Sequence Model

Abstract Data pre-processing presents the most time consuming phase in the whole process of knowledge discovery. The complexity of data pre-processing depends on the data sources used. The aim of this work is to determine to what extent it is necessary to carry out the time consuming data pre-processing in the process of discovering sequential patterns in e-documents. We used the transaction/sequence model for text representation and sequence rule analysis as a method of modelling. We compare four datasets of different quality obtained from texts and pre-processed in different ways: data with identified the paragraph sequences, data with identified the sentence sequences, data with identified the paragraph sequences without stop words and data with identified the sentence sequences without stop words. We try to assess the impact of these advanced techniques of data pre-processing on the quantity and quality of the extracted rules. The results confirm some initial assumptions, but they also show that the stop words removal has a substantial impact on the quantity and quality of extracted rules in case of paragraph sequence identification. Contrary, in case of sentence sequence identification, removing the stop words has not any significant impact on the quantity and quality of extracted rules.

Influence of Stop-Words Removal on Sequence Patterns Identification within Comparable Corpora

Short texts like advertisements are characterised by a number of slogans, phrases, words, symbols etc. To improve the quality of textual data, it is necessary to filter out noise textual data from important data. The aim of this work is to determine to what extent it is necessary to carry out the time consuming data pre-processing in the process of discovering sequential patterns in English and Slovak advertisement corpora. For this purpose, an experiment was conducted focusing on data pre-processing in these two comparable corpora. We try to find out to what extent removing the stop words has an influence on a quantity and quality of extracted rules. Stop words removal has no impact on the quantity and quality of extracted rules in English as well as in Slovak advertisement corpora. Only language has a significant impact on the quantity and quality of extracted rules.

Modelling of Language Processing Dependence on Morphological Features

The order, association and variability of the advertising language is different in every language and culture, because it is based on different rules in the given culture. Therefore, the study is focused on comparative linguistic data analysis of advertisements written in Slovak and English randomly collected from online sources. The transaction/sequence model for text representation was used and an association rules analysis was applied as the research method. The results are significant mainly in terms of the differences in the incidence of parts of speech in English and Slovak written advertisements. Based on the morphological features of the examined languages, different models of language of advertising were being created.

An automatic evaluation of machine translation and Slavic languages

Evaluation plays a crucial role in the field of machine translation. The aim of our research was to assess the quality - reliability and validity - of automatic evaluation of machine translation, and in our case an on-line statistical machine translation system was used. The results of the reliability analysis showed that the chosen metrics of automatic evaluation are reliable and valid, whereby the validity and reliability were verified for the following language pairs: Slovak-English and Slovak-German.

Identification of Relevant and Redundant Automatic Metrics for MT Evaluation

The paper is aimed at automatic metrics for translation quality assessment (TQA), specifically at machine translation (MT) output and the metrics for the evaluation of MT output (Precision, Recall, F-measure, BLEU, PER, WER and CDER). We examine their reliability and we determine the metrics which show decreasing reliability of the automatic evaluation of MT output. Besides the traditional measures (Cronbach’s alpha and standardized alpha) we use entropy for assessing the reliability of the automatic metrics of MT output. The results were obtained on a dataset covering translation from a low resource language (SK) into English (EN). The main contribution consists of the identification of the redundant automatic MT evaluation metrics.

Automatic Metrics for Machine Translation Evaluation and Minority Languages

Translation quality and its evaluation play a crucial role in the field of machine translation (MT). This paper focuses on the quality assessment of automatic metrics for MT evaluation. In our study we assess the reliability and validity of the following automatic metrics: Position-independent Error Rate (PER), Word Error Rate (WER) and Cover Disjoint Error Rate (CDER). These metrics define an error rate of MT output and also of MT system itself, in our case it is an on-line statistical MT system. The results of the reliability analysis showed that these automatic metrics for MT evaluation are reliable and valid, whereby the validity and reliability were verified for one translation direction: from the minority language (Slovak) into English.

Automatic Evaluation of Machine Translation Through the Residual Analysis

In this study we aim at the automatic evaluation of machine translation through the residual analysis at the sentence level. We created a dataset, which covered one translation direction- a translation from an inflective language (Slovak) into an analytical language (English). BLEU (Bilingual Evaluation Understudy) as a state-of-the-art automatic metric for machine translation evaluation was used. The main contribution consists of rigorous technique (statistical method), novel to research of MT evaluation given by the residual analysis to identify differences between MT output and post-edited machine translation output.

Evaluation of machine translation output in context of inflectional languages

Objective of the paper is to evaluate metrics of automatic evaluation of machine translation output using manual metrics — fluency and adequacy. We tried to answer the question to which extent the manual evaluation correlates with the automatic evaluation of MT output from/to Slovak to/from English. We focused on metrics based on the similarity and statistical principles (WER, PER, CDER and BLEU-n). We found out, that the manual evaluation, namely fluency and adequacy metrics correlates with automatic metrics of MT evaluation for less spoken language and low resource language such as Slovak. The contribution also consists of system proposal for both, manual (based on POS tagging) and automatic (based on reference) evaluation of MT output.

Language Processing and Human Cognition

The order, association and variability of the language of request are different in every language and culture, because it is based not only on different rules in the given culture but also on the dichotomy of language processing- global and detail language processing. The study is focused on the investigation of modelling language processing in terms of human cognition. The transaction/sequence model for text representation was used and an association rules analysis was applied as the research method. Based on the human cognition (category width cognitive style) of examined texts, different models of language processing were being created.