Federico Gaspari

A survey of machine translation competences: Insights for translation technology educators and practitioners

This paper describes a large-scale survey of machine translation (MT) competencies conducted by a non-commercial and publicly funded European research project. Firstly, we highlight the increased prevalence of translation technologies in the translation and localisation industry, and develop upon this by reporting on survey data derived from 438 validated respondents, including freelance translators, language service providers, translator trainers, and academics. We then focus on ascertaining the prevalence of translation technology usage on a fine-grained scale to address aspects of MT, quality assessment techniques and post-editing. We report a strong need for an improvement in quality assessment methods, tools, and training, partly due to the large variance in approaches and combinations of methods, and to the lack of knowledge and resources. We note the growing uptake of MT and the perceived increase of its prevalence in future workflows. We find that this adoption of MT has led to significant changes in the human translation process, in which post-editing appears to be exclusively used for high-quality content publication. Lastly, we echo the needs of the translation industry and community in an attempt to provide a more comprehensive snapshot to inform the provision of translation training and the need for increased technical competencies.

Is Neural Machine Translation the New State of the Art

Abstract This paper discusses neural machine translation (NMT), a new paradigm in the MT field, comparing the quality of NMT systems with statistical MT by describing three studies using automatic and human evaluation methods. Automatic evaluation results presented for NMT are very promising, however human evaluations show mixed results. We report increases in fluency but inconsistent results for adequacy and post-editing effort. NMT undoubtedly represents a step forward for the MT field, but one that the community should be careful not to oversell.

Online MT Services and Real Users’ Needs: An Empirical Usability Evaluation

This paper presents an empirical evaluation of the main usability factors that play a significant role in the interaction with on-line Machine Translation (MT) services. The investigation is carried out from the point of view of typical users with an emphasis on their real needs, and focuses on a set of key usability criteria that have an impact on the successful deployment of Internet-based MT technology. A small-scale evaluation of the performance of five popular web-based MT systems against the selected usability criteria shows that different approaches to interaction design can dramatically affect the level of user satisfaction. There are strong indications that the results of this study can be fed back into the development of on-line MT services to enhance their design, thus ensuring that they meet the requirements and expectations of a wide range of Internet users.

DELiC4MT: A Tool for Diagnostic MT Evaluation over User-defined Linguistic Phenomena

DELiC4MT: A Tool for Diagnostic MT Evaluation over User-defined Linguistic Phenomena This paper demonstrates DELiC4MT, a piece of software that allows the user to perform diagnostic evaluation of machine translation systems over linguistic checkpoints, i.e., source-language lexical elements and grammatical constructions specified by the user. Our integrated tool builds upon best practices, software components and formats developed under different projects and initiatives, focusing on enabling easy adaptation to any language pair and linguistic phenomenon. We provide a description of the different modules that make up the tool, introduce a web demo and present a step-by-step case study of how it can be applied to a specific language pair and linguistic phenomenon.

On Education and Training in Translation Quality Assessment

In this chapter, we argue that education and training in translation quality assessment (TQA)is being neglected for most, if not all, stakeholders of the translation process, from translators, post-editors, and reviewers to buyers and end-users of translation products and services. Within academia, there is a lack of education and training opportunities to equip translation students, even at postgraduate level, with the knowledge and skills required to understand and use TQA. This has immediate effects on their employability and long-term effects on professional practice. In discussing and building upon previous initiatives to tackle this issue, we provide a range of viewpoints and resources for the provision of such opportunities in collaborative and independent contexts across all modes and academic settings, focusing not just on TQA and machine translation training, but also on the use of assessment strategies in educational contexts that are directly relevant to those used in industry. In closing, we reiterate our argument for the importance of education and training in TQA, on the basis of all the contributions and perspectives presented in the volume.

Exploring Expo Milano 2015: a cross-linguistic comparison of food-related phraseology in translation using a comparallel corpus approach

This article compares the food-related phraseology used in the official online descriptions in English and Italian of the 116 national pavilions at the universal exhibition Expo Milano 2015, whose main theme was ‘Feeding the Planet, Energy for Life’. An innovative comparallel corpus methodology is adopted to investigate the bilingual phraseology celebrating the best culinary products and gastronomic traditions from all over the world; in spite of consisting of seemingly parallel texts, the pavilion descriptions in English and Italian are analysed as a comparable corpus: even though the corpus data were collected from corresponding localised versions of the Expo Milano 2015 website, a classic analysis of the English and Italian parallel texts was impracticable, because many of these descriptions were bound to be themselves translations from the official languages of the respective countries, resulting in a chaotic mix of language pairs and translation directions. The cross-linguistic phraseological comparison of the formal and functional features of the most frequent lexical items used to describe food cultures from across the world in the two languages shows that the English keywords tend to have a much broader and more diverse collocational range than their prima facie translation equivalents in Italian, which are subject to more severe co-selectional restrictions. This sheds light not only on how Expo Milano 2015 presented itself to the online global audience, but also, and more crucially, on how multilingual translated food-related phraseology shaped the perception of the key themes of this world-scale event across different linguistic and cultural communities.

Approaches to Human and Machine Translation Quality Assessment

In both research and practice, translation quality assessment is a complex task involving a range of linguistic and extra-linguistic factors. This chapter provides a critical overview of the established and developing approaches to the definition and measurement of translation quality in human and machine translation workflows across a range of research, educational, and industry scenarios. We intertwine literature from several interrelated disciplines dealing with contemporary translation quality assessment and, while we acknowledge the need for diversity in these approaches, we argue that there are fundamental and widespread issues that remain to be addressed, if we are to consolidate our knowledge and practice of translation quality assessment in increasingly technologised environments across research, teaching, and professional practice.

Evaluating MT for massive open online courses

This article reports a multifaceted comparison between statistical and neural machine translation (MT) systems that were developed for translation of data from massive open online courses (MOOCs). The study uses four language pairs: English to German, Greek, Portuguese, and Russian. Translation quality is evaluated using automatic metrics and human evaluation, carried out by professional translators. Results show that neural MT is preferred in side-by-side ranking, and is found to contain fewer overall errors. Results are less clear-cut for some error categories, and for temporal and technical post-editing effort. In addition, results are reported based on sentence length, showing advantages and disadvantages depending on the particular language pair and MT paradigm.

Enhancing Machine Translation of Academic Course Catalogues with Terminological Resources

This paper describes an approach to translating course unit descriptions from Italian and German into English, using a phrase-based machine translation (MT) system. The genre is very prominent among those requiring translation by universities in European countries in which English is a non-native language. For each language combination, an in-domain bilingual corpus including course unit and degree program descriptions is used to train an MT engine, whose output is then compared to a baseline engine trained on the Europarl corpus. In a subsequent experiment, a bilingual terminology database is added to the training sets in both engines and its impact on the output quality is evaluated based on BLEU and postediting score. Results suggest that the use of domain-specific corpora boosts the engines quality for both language combinations, especially for German-English, whereas adding terminological resources does not seem to bring notable benefits.