Many applications within natural language processing involve performing text-to-text transformations, i.e., given a text in natural language as input, systems are required to produce a version of this text (e.g., a translation), also in natural language, as output. Automatically evaluating the output of such systems is an important component in developing text-to-text applications. Two approaches have been proposed for this problem: (i) to compare the system outputs against one or more reference outputs using string matching-based evaluation metrics and (ii) to build models based on human feedback to predict the quality of system outputs without reference texts. Despite their popularity, reference-based evaluation metrics are faced with the challenge that multiple good (and bad) quality outputs can be produced by text-to-text approaches for the same input. This variation is very hard to capture, even with multiple reference texts. In addition, reference-based metrics cannot be used in production (e.g., online machine translation systems), when systems are expected to produce outputs for any unseen input. In this book, we focus on the second set of metrics, so-called Quality Estimation (QE) metrics, where the goal is to provide an estimate on how good or reliable the texts produced by an application are without access to gold-standard outputs. QE enables different types of evaluation that can target different types of users and applications. Machine learning techniques are used to build QE models with various types of quality labels and explicit features or learnt representations, which can then predict the quality of unseen system outputs. This book describes the topic of QE for text-to-text applications, covering quality labels, features, algorithms, evaluation, uses, and state-of-the-art approaches. It focuses on machine translation as application, since this represents most of the QE work done to date. It also briefly describes QE for several other applications, including text simplification, text summarization, grammatical error correction, and natural language generation.

Recent Advances in Example-Based Machine Translation is of relevance to researchers and program developers in the field of Machine Translation and especially Example-Based Machine Translation, bilingual text processing and cross-linguistic information retrieval. It is also of interest to translation technologists and localisation professionals. Recent Advances in Example-Based Machine Translation fills a void, because it is the first book to tackle the issue of EBMT in depth. It gives a state-of-the-art overview of EBMT techniques and provides a coherent structure in which all aspects of EBMT are embedded. Its contributions are written by long-standing researchers in the field of MT in general, and EBMT in particular. This book can be used in graduate-level courses in machine translation and statistical NLP.

How Machine Learning can improve machine translation: enabling technologies and new statistical techniques.

Machine translation of natural languages is one of the most complex and comprehensive applications of computational linguistics and artificial intelligence. This is especially true of knowledge-based machine translation (KBMT) systems, which require many knowledge resources and processing modules to carry out the necessary levels of analysis, representation and generation of meaning and form. The number of real-world problems, tasks, and solutions involved in developing any realistic-size knowledge-based machine translation system is enormous. It is thus difficult for researchers in the field to learn what a system "really does". This book fills that need with a detailed case study of a KBMT system implemented at the Center for Machine Translation at Carnegie Mellon University. The research consists in part of the creation of a system for translation between English and Japanese. The corpora used in the project were manuals for installing and maintaining IBM personal computers (sponsorship by IBM, through its Tokyo Research Laboratory) Individual chapters describe the interlingua texts used in knowledge-based machine translation, the grammar formalism embodied in the system, the grammars and lexicons and their roles in the translation process, the process of source language analysis, an augmentation module that interactively and automatically resolves ambiguities remaining after source language analysis, and the generator, which produces target language sentences. Detailed appendices illustrate the process from analysis through generation. This book is intended for developers, researchers and advanced students in natural language processing and computational linguistics, including all those who have an interest in machine translation and machine-aided translation.

This book reviews ways to improve statistical machine speech translation between Polish and English. Research has been conducted mostly on dictionary-based, rule-based, and syntax-based, machine translation techniques. Most popular methodologies and tools are not well-suited for the Polish language and therefore require adaptation, and language resources are lacking in parallel and monolingual data. The main objective of this volume to develop an automatic and robust Polish-to-English translation system to meet specific translation requirements and to develop bilingual textual resources by mining comparable corpora.