Accompanying continued industrial production and sales of artificial intelligence and expert systems is the risk that difficult and resistant theoretical problems and issues will be ignored. The participants at the Third Tinlap Workshop, whose contributions are contained in Theoretical Issues in Natural Language Processing, remove that risk. They discuss and promote theoretical research on natural language processing, examinations of solutions to current problems, development of new theories, and representations of published literature on the subject. Discussions among these theoreticians in artificial intelligence, logic, psychology, philosophy, and linguistics draw a comprehensive, up-to-date picture of the natural language processing field.

Four separate essays address the complex and difficult connections among grammatical theory, mathematical linguistics, and the operation of real natural-language-processing systems, both human and electronic.William Rounds, Avarind Joshi, Janet Fodor, and Robert Berwick are leading scholars in the multidisciplinary field of natural language processing. In four separate essays they address the complex and difficult connections among grammatical theory, mathematical linguistics, and the operation of real natural-language-processing systems, both human and electronic. The editors' substantial introduction details the progress and problems involved in attempts to relate these four areas of research. William Rounds discusses the relevance of complexity results to linguistics and computational linguistics, providing useful caveats about how results might be misinterpreted and pointing out promising avenues of future research. Avarind Joshi (with K. Vijay-Shanker and David Weir) surveys results showing the equivalence of several different grammatical formalisms, all of which are mildly context-sensitive, with special attention to variants of tree adjoining grammar. Janet Fodor discusses how psycholinguistic results can bear on the choice among competing grammatical theories, surveying a number of recent experiments and their relevance to issues in grammatical theory. Robert Berwick considers the relationship between issues in linguistic theory and the construction of computational parsing systems, in particular the question of what it means to implement a theory of grammar in a computational system. He argues for the advantages of a principle-based approach over a rule-based one, and surveys several recent parsing systems based on the theory of government and binding.

Embeddings have undoubtedly been one of the most influential research areas in Natural Language Processing (NLP). Encoding information into a low-dimensional vector representation, which is easily integrable in modern machine learning models, has played a central role in the development of NLP. Embedding techniques initially focused on words, but the attention soon started to shift to other forms: from graph structures, such as knowledge bases, to other types of textual content, such as sentences and documents. This book provides a high-level synthesis of the main embedding techniques in NLP, in the broad sense. The book starts by explaining conventional word vector space models and word embeddings (e.g., Word2Vec and GloVe) and then moves to other types of embeddings, such as word sense, sentence and document, and graph embeddings. The book also provides an overview of recent developments in contextualized representations (e.g., ELMo and BERT) and explains their potential in NLP. Throughout the book, the reader can find both essential information for understanding a certain topic from scratch and a broad overview of the most successful techniques developed in the literature.

This comprehensive reference work provides an overview of the concepts, methodologies, and applications in computational linguistics and natural language processing (NLP). Features contributions by the top researchers in the field, reflecting the work that is driving the discipline forward Includes an introduction to the major theoretical issues in these fields, as well as the central engineering applications that the work has produced Presents the major developments in an accessible way, explaining the close connection between scientific understanding of the computational properties of natural language and the creation of effective language technologies Serves as an invaluable state-of-the-art reference source for computational linguists and software engineers developing NLP applications in industrial research and development labs of software companies