Weighted finite-state transducers (WFSTs) are commonly used by engineers and computational linguists for processing and generating speech and text. This book first provides a detailed introduction to this formalism. It then introduces Pynini, a Python library for compiling finite-state grammars and for combining, optimizing, applying, and searching finite-state transducers. This book illustrates this library's conventions and use with a series of case studies. These include the compilation and application of context-dependent rewrite rules, the construction of morphological analyzers and generators, and text generation and processing applications.

Finite-state devices, such as finite-state automata, graphs, and finite-state transducers, have been present since the emergence of computer science and are extensively used in areas as various as program compilation, hardware modeling, and database management. Although finite-state devices have been known for some time in computational linguistics, more powerful formalisms such as context-free grammars or unification grammars have typically been preferred. Recent mathematical and algorithmic results in the field of finite-state technology have had a great impact on the representation of electronic dictionaries and on natural language processing, resulting in a new technology for language emerging out of both industrial and academic research. This book presents a discussion of fundamental finite-state algorithms, and constitutes an approach from the perspective of natural language processing.

Weighted finite-state transducers (WFSTs) are commonly used by engineers and computational linguists for processing and generating speech and text. This book first provides a detailed introduction to this formalism. It then introduces Pynini, a Python library for compiling finite-state grammars and for combining, optimizing, applying, and searching finite-state transducers. This book illustrates this library's conventions and use with a series of case studies. These include the compilation and application of context-dependent rewrite rules, the construction of morphological analyzers and generators, and text generation and processing applications.

Covers the whole spectrum of finite-state methods, from theory to practical applications.

This handbook provides a comprehensive account of current research on the finite-state morphology of Georgian and enables the reader to enter quickly into Georgian morphosyntax and its computational processing. It combines linguistic analysis with application of finite-state technology to processing of the language. The book opens with the author’s synoptic overview of the main lines of research, covers the properties of the word and its components, then moves up to the description of Georgian morphosyntax and the morphological analyzer and generator of Georgian.The book comprises three chapters and accompanying appendices. The aim of the first chapter is to describe the morphosyntactic structure of Georgian, focusing on differences between Old and Modern Georgian. The second chapter focuses on the application of finite-state technology to the processing of Georgian and on the compilation of a tokenizer, a morphological analyzer and a generator for Georgian. The third chapter discusses the testing and evaluation of the analyzer’s output and the compilation of the Georgian Language Corpus (GLC), which is now accessible online and freely available to the research community.Since the development of the analyzer, the field of computational linguistics has advanced in several ways, but the majority of new approaches to language processing has not been tested on Georgian. So, the organization of the book makes it easier to handle new developments from both a theoretical and practical viewpoint.The book includes a detailed index and references as well as the full list of morphosyntactic tags. It will be of interest and practical use to a wide range of linguists and advanced students interested in Georgian morphosyntax generally as well as to researchers working in the field of computational linguistics and focusing on how languages with complicated morphosyntax can be handled through finite-state approaches.

These proceedings contain the final versions of the papers presented at the 7th International Workshop on Finite-State Methods and Natural Language Processing (FSMNLP), held in Ispra, Italy, on September 11–12, 2008. The aim of the FSMNLP workshops is to bring together members of the research and industrial community working on finite-state based models in language technology, computational linguistics, web mining, linguistics and cognitive science on one hand, and on related theory and methods in fields such as computer science and mathematics on the other. Thus, the workshop series is a forum for researchers and practitioners working on applications as well as theoretical and implementation aspects. The special theme of FSMNLP 2008 was high performance finite-state devices in large-scale natural language text processing systems and applications. The papers in this publication cover a range of interesting NLP applications, including machine learning and translation, logic, computational phonology, morphology and semantics, data mining, information extraction and disambiguation, as well as programming, optimization and compression of finite-state networks. The applied methods include weighted algorithms, kernels and tree automata. In addition, relevant aspects of software engineering, standardization and European funding programmes are discussed.

This book constitutes the refereed proceedings of the 10th International Conference on Computational Linguistics and Intelligent Text Processing, CICLing 2009, held in Mexico City, Mexico in March 2009. The 44 revised full papers presented together with 4 invited papers were carefully reviewed and selected from numerous submissions. The papers cover all current issues in computational linguistics research and present intelligent text processing applications.