This volume presents the proceedings of the Fourth International Workshop on Analogical and Inductive Inference (AII '94) and the Fifth International Workshop on Algorithmic Learning Theory (ALT '94), held jointly at Reinhardsbrunn Castle, Germany in October 1994. (In future the AII and ALT workshops will be amalgamated and held under the single title of Algorithmic Learning Theory.) The book contains revised versions of 45 papers on all current aspects of computational learning theory; in particular, algorithmic learning, machine learning, analogical inference, inductive logic, case-based reasoning, and formal language learning are addressed.

Algorithmic learning theory is mathematics about computer programs which learn from experience. This involves considerable interaction between various mathematical disciplines including theory of computation, statistics, and c- binatorics. There is also considerable interaction with the practical, empirical ?elds of machine and statistical learning in which a principal aim is to predict, from past data about phenomena, useful features of future data from the same phenomena. The papers in this volume cover a broad range of topics of current research in the ?eld of algorithmic learning theory. We have divided the 29 technical, contributed papers in this volume into eight categories (corresponding to eight sessions) re?ecting this broad range. The categories featured are Inductive Inf- ence, Approximate Optimization Algorithms, Online Sequence Prediction, S- tistical Analysis of Unlabeled Data, PAC Learning & Boosting, Statistical - pervisedLearning,LogicBasedLearning,andQuery&ReinforcementLearning. Below we give a brief overview of the ?eld, placing each of these topics in the general context of the ?eld. Formal models of automated learning re?ect various facets of the wide range of activities that can be viewed as learning. A ?rst dichotomy is between viewing learning as an inde?nite process and viewing it as a ?nite activity with a de?ned termination. Inductive Inference models focus on inde?nite learning processes, requiring only eventual success of the learner to converge to a satisfactory conclusion.

An accessible introduction and essential reference for an approach to machine learning that creates highly accurate prediction rules by combining many weak and inaccurate ones. Boosting is an approach to machine learning based on the idea of creating a highly accurate predictor by combining many weak and inaccurate “rules of thumb.” A remarkably rich theory has evolved around boosting, with connections to a range of topics, including statistics, game theory, convex optimization, and information geometry. Boosting algorithms have also enjoyed practical success in such fields as biology, vision, and speech processing. At various times in its history, boosting has been perceived as mysterious, controversial, even paradoxical. This book, written by the inventors of the method, brings together, organizes, simplifies, and substantially extends two decades of research on boosting, presenting both theory and applications in a way that is accessible to readers from diverse backgrounds while also providing an authoritative reference for advanced researchers. With its introductory treatment of all material and its inclusion of exercises in every chapter, the book is appropriate for course use as well. The book begins with a general introduction to machine learning algorithms and their analysis; then explores the core theory of boosting, especially its ability to generalize; examines some of the myriad other theoretical viewpoints that help to explain and understand boosting; provides practical extensions of boosting for more complex learning problems; and finally presents a number of advanced theoretical topics. Numerous applications and practical illustrations are offered throughout.

This book constitutes the refereed proceedings of the 16th International Conference on Algorithmic Learning Theory, ALT 2005, held in Singapore in October 2005. The 30 revised full papers presented together with 5 invited papers and an introduction by the editors were carefully reviewed and selected from 98 submissions. The papers are organized in topical sections on kernel-based learning, bayesian and statistical models, PAC-learning, query-learning, inductive inference, language learning, learning and logic, learning from expert advice, online learning, defensive forecasting, and teaching.