Anticipatory Learning Classifier Systems describes the state of the art of anticipatory learning classifier systems-adaptive rule learning systems that autonomously build anticipatory environmental models. An anticipatory model specifies all possible action-effects in an environment with respect to given situations. It can be used to simulate anticipatory adaptive behavior. Anticipatory Learning Classifier Systems highlights how anticipations influence cognitive systems and illustrates the use of anticipations for (1) faster reactivity, (2) adaptive behavior beyond reinforcement learning, (3) attentional mechanisms, (4) simulation of other agents and (5) the implementation of a motivational module. The book focuses on a particular evolutionary model learning mechanism, a combination of a directed specializing mechanism and a genetic generalizing mechanism. Experiments show that anticipatory adaptive behavior can be simulated by exploiting the evolving anticipatory model for even faster model learning, planning applications, and adaptive behavior beyond reinforcement learning. Anticipatory Learning Classifier Systems gives a detailed algorithmic description as well as a program documentation of a C++ implementation of the system.

Anticipatory Learning Classifier Systems describes the state of the art of anticipatory learning classifier systems-adaptive rule learning systems that autonomously build anticipatory environmental models. An anticipatory model specifies all possible action-effects in an environment with respect to given situations. It can be used to simulate anticipatory adaptive behavior. Anticipatory Learning Classifier Systems highlights how anticipations influence cognitive systems and illustrates the use of anticipations for (1) faster reactivity, (2) adaptive behavior beyond reinforcement learning, (3) attentional mechanisms, (4) simulation of other agents and (5) the implementation of a motivational module. The book focuses on a particular evolutionary model learning mechanism, a combination of a directed specializing mechanism and a genetic generalizing mechanism. Experiments show that anticipatory adaptive behavior can be simulated by exploiting the evolving anticipatory model for even faster model learning, planning applications, and adaptive behavior beyond reinforcement learning. Anticipatory Learning Classifier Systems gives a detailed algorithmic description as well as a program documentation of a C++ implementation of the system.

Learning Classifier Systems (LCS) are a machine learning paradigm introduced by John Holland in 1976. They are rule-based systems in which learning is viewed as a process of ongoing adaptation to a partially unknown environment through genetic algorithms and temporal difference learning. This book provides a unique survey of the current state of the art of LCS and highlights some of the most promising research directions. The first part presents various views of leading people on what learning classifier systems are. The second part is devoted to advanced topics of current interest, including alternative representations, methods for evaluating rule utility, and extensions to existing classifier system models. The final part is dedicated to promising applications in areas like data mining, medical data analysis, economic trading agents, aircraft maneuvering, and autonomous robotics. An appendix comprising 467 entries provides a comprehensive LCS bibliography.

This book constitutes the thoroughly refereed post-proceedings of the 4th International Workshop on Learning Classifier Systems, IWLCS 2001, held in San Francisco, CA, USA, in July 2001. The 12 revised full papers presented together with a special paper on a formal description of ACS have gone through two rounds of reviewing and improvement. The first part of the book is devoted to theoretical issues of learning classifier systems including the influence of exploration strategy, self-adaptive classifier systems, and the use of classifier systems for social simulation. The second part is devoted to applications in various fields such as data mining, stock trading, and power distributionn networks.

This book constitutes the refereed proceedings of the 5th International Workshop on Learning Classifier Systems, IWLCS 2003, held in Granada, Spain in September 2003 in conjunction with PPSN VII. The 10 revised full papers presented together with a comprehensive bibliography on learning classifier systems were carefully reviewed and selected during two rounds of refereeing and improvement. All relevant issues in the area are addressed.

Anticipatory behavior in adaptive learning systems continues attracting attention of researchers in many areas, including cognitive systems, neuroscience, psychology, and machine learning. This book constitutes the thoroughly refereed post-workshop proceedings of the 4th International Workshop on Anticipatory Behavior in Adaptive Learning Systems, ABiALS 2008, held in Munich, Germany, in June 2008, in collaboration with the six-monthly Meeting of euCognition 'The Role of Anticipation in Cognition'. The 18 revised full papers presented were carefully selected during two rounds of reviewing and improvement for inclusion in the book. The introductory chapter of this state-of-the-art survey not only provides an overview of the contributions included in this volume but also revisits the current available terminology on anticipatory behavior and relates it to the available system approaches. The papers are organized in topical sections on anticipation in psychology with focus on the ideomotor view, conceptualizations, anticipation and dynamical systems, computational modeling of psychological processes in the individual and social domains, behavioral and cognitive capabilities based on anticipation, and computational frameworks and algorithms for anticipation, and their evaluation.

The significantly expanded and updated new edition of a widely used text on reinforcement learning, one of the most active research areas in artificial intelligence. Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms. This second edition has been significantly expanded and updated, presenting new topics and updating coverage of other topics. Like the first edition, this second edition focuses on core online learning algorithms, with the more mathematical material set off in shaded boxes. Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found. Many algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on such topics as artificial neural networks and the Fourier basis, and offers expanded treatment of off-policy learning and policy-gradient methods. Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience, as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero, Atari game playing, and IBM Watson's wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.