This book looks at multiagent systems that consist of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments. This book looks at multiagent systems that consist of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments. The book makes four main contributions to the fields of machine learning and multiagent systems. First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods. Third, the book introduces a new multiagent reinforcement learning algorithm—team-partitioned, opaque-transition reinforcement learning (TPOT-RL)—designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multiagent system that incorporates learning in a real-time, noisy domain with teammates and adversaries—a computer-simulated robotic soccer team. Peter Stone's work is the basis for the CMUnited Robotic Soccer Team, which has dominated recent RoboCup competitions. RoboCup not only helps roboticists to prove their theories in a realistic situation, but has drawn considerable public and professional attention to the field of intelligent robotics. The CMUnited team won the 1999 Stockholm simulator competition, outscoring its opponents by the rather impressive cumulative score of 110-0.

Multi-agent systems in complex, real-time domains require agents to act effectively both autonomously and as part of a team. This dissertation addresses multi-agent systems consisting of teams of autonomous agents acting in real-time, noisy, collaborative, and adversarial environments. Because of the inherent complexity of this type of multi-agent system, this thesis investigates the use of machine learning within multi-agent systems. The dissertation makes four main contributions to the fields of Machine Learning and Multi-Agent Systems. First, the thesis defines a team member agent architecture within which a flexible team structure is presented, allowing agents to decompose the task space into flexible roles and allowing them to smoothly switch roles while acting. Team organization is achieved by the introduction of a locker-room agreement as a collection of conventions followed by all team members. It defines agent roles, team formations, and pre-compiled multi-agent plans. In addition, the team member agent architecture includes a communication paradigm for domains with single-channel, low-bandwidth, unreliable communication. The communication paradigm facilitates team coordination while being robust to lost messages and active interference from opponents. Second, the thesis introduces layered learning, a general-purpose machine learning paradigm for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable. Given a hierarchical task decomposition, layered learning allows for learning at each level of the hierarchy, with learning at each level directly affecting learning at the next higher level. Third, the thesis introduces a new multi-agent reinforcement learning algorithm, namely team-partitioned, opaque-transition reinforcement learning (TPOT-RL). TPOT-RL is designed for domains in which agents cannot necessarily observe the state changes when other team members act.

This book constitutes the thoroughly refereed post-proceedings of the First International Workshop on Learning and Adaption in Multi-Agent Systems, LAMAS 2005, held in The Netherlands, in July 2005, as an associated event of AAMAS 2005. The 13 revised papers presented together with two invited talks were carefully reviewed and selected from the lectures given at the workshop.

This book constitutes the refereed proceedings of the First German Conference on Multiagent System Technologies, MATES 2003, held in Erfurt, Germany, in September 2003. The 18 revised full papers presented together with an invited paper were carefully reviewed and selected from 49 submissions. The papers are organized in topical sections on engineering agent-based systems, systems and applications, models and architectures, the semantic Web and interoperability, and collaboration and negotiation.

The new edition of an introduction to multiagent systems that captures the state of the art in both theory and practice, suitable as textbook or reference. Multiagent systems are made up of multiple interacting intelligent agents—computational entities to some degree autonomous and able to cooperate, compete, communicate, act flexibly, and exercise control over their behavior within the frame of their objectives. They are the enabling technology for a wide range of advanced applications relying on distributed and parallel processing of data, information, and knowledge relevant in domains ranging from industrial manufacturing to e-commerce to health care. This book offers a state-of-the-art introduction to multiagent systems, covering the field in both breadth and depth, and treating both theory and practice. It is suitable for classroom use or independent study. This second edition has been completely revised, capturing the tremendous developments in multiagent systems since the first edition appeared in 1999. Sixteen of the book's seventeen chapters were written for this edition; all chapters are by leaders in the field, with each author contributing to the broad base of knowledge and experience on which the book rests. The book covers basic concepts of computational agency from the perspective of both individual agents and agent organizations; communication among agents; coordination among agents; distributed cognition; development and engineering of multiagent systems; and background knowledge in logics and game theory. Each chapter includes references, many illustrations and examples, and exercises of varying degrees of difficulty. The chapters and the overall book are designed to be self-contained and understandable without additional material. Supplemental resources are available on the book's Web site. Contributors Rafael Bordini, Felix Brandt, Amit Chopra, Vincent Conitzer, Virginia Dignum, Jürgen Dix, Ed Durfee, Edith Elkind, Ulle Endriss, Alessandro Farinelli, Shaheen Fatima, Michael Fisher, Nicholas R. Jennings, Kevin Leyton-Brown, Evangelos Markakis, Lin Padgham, Julian Padget, Iyad Rahwan, Talal Rahwan, Alex Rogers, Jordi Sabater-Mir, Yoav Shoham, Munindar P. Singh, Kagan Tumer, Karl Tuyls, Wiebe van der Hoek, Laurent Vercouter, Meritxell Vinyals, Michael Winikoff, Michael Wooldridge, Shlomo Zilberstein

This volume contains the papers selected for presentation at CEEMAS 2001. The wo- shop was the fourth in a series of international conferences devoted to autonomous agents and multi-agent systems organized in Central-Eastern Europe. Its predecessors wereCEEMAS’99andDAIMAS’97,whichtookplaceinSt. Petersburg,Russia,aswell as DIMAS’95, which took place in Cracow, Poland. Organizers of all these events made efforts to make them wide-open to participants from all over the world. This would have been impossible without some help from friendly centers in the Czech Republic, England, France, Japan, and The Netherlands. DIMAS’95 featured papers from 15 countries, while CEEMAS’99 from 18 co- tries. A total of 61 papers were submitted to CEEMAS 2001 from 17 countries. Out of these papers, 31 were selected for regular presentation, while 14 were quali ed as posters. The motto of the meeting was “Diversity is the core of multi-agent systems". This variety of subjects was clearly visible in the CEEMAS 2001 program, addressing the following major areas of multi-agent systems: – Organizations and social aspects of multi-agent systems – Agent and multi-agent system architectures, models, and formalisms – Communication languages, protocols, and negotiation – Applications of multi-agent systems – Agent and multi-agent development tools – Theoretical foundations of DistributedAI – Learning in multi-agent systems The richness of workshop subjects was ensured thanks to the CEEMAS 2001 contributing authors as well as the keynote speakers.

This book presents selected tutorial lectures given at the summer school on Multi-Agent Systems and Their Applications held in Prague, Czech Republic, in July 2001 under the sponsorship of ECCAI and Agent Link. The 20 lectures by leading researchers in the field presented in the book give a competent state-of-the-art account of research and development in the field of multi-agent systems and advanced applications. The book offers parts on foundations of MAS; social behaviour, meta-reasoning, and learning; and applications.