This book contains three well-written research tutorials that inform the graduate reader about the forefront of current research in multi-agent optimization. These tutorials cover topics that have not yet found their way in standard books and offer the reader the unique opportunity to be guided by major researchers in the respective fields. Multi-agent optimization, lying at the intersection of classical optimization, game theory, and variational inequality theory, is at the forefront of modern optimization and has recently undergone a dramatic development. It seems timely to provide an overview that describes in detail ongoing research and important trends. This book concentrates on Distributed Optimization over Networks; Differential Variational Inequalities; and Advanced Decomposition Algorithms for Multi-agent Systems. This book will appeal to both mathematicians and mathematically oriented engineers and will be the source of inspiration for PhD students and researchers.

This volume presents a collection of original research works by leading specialists focusing on novel and promising approaches in which the multi-agent system paradigm is used to support, enhance or replace traditional approaches to solving difficult optimization problems. The editors have invited several well-known specialists to present their solutions, tools, and models falling under the common denominator of the agent-based optimization. The book consists of eight chapters covering examples of application of the multi-agent paradigm and respective customized tools to solve difficult optimization problems arising in different areas such as machine learning, scheduling, transportation and, more generally, distributed and cooperative problem solving.

The paradigm of ‘multi-agent’ cooperative control is the challenge frontier for new control system application domains, and as a research area it has experienced a considerable increase in activity in recent years. This volume, the result of a UCLA collaborative project with Caltech, Cornell and MIT, presents cutting edge results in terms of the “dimensions” of cooperative control from leading researchers worldwide. This dimensional decomposition allows the reader to assess the multi-faceted landscape of cooperative control. Cooperative Control of Distributed Multi-Agent Systems is organized into four main themes, or dimensions, of cooperative control: distributed control and computation, adversarial interactions, uncertain evolution and complexity management. The military application of autonomous vehicles systems or multiple unmanned vehicles is primarily targeted; however much of the material is relevant to a broader range of multi-agent systems including cooperative robotics, distributed computing, sensor networks and data network congestion control. Cooperative Control of Distributed Multi-Agent Systems offers the reader an organized presentation of a variety of recent research advances, supporting software and experimental data on the resolution of the cooperative control problem. It will appeal to senior academics, researchers and graduate students as well as engineers working in the areas of cooperative systems, control and optimization.

An autonomous agent is a computational system that acquires sensory data from its environment and decides by itself how to relate the external stimulus to its behaviors in order to attain certain goals. Responding to different stimuli received from its task environment, the agent may select and exhibit different behavioral patterns. The behavioral patterns may be carefully predefined or dynamically acquired by the agent based on some learning and adaptation mechanism(s). In order to achieve structural flexibility, reliability through redundancy, adaptability, and reconfigurability in real-world tasks, some researchers have started to address the issue of multiagent cooperation. Broadly speaking, the power of autonomous agents lies in their ability to deal with unpredictable, dynamically changing environments. Agent-based systems are becoming one of the most important computer technologies, holding out many promises for solving real-world problems. The aims of this book are to provide a guided tour to the pioneering work and the major technical issues in agent research, and to give an in-depth discussion on the computational mechanisms for behavioral engineering in autonomous agents. Through a systematic examination, the book attempts to provide the general design principles for building autonomous agents and the analytical tools for modeling the emerged behavioral properties of a multiagent system.

The MATSim (Multi-Agent Transport Simulation) software project was started around 2006 with the goal of generating traffic and congestion patterns by following individual synthetic travelers through their daily or weekly activity programme. It has since then evolved from a collection of stand-alone C++ programs to an integrated Java-based framework which is publicly hosted, open-source available, automatically regression tested. It is currently used by about 40 groups throughout the world. This book takes stock of the current status. The first part of the book gives an introduction to the most important concepts, with the intention of enabling a potential user to set up and run basic simulations. The second part of the book describes how the basic functionality can be extended, for example by adding schedule-based public transit, electric or autonomous cars, paratransit, or within-day replanning. For each extension, the text provides pointers to the additional documentation and to the code base. It is also discussed how people with appropriate Java programming skills can write their own extensions, and plug them into the MATSim core. The project has started from the basic idea that traffic is a consequence of human behavior, and thus humans and their behavior should be the starting point of all modelling, and with the intuition that when simulations with 100 million particles are possible in computational physics, then behavior-oriented simulations with 10 million travelers should be possible in travel behavior research. The initial implementations thus combined concepts from computational physics and complex adaptive systems with concepts from travel behavior research. The third part of the book looks at theoretical concepts that are able to describe important aspects of the simulation system; for example, under certain conditions the code becomes a Monte Carlo engine sampling from a discrete choice model. Another important aspect is the interpretation of the MATSim score as utility in the microeconomic sense, opening up a connection to benefit cost analysis. Finally, the book collects use cases as they have been undertaken with MATSim. All current users of MATSim were invited to submit their work, and many followed with sometimes crisp and short and sometimes longer contributions, always with pointers to additional references. We hope that the book will become an invitation to explore, to build and to extend agent-based modeling of travel behavior from the stable and well tested core of MATSim documented here.

This is the first comprehensive introduction to multiagent systems and contemporary distributed artificial intelligence that is suitable as a textbook.

The purpose of this book is to develop in greater depth some of the methods from the author's Reinforcement Learning and Optimal Control recently published textbook (Athena Scientific, 2019). In particular, we present new research, relating to systems involving multiple agents, partitioned architectures, and distributed asynchronous computation. We pay special attention to the contexts of dynamic programming/policy iteration and control theory/model predictive control. We also discuss in some detail the application of the methodology to challenging discrete/combinatorial optimization problems, such as routing, scheduling, assignment, and mixed integer programming, including the use of neural network approximations within these contexts. The book focuses on the fundamental idea of policy iteration, i.e., start from some policy, and successively generate one or more improved policies. If just one improved policy is generated, this is called rollout, which, based on broad and consistent computational experience, appears to be one of the most versatile and reliable of all reinforcement learning methods. In this book, rollout algorithms are developed for both discrete deterministic and stochastic DP problems, and the development of distributed implementations in both multiagent and multiprocessor settings, aiming to take advantage of parallelism. Approximate policy iteration is more ambitious than rollout, but it is a strictly off-line method, and it is generally far more computationally intensive. This motivates the use of parallel and distributed computation. One of the purposes of the monograph is to discuss distributed (possibly asynchronous) methods that relate to rollout and policy iteration, both in the context of an exact and an approximate implementation involving neural networks or other approximation architectures. Much of the new research is inspired by the remarkable AlphaZero chess program, where policy iteration, value and policy networks, approximate lookahead minimization, and parallel computation all play an important role.