It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and interest in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem. This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.

This book presents and develops new reinforcement learning methods that enable fast and robust learning on robots in real-time. Robots have the potential to solve many problems in society, because of their ability to work in dangerous places doing necessary jobs that no one wants or is able to do. One barrier to their widespread deployment is that they are mainly limited to tasks where it is possible to hand-program behaviors for every situation that may be encountered. For robots to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes and could solve the problems of learning and adaptation on robots. This book identifies four key challenges that must be addressed for an RL algorithm to be practical for robotic control tasks. These RL for Robotics Challenges are: 1) it must learn in very few samples; 2) it must learn in domains with continuous state features; 3) it must handle sensor and/or actuator delays; and 4) it should continually select actions in real time. This book focuses on addressing all four of these challenges. In particular, this book is focused on time-constrained domains where the first challenge is critically important. In these domains, the agent’s lifetime is not long enough for it to explore the domains thoroughly, and it must learn in very few samples.

Neuromorphic and brain-based robotics have enormous potential for furthering our understanding of the brain. By embodying models of the brain on robotic platforms, researchers can investigate the roots of biological intelligence and work towards the development of truly intelligent machines. This book provides a broad introduction to this groundbreaking area for researchers from a wide range of fields, from engineering to neuroscience. Case studies explore how robots are being used in current research, including a whisker system that allows a robot to sense its environment and neurally inspired navigation systems that show impressive mapping results. Looking to the future, several chapters consider the development of cognitive, or even conscious robots that display the adaptability and intelligence of biological organisms. Finally, the ethical implications of intelligent robots are explored, from morality and Asimov's three laws to the question of whether robots have rights.

This book constitutes the proceedings of the International Conference on Adaptive and Intelligent Systems, ICAIS 2011, held in Klagenfurt, Austria, in September 2011. The 36 full papers included in these proceedings together with the abstracts of 4 invited talks, were carefully reviewed and selected from 72 submissions. The contributions are organized under the following topical sections: incremental learning; adaptive system architecture; intelligent system engineering; data mining and pattern recognition; intelligent agents; and computational intelligence.

This book constitutes the refereed proceedings of the 9th European Conference on Artificial Life, ECAL 2007, held in Lisbon, Portugal. The 125 revised full papers cover morphogenesis and development, robotics and autonomous agents, evolutionary computation and theory, cellular automata, models of biological systems and their applications, ant colony and swarm systems, evolution of communication, simulation of social interactions, self-replication, artificial chemistry.

This book constitutes the refereed proceedings of the 9th International Conference on Simulation of Adaptive Behavior, SAB 2006. The 35 revised full papers and 35 revised poster papers presented are organized in topical sections on the animat approach to adaptive behaviour, perception and motor control, action selection and behavioral sequences, navigation and internal world models, learning and adaptation, evolution, collective and social behaviours, applied adaptive behavior and more.