It has become clear to researchers in robotics and adaptive behaviour that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities that are not directed to pursue goals of immediate relevance for survival and reproduction but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and interest in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be used later to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human well-being, such as the sense of competence, self-determination, and self-esteem. This book has two aims: to present the state of the art in research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and most promising research directions. The book introduces the concept of intrinsic motivation in artificial systems, reviews the relevant literature, offers insights from the neural and behavioural sciences, and presents novel tools for research. The book is organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on three classes of intrinsic motivation mechanisms, those based on predictors, on novelty, and on competence; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots. The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robots.

This book presents and develops new reinforcement learning methods that enable fast and robust learning on robots in real-time. Robots have the potential to solve many problems in society, because of their ability to work in dangerous places doing necessary jobs that no one wants or is able to do. One barrier to their widespread deployment is that they are mainly limited to tasks where it is possible to hand-program behaviors for every situation that may be encountered. For robots to meet their potential, they need methods that enable them to learn and adapt to novel situations that they were not programmed for. Reinforcement learning (RL) is a paradigm for learning sequential decision making processes and could solve the problems of learning and adaptation on robots. This book identifies four key challenges that must be addressed for an RL algorithm to be practical for robotic control tasks. These RL for Robotics Challenges are: 1) it must learn in very few samples; 2) it must learn in domains with continuous state features; 3) it must handle sensor and/or actuator delays; and 4) it should continually select actions in real time. This book focuses on addressing all four of these challenges. In particular, this book is focused on time-constrained domains where the first challenge is critically important. In these domains, the agent’s lifetime is not long enough for it to explore the domains thoroughly, and it must learn in very few samples.

The aim of this Research Topic for Frontiers in Psychology under the section of Cognitive Science and Frontiers in Neurorobotics is to present state-of-the-art research, whether theoretical, empirical, or computational investigations, on open-ended development driven by intrinsic motivations. The topic will address questions such as: How do motivations drive learning? How are complex skills built up from a foundation of simpler competencies? What are the neural and computational bases for intrinsically motivated learning? What is the contribution of intrinsic motivations to wider cognition? Autonomous development and lifelong open-ended learning are hallmarks of intelligence. Higher mammals, and especially humans, engage in activities that do not appear to directly serve the goals of survival, reproduction, or material advantage. Rather, a large part of their activity is intrinsically motivated - behavior driven by curiosity, play, interest in novel stimuli and surprising events, autonomous goal-setting, and the pleasure of acquiring new competencies. This allows the cumulative acquisition of knowledge and skills that can later be used to accomplish fitness-enhancing goals. Intrinsic motivations continue during adulthood, and in humans artistic creativity, scientific discovery, and subjective well-being owe much to them. The study of intrinsically motivated behavior has a long history in psychological and ethological research, which is now being reinvigorated by perspectives from neuroscience, artificial intelligence and computer science. For example, recent neuroscientific research is discovering how neuromodulators like dopamine and noradrenaline relate not only to extrinsic rewards but also to novel and surprising events, how brain areas such as the superior colliculus and the hippocampus are involved in the perception and processing of events, novel stimuli, and novel associations of stimuli, and how violations of predictions and expectations influence learning and motivation. Computational approaches are characterizing the space of possible reinforcement learning algorithms and their augmentation by intrinsic reinforcements of different kinds. Research in robotics and machine learning is yielding systems with increasing autonomy and capacity for self-improvement: artificial systems with motivations that are similar to those of real organisms and support prolonged autonomous learning. Computational research on intrinsic motivation is being complemented by, and closely interacting with, research that aims to build hierarchical architectures capable of acquiring, storing, and exploiting the knowledge and skills acquired through intrinsically motivated learning. Now is an important moment in the study of intrinsically motivated open-ended development, requiring contributions and integration across a large number of fields within the cognitive sciences. This Research Topic aims to contribute to this effort by welcoming papers carried out with ethological, psychological, neuroscientific and computational approaches, as well as research that cuts across disciplines and approaches.

Originating from a Dagstuhl seminar, the collection of papers presented in this book constitutes on the one hand a representative state-of-the-art survey of embodied artificial intelligence, and on the other hand the papers identify the important research trends and directions in the field. Following an introductory overview, the 23 papers are organized into topical sections on - philosophical and conceptual issues - information, dynamics, and morphology - principles of embodiment for real-world applications - developmental approaches - artificial evolution and self-reconfiguration

​This book constitutes revised selected papers from the refereed proceedings of the 20th International Conference of the Italian Association for Artificial Intelligence, AIxIA 2021, which was held virtually in December 2021. The 36 full papers included in this book were carefully reviewed and selected from 58 submissions; the volume also contains 12 extended and revised workshop contributions. The papers were organized in topical sections as follows: Planning and strategies; constraints, argumentation, and logic programming; knowledge representation, reasoning, and learning; natural language processing; AI for content and social media analysis; signal processing: images, videos and speech; machine learning for argumentation, explanation, and exploration; machine learning and applications; and AI applications.

The current state of the art in cognitive robotics, covering the challenges of building AI-powered intelligent robots inspired by natural cognitive systems. A novel approach to building AI-powered intelligent robots takes inspiration from the way natural cognitive systems—in humans, animals, and biological systems—develop intelligence by exploiting the full power of interactions between body and brain, the physical and social environment in which they live, and phylogenetic, developmental, and learning dynamics. This volume reports on the current state of the art in cognitive robotics, offering the first comprehensive coverage of building robots inspired by natural cognitive systems. Contributors first provide a systematic definition of cognitive robotics and a history of developments in the field. They describe in detail five main approaches: developmental, neuro, evolutionary, swarm, and soft robotics. They go on to consider methodologies and concepts, treating topics that include commonly used cognitive robotics platforms and robot simulators, biomimetic skin as an example of a hardware-based approach, machine-learning methods, and cognitive architecture. Finally, they cover the behavioral and cognitive capabilities of a variety of models, experiments, and applications, looking at issues that range from intrinsic motivation and perception to robot consciousness. Cognitive Robotics is aimed at an interdisciplinary audience, balancing technical details and examples for the computational reader with theoretical and experimental findings for the empirical scientist.