Visual perception is one of the most important sources of information for both humans and robots. A particular challenge is the acquisition and interpretation of complex unstructured scenes. This work contributes to active vision for humanoid robots. A semantic model of the scene is created, which is extended by successively changing the robot's view in order to explore interaction possibilities of the scene.

Robot vision refers to the capability of a robot to visually perceive the environment and use this information for execution of various tasks. Visual feedback has been used extensively for robot navigation and obstacle avoidance. In the recent years, there are also examples that include interaction with people and manipulation of objects. In this paper, we review some of the work that goes beyond of using artificial landmarks and fiducial markers for the purpose of implementing visionbased control in robots. We discuss different application areas, both from the systems perspective and individual problems such as object tracking and recognition.

Intelligent robotics has become the focus of extensive research activity. This effort has been motivated by the wide variety of applications that can benefit from the developments. These applications often involve mobile robots, multiple robots working and interacting in the same work area, and operations in hazardous environments like nuclear power plants. Applications in the consumer and service sectors are also attracting interest. These applications have highlighted the importance of performance, safety, reliability, and fault tolerance. This volume is a selection of papers from a NATO Advanced Study Institute held in July 1989 with a focus on active perception and robot vision. The papers deal with such issues as motion understanding, 3-D data analysis, error minimization, object and environment modeling, object detection and recognition, parallel and real-time vision, and data fusion. The paradigm underlying the papers is that robotic systems require repeated and hierarchical application of the perception-planning-action cycle. The primary focus of the papers is the perception part of the cycle. Issues related to complete implementations are also discussed.

Active vision and perception for resource-constrained autonomous vehicles, such as small ground robots and quadrotors, are limited in their allowable algorithmic complexity and slow reaction times. For an autonomous mobile robot to safely and reliably perform a useful task or behavior, real-time visual perception that informs a controller with a fast reaction time is needed. This dissertation covers new research developments in the areas of active vision, planning, and control for directional sensors with a focus on event-cameras and RGB cameras. Event-cameras, also known as neuromorphic cameras, are biologically inspired visual sensors that measure local changes in light intensity, mitigating latency and redundant data. Several high-level active vision algorithms, interfaced with autonomous vehicle controllers, are developed for event-cameras and quantitatively compared to analogous RGB camera algorithms, in terms of both accuracy and computational cost. In particular, motion-based perception algorithms for object recognition and tracking, action recognition, and depth estimation are developed for use on a moving quadrotor tasked with reacting to the perceived environment. Novel active vision algorithms for RGB cameras are also developed in which an autonomous ground vehicle or quadrotor interact with a human target of interest using novel action recognition and tracking perception capabilities paralleled with new control methods for target following. Furthermore, a novel occlusion-avoiding path planning algorithm that is applicable to both event-cameras and RGB cameras is developed. The proposed method computes a closed-form collection of subsets of the sensor's configuration space, referred to as visibility regions, that quantify the visibility of targets subject to the sensor field of view geometry and line of sigh visibility. This method is quantitatively compared to several existing sensor path planning methods in terms of analytical computational complexity, experimental path performance, and experimental computational cost analysis. The results of this work enable active vision, perception, and planning for resource-constrained mobile robots equipped with directional sensors such as an event-camera or RGB camera.

Now in its third edition, this textbook is a comprehensive introduction to the multidisciplinary field of mobile robotics, which lies at the intersection of artificial intelligence, computational vision, and traditional robotics. Written for advanced undergraduates and graduate students in computer science and engineering, the book covers algorithms for a range of strategies for locomotion, sensing, and reasoning. The new edition includes recent advances in robotics and intelligent machines, including coverage of human-robot interaction, robot ethics, and the application of advanced AI techniques to end-to-end robot control and specific computational tasks. This book also provides support for a number of algorithms using ROS 2, and includes a review of critical mathematical material and an extensive list of sample problems. Researchers as well as students in the field of mobile robotics will appreciate this comprehensive treatment of state-of-the-art methods and key technologies.