Human action or gesture recognition has been extensively studied in the literature spanning a wide variety of human-computer interaction applications including gaming, surveillance, healthcare monitoring, and assistive living. Sensors used for action or gesture recognition are primarily either vision-based sensors or inertial sensors. Compared to the great majority of previous works where a single modality sensor is used for action or gesture recognition, the simultaneous utilization of a depth camera and a wearable inertial sensor is considered in this dissertation. Furthermore, compared to the great majority of previous works in which actions are assumed to be segmented actions, this dissertation addresses a more realistic and practical scenario in which actions of interest occur continuously and randomly amongst arbitrary actions of non-interest. In this dissertation, computationally efficient solutions are presented to recognize actions of interest from continuous data streams captured simultaneously by a depth camera and a wearable inertial sensor. These solutions comprise three main steps of segmentation, detection, and classification. In the segmentation step, all motion segments are extracted from continuous action streams. In the detection step, the segmented actions are separated into actions of interest and actions of non- interest. In the classification step, the detected actions of interest are classified. The features considered include skeleton joint positions, depth motion maps, and statistical attributes of acceleration and angular velocity inertial signals. The classifiers considered include maximum entropy Markov model, support vector data description, collaborative representation classifier, convolutional neural network, and long short-term memory network. These solutions are applied to the two applications of smart TV hand gestures and transition movements for home healthcare monitoring. The results obtained indicate the effectiveness of the developed solutions in detecting and recognizing actions of interest in continuous data streams. It is shown that higher recognition rates are achieved when fusing the decisions from the two sensing modalities as compared to when each sensing modality is used individually. The results also indicate that the deep learning-based solution provides the best outcome among the solutions developed.

Human action recognition is an active research area benefitting many applications. Example applications include human-computer interaction, assistive-living, rehabilitation, and gaming. Action recognition can be broadly categorized into vision-based and inertial sensor-based. Under realistic operating conditions, it is well known that there are recognition rate limitations when using a single modality sensor due to the fact that no single sensor modality can cope with various situations that occur in practice. The hypothesis addressed in this dissertation is that by using and fusing the information from two differing modality sensors that provide 3D data (a Microsoft Kinect depth camera and a wearable inertial sensor), a more robust human action recognition is achievable. More specifically, effective and computationally efficient features have been devised and extracted from depth images. Both feature-level fusion and decision-level fusion approaches have been investigated for a dual-modality sensing incorporating a depth camera and an inertial sensor. Experimental results obtained indicate that the developed fusion approaches generate higher recognition rates compared to the situations when an individual sensor is used. Moreover, an actual working action recognition system using depth and inertial sensing has been devised which runs in real-time on laptop platforms. In addition, the developed fusion framework has been applied to a medical application.

Action recognition technology has many real-world applications in human-computer interaction, surveillance, video retrieval, retirement home monitoring, and robotics. The commoditization of depth sensors has also opened up further applications that were not feasible before. This text focuses on feature representation and machine learning algorithms for action recognition from depth sensors. After presenting a comprehensive overview of the state of the art, the authors then provide in-depth descriptions of their recently developed feature representations and machine learning techniques, including lower-level depth and skeleton features, higher-level representations to model the temporal structure and human-object interactions, and feature selection techniques for occlusion handling. This work enables the reader to quickly familiarize themselves with the latest research, and to gain a deeper understanding of recently developed techniques. It will be of great use for both researchers and practitioners.

The book presents the proceedings of the 11th International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA 2023), held at Cardiff School of Technologies, Cardiff Metropolitan University, Cardiff, Wales, UK, during April 11–12, 2023. Researchers, scientists, engineers, and practitioners exchange new ideas and experiences in the domain of intelligent computing theories with prospective applications in various engineering disciplines in the book. This book is divided into two volumes. It covers broad areas of information and decision sciences, with papers exploring both the theoretical and practical aspects of data-intensive computing, data mining, evolutionary computation, knowledge management and networks, sensor networks, signal processing, wireless networks, protocols, and architectures. This book is a valuable resource for postgraduate students in various engineering disciplines.

This book illustrates how to use description logic-based formalisms to their full potential in the creation, indexing, and reuse of multimedia semantics. To do so, it introduces researchers to multimedia semantics by providing an in-depth review of state-of-the-art standards, technologies, ontologies, and software tools. It draws attention to the importance of formal grounding in the knowledge representation of multimedia objects, the potential of multimedia reasoning in intelligent multimedia applications, and presents both theoretical discussions and best practices in multimedia ontology engineering. Readers already familiar with mathematical logic, Internet, and multimedia fundamentals will learn to develop formally grounded multimedia ontologies, and map concept definitions to high-level descriptors. The core reasoning tasks, reasoning algorithms, and industry-leading reasoners are presented, while scene interpretation via reasoning is also demonstrated. Overall, this book offers readers an essential introduction to the formal grounding of web ontologies, as well as a comprehensive collection and review of description logics (DLs) from the perspectives of expressivity and reasoning complexity. It covers best practices for developing multimedia ontologies with formal grounding to guarantee decidability and obtain the desired level of expressivity while maximizing the reasoning potential. The capabilities of such multimedia ontologies are demonstrated by DL implementations with an emphasis on multimedia reasoning applications.

This book provides a unique view of human activity recognition, especially fine-grained human activity structure learning, human-interaction recognition, RGB-D data based action recognition, temporal decomposition, and causality learning in unconstrained human activity videos. The techniques discussed give readers tools that provide a significant improvement over existing methodologies of video content understanding by taking advantage of activity recognition. It links multiple popular research fields in computer vision, machine learning, human-centered computing, human-computer interaction, image classification, and pattern recognition. In addition, the book includes several key chapters covering multiple emerging topics in the field. Contributed by top experts and practitioners, the chapters present key topics from different angles and blend both methodology and application, composing a solid overview of the human activity recognition techniques.