This dissertation, "Novel Techniques for Visual Object Tracking and Depth-aware Video Processing" by Shuai, Zhang, 張帥, was obtained from The University of Hong Kong (Pokfulam, Hong Kong) and is being sold pursuant to Creative Commons: Attribution 3.0 Hong Kong License. The content of this dissertation has not been altered in any way. We have altered the formatting in order to facilitate the ease of printing and reading of the dissertation. All rights not granted by the above license are retained by the author. Abstract: Visual object tracking is frequently employed in applications, such as intelligent video surveillance, human body tracking, and many other related problems. Therefore, it is a fundamental problem in video processing and computer vision. The general procedure of automatic object tracking consists of object detection, object representation, tracking strategy and model updating. Tracking strategy, in particular, is an important component because it performs prediction and inference of useful object information such as object location, object orientation and object size, from one frame to another. In this dissertation, a new visual object tracking algorithm using a novel Bayesian Kalman filter (BKF) with simplified Gaussian mixture (BKF-SGM) tracking strategy is proposed. The new BKF-SGM employs a GM representation of the state and noise densities and a novel direct density simplifying algorithm for avoiding the exponential complexity growth of conventional Kalman filters using GM. As the GM is simplified directly without resampling using particles, the proposed BKF-SGM considerably reduces the exponential arithmetic complexity and avoids performance degradation due to sampling degeneracy and impoverishment in conventional particle filtering (PF). When coupled with an improved mean shift (MS) algorithm, the original MS tracker is extended under the BKF-SGM framework above to a bank of parallel MS trackers, which offer a more robust tracking performance. The resultant algorithm, which is called the BKF-SGM with improved MS (BKF-SGM-IMS), is inherently parallel in nature and hence can be readily accelerated using Graphics Processing Unit (GPU) to meet the high computational requirement in real-time applications. The proposed BKF-SGM-IMS algorithm can successfully handle complex scenarios with good performance and low arithmetic complexity. Moreover, the performance of both non-training/training-based object recognition algorithms can be improved by using our tracking results as input. As depth information make machine vision one step closer to human vision by combining color and depth information, there is a recent interest in depth-aware video processing and computer vision both in the academic and industrial fields. However, high quality and high resolution depth map acquisition for real world scene is a challenging problem. Conventional depth acquisition algorithms which rely on stereo/multi-view vision (passive method) or depth sensing device (active method) alone are limited by complicated scenes or imperfections of the depth sensing devices. In this dissertation, a new system for indoor high resolution and high quality depth estimation using joint fusion of stereo and depth sensing data is proposed. By modeling the observations using Markov random field (MRF), the fusion problem is formulated as a maximum a posteriori probability (MAP) estimation problem. The reliability and the probability density functions for describing the observations from the two devices are also derived. The MAP problem is solved using a multi-scale belief propagation (BP) algorithm. To suppress possible estimation noise, the depth map estimated is further refined by color image guided depth matting and a 2D polynomial regression (LPR)-based filtering. Experimental results and numerical comparisons show that our system can provide high quality and high resolution depth maps, thanks to the complementary str

Visual object tracking (VOT) and face recognition (FR) are essential tasks in computer vision with various real-world applications including human-computer interaction, autonomous vehicles, robotics, motion-based recognition, video indexing, surveillance and security. This book presents the state-of-the-art and new algorithms, methods, and systems of these research fields by using deep learning. It is organized into nine chapters across three sections. Section I discusses object detection and tracking ideas and algorithms; Section II examines applications based on re-identification challenges; and Section III presents applications based on FR research.

The book focuses on visual object tracking systems and approaches based on correlation filter and deep learning. Both foundations and implementations have been addressed. The algorithm, system design and performance evaluation have been explored for three kinds of tracking methods including correlation filter based methods, correlation filter with deep feature based methods, and deep learning based methods. Firstly, context aware and multi-scale strategy are presented in correlation filter based trackers; then, long-short term correlation filter, context aware correlation filter and auxiliary relocation in SiamFC framework are proposed for combining correlation filter and deep learning in visual object tracking; finally, improvements in deep learning based trackers including Siamese network, GAN and reinforcement learning are designed. The goal of this book is to bring, in a timely fashion, the latest advances and developments in visual object tracking, especially correlation filter and deep learning based methods, which is particularly suited for readers who are interested in the research and technology innovation in visual object tracking and related fields.

This book covers the description of both conventional methods and advanced methods. In conventional methods, visual tracking techniques such as stochastic, deterministic, generative, and discriminative are discussed. The conventional techniques are further explored for multi-stage and collaborative frameworks. In advanced methods, various categories of deep learning-based trackers and correlation filter-based trackers are analyzed. The book also: Discusses potential performance metrics used for comparing the efficiency and effectiveness of various visual tracking methods Elaborates on the salient features of deep learning trackers along with traditional trackers, wherein the handcrafted features are fused to reduce computational complexity Illustrates various categories of correlation filter-based trackers suitable for superior and efficient performance under tedious tracking scenarios Explores the future research directions for visual tracking by analyzing the real-time applications The book comprehensively discusses various deep learning-based tracking architectures along with conventional tracking methods. It covers in-depth analysis of various feature extraction techniques, evaluation metrics and benchmark available for performance evaluation of tracking frameworks. The text is primarily written for senior undergraduates, graduate students, and academic researchers in the fields of electrical engineering, electronics and communication engineering, computer engineering, and information technology.

The book focuses on visual object tracking systems and approaches based on correlation filter and deep learning. Both foundations and implementations have been addressed. The algorithm, system design and performance evaluation have been explored for three kinds of tracking methods including correlation filter based methods, correlation filter with deep feature based methods, and deep learning based methods. Firstly, context aware and multi-scale strategy are presented in correlation filter based trackers; then, long-short term correlation filter, context aware correlation filter and auxiliary relocation in SiamFC framework are proposed for combining correlation filter and deep learning in visual object tracking; finally, improvements in deep learning based trackers including Siamese network, GAN and reinforcement learning are designed. The goal of this book is to bring, in a timely fashion, the latest advances and developments in visual object tracking, especially correlation filter and deep learning based methods, which is particularly suited for readers who are interested in the research and technology innovation in visual object tracking and related fields.