Vladislav Golyanik proposes several new methods for dense non-rigid structure from motion (NRSfM) as well as alignment of point clouds. The introduced methods improve the state of the art in various aspects, i.e. in the ability to handle inaccurate point tracks and 3D data with contaminations. NRSfM with shape priors obtained on-the-fly from several unoccluded frames of the sequence and the new gravitational class of methods for point set alignment represent the primary contributions of this book. About the Author: Vladislav Golyanik is currently a postdoctoral researcher at the Max Planck Institute for Informatics in Saarbrücken, Germany. The current focus of his research lies on 3D reconstruction and analysis of general deformable scenes, 3D reconstruction of human body and matching problems on point sets and graphs. He is interested in machine learning (both supervised and unsupervised), physics-based methods as well as new hardware and sensors for computer vision and graphics (e.g., quantum computers and event cameras).

Abstract: 3D reconstruction of deformable (or non-rigid) scenes from a set of monocular 2D image observations is a long-standing and actively researched area of computer vision and graphics. It is an ill-posed inverse problem, since--without additional prior assumptions--it permits infinitely many solutions leading to accurate projection to the input 2D images. Non-rigid reconstruction is a foundational building block for downstream applications like robotics, AR/VR, or visual content creation. The key advantage of using monocular cameras is their omnipresence and availability to the end users as well as their ease of use compared to more sophisticated camera set-ups such as stereo or multi-view systems. This survey focuses on state-of-the-art methods for dense non-rigid 3D reconstruction of various deformable objects and composite scenes from monocular videos or sets of monocular views. It reviews the fundamentals of 3D reconstruction and deformation modeling from 2D image observations. We then start from general methods--that handle arbitrary scenes and make only a few prior assumptions--and proceed towards techniques making stronger assumptions about the observed objects and types of deformations (e.g. human faces, bodies, hands, and animals). A significant part of this STAR is also devoted to classification and a high-level comparison of the methods, as well as an overview of the datasets for training and evaluation of the discussed techniques. We conclude by discussing open challenges in the field and the social aspects associated with the usage of the reviewed methods

This book constitutes the refereed proceedings of the 42nd German Conference on Pattern Recognition, DAGM GCPR 2020, which took place during September 28 until October 1, 2020. The conference was planned to take place in Tübingen, Germany, but had to change to an online format due to the COVID-19 pandemic. The 34 papers presented in this volume were carefully reviewed and selected from a total of 89 submissions. They were organized in topical sections named: Normalizing Flow, Semantics, Physics, Camera Calibration and Computer Vision, Pattern Recognition, Machine Learning.

This SpringerBrief presents novel methods of approaching challenging problems in the reconstruction of accurate 3D models and serves as an introduction for further 3D reconstruction methods. It develops a 3D reconstruction system that produces accurate results by cascading multiple novel loop detection, sifting, and optimization methods. The authors offer a fast point cloud registration method that utilizes optimized randomness in random sample consensus for surface loop detection. The text also proposes two methods for surface-loop sifting. One is supported by a sparse-feature-based optimization graph. This graph is more robust to different scan patterns than earlier methods and can cope with tracking failure and recovery. The other is an offline algorithm that can sift loop detections based on their impact on loop optimization results and which is enabled by a dense map posterior metric for 3D reconstruction and mapping performance evaluation works without any costly ground-truth data. The methods presented in Towards Optimal Point Cloud Processing for 3D Reconstruction will be of assistance to researchers developing 3D modelling methods and to workers in the wide variety of fields that exploit such technology including metrology, geological animation and mass customization in smart manufacturing.

Being able to recover the shape of 3D deformable surfaces from a single video stream would make it possible to field reconstruction systems that run on widely available hardware without requiring specialized devices. However, because many different 3D shapes can have virtually the same projection, such monocular shape recovery is inherently ambiguous. In this survey, we will review the two main classes of techniques that have proved most effective so far: The template-based methods that rely on establishing correspondences with a reference image in which the shape is already known, and non-rigid structure-from-motion techniques that exploit points tracked across the sequences to reconstruct a completely unknown shape. In both cases, we will formalize the approach, discuss its inherent ambiguities, and present the practical solutions that have been proposed to resolve them. To conclude, we will suggest directions for future research. Table of Contents: Introduction / Early Approaches to Non-Rigid Reconstruction / Formalizing Template-Based Reconstruction / Performing Template-Based Reconstruction / Formalizing Non-Rigid Structure from Motion / Performing Non-Rigid Structure from Motion / Future Directions

We present a 3D reconstruction algorithm designed to support various automation and navigation applications. The algorithm presented focuses on the 3D reconstruction of a scene using only a single moving camera. Utilizing video frames captured at different points in time allows us to determine the depths of a scene. In this way, the system can be used to construct a point cloud model of its unknown surroundings. In this thesis, we present the step by step methodology of the development of a reconstruction technique. The original reconstruction process, resulting with a point cloud was computed based on feature matching and depth triangulation analysis. In an improved version of the algorithm, we utilized optical flow features to create an extremely dense representation model. Although dense, this model is hindered due to its low disparity resolution. As feature points were matched from frame to frame, the resolution of the input images and the discrete nature of disparities limited the depth computations within a scene. With the third algorithmic modification, we introduce the addition of the preprocessing step of nonlinear super resolution. With this addition, the accuracy of the point cloud which relies on precise disparity measurement has significantly increased. Using a pixel by pixel approach, the super resolution technique computes the phase congruency of each pixel's neighborhood and produces nonlinearly interpolated high resolution input frames. Thus, a feature point travels a more precise discrete disparity. Also, the quantity of points within the 3D point cloud model is significantly increased since the number of features is directly proportional to the resolution and high frequencies of the input image. Our final contribution of additional preprocessing steps is designed to filter noise points and mismatched features, giving birth to the complete Dense Point-cloud Representation (DPR) technique. We measure the success of DPR by evaluating the visual appeal, density, accuracy and computational expense of the reconstruction technique and compare with two state-of-the-arts techniques. After the presentation of rigorous analysis and comparison, we conclude by presenting the future direction of development and its plans for deployment in real-world applications.

Monocular deformable 3D reconstruction is the general problem of recovering the 3D shape of a deformable object from monocular 2D images. Several scenarios have emerged: the Shape-from-Template (SfT) and the Non-Rigid Structure-from-Motion (NRSfM) are two approaches intensively studied for their practicability. The former uses a single image depicting the deforming object and a template (a textured 3D shape of this object in a reference pose). The latter does not use a template, but uses several images and recovers the 3D shape in each image. Both approaches rely on the motion of correspondences between the images and deformation priors, which restrict their use to well-textured surfaces which deform smoothly. This thesis advances the state-of-the-art in SfT and NRSfM in two main directions. The first direction is to study SfT for the case of 1D templates (i.e. curved, thin structures such as ropes and cables). The second direction is to develop algorithms in SfT and NRSfM that exploit multiple visual cues and can solve complex, real-world cases which were previously unsolved. We focus on isometric deformations and reconstruct the outer part of the object. The technical and scientific contributions of this thesis are divided into four parts. The first part of this thesis studies the case of a curvilinear template embedded in 2D or 3D space, referred to Curve SfT. We propose a thorough theoretical analysis and practical solutions for Curve SfT. Despite its apparent simplicity, Curve SfT appears to be a complex problem: it cannot be solved locally using exact non-holonomic partial differential equation and is only solvable up to a finite number of ambiguous solutions. A major technical contribution is a computational solution based on our theory, which generates all the ambiguous solutions.The second part of this thesis deals with a limitation of SfT methods: reconstructing creases. This is due to the sparsity of the motion constraint and regularization. We propose two contributions which rely on a non-convex energy minimization framework. First, we complement the motion constraint with a robust boundary contour constraint. Second, we implicitly model creases with a dense mesh-based surface representation and an associated robust smoothing constraint, which deactivates curvature smoothing automatically where needed, without knowing a priori the crease location. The third part of this thesis is dedicated to another limitation of SfT: reconstructing poorly-textured surfaces. This is due to correspondences which cannot be obtained so easily on poorly-textured surfaces (either sparse or dense). As shading reveals details on poorly-textured surfaces, we propose to combine shading and SfT. We have two contributions. The first is a cascaded initialization which estimates sequentially the surface's deformation, the scene illumination, the camera response and then the surface albedos from deformed monocular images. The second is to integrate shading to our previous energy minimization framework for simultaneously refining deformation and photometric parameters.The last part of this thesis relaxes the knowledge of the template and addresses two limitations of NRSfM: reconstructing poorly-textured surfaces with creases. Our major contribution is an extension of the second framework to recover jointly the 3D shapes of all input images and the surface albedos without any template.