The issue discusses methods to extract 3-dimensional (3D) models from plain images. In particular, the 3D information is obtained from images for which the camera parameters are unknown. The principles underlying such uncalibrated structure-from-motion methods are outlined. First, a short review of 3D acquisition technologies puts such methods in a wider context, and highlights their important advantages. Then, the actual theory behind this line of research is given. The authors have tried to keep the text maximally self-contained, therefore also avoiding to rely on an extensive knowledge of the projective concepts that usually appear in texts about self-calibration 3D methods. Rather, mathematical explanations that are more amenable to intuition are given. The explanation of the theory includes the stratification of reconstructions obtained from image pairs as well as metric reconstruction on the basis of more than 2 images combined with some additional knowledge about the cameras used. Readers who want to obtain more practical information about how to implement such uncalibrated structure-from-motion pipelines may be interested in two more Foundations and Trends issues written by the same authors. Together with this issue they can be read as a single tutorial on the subject.

This dissertation, "3D Metric Reconstruction From Uncalibrated Circular Motion Image Sequences" by Huang, Zhong, 鐘煌, was obtained from The University of Hong Kong (Pokfulam, Hong Kong) and is being sold pursuant to Creative Commons: Attribution 3.0 Hong Kong License. The content of this dissertation has not been altered in any way. We have altered the formatting in order to facilitate the ease of printing and reading of the dissertation. All rights not granted by the above license are retained by the author. Abstract: Abstract of thesis entitled 3D Metric Reconstruction from Uncalibrated Circular Motion Image Sequences submitted by Zhong Huang for the degree of Doctor of Philosophy at The University of Hong Kong May 2006 Circular motion is a practical motion for model acquisition. This thesis addresses the problem of metric reconstruction from uncalibrated circular motion image sequences from both the theoretical and practical viewpoints. The cameras are assumed to have constant intrinsic parameters, but the actual camera motion need not be known. A stratified geometric approach is taken to devise solutions. Algorithms for motion estimation, metric reconstruction and surface extraction are developed, by which an object model can be reconstructed systematically. First, a reconstruction method for circular motion is developed with the aim of estimating the unknown rotation angles of the camera robustly. This is achieved by enforcing the knowledge of the type of motion (i.e. the rotational motion constraint) in a factorization-based projective reconstruction, yielding what is called a circular projective reconstruction. Furthermore, a three-stage reconstruction approach with image points incrementally added in different stages of reconstructions is presented for computing a circular projective reconstruction from a long circular motion image sequence. This approach can cope with the missing data problem which inherently exists in a long image sequence. Second, the problem of computing a metric reconstruction from a circular projective reconstruction is considered. Two camera calibration methods are II proposed for resolving the metric reconstruction ambiguity in a circular projective reconstruction by exploring available calibration constraints. The methods are based on a new decomposition of a rectifying homography and developed for the cases involving one circular motion image sequence or one circular motion image sequence with one additional image. Finally, a fast model reconstruction method is proposed to extract the surface of an object from a calibrated circular motion image sequence. It is shown that 3D rim curves with known order enclosing the object can be reconstructed by means of the object silhouettes and feature points. These ordered rim curves allow a triangulated surface mesh of the object model to be constructed efficiently. Throughout the thesis, experimental results are given to demonstrate the performance of the proposed algorithms, and these results are shown to be both accurate and stable. III DOI: 10.5353/th_b3704379 Subjects: Three-dimensional imaging Image reconstruction Cameras - Calibration Algorithms

When planning road construction measures, it is essential to have up-to-date information on road conditions. If this information is not to be obtained manually, it is currently obtained using laser scanners mounted on mobile mapping vehicles, which can measure the 3D road profile. However, a large number of mobile mapping vehicles would be necessary to record an entire road network on a regular basis. Since 2D road damages can be found automatically on monocular camera images, the idea was born to use a stereo camera system to capture the 3D profile of roads. With stereo camera systems, it would be possible to equip a large number of vehicles and regularly collect data from large road networks. In this thesis, the potential applications of a stereo camera system for measuring road profiles, which is mounted behind the windshield of a vehicle, are investigated. Since this requires a calibration of the stereo camera system, but the effort for the user should be kept low, the camera self-calibration for this application is also examined. 3D reconstruction from stereoscopic images is a well-studied topic, but its application on road surfaces with little and repetitive textures requires special algorithms. For this reason, a new stereo method was developed. It is based on the plane-sweep approach in combination with semi-global matching. It was tested with different measures for pixel comparison. Furthermore, the plane-sweep approach was implemented in a neural network that solves the stereo correspondence problem in a single step. It uses the stereoscopic images as input and provides an elevation image as output. A completely new approach was developed for the self-calibration of mono cameras and stereo camera systems. Previous methods search for feature points in several images of the same scene. The points are matched between the images and used for the calibration. In contrast to these methods, the proposed method uses feature maps instead of feature points to compare multiple views of one and the same plane. To estimate the unknown parameters, the backpropagation algorithm is used together with the gradient descent method. The measurements obtained by stereoscopic image processing were compared with those obtained by industrial laser scanners. They show that both measurements are very close to each other and that a stereoscopic camera system is in principle suitable for capturing the surface profile of a road. Experiments show that the proposed self-calibration method is capable of estimating all parameters of a complex camera model, including lens distortion, with high precision. Bei der Planung von Straßenbaumaßnahmen ist es unabdingbar, über aktuelle Informationen über den Straßenzustand zu verfügen. Sollen diese Informationen nicht manuell gewonnen werden, werden derzeit Messfahrzeug mit Laserscannern verwendet, welche das 3D-Straßenprofil vermessen können. Für die regelmäßige Erfassung eines gesamten Straßennetzes wäre jedoch eine große Anzahl von Messfahrzeugen erforderlich. Da 2D-Straßenschäden automatisch auf monokularen Kamerabildern gefunden werden können, entstand die Idee, ein Stereokamerasystem zur Erfassung des 3D-Profils zu verwenden. Eine große Anzahl von Fahrzeugen könnte damit ausgerüstet werden und es könnten regelmäßig Daten von großen Straßennetzen erfasst werden. In dieser Arbeit werden die Einsatzmöglichkeiten eines Stereokamerasystems zur Messung von Straßenprofilen untersucht, dass sich hinter der Windschutzscheibe eines Fahrzeugs befindet. Da hierzu das Stereokamerasystems kalibriert sein muss, der Aufwand für den Anwender aber geringgehalten werden soll, wird außerdem die Selbstkalibrierung für diesen Einsatzzweck untersucht. Die 3D-Rekonstruktion aus stereoskopischen Bildern ist ein viel untersuchtes Thema, aber ihre Anwendung auf Straßenoberflächen mit wenig und sich wiederholenden Texturen erfordert spezielle Algorithmen. Aus diesem Grund wurde ein neues Stereoverfahren entwickelt. Es basiert auf dem Plane-sweep-Ansatz in Kombination mit Semi-global Matching. Es wurde mit verschiedene Maßen für den Vergleich von Pixeln getestet. Darüber hinaus wurde der Plane-sweep-Ansatz in einem neuronalen Netzwerk implementiert, das das Stereo-Korrespondenzproblem in einem einzigen Schritt löst. Es verwendet die stereoskopischen Bilder als Eingabe und liefert als Ausgabe ein Höhenbild. Für die Selbstkalibrierung von Monokameras und Stereokamerasystemen wurde ein völlig neuer Ansatz entwickelt. Bisherige Methoden suchen nach Merkmalspunkten in mehreren Bildern der gleichen Szene. Die Punkte werden zwischen den Bildern zugeordnet und für die Kalibrierung verwendet. Die vorgeschlagene Methode verwendet anstelle von Merkmalspunkten Feature-Maps um mehrere Ansichten derselben Ebene zu vergleichen. Zur Schätzung der unbekannten Parameter wird der Backpropagation-Algorithmus zusammen mit dem Gradientenabstiegsverfahren verwendet. Die durch stereoskopische Bildverarbeitung erhaltenen Messungen wurden mit Messungen von industriellen Laserscannern verglichen. Sie zeigen, dass beide sehr nahe beieinander liegen und dass ein Stereokamerasystem für die Erfassung des Oberflächenprofils einer Straße grundsätzlich geeignet ist. Experimente zeigen, dass die neue Selbstkalibrierungsmethode in der Lage ist, alle Parameter eines komplexen Kameramodells, einschließlich der Linsenverzerrung, mit hoher Präzision abzuschätzen.