The primary focus of this work is to develop a vehicle pose estimation algorithm using a-priori knowledge of the environment. Specifically, this work focuses on the problem of achieving accurate localization of a vehicle within a map and on the road, using a map as a feedforward sensor to help estimate the location of the vehicle. Presented here is a method for improving localization over standard GPS and inertial-based methods via a map-based, monocular vision, pose estimation algorithm. The presented algorithm creates a tractable method for determining the observability of three-dimensional road features for localizing a vehicle, as well as the bounded region wherein the pose estimator can converge to the true vehicle pose. This thesis presents simulation and experimental results to determine this bounded region, or region of attraction, under common road scenes using a map of the lane marker features.

The primary focus of this work is to develop a vehicle state estimation algorithm using a-priori knowledge of the environment. Specifically, this work focuses on the problem of achieving accurate localization of a vehicle within a map and on the road, using a map as a feedforward sensor to help estimate the location of the vehicle using image features. Presented here is a method for improving localization over standard GPS and inertial-based methods via map-based, monocular vision, state estimation algorithms. The measurements obtained from a camera pose estimation algorithm are fused with a dynamic vehicle model to improve vehicle state estimation in a real-time implementable algorithm. The presented methods, utilizing kinematic and dynamic modeling, allow for the calculation of the influence of specific three-dimensional road features when measuring a vehicle's pose. Additionally, the combined simulation and experimental implementation of these methods enabled comparative evaluations of the bounded region wherein the pose estimator can converge to the true vehicle pose under common road scenes using a map of the lane marker features. Finally, this work examines the use of a map-based Kalman filtering method using previewed road features and vehicle steering inputs, in coordination with the image-based pose estimation, to further improve the vehicle's state estimate.

Estimating the pose (position and attitude) in real-time is a key function for road autonomous vehicles. This thesis aims at studying vehicle localization performance using low cost automotive sensors. Three kinds of sensors are considered : dead reckoning (DR) sensors that already exist in modern vehicles, mono-frequency GNSS (Global navigation satellite system) receivers with patch antennas and a frontlooking lane detection camera. Highly accurate maps enhanced with road features are also key components for autonomous vehicle navigation. In this work, a lane marking map with decimeter-level accuracy is considered. The localization problem is studied in a local East-North-Up (ENU) working frame. Indeed, the localization outputs are used in real-time as inputs to a path planner and a motion generator to make a valet vehicle able to drive autonomously at low speed with nobody on-board the car. The use of a lane detection camera makes possible to exploit lane marking information stored in the georeferenced map. A lane marking detection module detects the vehicle's host lane and provides the lateral distance between the detected lane marking and the vehicle. The camera is also able to identify the type of the detected lane markings (e.g., solid or dashed). Since the camera gives relative measurements, the important step is to link the measures with the vehicle's state. A refined camera observation model is proposed. It expresses the camera metric measurements as a function of the vehicle's state vector and the parameters of the detected lane markings. However, the use of a camera alone has some limitations. For example, lane markings can be missing in some parts of the navigation area and the camera sometimes fails to detect the lane markings in particular at cross-roads. GNSS, which is mandatory for cold start initialization, can be used also continuously in the multi-sensor localization system as done often when GNSS compensates for the DR drift. GNSS positioning errors can't be modeled as white noises in particular with low cost mono-frequency receivers working in a standalone way, due to the unknown delays when the satellites signals cross the atmosphere and real-time satellites orbits errors. GNSS can also be affected by strong biases which are mainly due to multipath effect. This thesis studies GNSS biases shaping models that are used in the localization solver by augmenting the state vector. An abrupt bias due to multipath is seen as an outlier that has to be rejected by the filter. Depending on the information flows between the GNSS receiver and the other components of the localization system, data-fusion architectures are commonly referred to as loosely coupled (GNSS fixes and velocities) and tightly coupled (raw pseudoranges and Dopplers for the satellites in view). This thesis investigates both approaches. In particular, a road-invariant approach is proposed to handle a refined modeling of the GNSS error in the loosely coupled approach since the camera can only improve the localization performance in the lateral direction of the road. Finally, this research discusses some map-matching issues for instance when the uncertainty domain of the vehicle state becomes large if the camera is blind. It is challenging in this case to distinguish between different lanes when the camera retrieves lane marking measurements.As many outdoor experiments have been carried out with equipped vehicles, every problem addressed in this thesis is evaluated with real data. The different studied approaches that perform the data fusion of DR, GNSS, camera and lane marking map are compared and several conclusions are drawn on the fusion architecture choice.

Autonomous Vehicles (AV) applications and Advanced Driving Assistance Systems (ADAS) relay in scene understanding processes allowing high level systems to carry out decision marking. For such systems, the localization of a vehicle evolving in a structured dynamic environment constitutes a complex problem of crucial importance. Our research addresses scene structure detection, localization and error modeling. Taking into account the large functional spectrum of vision systems, the accessibility of Open Geographical Information Systems (GIS) and the widely presence of Global Positioning Systems (GPS) onboard vehicles, we study the performance and the reliability of a vehicle localization method combining such information sources. Monocular vision-based lane marking detection provides key information about the scene structure. Using an enhanced multi-kernel framework with hierarchical weights, the proposed parametric method performs, in real time, the detection and tracking of the ego-lane marking. A self-assessment indicator quantifies the confidence of this information source. We conduct our investigations in a localization system which tightly couples GPS, GIS and lane makings in the probabilistic framework of Particle Filter (PF). To this end, it is proposed the use of lane markings not only during the map-matching process but also to model the expected ego-vehicle motion. The reliability of the localization system, in presence of unusual errors from the different information sources, is enhanced by taking into account different confidence indicators. Such a mechanism is later employed to identify error sources. This research concludes with an experimental validation in real driving situations of the proposed methods. They were tested and its performance was quantified using an experimental vehicle and publicly available datasets.

Monocular Model-Based 3D Tracking of Rigid Objects reviews the different techniques and approaches that have been developed by industry and research.

This work presents a new concept of a Collaborative Assistance Vehicle with high interaction capabilities for collaboration with external users outside the vehicle. This work proposes a functional architecture for level 4 automated driving that focuses on an interaction framework, along with algorithmic solutions for implementing core function modules. Perception, command extraction, and behavior planning are part of the core function modules. All of these modules will be implemented and evaluated.