Autonomous Vehicles (AV) applications and Advanced Driving Assistance Systems (ADAS) relay in scene understanding processes allowing high level systems to carry out decision marking. For such systems, the localization of a vehicle evolving in a structured dynamic environment constitutes a complex problem of crucial importance. Our research addresses scene structure detection, localization and error modeling. Taking into account the large functional spectrum of vision systems, the accessibility of Open Geographical Information Systems (GIS) and the widely presence of Global Positioning Systems (GPS) onboard vehicles, we study the performance and the reliability of a vehicle localization method combining such information sources. Monocular vision-based lane marking detection provides key information about the scene structure. Using an enhanced multi-kernel framework with hierarchical weights, the proposed parametric method performs, in real time, the detection and tracking of the ego-lane marking. A self-assessment indicator quantifies the confidence of this information source. We conduct our investigations in a localization system which tightly couples GPS, GIS and lane makings in the probabilistic framework of Particle Filter (PF). To this end, it is proposed the use of lane markings not only during the map-matching process but also to model the expected ego-vehicle motion. The reliability of the localization system, in presence of unusual errors from the different information sources, is enhanced by taking into account different confidence indicators. Such a mechanism is later employed to identify error sources. This research concludes with an experimental validation in real driving situations of the proposed methods. They were tested and its performance was quantified using an experimental vehicle and publicly available datasets.

Estimating the pose (position and attitude) in real-time is a key function for road autonomous vehicles. This thesis aims at studying vehicle localization performance using low cost automotive sensors. Three kinds of sensors are considered : dead reckoning (DR) sensors that already exist in modern vehicles, mono-frequency GNSS (Global navigation satellite system) receivers with patch antennas and a frontlooking lane detection camera. Highly accurate maps enhanced with road features are also key components for autonomous vehicle navigation. In this work, a lane marking map with decimeter-level accuracy is considered. The localization problem is studied in a local East-North-Up (ENU) working frame. Indeed, the localization outputs are used in real-time as inputs to a path planner and a motion generator to make a valet vehicle able to drive autonomously at low speed with nobody on-board the car. The use of a lane detection camera makes possible to exploit lane marking information stored in the georeferenced map. A lane marking detection module detects the vehicle's host lane and provides the lateral distance between the detected lane marking and the vehicle. The camera is also able to identify the type of the detected lane markings (e.g., solid or dashed). Since the camera gives relative measurements, the important step is to link the measures with the vehicle's state. A refined camera observation model is proposed. It expresses the camera metric measurements as a function of the vehicle's state vector and the parameters of the detected lane markings. However, the use of a camera alone has some limitations. For example, lane markings can be missing in some parts of the navigation area and the camera sometimes fails to detect the lane markings in particular at cross-roads. GNSS, which is mandatory for cold start initialization, can be used also continuously in the multi-sensor localization system as done often when GNSS compensates for the DR drift. GNSS positioning errors can't be modeled as white noises in particular with low cost mono-frequency receivers working in a standalone way, due to the unknown delays when the satellites signals cross the atmosphere and real-time satellites orbits errors. GNSS can also be affected by strong biases which are mainly due to multipath effect. This thesis studies GNSS biases shaping models that are used in the localization solver by augmenting the state vector. An abrupt bias due to multipath is seen as an outlier that has to be rejected by the filter. Depending on the information flows between the GNSS receiver and the other components of the localization system, data-fusion architectures are commonly referred to as loosely coupled (GNSS fixes and velocities) and tightly coupled (raw pseudoranges and Dopplers for the satellites in view). This thesis investigates both approaches. In particular, a road-invariant approach is proposed to handle a refined modeling of the GNSS error in the loosely coupled approach since the camera can only improve the localization performance in the lateral direction of the road. Finally, this research discusses some map-matching issues for instance when the uncertainty domain of the vehicle state becomes large if the camera is blind. It is challenging in this case to distinguish between different lanes when the camera retrieves lane marking measurements.As many outdoor experiments have been carried out with equipped vehicles, every problem addressed in this thesis is evaluated with real data. The different studied approaches that perform the data fusion of DR, GNSS, camera and lane marking map are compared and several conclusions are drawn on the fusion architecture choice.

Advanced Driver Assistance Systems (ADAS) can improve road safety by supporting the driver through warnings in hazardous circumstances or triggering appropriate actions when facing imminent collision situations (e.g. airbags, emergency brake systems, etc). In this context, the knowledge of the location and the speed of the surrounding mobile objects constitute a key information. Consequently, in this work, we focus on object detection, localization and tracking in dynamic scenes. Noticing the increasing presence of embedded multi-camera systems on vehicles and recognizing the effectiveness of lidar automotive systems to detect obstacles, we investigate stereo vision systems contributions to multi-modal perception of the environment geometry. In order to fuse geometrical information between lidar and vision system, we propose a calibration process which determines the extrinsic parameters between the exteroceptive sensors and quantifies the uncertainties of this estimation. We present a real-time visual odometry method which estimates the vehicle ego-motion and simplifies dynamic object motion analysis. Then, the integrity of the lidar-based object detection and tracking is increased by the means of a visual confirmation method that exploits stereo-vision 3D dense reconstruction in focused areas. Finally, a complete full scale automotive system integrating the considered perception modalities was implemented and tested experimentally in open road situations with an experimental car.

This dissertation presents several related algorithms that enable important capabilities for self-driving vehicles. Using a rotating multi-beam laser rangefinder to sense the world, our vehicle scans millions of 3D points every second. Calibrating these sensors plays a crucial role in accurate perception, but manual calibration is unreasonably tedious, and generally inaccurate. As an alternative, we present an unsupervised algorithm for automatically calibrating both the intrinsics and extrinsics of the laser unit from only seconds of driving in an arbitrary and unknown environment. We show that the results are not only vastly easier to obtain than traditional calibration techniques, they are also more accurate. A second key challenge in autonomous navigation is reliable localization in the face of uncertainty. Using our calibrated sensors, we obtain high resolution infrared reflectivity readings of the world. From these, we build large-scale self-consistent probabilistic laser maps of urban scenes, and show that we can reliably localize a vehicle against these maps to within centimeters, even in dynamic environments, by fusing noisy GPS and IMU readings with the laser in realtime. We also present a localization algorithm that was used in the DARPA Urban Challenge, which operated without a prerecorded laser map, and allowed our vehicle to complete the entire six-hour course without a single localization failure. Finally, we present a collection of algorithms for the mapping and detection of traffic lights in realtime. These methods use a combination of computer-vision techniques and probabilistic approaches to incorporating uncertainty in order to allow our vehicle to reliably ascertain the state of traffic-light-controlled intersections.

The information stored in digital road maps has become very important for intelligent vehicles. As intelligent vehicles address more complex environments, the accuracy requirements for this information have increased. Regarded as a geographic database, digital road maps contain contextual information about the road network, crucial for a good understanding of the environment. When combined with data acquired from on-board sensors, a better representation of the environment can be made, improving the vehicle's situation understanding. Sensors performance can vary drastically depending on the location of the vehicle, mainly due to environmental factors. Comparatively, a map can provide prior information more reliably but to do so, it depends on another essential component: a localization system. Global Navigation Satellite Systems (GNSS) are commonly used in automotive to provide an absolute positioning of the vehicle, but its accuracy is not perfect: GNSS are prone to errors, also depending greatly on the environment (e.g., multipaths). Perception and localization systems are two important components of an intelligent vehicle whose performances vary in function of the vehicle location. This research focuses on their common denominator, the digital road map, and its use as a tool to assess their performance. The idea developed during this thesis is to use the map as a learning canvas, to store georeferenced information about the performance of the sensors during repetitive travels. This requires a robust localization with respect to the map to be available, through a process of map-matching. The main problematic is the discrepancy between the accuracy of the map and of the GNSS, creating ambiguous situations. This thesis develops a map-matching algorithm designed to cope with these ambiguities by providing multiple hypotheses when necessary. The objective is to ensure the integrity of the result by returning a hypothesis set containing the correct matching with high probability. The method relies on proprioceptive sensors via a dead-reckoning approach aided by the map. A coherence checking procedure using GNSS redundant information is then applied to isolate a single map-matching result that can be used to write learning data with confidence in the map. The possibility to handle the digital map in read/write operation has been assessed and the whole writing procedure has been tested on data recorded by test vehicles on open roads.