Nicolas Hautiere

Fast visibility restoration from a single color or gray level image

One source of difficulties when processing outdoor images is the presence of haze, fog or smoke which fades the colors and reduces the contrast of the observed objects. We introduce a novel algorithm and variants for visibility restoration from a single image. The main advantage of the proposed algorithm compared with other is its speed: its complexity is a linear function of the number of image pixels only. This speed allows visibility restoration to be applied for the first time within real-time processing applications such as sign, lane-marking and obstacle detection from an in-vehicle camera. Another advantage is the possibility to handle both color images or gray level images since the ambiguity between the presence of fog and the objects with low color saturation is solved by assuming only small objects can have colors with low saturation. The algorithm is controlled only by a few parameters and consists in: atmospheric veil inference, image restoration and smoothing, tone mapping. A comparative study and quantitative evaluation is proposed with a few other state of the art algorithms which demonstrates that similar or better quality results are obtained. Finally, an application is presented to lane-marking extraction in gray level images, illustrating the interest of the approach.

Automatic fog detection and estimation of visibility distance through use of an onboard camera

In this paper, we will present a technique for measuring visibility distances under foggy weather conditions using a camera mounted onboard a moving vehicle. Our research has focused in particular on the problem of detecting daytime fog and estimating visibility distances; thanks to these efforts, an original method has been developed, tested and patented. The approach consists of dynamically implementing Koschmieders law. Our method enables computing the meteorological visibility distance, a measure defined by the International Commission on Illumination (CIE) as the distance beyond which a black object of an appropriate dimension is perceived with a contrast of less than 5%. Our proposed solution is an original one, featuring the advantage of utilizing a single camera and necessitating the presence of just the road and sky in the scene. As opposed to other methods that require the explicit extraction of the road, this method offers fewer constraints by virtue of being applicable with no more than the extraction of a homogeneous surface containing a portion of the road and sky within the image. This image preprocessing also serves to identify the level of compatibility of the processed image with the set of Koschmieders model hypotheses.

Towards Fog-Free In-Vehicle Vision Systems through Contrast Restoration

In foggy weather, the contrast of images grabbed by in-vehicle cameras in the visible light range is drastically degraded, which makes the current applications very sensitive to weather conditions. An onboard vision system should take fog effects into account. The effects of fog varies across the scene and are exponential with respect to the depth of scene points. Because it is not possible in this context to compute the road scene structure beforehand contrary to fixed camera surveillance, a new scheme is proposed. Weather conditions are first estimated and then used to restore the contrast according to a scene structure which is inferred a priori and refined during the restoration process. Based on the aimed application, different algorithms with increasing complexities are proposed. Results are presented using sample road scenes under foggy weather and assessed by computing the contrast before and after restoration.

Improved visibility of road scene images under heterogeneous fog

One source of accidents when driving a vehicle is the presence of homogeneous and heterogeneous fog. Fog fades the colors and reduces the contrast of the observed objects with respect to their distances. Various camera-based Advanced Driver Assistance Systems (ADAS) can be improved if efficient algorithms are designed for visibility enhancement of road images. The visibility enhancement algorithm proposed in [1] is not dedicated to road images and thus it leads to limited quality results on images of this kind. In this paper, we interpret the algorithm in [1] as the inference of the local atmospheric veil subject to two constraints. From this interpretation, we propose an extended algorithm which better handles road images by taking into account that a large part of the image can be assumed to be a planar road. The advantages of the proposed local algorithm are its speed, the possibility to handle both color images or gray-level images, and its small number of parameters. A comparative study and quantitative evaluation with other state-of-the-art algorithms is proposed on synthetic images with several types of generated fog. This evaluation demonstrates that the new algorithm produces similar quality results with homogeneous fog and that it is able to better deal with the presence of heterogeneous fog.

Real-time disparity contrast combination for onboard estimation of the visibility distance

An atmospheric visibility measurement system capable of quantifying the most common operating range of onboard exteroceptive sensors is a key parameter in the creation of driving assistance systems. This information is then utilized to adapt sensor operations and processing or to alert the driver that his onboard assistance system is momentarily inoperative. Moreover, a system capable of either detecting the presence of fog or estimating visibility distances constitutes in itself a driving assistance. In this paper, the authors present a technique to estimate the mobilized visibility distance through a use of onboard charge-coupled device cameras. The latter represents the distance to the most distant object on the road surface having a contrast above 5%. This definition is very close to the definition of the meteorological visibility distance proposed by the International Commission on Illumination. The method combines the computations of local contrasts above 5% and of a depth map of the vehicle environment using stereovision within 60 ms on a current-day computer. In this paper, both methods are described separately. Then, their combination is detailed. The method is operative night and day in every kind of meteorological condition and is evaluated; thanks to video sequences under sunny weather and foggy weather.

Road Segmentation Supervised by an Extended V-Disparity Algorithm for Autonomous Navigation

This paper presents an original approach of road segmentation supervised by stereovision. It deals with free space estimation by stereovision and road detection by color segmentation. The v-disparity algorithm is extended to provide a reliable and precise road profile on all types of roads. The free space is estimated by classifying the pixels of the disparity map. This classification is performed by using the road profile and the u-disparity image. Then a color segmentation is performed on the free space. Here is the supervision. Each stage of the algorithm is presented and experimental results are shown.

Long Range Obstacle Detection Using Laser Scanner and Stereovision

To be exploited for driving assistance purpose, a road obstacle detection system must have a good detection rate and an extremely low false detection rate. Moreover, the field of possible applications depends on the detection range of the system. With these ideas in mind, we propose in this paper a long range generic road obstacle detection system based on fusion between stereovision and laser scanner. The obstacles are detected and tracked by the laser sensor. Afterwards, stereovision is used to confirm the detections. An overview of the whole method is given. Then the confirmation process is detailed: three algorithms are proposed and compared on real road situations

Mitigation of Visibility Loss for Advanced Camera-Based Driver Assistance

In adverse weather conditions, in particular, in daylight fog, the contrast of images grabbed by in-vehicle cameras in the visible light range is drastically degraded, which makes current driver assistance that relies on cameras very sensitive to weather conditions. An onboard vision system should take weather effects into account. The effects of daylight fog vary across the scene and are exponential with respect to the depth of scene points. Because it is not possible in this context to compute the road scene structure beforehand, contrary to fixed camera surveillance, a new scheme is proposed. Fog density is first estimated and then used to restore the contrast using a flat-world assumption on the segmented free space in front of a moving vehicle. A scene structure is estimated and used to refine the restoration process. Results are presented using sample road scenes under foggy weather and assessed by computing the visibility level enhancement that is gained by the method. Finally, we show applications to the enhancement in daylight fog of low-level algorithms that are used in advanced camera-based driver assistance.

Contrast restoration of foggy images through use of an onboard camera

Perception sensors (cameras, laser, radar) are being introduced into certain vehicles. These sensors have been designed to operate within a wide range of situations and conditions (weather, luminosity, etc.) with a prescribed set of variation thresholds. Effectively detecting when a given operating threshold has been surpassed constitutes a key parameter in the creation of driving assistance systems that meet required reliability levels. With this context in mind, an atmospheric visibility measurement system may be capable of quantifying the most common operating range of onboard exteroceptive sensors. In particular, foggy images suffer from poor contrast. Furthermore, this decay varies across the scene and is exponential in the depths of scene points. In this paper, we present a physics based method to restore contrast of foggy images without any a priori weather-specific prediction. Our method only uses a simple black-and-white camera mounted onboard a moving vehicle. Furthermore, we are able to estimate the meteorological visibility distance, which characterizes the weather condition.

Enhanced fog detection and free-space segmentation for car navigation

Free-space detection is a primary task for car navigation. Unfortunately, classical approaches have difficulties in adverse weather conditions, in particular in daytime fog. In this paper, a solution is proposed thanks to a contrast restoration approach on images grabbed by an in-vehicle camera. The proposed method improves the state-of-the-art in several ways. First, the segmentation of the fog region of interest is better segmented thanks to the computation of the shortest routes maps. Second, the fog density as well as the position of the horizon line is jointly computed. Then, the method restores the contrast of the road by only assuming that the road is flat and, at the same time, detects the vertical objects. Finally, a segmentation of the connected component in front of the vehicle gives the free-space area. An experimental validation was carried out to foresee the effectiveness of the method. Different results are shown on sample images extracted from video sequences acquired from an in-vehicle camera. The proposed method is complementary to existing free-space area detection methods relying on color segmentation and stereovision.