Samia Ainouz

A robust cost function for stereo matching of road scenes

In this paper different matching cost functions used for stereo matching are evaluated in the context of intelligent vehicles applications. Classical costs are considered, like: sum of squared differences, normalised cross correlation or Census Transform that were already evaluated in previous studies, together with some recent functions that try to enhance the discriminative power of Census Transform (CT). These are evaluated with two different stereo matching algorithms: a global method based on graph cuts and a fast local one based on cross aggregation regions. Furthermore we propose a new cost function that combines the CT and alternatively a variant of CT called Cross-Comparison Census (CCC), with the mean sum of relative pixel intensity differences (DIFFCensus). Among all the tested cost functions, under the same constraints, the proposed DIFFCensus produces the lower error rate on the KITTI road scenes dataset with both global and local stereo matching algorithms.

Intensity self similarity features for pedestrian detection in Far-Infrared images

Pedestrian detection is an important but challenging component of an Intelligent Transportation System. In this paper, we describe a pedestrian detection system based on a monocular vision with a Far-Infrared camera (FIR). We propose an original feature representation, called Intensity Self Similarity (ISS), adapted to pedestrian detection in FIR images. The ISS representation is based on the relative intensity self similarity within a pedestrian region of interest (ROI) hypothesis. Our system consists of two components. The first component generates pedestrian ROI hypothesis by exploiting the specific characteristics of FIR images, where pedestrian shapes may vary in large scale, but heads appear usually as light regions. Pedestrian ROI are detected, with high recall rate, due to a Hierarchical Codebook (HC) of Speeded-Up Robust Features (SURF) located in light head regions. The second component consists of pedestrian hypothesis validation, by using a pedestrian full-body classification based on the ISS representation, with Support Vector Machine (SVM). For classification, we retained two feature descriptors: the Histogram of Oriented Gradients (HOG) descriptor and the original ISS feature representation that we proposed for FIR images. The early fusion of these two features enhances significantly the system precision, attaining an F-measure for the pedestrian class of 97.7%. Moreover, this feature fusion outperforms the state-of-the-art SURF descriptor proposed previously. The experimental evaluation shows that our pedestrian detector is also robust, since it performs well in detecting pedestrians even in large scale and crowded real-world scenes.

Polarization-based specularity removal method with global energy minimization

A new image specularity removal method is presented in this paper. This method is based on the polarization imaging through global energy minimization. Traditional color-based methods generate severe color distortions, and local-patch-based algorithms produce limited results without integrating the long range information. To handle these limitations, the proposed method uses polarization images to provide complementary information and reduce the color distortions. By minimizing a global energy function, our algorithm properly takes into account the long range cue and produces accurate and stable results. Experimental results show that our method outperforms the other polarization-based methods.

An evaluation of the pedestrian classification in a multi-domain multi-modality setup

The objective of this article is to study the problem of pedestrian classification across different light spectrum domains (visible and far-infrared (FIR)) and modalities (intensity, depth and motion). In recent years, there has been a number of approaches for classifying and detecting pedestrians in both FIR and visible images, but the methods are difficult to compare, because either the datasets are not publicly available or they do not offer a comparison between the two domains. Our two primary contributions are the following: (1) we propose a public dataset, named RIFIR , containing both FIR and visible images collected in an urban environment from a moving vehicle during daytime; and (2) we compare the state-of-the-art features in a multi-modality setup: intensity, depth and flow, in far-infrared over visible domains. The experiments show that features families, intensity self-similarity (ISS), local binary patterns (LBP), local gradient patterns (LGP) and histogram of oriented gradients (HOG), computed from FIR and visible domains are highly complementary, but their relative performance varies across different modalities. In our experiments, the FIR domain has proven superior to the visible one for the task of pedestrian classification, but the overall best results are obtained by a multi-domain multi-modality multi-feature fusion.

PHROG: A Multimodal Feature for Place Recognition

Long-term place recognition in outdoor environments remains a challenge due to high appearance changes in the environment. The problem becomes even more difficult when the matching between two scenes has to be made with information coming from different visual sources, particularly with different spectral ranges. For instance, an infrared camera is helpful for night vision in combination with a visible camera. In this paper, we emphasize our work on testing usual feature point extractors under both constraints: repeatability across spectral ranges and long-term appearance. We develop a new feature extraction method dedicated to improve the repeatability across spectral ranges. We conduct an evaluation of feature robustness on long-term datasets coming from different imaging sources (optics, sensors size and spectral ranges) with a Bag-of-Words approach. The tests we perform demonstrate that our method brings a significant improvement on the image retrieval issue in a visual place recognition context, particularly when there is a need to associate images from various spectral ranges such as infrared and visible: we have evaluated our approach using visible, Near InfraRed (NIR), Short Wavelength InfraRed (SWIR) and Long Wavelength InfraRed (LWIR).

Multimodality semantic segmentation based on polarization and color images

Semantic segmentation gives a meaningful class label to every pixel in an image. It enables intelligent devices to understand the scene and has received sufficient attention during recent years. Traditional imaging systems always apply their methods on RGB, RGB-D or even RGB combined with geometric information. However, for outdoor applications, strong reflection or poor illumination appears to reduce the visualization of the real shape or texture of the objects, thus limiting the performance of semantic segmentation algorithms. To tackle this problem, this paper adopts polarization imaging as it can provide complementary information by describing some imperceptible light properties, which varies from different materials. For acceleration, SLIC superpixel segmentation is used to speed up the system. HOG and LBP features are extracted from both color and polarization images. After quantization using visual codebooks, Joint Boosting classifier is trained to label each pixel based on the quantized features. The proposed method was evaluated both on Day-set and Dusk-set. The experimental results show that using polarization setup can provide complementary information to improve the semantic segmentation accuracy. Especially, a large improvement on Dusk-set shows its capacity for intelligent vehicle applications under dark illumination condition.

Specularity removal

Spatial varying coefficient of diffuse and specular image.A global energy function is constructed based on the independent assumption.A specific constraint to ensure the effectiveness of the final solution.Quantitive result evaluation based on histogram. Concentration of light energy in images causes strong highlights (specular reflection), and challenges the robustness of a large variety of vision algorithms, such as feature extraction and object detection. Many algorithms indeed assume perfect diffuse surfaces and ignore the specular reflections; specularity removal may thus be a preprocessing step to improve the accuracy of such algorithms. Regarding specularity removal, traditional color-based methods generate severe color distortions and local patch-based algorithms do not integrate long range information, which may result in artifacts. In this paper, we present a new image specularity removal method which is based on polarization imaging through global energy minimization. Polarization images provide complementary information and reduce color distortions. By minimizing a global energy function, our algorithm properly takes into account the long range cue and produces accurate and stable results. Compared to other polarization-based methods of the literature, our method obtains encouraging results, both in terms of accuracy and robustness.

Towards a robust and fast color stereo matching for intelligent vehicle application

A fast and efficient color stereo matching algorithm is presented in this paper. The intended application is in the field of intelligent vehicles. The algorithm is initialized with selecting an appropriate color space giving the smallest disparity error. Dynamic cross-based aggregation region is then applied. To be fast and also robust to noise and illumination variation, the algorithm explores sparse and strategic census mask. The algorithm is accelerated with GPU implementation. The performances of the proposed algorithm are tested on Middlebury1 dataset images as well as on simulated road traffic scenes of TNO MARS/Prescan2 database. Experiments show that our results perform better than the top performer method in the literature. Some limitations of the method are discussed at the end of the paper.

SoC Design Of A Novel Cluster-Based Approach for Real-Time Lane Detection in Low Quality Images

In this paper, we present a novel System on Chip design for real time lane detection approach on low-quality grayscale images. The proposed method leverages the sequential read out from image sensors to progressively build clusters. The decision to allocate pixels to existing lines (clusters) is made on the fly as pixels flow from the image sensor into the system. We propose a hardware/software partitioning that places low-level computational intensive parts in a pipelined chain in hardware. The pipeline first applies morphological operations on incoming images to enhance their quality. Later canny edge detection followed by Probabilistic Hough Transform is used for accurate line detection. The lines are then filtered and clustered before being fitted into road lanes using weighted least squares method. We prototype our design on a system on FPGA with a precision above 90% and demonstrate a speedup of 2.09x compared to a software only implementation on an embedded processor.

A novel global image description approach for long term vehicle localization

Long-term place recognition for vehicles or robots in outdoor environment is still a tackling issue: numerous changes occur in appearance due to illumination variations or weather phenomena for instance, when using visual sensors. Few methods from the literature try to manage different visual sources while it could favor data interoperability across variable sensors. In this paper, we emphasis our works on cases where there is a need to associate data from different imaging sources (optics, sensors size and even spectral ranges). We developed a method with a first camera which composes the visual memory. Afterwards, we consider another camera which partially covers the same journey. Our goal is to associate live images to the prior visual memory thanks to visual features invariant to sensors changes, with the help of a probabilistic approach for the implementation part.