Lotfi Abdi

Augmented Reality Based Traffic Sign Recognition for Improved Driving Safety

In recent years, automotive active safety systems have become increasingly common in road vehicles since they provide an opportunity to significantly reduce traffic fatalities by active vehicle control. Augmented Reality (AR) applications can enhance intelligent transportation systems by superimposing surrounding traffic information on the users view and keep drivers and pedestrians view on roads. However, due to the existence of a complex environment such as weather conditions, illuminations and geometric distortions, Traffic Sign Recognition(TSR) systems has always been considered as a challenging task. The aim of this paper is to evaluate the effectiveness of AR cues in improving driving safety by deploying an on-board camera-based driver alert system against approaching traffic signs such as stop, speed limit, unique, danger signs, etc. A new approach is presented for marker-less AR-TSR system that superimposes augmented virtual objects onto a real scene under all types of driving situations including unfavorable weather conditions. Our method is composed of both online and offline stages. An intrinsic camera parameter change depending on the zoom values is calibrated. A Haar-like feature with Adaboost has been used to train a Haar detector in the offline stage. Extrinsic camera parameters are then estimated based on homography method in the online stage. With the complete set of camera parameters, virtual objects can be coherently inserted into the video sequence captured by the camera so that synthetic traffic signs may be added to increase safety.

In-vehicle augmented reality system to provide driving safety information

Improving traffic safety is one of the important goals of intelligent transportation systems. Traffic signs play a very vital role in safe driving and in avoiding accidents by informing the driver about the speed limits or possible dangers such as icy roads, imminent road works or pedestrian crossings. In-vehicle contextual Augmented reality (AR) has the potential to provide novel visual feedbacks to drivers for an enhanced driving experience. In this paper, we propose a new AR traffic sign recognition system (AR-TSR) to improve driving safety and enhance the driver’s experience based on the Haar cascade and the bag-of-visual-words approach, using spatial information to improve accuracy and an overview of studies related to the driver’s perception and the effectiveness of the AR in improving driving safety. In the first step, the region of interest (ROI) is extracted using a scanning window with a Haar cascade detector and an AdaBoost classifier to reduce the computational region in the hypothesis-generation step. Second, we proposed a new computationally efficient method to model global spatial distribution of visual words by taking into consideration the spatial relationships of its visual words. Finally, a multiclass sign classifier takes the positive ROIs and assigns a 3D traffic sign for each one using a linear SVM. Experimental results show that the suggested method could reach comparable performance of the state-of-the-art approaches with less computational complexity and shorter training time, and the AR-TSR more strongly impacts the allocation of visual attention during the decision-making phase.Graphical Abstract

Deep learning traffic sign detection, recognition and augmentation

Driving is a complex, continuous, and multitask process that involves drivers cognition, perception, and motor movements. The way road traffic signs and vehicle information is displayed impacts strongly drivers attention with increased mental workload leading to safety concerns. Drivers must keep their eyes on the road, but can always use some assistance in maintaining their awareness and directing their attention to potential emerging hazards. Research in perceptual and human factors assessment is needed for relevant and correct display of this information for maximal road traffic safety as well as optimal driver comfort. In-vehicle contextual Augmented Reality (AR) has the potential to provide novel visual feedbacks to drivers for an enhanced driving experience. In this paper, we present a new real-time approach for fast and accurate framework for traffic sign recognition, based on Cascade Deep learning and AR, which superimposes augmented virtual objects onto a real scene under all types of driving situations, including unfavorable weather conditions. Experiments results show that, by combining the Haar Cascade and deep convolutional neural networks show that the joint learning greatly enhances the capability of detection and still retains its realtime performance.

A robust video watermarking algorithm in H.264/AVC compressed domain

In this paper, a low complexity video watermarking scheme for H.264 has been presented. Our contribution is to attain lower complexity in embedding procedure and extracting watermark. At the same time, we avoid a bit-rate increase and improve the runtime-efficiency and embedding capacity without sacrificing quality. The watermark is embedded into a video sequence by modifying the number of nonzero-quantized AC coefficients in a 4x4 block of I frames. The experimental results show that the proposed method can prevent a bit-rate increase and improve the runtime-efficiency and embedding capacity without sacrificing the perceptual quality.

In-vehicle augmented reality TSR to improve driving safety and enhance the driver’s experience

In-vehicle contextual augmented reality (AR) has the potential to provide novel visual feedbacks to drivers for an enhanced driving experience. In this paper, we propose a new AR traffic sign recognition system (AR-TSR) to improve driving safety and enhance the driver’s experience based on the Haar cascade and the Bag-of-Visual-Words approach, using spatial information to improve accuracy and an overview of studies related to the driver’s perception and the effectiveness of the AR in improving driving safety. In the first step, the region of interest (ROI) is extracted using a scanning window with a Haar cascade detector and an AdaBoost classifier to reduce the computational region in the hypothesis generation step. Second, we proposed a new computationally efficient method to model global spatial distribution of visual words by taking into consideration the spatial relationships of its visual words. Finally, a multiclass sign classifier takes the positive ROIs and assigns a 3D traffic sign for each one using a linear SVM. Experimental results show that the suggested method could reach comparable performance of the state-of-the-art approaches with less computational complexity and shorter training time, and the AR-TSR more strongly impacts the allocation of visual attention during the decision-making phase.

Driver information system: a combination of augmented reality, deep learning and vehicular Ad-hoc networks

Improving traffic safety is one of the important goals of Intelligent Transportation Systems (ITS). In vehicle-based safety systems, it is more desirable to prevent an accident than to reduce severity of injuries. Critical traffic problems such as accidents and traffic congestion require the development of new transportation systems. Research in perceptual and human factors assessment is needed for relevant and correct display of this information for maximal road traffic safety as well as optimal driver comfort. One of the solutions to prevent accidents is to provide information on the surrounding environment of the driver. Augmented Reality Head-Up Display (AR-HUD) can facilitate a new form of dialogue between the vehicle and the driver; and enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads. In this paper, we propose a fast deep-learning-based object detection approaches for identifying and recognizing road obstacles types, as well as interpreting and predicting complex traffic situations. A single convolutional neural network predicts region of interest and class probabilities directly from full images in one evaluation. We also investigated potential costs and benefits of using dynamic conformal AR cues in improving driving safety. A new AR-HUD approach to create real-time interactive traffic animations was introduced in terms of types of obstacle, rules for placement and visibility, and projection of these on an in-vehicle display. The novelty of our approach is that both global and local context information are integrated into a unified framework to distinguish the ambiguous detection outcomes, enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads.

Spatially Enhanced Bags of Visual Words Representation to Improve Traffic Signs Recognition

Traffic signs play a very vital role in safe driving and in avoiding accidents by informing the driver about the speed limits or possible dangers such as icy roads, imminent road works or pedestrian crossings. Considering the processing time and classification accuracy as a whole, a novel approach for visual words construction was presented, which takes the spatial information of keypoints into account in order to enhance the quality of visual words generated from extracted keypoints using the distance and angle information in the Bags of Visual Words (BoVW) representation. In this paper, we proposed a new computationally efficient method to model global spatial distribution of visual words by taking into consideration the spatial relationships of its visual words. In the first step, the region of interest is extracted using a scanning window with a Haar cascade detector and an AdaBoost classifier to reduce the computational region in the hypothesis generation step. Second, the regions are represented with BoVW and spatial information for classification. Experimental results show that the suggested method could reach comparable performance of the state-of-the-art approaches with less computational complexity and shorter training time. It clearly demonstrates the complementarity of the additional relative spatial information provided by our approach to improve accuracy while maintaining short retrieval time, and can obtain a better traffic sign recognition accuracy than the methods based on the traditional BoVW model.

Semantic recognition: unified framework for joint object detection and semantic segmentation

Understanding contents of an image, or scene labeling, is an important yet very challenging problem in artificial intelligence and computer vision to improve road safety. Semantic labeling and object detection in road scenes are strongly correlated tasks. Motivated by the complementary effect of the two tasks, we presented a novel framework to address the scene understanding problem. In this paper we propose a new framework for semantic labeling and object detection problem which is able to combine ideas from deep Convolutional Neural Network (CNN) for object detection and fully-connected Conditional Random Field (CRF) for segmenting and labeling. Specifically, we develop a new framework uses global image features to predict detection which drastically reduces its errors from background detections and a pairwise CRF is used as a post-processing step to enforce spatial consistency in the structured prediction. By combining the consistency between final detection results of CNN and CRF based graphical models, our unified framework can effectively leverage the advantages of leading techniques i.e., CRF and CNN for these two tasks. Extensive experiments on the PASCAL VOC 2007/2012 data sets demonstrate the effectiveness of our framework for Scene Understanding tasks.

Driver information system: a combination of augmented reality and deep learning

Improving traffic safety is one of the important goals of Intelligent Transportation Systems (ITS). In vehicle-based safety systems, it is more desirable to prevent an accident than to reduce severity of injuries. One of the solutions to prevent accidents is to provide information on the surrounding environment of the driver. Augmented Reality Head-Up Display (AR-HUD) can facilitate a new form of dialogue between the vehicle and the driver; and enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads. In this paper, we propose a fast deeplearning-based object detection approaches for identifying and recognizing road obstacles types, as well as interpreting and predicting complex traffic situations. A single Convolutional Neural Network (CNN) predicts region of interest and class probabilities directly from full images in one evaluation. We also investigated potential costs and benefits of using dynamic conformal AR cues in improving driving safety.

In-vehicle cooperative driver information systems

Critical traffic problems such as accidents and traffic congestion require the development of new transportation systems. Research in perceptual and human factors assessment is needed for relevant and correct display of this information for maximal road traffic safety as well as optimal driver comfort. One of the solutions to prevent accidents is to provide information on the surrounding environment of the driver. The development and deployment of cooperative vehicular safety systems undeniably require a combination of dedicated wireless communications, computer vision, and AR technologies as the building blocks of cooperative safety systems. Augmented Reality Head-Up Display (AR-HUD) can facilitate a new form of dialogue between the vehicle and the driver; and enhance ITS by superimposing surrounding traffic information on the users view and keep drivers view on roads. In this paper, we propose a fast deep-learning-based object detection approaches for identifying and recognizing road obstacles types, as well as interpreting and predicting complex traffic situations. A single Convolutional Neural Network (CNN) predicts region of interest and class probabilities directly from full images in one evaluation. We also investigated potential costs and benefits of using dynamic conformal AR cues in improving driving safety. A new AR-HUD approach to create real-time interactive traffic animations was introduced in terms of types of obstacle, rules for placement and visibility, and projection of these on an in-vehicle display.