See-Through a Vehicle: Augmenting Road Safety Information Using Visual Perception and Camera Communication in Vehicles

Abstract

In this paper, we present the design, implementation and evaluation of non-line-of-sight (NLOS) perception to achieve a virtual see-through functionality for road vehicles. In this system, a safety event, such as pedestrian crossing or traffic list status or vehicle merge, that are occluded to a driver due to another vehicle on the driving lane, are perceived and communicated by the occluding vehicle. Each vehicle is equipped with a camera, placed on the dashboard, that perceives the scene in a drivers view. This scene is analyzed and mapped into specific warning codes that are specific to a safety event, and communicated as short packets using visible light communication. The camera in the following vehicle captures this information and generates a recommendation for safety action to the driver by comparing the warning from the packet and from its own scene perception. Through experimental evaluations of a proof-of-concept implementation, we show that our system is able to achieve up to 90% accuracy in identifying nine occluded safety events, which correspond to traffic light statuses (red, green, yellow), other vehicles’ lane change behaviors (merge/leave lane left/right), and pedestrian detection.