SMART on-board multi-sensor obstacle detection system for improvement of rail transport safety

Abstract

This paper presents an on-board multi-sensor system which is able to detect obstacles and estimate their distances in railway scenes in different illumination conditions. The system was developed within the H2020 Shift2Rail project SMART (Smart Automation of Rail Transport) and aims at increasing the safety of rail transport by detecting obstacles on the rail tracks ahead of a moving train in order to reduce the number of collisions. The system hardware consists of cameras of different types integrated into a specially designed housing, mounted on the front of the train. Multiple vision sensors complement each other in order to handle different illumination and environmental conditions. The system software uses a novel machine learning-based method that is suited to a particular challenge of railway operations, the need for long-range obstacle detection and distance estimation. The development of this method used a long-range railway dataset, which was specifically generated for this project. Evaluation results of reliable obstacle detection in various environmental conditions using the SMART RGB camera in day light illumination conditions and using the SMART Night Vision sensor in poor (night) illumination conditions are presented. The results demonstrate both the potential of the on-board SMART obstacle detection system in the operational railway environment and the benefit of the use of different cameras to be more independent of light and environmental conditions.