Pedestrian Re-identification using a Surround-view Fisheye Camera System*

Abstract

In a multi-camera system, matching the same pedestrian across different camera views is a challenging problem. Pedestrian detection and ReIDentification (ReID) plays an important role in preventing traffic accidents involving pedestrians, for both conventional and autonomous vehicles. To the best of our knowledge, there is no existing work which addresses the problem of pedestrian ReID in a typical vehicle setting, where the vehicle is equipped with a 360° surround-view with fisheye cameras. In this paper, we propose a deep learning system, Seeing Pedestrians in Surround-View Moving Cameras (SPinSVMC), that consists of a Single Camera Detection and Tracking (Single-Cam-D&T) module and Two Cameras ReID (2Cam-ReID) module applied to multi-camera views. The Single-Cam module uses a YOLOv3 model to detect the pedestrians in single camera view videos, and a model that combines OSnet with DeepSORT to track pedestrians and assign an ID for each detected pedestrian. Both models were adapted to the fisheye images through transfer learning processes. The 2Cam-ReID module consists of a camera constraint model and a pedestrian ReID model developed for tracking pedestrians in images captured by a pair of adjacent cameras with overlapping views and assigning unique IDs to the pedestrians captured by both cameras. We evaluated both models on a real-world traffic dataset captured by surround-view fisheye cameras mounted on top of a vehicle. Our experiments demonstrate that the proposed two modules in SPinSVMC achieve high accuracy in both pedestrian detection, tracking and ReID in fisheye image domains.