Kuan-Hui Lee

A Quality-of-Content-Based Joint Source and Channel Coding for Human Detections in a Mobile Surveillance Cloud

More than 70% of consumer mobile Internet traffic will be mobile video transmissions by 2019. The development of wireless video transmission technologies has been boosted by the rapidly increasing demand of video streaming applications. Although more and more videos are delivered for video analysis (e.g., object detection/tracking and action recognition), most existing wireless video transmission schemes are developed to optimize human perception quality and are suboptimal for video analysis. In mobile surveillance networks, a cloud server collects videos from multiple moving cameras and detects suspicious persons in all camera views. Camera mobility in smartphones or dash cameras implies that video is to be uploaded through bandwidth-limited and error-prone wireless networks, which may cause quality degradation of the decoded videos and jeopardize the performance of video analyses. In this paper, we propose an effective rate-allocation scheme for multiple moving cameras in order to improve human detection (content) performance. Therefore, the optimization criterion of the proposed rate-allocation scheme is driven by quality of content (QoC). Both video source coding and application layer forward error correction coding rates are jointly optimized. Moreover, the proposed rate-allocation problem is formulated as a convex optimization problem and can be efficiently solved by standard solvers. Many simulations using High Efficiency Video Coding standard compression of video sequences and the deformable part model object detector are carried, and results demonstrate the effectiveness and favorable performance of our proposed QoC-driven scheme under different pedestrian densities and wireless conditions.

Driving Recorder Based On-road Pedestrian Tracking Using Visual SLAM and Constrained Multiple-Kernel

This paper proposes a robust driving recorder based on-road pedestrian tracking system, which effectively integrates Visual Simultaneous Localization And Mapping (V-SLAM), pedestrian detection, ground plane estimation, and kernel-based tracking techniques. The proposed system systematically detects the pedestrians from recorded video frames and tracks the pedestrians in the V-SLAM inferred 3-D space via a tracking-by-detection scheme. In order to efficiently associate the detected pedestrian frame-by-frame, we propose a novel tracking framework, combining the Constrained Multiple-Kernel (CMK) tracking and the estimated 3-D (depth) information, to globally optimize the data association between consecutive frames. By taking advantage of the appearance model and 3-D information, the proposed system not only achieves high effectiveness but also well handles occlusion in the tracking. Experimental results show the favorable performance of the proposed system which efficiently tracks on-road pedestrian in a moving camera equipped on a driving vehicle.

Robust Human Tracking Based on DPM Constrained Multiple-Kernel from a Moving Camera

In this paper, we attempt to solve the challenging task of precise and robust human tracking from a moving camera. We propose an innovative human tracking approach, which efficiently integrates the deformable part model (DPM) into multiple-kernel tracking from a moving camera. The proposed approach consists of a two-stage tracking procedure. For each frame, we first iteratively mean-shift several spatially weighted color histograms, called kernels, from the current frame to the next frame. Each kernel corresponds to a part model of a DPM-detected human. In the second step, conditioned on the tracking results of these kernels on the later frame, we then iteratively mean-shift the part models on that frame. The part models are represented by histogram of gradient (HOG) features, and the deformation cost of each part model provided by the trained DPM detector is used to constrain the movement of each detected body part from the first step. The proposed approach takes advantage of not only low computation owing to the kernel-based tracking, but also robustness of the DPM detector without the need of laborious human detection for each frame. Experimental results have shown that the proposed approach makes it possible to successfully track humans robustly with high accuracy under different scenarios from a moving camera.

Vehicle tracking iterative by Kalman-based constrained multiple-kernel and 3-D model-based localization

In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel facilitated with Kalman filtering, and then continuously updates the position and the orientation by adopting a systematically built 3-D vehicle model in an evolutionary computing framework. The proposed system can thus successfully track vehicles under occlusion as facilitated by the obtained 3-D geometry of vehicles. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles while maintaining the knowledge of 3-D vehicle geometry.

Self-organized and scalable camera networks for systematic human tracking across nonoverlapping cameras

We present a self-organized and scalable multiple-camera tracking system that tracks human across the cameras with nonoverlapping views. Given the GPS locations of uncalibrated cameras, the system automatically detects the existence of camera link within the camera network based on the routing information provided by Google Maps. The connected zones in any pair of directly-connected cameras are identified based on the feature matching between the cameras view and Google Street View. The camera link model is further estimated by an unsupervised learning scheme. Finally, multiple-camera tracking is performed. Thanks to the unsupervised pairwise learning and tracking in our system, the camera network is self-organized, and our proposed system is able to be scaled up efficiently when more cameras are added into the network.

Quality-of-content (QoC)-driven rate allocation for video analysis in mobile surveillance networks

Nowadays, more and more videos are transmitted for video analytics purposes rather than human perceptions. In mobile surveillance networks, a cloud server collects videos delivered from multiple moving cameras and detects suspicious people in all the camera views. However, all the videos recorded by moving cameras such as phone or dash cameras are uploaded through bandwidth-limited wireless networks. Therefore, videos are required to be encoded with high compression ratio to satisfy the total data rate constraint, which may affect the video analyses (e.g., human detection/tracking and action recognition, etc.) performance due to the degraded video decoding qualities at the server side. In this paper, we propose an effective content-driven video source coding rate allocation scheme, which can improve the human detection success rate in mobile surveillance networks under a total data rate constraint. The proposed scheme allocates appropriate amount of data rate to each moving camera based on the corresponding content information (i.e., human detection results). A model of human detection accuracy based on object area and video quality is provided. The rate allocation problem is formulated as a convex optimization problem and can be solved by standard solvers. Simulations with real video sequences demonstrate the effectiveness of our proposed scheme.

Deformable multiple-kernel based human tracking using a moving camera

In this paper, we propose an innovative human tracking algorithm, which efficiently integrates the deformable part model (DPM) into the multiple-kernel based tracking using a moving camera. By representing each part model of a DPM detected human as a kernel, the proposed algorithm iteratively mean-shift the kernels (i.e., part models) based on color appearance and histogram of gradient (HOG) features. More specifically, the color appearance features, in terms of kernel histogram, are used for tracking each body part from one frame to the next, the deformation cost provided by DPM detector is further used to constrain the movement of each body kernel based on the HOG features. The proposed deformable multiple-kernel (DMK) tracking algorithm takes advantage of not only low computation owing to the kernel-based tracking, but also robustness of the DPM detector. Experimental results have shown the favorable performance of the proposed algorithm, which can successfully track human using a moving camera more accurately under different scenarios.

Multiple-kernel based vehicle tracking using 3-D deformable model and license plate self-similarity

In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel, facilitated with Kalman filtering, to continuously update the position and the orientation of the moving vehicles. To further reliably track vehicles under partial occlusion or even total occlusion, our tracking algorithm also systematically builds 3-D vehicle model, from which the license plate region is identified and a self-similarity descriptor is further used for low-resolution license plate matching. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles under serious occlusion while maintaining the knowledge of 3-D geometry of the tracked vehicles.

Ground-Moving-Platform-Based Human Tracking Using Visual SLAM and Constrained Multiple Kernels

This paper proposes a robust ground-moving-platform-based human tracking system, which effectively integrates visual simultaneous localization and mapping (V-SLAM), human detection, ground plane estimation, and kernel-based tracking techniques. The proposed system systematically detects humans from recorded video frames of a moving camera and tracks the humans in the V-SLAM-inferred 3-D space via a tracking-by-detection scheme. To efficiently associate the detected human frame by frame, we propose a novel human tracking framework, combining the constrained-multiple-kernel tracking and the estimated 3-D information (depth), to globally optimize the data association between consecutive frames. By taking advantage of the appearance model and 3-D information, the proposed system not only achieves high effectiveness but also well handles occlusion in the tracking. Experimental results show the favorable performance of the proposed system, which efficiently tracks humans in a camera equipped on a ground-moving platform such as a dash camera and an unmanned ground vehicle.

Camera self-calibration from tracking of moving persons

In a video surveillance system with a single static camera, tracking results of moving persons can be effectively used for camera self-calibration. However, the current methods need to depend on robustness of both tracking and segmentation procedures. RANSAC has been widely used to remove outliers in finding the vertical vanishing point and the horizon line, but the performance is degraded when the proportion of outliers is high. Last but not least, all of them require excessive simplifications in the algorithmic procedures resulting in increasing reprojection error. In this paper, a robust segmentation and tracking system is applied to provide accurate estimation of head and foot locations of moving persons. The noise in the computation of vanishing points is handled by mean shift clustering and Laplace linear regression through convex optimization. We also propose to use the estimation of distribution algorithm (EDA) to search for the local optimal solution for camera calibration that minimizes average reprojection error on the ground plane, while relaxing the assumptions on camera parameters. Promising evaluations of the performance of our proposed method on real scenes are presented.