Sangho Park

Multi-person interaction and activity analysis: a synergistic track- and body-level analysis framework

This paper presents a synergistic track- and body-level analysis framework for multi-person interaction and activity analysis in the context of video surveillance. The proposed two-level analysis framework covers human activities both in wide and narrow fields of view with distributed camera sensors. The track-level analysis deals with the gross-level activity patterns of multiple tracks in various wide-area surveillance situations. The body-level analysis focuses on detailed-level activity patterns of individuals in isolation or in groups. ‘Spatio-temporal personal space’ is introduced to model various patterns of grouping behavior between persons. ‘Adaptive context switching’ is proposed to mediate the track-level and body-level analysis depending on the interpersonal configuration and imaging fidelity. Our approach is based on the hierarchy of action concepts: static pose, dynamic gesture, body-part action, single-person activity, and group interaction. Event ontology with human activity hierarchy combines the multi-level analysis results to form a semantically meaningful event description. Experimental results with real-world data show the effectiveness of the proposed framework.

Multiperspective Thermal IR and Video Arrays for 3D Body Tracking and Driver Activity Analysis

This paper presents a multi-perspective (i.e., four camera views) multi-modal (i.e., thermal infrared and color) video based system for robust and real-time 3D tracking of important body parts.The multi-perspective characteristics of the system provides 3Dtrajectory of the body parts, while the multi-modal characteristics of the system provides robustness and reliability of feature detection and tracking. The application context for this research is that of intelligent vehicles and driver assistance systems. Experimental results demonstrate effectiveness of the proposed system.

Understanding human interactions with track and body synergies (TBS) captured from multiple views

This paper presents a new two-stage multi-view framework for the analysis of human interactions and activities. The analysis is performed in a distributed multi-view vision system that synergistically integrates track- and body-level processing. The proposed framework is geared toward versatile and easily-deployable systems that do not require careful camera calibration. The main contributions of the paper are as follows; (1) context-dependent view switching for occlusion handling, (2) a method for switching the two-stage analysis between the track- and body-level processing, and (3) a hypothesis-verification paradigm for top-down feedback that exploits the spatio-temporal constraints inherent in human interaction. An experimental evaluation shows the efficacy of the proposed system for analyzing multi-person interactions.

Driver activity analysis for intelligent vehicles: issues and development framework

This paper examines the feasibility of a semantic-level driver activity analysis system. Several new considerations are made to construct the hierarchy of driver activity. Driver activity is represented and recognized at multiple levels: individual body-part pose/gesture at the low level, single body-part action at the middle level, and the driver interaction with the vehicle at the high level. Driving is represented in terms of the interactions among driver, vehicle, and surround, and driver activity is recognized by a rule-based decision tree. Our system works with a single color camera data, and it can be easily expanded to incorporate multimodal sensor data.

Tracking of individuals in very long video sequences

In this paper we present an approach for automatically detecting and tracking humans in very long video sequences. The detection is based on background subtraction using a multi-mode Codeword method. We enhance this method both in terms of representation and in terms of automatically updating the background allowing for handling gradual and rapid changes. Tracking is conducted by building appearance-based models and matching these over time. Tests show promising detection and tracking results in a ten hour video sequence.

Homography-based Analysis of People and Vehicle Activities in Crowded Scenes

This paper presents an new framework for homography-based analysis of pedestrian-vehicle activity in crowded scenes. Planar homography constraint is exploited to extract view-invariant object features including footage area and velocity of objects on the ground plane. Spatio-temporal relationships between people- and vehicle- tracks are represented by a semantic event. Context awareness of the situation is achieved by the estimated density distribution of objects and the anticipation of possible directions of near-future tracks using piecewise velocity history. Single-view and multi-view based homography mapping options are compared. Our framework can be used to enhance situational awareness for disaster prevention, human interactions in structured environments, and crowd movement analysis at wide regions

Analysis and query of person-vehicle interactions in homography domain

This paper presents an efficient and robust paradigm for analysis and query of moving-object interactions in planar homography domain.People and vehicle activities/interactions are analyzed for situational awareness by using a multi-perspective approach.Planar homography constraints are exploited to extract view-invariant object features including footage area and velocity of objects on the ground plane. Spatio-temporal relationships between person-and vehicle-tracks are represented by a semantic event grammar. Semantic-level information of the situation is achieved with the anticipation of possible directions of near-future tracks using piecewise velocity history. An efficient query paradigm is proposed by histogram-based approximation of probability density functions of objects and by quad-tree indexing. Experimental data show promising results.Our framework can be applied to applications for enhanced situational awareness such as disaster prevention,human interactions in structured environments,and crowd movement analysis in wide-view areas.

Video Analysis of Vehicles and Persons for Surveillance

This chapter presents a multi-perspective vision-based analysis of the activities of vehicles and persons for the enhancement of situational awareness in surveillance. Multiple perspectives provide a useful invariant feature of the object in the image, i.e., the footage area on the ground. Moving objects are detected in the image domain, and the tracking results of the objects are represented in the projection domain using planar homography. Spatio-temporal relationships between human and vehicle tracks are categorized as safe or unsafe situation depending on the site context such as walkway and driveway locations. Semantic-level information of the situation is achieved with the anticipation of possible directions of near-future tracks using piecewise velocity history. Crowd density is estimated from the footage on the homography plane. Experimental data show promising results. Our framework can be applied to broad range of situational awareness for emergency response, disaster prevention, human interactions in structured environments, and crowd movement analysis in a wide field of view.

A Two-stage Multi-view Analysis Framework for Human Activity and Interactions

This paper presents a new framework for a multi-stage multi-view approach for human interactions and activity analysis. The analysis is performed in a distributed vision system that synergistically integrate track- and body-level representations across multiple cameras. Our system aims at versatile and easily-deployable system that does not require careful camera calibration. Main contributions of the paper are: (1) context-dependent camera handover for occlusion handling, (2) switching the multi-stage analysis between track- and body-level representations, and (3) a hypothesis-verification paradigm for top-down feedback exploiting spatio-temporal constraints inherent in human interaction. Experimental evaluation shows the efficacy of the proposed system for analyzing multi-person interactions. Current implementation uses two views, but extension to more views is straightforward.

Multi-view Video Analysis of Humans and Vehicles in an Unconstrained Environment

This paper presents an automatic visual analysis system for simultaneously tracking humans and vehicles using multiple cameras in an unconstrained outdoor environment. The system establishes correspondence between views using a principal axis approach for humans and a footage region approach for vehicles. Novel methods for locating humans in groups and solving ambiguity when matching vehicles across views are presented. Foreground segmentation for each view is performed using the codebook method and HSV shadow suppression. The tracking of objects is performed in each view, and occlusion situations are resolved by probabilistic appearance models. The system is tested on hours of video and on three different datasets.