Chan-Su Lee

Tracking People on a Torus

We present a framework for monocular 3D kinematic pose tracking and viewpoint estimation of periodic and quasi-periodic human motions from an uncalibrated camera. The approach we introduce here is based on learning both the visual observation manifold and the kinematic manifold of the motion using a joint representation. We show that the visual manifold of the observed shape of a human performing a periodic motion, observed from different viewpoints, is topologically equivalent to a torus manifold. The approach we introduce here is based on the supervised learning of both the visual and kinematic manifolds. Instead of learning an embedding of the manifold, we learn the geometric deformation between an ideal manifold (conceptual equivalent topological structure) and a twisted version of the manifold (the data). Experimental results show accurate estimation of the 3D body posture and the viewpoint from a single uncalibrated camera.

Coupled Visual and Kinematic Manifold Models for Tracking

In this paper, we consider modeling data lying on multiple continuous manifolds. In particular, we model the shape manifold of a person performing a motion observed from different viewpoints along a view circle at a fixed camera height. We introduce a model that ties together the body configuration (kinematics) manifold and visual (observations) manifold in a way that facilitates tracking the 3D configuration with continuous relative view variability. The model exploits the low-dimensionality nature of both the body configuration manifold and the view manifold, where each of them are represented separately. The resulting representation is used for tracking complex motions within a Bayesian framework, in which the model provides a low-dimensional state representation as well as a constrained dynamic model for both body configuration and view variations. Experimental results estimating the 3D body posture from a single camera are presented for the HUMANEVA dataset and other complex motion video sequences.

Learning a Joint Manifold Representation from Multiple Data Sets

The problem we address in the paper is how to learn a joint representation from data lying on multiple manifolds. We are given multiple data sets and there is an underlying common manifold among the different data set. We propose a framework to learn an embedding of all the points on all the manifolds in a way that preserves the local structure on each manifold and, in the same time, collapses all the different manifolds into one manifold in the embedding space, while preserving the implicit correspondences between the points across different data sets. The proposed solution works as extensions to current state of the art spectral-embedding approaches to handle multiple manifolds.

Editor's choice article: Homeomorphic Manifold Analysis (HMA): Generalized separation of style and content on manifolds

The problem of separation of style and content is an essential element of visual perception, and is a fundamental mystery of perception. This problem appears extensively in different computer vision applications. The problem we address in this paper is the separation of style and content when the content lies on a low-dimensional nonlinear manifold representing a dynamic object. We show that such a setting appears in many human motion analysis problems. We introduce a framework for learning parameterization of style and content in such settings. Given a set of topologically equivalent manifolds, the Homeomorphic Manifold Analysis (HMA) framework models the variation in their geometries in the space of functions that maps between a topologically-equivalent common representation and each of them. The framework is based on decomposing the style parameters in the space of nonlinear functions that map between a unified embedded representation of the content manifold and style-dependent visual observations. We show the application of the framework in synthesis, recognition, and tracking of certain human motions that follow this setting, such as gait and facial expressions.

Dynamic shape outlier detection for human locomotion

Dynamic human shape in video contains rich perceptual information, such as the body posture, identity, and even the emotional state of a person. Human locomotion activities, such as walking and running, have familiar spatiotemporal patterns that can easily be detected in arbitrary views. We present a framework for detecting shape outliers for human locomotion using a dynamic shape model that factorizes the body posture, the viewpoint, and the individuals shape style. The model uses a common embedding of the kinematic manifold of the motion and factorizes the shape variability with respect to different viewpoints and shape styles in the space of the coefficients of the nonlinear mapping functions that are used to generate the shapes from the kinematic manifold representation. Given a corrupted input silhouette, an iterative procedure is used to recover the body posture, viewpoint, and shape style. We use the proposed outlier detection approach to fill in the holes in the input silhouettes, and detect carried objects, shadows, and abnormal motions.

Analysis of the Effect on Attention and Relaxation Level by Correlated Color Temperature and Illuminance of LED Lighting using EEG Signal

Preferred combinations of illuminance and color temperature of lighting depend on daily living activities. We investigated whether the illumination stimuli of LED lighting can enhance attention and relaxation level by controlling color temperature and illuminance level according to activities. Illuminations and color temperatures of LED flat panels are controlled in accordance with activities such as office work and resting. The attention and relaxation level under the task specific lightings are compared with those under normal lighting condition. Single channel EEG signals from the NeuroSky`s Mindset are used to estimate attention and relaxation level of human subjects under different lighting conditions. Experiment results show that high color temperature with high illuminance of LED lightings (6600K, 800lx) shows improved attention level compared with conventional lighting conditions (4000K, 500lx).

Tracking Hand Rotation and Grasping from an IR Camera Using Cylindrical Manifold Embedding

This paper presents a new approach for hand rotation and grasping tracking from a single IR camera. For the complexity and ambiguity of hand pose, it is difficult to track hand pose and view variations simultaneously from a single camera. We propose a cylindrical manifold embedding for one dimensional hand pose variation and cyclic viewpoint variation. A hand pose shape from a specific viewpoint can be generated from an embedding point on the cylindrical manifold after learning nonlinear generative models from the embedding space to the corresponding observed shape. Hand grasping with simultaneous hand rotation is tracked using particle filter on the manifold space. Experimental results for synthetic and real data show accurate tracking of grasping hand with rotation. The proposed approach shows potentials for advanced user interface in dark environments.

Tracking hand rotation and various grasping gestures from an IR camera using extended cylindrical manifold embedding

This paper presents a new approach for tracking hand rotation and various grasping gestures through an infrared camera. For the complexity and ambiguity of an observed hand shape, it is difficult to simultaneously estimate hand configuration and orientation from a silhouette image of a grasping hand gesture. This paper proposes a dynamic shape model for hand grasping gestures using cylindrical manifold embedding to analyze variations of hand shape in different hand configurations between two key hand poses and in simultaneous circular view change by hand rotation. An arbitrary hand shape between two key hand poses from any view can be generated using a cylindrical manifold embedding point after learning nonlinear generative models from the embedding space to the corresponding hand shape observed. The cylindrical manifold embedding model is extended to various grasping gestures by decomposing multiple cylindrical manifold embeddings through grasping style analysis. Grasping hand gestures with simultaneous hand rotation are tracked using particle filters on the manifold space with grasping style estimation. Experimental results for synthetic and real data indicate that the proposed model can accurately track various grasping gestures with hand rotation. The proposed approach may be applied to advanced user interfaces in dark environments by using images beyond the visible spectrum.

Style adaptive contour tracking of human gait using explicit manifold models

In the domain of human motion analysis, the observed contours of the body contain rich information about the body configuration, the motion performed, the person’s identity, and even the emotional states of the person. In this paper, we introduce a framework for Bayesian tracking of the dynamic contours of the articulated human motion. We propose a factorized generative model for walking shape contour sequences that separates the dynamic deformation due to a motion, from the static variability due to the appearance of the person performing that motion. This results in an efficient tracking of gait sequence with a low-dimensional representation of body configuration and simultaneous adaptation to highly nonlinear static and dynamic shape deformations. Experimental results using the CMU Mobo, the University of Southampton (UoS), and M. Black’s walking sequence, show accurate contour tracking of a person walking by dynamic body configuration estimation on a low-dimensional manifold space and personal style estimation to fit the contour to the individual characteristics. In addition, the estimated shape style provides a good descriptors for human identification from gait.

A Development of a LED Stand Using Illuminance Sensor for Efficient Energy Saving

In this paper, we present a new lighting control method considering ambient light in addition to the required lighting illumination for efficient energy saving of a LED stand. We estimate accurate environmental illuminance using a cheap illuminance sensor by modeling measured- and actual-illuminance using quadratic polynomial approximation. The relation between PWM(Pulse Width Modulation) duty ratio and illuminance intensity is modeled by a linear model. Illumination of the LED stand is controlled by estimating the difference of required illumination and the estimated ambient illumination. The developed LED stand has reduced electric energy consumption compared with a conventional manually controlled LED stand with the same lighting source. In addition, human subject evaluation shows that the LED stand, which is applied the proposed method, is more satisfactory than conventional ones since the proposed automatic controlled illumination produce more accurately required lighting and it is convenient.