Jungchan Cho

Procrustean Normal Distribution for Non-rigid Structure from Motion

A well-defined deformation model can be vital for non-rigid structure from motion (NRSfM). Most existing methods restrict the deformation space by assuming a fixed rank or smooth deformation, which are not exactly true in the real world, and they require the degree of deformation to be predetermined, which is impractical. Meanwhile, the errors in rotation estimation can have severe effects on the performance, i.e., these errors can make a rigid motion be misinterpreted as a deformation. In this paper, we propose an alternative to resolve these issues, motivated by an observation that non-rigid deformations, excluding rigid changes, can be concisely represented in a linear subspace without imposing any strong constraints, such as smoothness or low-rank. This observation is embedded in our new prior distribution, the Procrustean normal distribution (PND), which is a shape distribution exclusively for non-rigid deformations. Because of this unique characteristic of the PND, rigid and non-rigid changes can be strictly separated, which leads to better performance. The proposed algorithm, EM-PND, fits a PND to given 2D observations to solve NRSfM without any user-determined parameters. The experimental results show that EM-PND gives the state-of-the-art performance for the benchmark data sets, confirming the adequacy of the new deformation model.

Robust action recognition using local motion and group sparsity

Recognizing actions in a video is a critical step for making many vision-based applications possible and has attracted much attention recently. However, action recognition in a video is a challenging task due to wide variations within an action, camera motion, cluttered background, and occlusions, to name a few. While dense sampling based approaches are currently achieving the state-of-the-art performance in action recognition, they do not perform well for many realistic video sequences since, by considering every motion found in a video equally, the discriminative power of these approaches is often reduced due to clutter motions, such as background changes and camera motions. In this paper, we robustly identify local motions of interest in an unsupervised manner by taking advantage of group sparsity. In order to robustly classify action types, we emphasize local motion by combining local motion descriptors and full motion descriptors and apply group sparsity to the emphasized motion features using the multiple kernel method. In experiments, we show that different types of actions can be well recognized using a small number of selected local motion descriptors and the proposed algorithm achieves the state-of-the-art performance on popular benchmark datasets, outperforming existing methods. We also demonstrate that the group sparse representation with the multiple kernel method can dramatically improve the action recognition performance.

Complex Non-rigid 3D Shape Recovery Using a Procrustean Normal Distribution Mixture Model

Recovering the 3D shape of a non-rigid object is a challenging problem. Existing methods make the low-rank assumption and do not scale well with the increased degree of freedom found in complex non-rigid deformations or shape variations. Moreover, in general, the degree of freedom of deformation is assumed to be known in advance, which limits the applicability of non-rigid structure from motion algorithms in a practical situation. In this paper, we propose a method for handling complex shape variations based on the assumption that complex shape variations can be represented probabilistically by a mixture of primitive shape variations. The proposed model is a generative probabilistic model, called a Procrustean normal distribution mixture model, which can model complex shape variations without rank constraints. Experimental results show that the proposed method significantly outperforms existing methods.

Privacy-Aware Communication for Smartphones Using Vibration

We propose a novel communication method between smart devices using a built-in vibrator and accelerometer. The proposed approach is ideal for low-rate private communication and its communication medium is an object on which smart devices are placed, such as tables and desks. When more than two smart devices are placed on an object and one device wants to transmit a message to the other devices, the transmitting device generates a sequence of vibrations. The vibrations are propagated through the object on which devices are placed. The receiving devices analyze their accelerometer readings to decode incoming messages. Unlike radio-based wireless communication where eavesdropping of private communication is possible without the knowledge of the user, the proposed method can guarantee privacy as long as the object used for communication is secured. The proposed method is implemented on Android smart phones and comprehensive experiments are conducted to show its feasibility.

Selective generation of Gabor features for fast face recognition on mobile devices

In this paper, we propose a robust face recognition method to provide fast response on a mobile device by selectively generating Gabor features. The Gabor filter has been popularly used in face recognition to improve recognition performance. Since the computational effort for generating a Gabor feature is very large, it is important to use only the discriminative Gabor features on mobile devices which do not have sufficient computing power. At the same time, it is also important to maintain the recognition performance at an acceptable level. To reduce computational effort without degrading the recognition performance, the proposed method selectively generates Gabor features based on a contribution measure obtained by discriminant analysis. Face recognition is performed using only the selectively generated Gabor features, and the experimental results for the CMU Multi-PIE database and a real world data set show that the number of Gabor features can be reduced by more than 50% while keeping almost the same recognition performance. On a 624MHz mobile phone, the execution time of feature extraction can be reduced to 19ms from 46ms on average.

Vibration-Based Surface Recognition for Smartphones

With various sensors in a smart phone, it is now possible to obtain information about a user and her surroundings, such as the location of a smartphone and the activity of the smartphone user, and the obtained context information is being used to provide new services to the users. In this paper, we propose VibePhone, which uses a built-in vibrator and accelerometer, for recognizing the type of surfaces contacted by a smart phone, enabling the sense of touch in smartphones. For humans and animals, the sense of touch is fundamental for both recognizing and learning the properties of objects. The sense of touch is obtained from the texture of an object and humans recognize the type of an object by scrubbing the surface with fingers. Since a smartphone cannot physically scrub the contacting surface, we emulate the touch by generating vibrations using a smartphone and propose a method to recognize the type of contacting objects. The recognition of the object type by vibration alone is an extremely difficult task, even for a human. However, we demonstrate that it is possible to distinguish object types into broad categories where a phone is usually placed, e.g., sofas, plastic tables, wooden tables, hands, backpacks, and pants pockets. The proposed VibePhone system achieves an accuracy over 85% on average. We have prototyped VibePhone on an Android-based smart phone which changes its background display based on the contacting surface. We envision that the haptic perception in future smart phones will enable new experiences to the users.

Consensus of Non-rigid Reconstructions

Recently, there have been many progresses for the problem of non-rigid structure reconstruction based on 2D trajectories, but it is still challenging to deal with complex deformations or restricted view ranges. Promising alternatives are the piecewise reconstruction approaches, which divide trajectories into several local parts and stitch their individual reconstructions to produce an entire 3D structure. These methods show the state-of-the-art performance, however, most of them are specialized for relatively smooth surfaces and some are quite complicated. Meanwhile, it has been reported numerously in the field of pattern recognition that obtaining consensus from many weak hypotheses can give a strong, powerful result. Inspired by these reports, in this paper, we push the concept of part-based reconstruction to the limit: Instead of considering the parts as explicitly-divided local patches, we draw a large number of small random trajectory sets. From their individual reconstructions, we pull out a statistic of each 3D point to retrieve a strong reconstruction, of which the procedure can be expressed as a sparse l1-norm minimization problem. In order to resolve the reflection ambiguity between weak (and possibly bad) reconstructions, we propose a novel optimization framework which only involves a single eigenvalue decomposition. The proposed method can be applied to any type of data and outperforms the existing methods for the benchmark sequences, even though it is composed of a few, simple steps. Furthermore, it is easily parallelizable, which is another advantage.

EM-GPA: Generalized Procrustes analysis with hidden variables for 3D shape modeling

Aligning shapes is essential in many computer vision problems and generalized Procrustes analysis (GPA) is one of the most popular algorithms to align shapes. However, if some of the shape data are missing, GPA cannot be applied. In this paper, we propose EM-GPA, which extends GPA to handle shapes with hidden (missing) variables by using the expectation-maximization (EM) algorithm. For example, 2D shapes can be considered as 3D shapes with missing depth information due to the projection of 3D shapes into the image plane. For a set of 2D shapes, EM-GPA finds scales, rotations and 3D shapes along with their mean and covariance matrix for 3D shape modeling. A distinctive characteristic of EM-GPA is that it does not enforce any rank constraint often appeared in other work and instead uses GPA constraints to resolve the ambiguity in finding scales, rotations, and 3D shapes. The experimental results show that EM-GPA can recover depth information accurately even when the noise level is high and there are a large number of missing variables. By using the images from the FRGC database, we show that EM-GPA can successfully align 2D shapes by taking the missing information into consideration. We also demonstrate that the 3D mean shape and its covariance matrix are accurately estimated. As an application of EM-GPA, we construct a 2D+3D AAM (active appearance model) using the 3D shapes obtained by EM-GPA, and it gives a similar success rate in model fitting compared to the method using real 3D shapes. EM-GPA is not limited to the case of missing depth information, but it can be easily extended to more general cases.

Procrustean Normal Distribution for Non-Rigid Structure from Motion

A well-defined deformation model can be vital for non-rigid structure from motion (NRSfM). Most existing methods restrict the deformation space by assuming a fixed rank or smooth deformation, which are not exactly true in the real world, and they require the degree of deformation to be predetermined, which is impractical. Meanwhile, the errors in rotation estimation can have severe effects on the performance, i.e., these errors can make a rigid motion be misinterpreted as a deformation. In this paper, we propose an alternative to resolve these issues, motivated by an observation that non-rigid deformations, excluding rigid changes, can be concisely represented in a linear subspace without imposing any strong constraints, such as smoothness or low-rank. This observation is embedded in our new prior distribution, the Procrustean normal distribution (PND), which is a shape distribution exclusively for non-rigid deformations. Because of this unique characteristic of the PND, rigid and non-rigid changes can be strictly separated, which leads to better performance. The proposed algorithm, EM-PND, fits a PND to given 2D observations to solve NRSfM without any user-determined parameters. The experimental results show that EM-PND gives the state-of-the-art performance for the benchmark data sets, confirming the adequacy of the new deformation model.A well-defined deformation model can be vital for non-rigid structure from motion (NRSfM). Most existing methods restrict the deformation space by assuming a fixed rank or smooth deformation, which are not exactly true in the real world, and they require the degree of deformation to be predetermined, which is impractical. Meanwhile, the errors in rotation estimation can have severe effects on the performance, i.e., these errors can make a rigid motion be misinterpreted as a deformation. In this paper, we propose an alternative to resolve these issues, motivated by an observation that non-rigid deformations, excluding rigid changes, can be concisely represented in a linear subspace without imposing any strong constraints, such as smoothness or low-rank. This observation is embedded in our new prior distribution, the Procrustean normal distribution (PND), which is a shape distribution exclusively for non-rigid deformations. Because of this unique characteristic of the PND, rigid and non-rigid changes can be strictly separated, which leads to better performance. The proposed algorithm, EM-PND, fits a PND to given 2D observations to solve NRSfM without any user-determined parameters. The experimental results show that EM-PND gives the state-of-the-art performance for the benchmark data sets, confirming the adequacy of the new deformation model.

Single image 3D human pose estimation using a procrustean normal distribution mixture model and model transformation

3D human pose estimation from a single image is an important problem.We use the Procrustean normal distribution mixture model as a 3D shape prior.We propose a model transformation to adjust limb lengths of the 3D shape prior model.Inaccuracies of 2D part detections are handled by multiple 2D pose candidates.We demonstrate that the proposed method outperforms existing methods. 3D human pose estimation from a single image is an important problem in computer vision with a number of applications, including action recognition and scene understanding. However, it is still challenging due to its ill-posedness and complex non-rigid shape variations of a human body. In this paper, we use the Procrustean normal distribution mixture model as a 3D shape prior and propose a model transformation method for adjusting limb lengths of the 3D shape prior model, by which the proposed method can be applied to a novel test image. Inaccuracies of 2D part detections are handled by selecting from a diverse set of 2D pose candidates considering both the 2D part model and 3D shape model. Experimental results show that the proposed method performs favorably compared with existing methods, despite inaccuracies of 2D part detections and 3D shape ambiguities.