Akihiro Sugimoto

Fast unsupervised ego-action learning for first-person sports videos

Portable high-quality sports cameras (e.g. head or helmet mounted) built for recording dynamic first-person video footage are becoming a common item among many sports enthusiasts. We address the novel task of discovering first-person action categories (which we call ego-actions) which can be useful for such tasks as video indexing and retrieval. In order to learn ego-action categories, we investigate the use of motion-based histograms and unsupervised learning algorithms to quickly cluster video content. Our approach assumes a completely unsupervised scenario, where labeled training videos are not available, videos are not pre-segmented and the number of ego-action categories are unknown. In our proposed framework we show that a stacked Dirichlet process mixture model can be used to automatically learn a motion histogram codebook and the set of ego-action categories. We quantitatively evaluate our approach on both in-house and public YouTube videos and demonstrate robust ego-action categorization across several sports genres. Comparative analysis shows that our approach outperforms other state-of-the-art topic models with respect to both classification accuracy and computational speed. Preliminary results indicate that on average, the categorical content of a 10 minute video sequence can be indexed in under 5 seconds.

3D Head Tracking using the Particle Filter with Cascaded Classifiers

We propose a method for real-time people tracking using multiple cameras. The particle filter framework is known to be effective for tracking people, but most of existing methods adopt only simple perceptual cues such as color histogram or contour similarity for hypothesis evaluation. To improve the robustness and accuracy of tracking more sophisticated hypothesis evaluation is indispensable. We therefore present a novel technique for human head tracking using cascaded classifiers based on AdaBoost and Haar-like features for hypothesis evaluation. In addition, we use multiple classifiers, each of which is trained respectively to detect one direction of a human head. During real-time tracking the most suitable classifier is adaptively selected by considering each hypothesis and known camera position. Our experimental results demonstrate the effectiveness and robustness of our method.

Using individuality to track individuals: Clustering individual trajectories in crowds using local appearance and frequency trait

In this work, we propose a method for tracking individuals in crowds. Our method is based on a trajectory-based clustering approach that groups trajectories of image features that belong to the same person. The key novelty of our method is to make use of a persons individuality, that is, the gait features and the temporal consistency of local appearance to track each individual in a crowd. Gait features in the frequency domain have been shown to be an effective biometric cue in discriminating between individuals, and our method uses such features for tracking people in crowds for the first time. Unlike existing trajectory-based tracking methods, our method evaluates the dissimilarity of trajectories with respect to a group of three adjacent trajectories. In this way, we incorporate the temporal consistency of local patch appearance to differentiate trajectories of multiple people moving in close proximity. Our experiments show that the use of gait features and the temporal consistency of local appearance contributes to significant performance improvement in tracking people in crowded scenes.

Cyclosporin-A potently induces highly cardiogenic progenitors from embryonic stem cells

Though cardiac progenitor cells should be a suitable material for cardiac regeneration, efficient ways to induce cardiac progenitors from embryonic stem (ES) cells have not been established. Extending our systematic cardiovascular differentiation method of ES cells, here we show efficient and specific expansion of cardiomyocytes and highly cardiogenic progenitors from ES cells. An immunosuppressant, cyclosporin-A (CSA), showed a novel effect specifically acting on mesoderm cells to drastically increase cardiac progenitors as well as cardiomyocytes by 10-20 times. Approximately 200 cardiomyocytes could be induced from one mouse ES cell using this method. Expanded progenitors successfully integrated into scar tissue of infracted heart as cardiomyocytes after cell transplantation to rat myocardial infarction model. CSA elicited specific induction of cardiac lineage from mesoderm in a novel mesoderm-specific, NFAT independent fashion. This simple but efficient differentiation technology would be extended to induce pluripotent stem (iPS) cells and broadly contribute to cardiac regeneration.

Attention prediction in egocentric video using motion and visual saliency

We propose a method of predicting human egocentric visual attention using bottom-up visual saliency and egomotion information. Computational models of visual saliency are often employed to predict human attention; however, its mechanism and effectiveness have not been fully explored in egocentric vision. The purpose of our framework is to compute attention maps from an egocentric video that can be used to infer a persons visual attention. In addition to a standard visual saliency model, two kinds of attention maps are computed based on a cameras rotation velocity and direction of movement. These rotation-based and translation-based attention maps are aggregated with a bottom-up saliency map to enhance the accuracy with which the persons gaze positions can be predicted. The efficiency of the proposed framework was examined in real environments by using a head-mounted gaze tracker, and we found that the egomotion-based attention maps contributed to accurately predicting human visual attention.

Visualization of Information Spaces to Retrieve and Browse Image Data

We have developed a user interface for similarity-based image retrieval, where the distribution of retrieved data in a high-dimensional feature space is represented as a dynamical scatter diagram of thumbnail images in a 3-dimensional visualization space and similarities between data are represented as sizes in the 3-dimensional space. Coordinate systems in the visualization space are obtained by statistical calculations on the distribution of feature vectors of retrieved images. Our system provides some different transformations from a high-dimensional feature space to a 3-dimensional space that give different coordinate systems to the visualization space. By changing the coordinates automatically at some intervals, a spatial-temporal pattern of the distribution of images is generated. Furthermore a hierarchical coordinate system that consists of some local coordinate systems based on key images can be defined in the visualization space. These methods can represent a large number of retrieved results in a way that users can grasp intuitively.

Recovering the Basic Structure of Human Activities from a Video-Based Symbol String

In recent years stochastic context-free grammars have been shown to be effective in modeling human activities because of the hierarchical structures they represent. However, most of the research in this area has yet to address the issue of learning the activity grammars from a noisy input source, namely, video. In this paper, we present a framework for identifying noise and recovering the basic activity grammar from a noisy symbol string produced by video. We identify the noise symbols by finding the set of non-noise symbols that optimally compresses the training data, where the optimality of compression is measured using an MDL criterion. We show the robustness of our system to noise and its effectiveness in learning the basic structure of human activity, through an experiment with real video from a local convenience store.

An approximation algorithm for the two-layered graph drawing problem

We present a polynomial-time approximation algorithm for the minimum edge crossings problem for two-layered graphs. We show the relationship between the approximation ratio of our algorithm and the maximum degree of the vertices in the lower layer of the input graph. When the maximum degree is not greater than four, the approximation ratio is two and this ratio monotonically increases to three as the maximum degree becomes larger. We also present our experiments, showing that our algorithm constructs better solutions than the barycenter method and the median method for dense graphs as well as sparse graphs.

Structure-from-motion based hand-eye calibration using L ∞ minimization

This paper presents a novel method for so-called hand-eye calibration. Using a calibration target is not possible for many applications of hand-eye calibration. In such situations Structure-from-Motion approach of hand-eye calibration is commonly used to recover the camera poses up to scaling. The presented method takes advantage of recent results in the L∞-norm optimization using Second-Order Cone Programming (SOCP) to recover the correct scale. Further, the correctly scaled displacement of the hand-eye transformation is recovered solely from the image correspondences and robot measurements, and is guaranteed to be globally optimal with respect to the L∞-norm. The method is experimentally validated using both synthetic and real world datasets.

A Linear Algorithm for Computing the Homography from Conics in Correspondence

This paper presents a study, based on conic correspondences, on the relationship between two perspective images acquired by an uncalibrated camera. We show that for a pair of corresponding conics, the parameters representing the conics satisfy a linear constraint. To be more specific, the parameters that represent a conic in one image are transformed by a five-dimensional projective transformation to the parameters that represent the corresponding conic in another image. We also show that this transformation is expressed as the symmetric component of the tensor product of the transformation based on point/line correspondences and itself. In addition, we present a linear algorithm for uniquely determining the corresponding point-based transformation from a given conic-based transformation up to a scale factor. Accordingly, conic correspondences enable us to easily handle both points and lines in uncalibrated images of a planar object.