Masao Izumi

Generating natural language description of human behavior from video images

In visual surveillance applications, it is becoming popular to perceive video images and to interpret them using natural language concepts. We propose an approach to generating a natural language description of human behavior appearing in real video images. First, a head region of a human, on behalf of the whole body, is extracted from each frame. Using a model based method, three dimensional pose and position of the head are estimated. Next, the trajectory of these parameters is divided into segments of monotonous motions. For each segment, we evaluate conceptual features such as degree of change of pose and position and that of relative distance to some objects in the surroundings, and so on. By calculating the product of these feature values, a most suitable verb is selected and other syntactic elements are supplied. Finally natural language text is generated using a technique of machine translation.

Blackboard segmentation using video image of lecture and its applications

We propose a method for segmentation of written regions on a blackboard in the lecture room using a video image. We first detect the static edges of which locations on the image are stationary. Next, we extract several rectangular regions in which these static edges are located densely. Finally, by use of fuzzy rules, the extracted rectangles are merges as contextual regions, where letters and figures in each contextual region explain each context. We apply our method to automatic production of lecture video, and archives segmentation of written regions on the blackboard in lecture rooms.

Production of video images by computer controlled camera operation based on distribution of spatiotemporal mutual information

This paper defines a spatiotemporal mutual information on the pixels of a given video image on the basis of information theory (Shannons communication theory), which can be interpreted as the theoretical estimation of interested features for human being. As an application of this spatiotemporal mutual information, we propose a method of producing a vivid video image of the distance learning by using the computer controlled camera operation and switching of plural camera images on the basis of the video image processing. Results of the questionnaire survey for the video image produced by the method confirm the effectiveness of our approach.

Image matching using structures of edge-image graphs

This paper proposes an algorithm for examining similarity and correspondence between a pair of images in which the same object appears in different scaling and with rotation such as stereo images or motion stereo images. As a method for finding correspondence and similarity in an efficient manner, an edge-line image is found first and then the relationship of connection among edgelines in a graph structure are represented and expressed in an algebraic expression. Using this representation a method is clarified for finding similarity between images and an algorithm is proposed for finding similarity and further correspondence between two images affected by scaling and rotation using the degree of similarity between their line structures. Finally, the algorithm is applied to motion stereo images to examine its effectiveness.

Light-Weight Safety Confirmation System for Large-Scale Disasters

We have been developing and operating safety confirmation systems for universities since 1999. The Great East Japan Earthquake in 2011 clarified some issues related to the safety confirmation systems. Thereby, we reviewed the safety confirmation systems, the result of which revealed that more carefully selected functionality, restriction on the volume of communication, and enhanced anti-disaster feature of the systems would be required. We have then developed the light-weight safety confirmation system that does not have the function to simultaneously transmit mass emails and is designed for easier system migration. Evaluation was conducted by performing demonstration experiments of the newly developed safety confirmation at a disaster prevention drill of a university.

Model-based object recognition using camera control

We propose a robust object recognition system based on the control of viewpoint. The objective of our system is to seek the optimum camera position where the unknown object can be recognized clearly. We define a degree of recognition-ambiguity based on basic probabilities, that is calculated by using an input image and model images generated from object model data. Our active vision system makes an action plan iteratively so as to decrease the degree of recognition-ambiguity and controls the camera to move to the optimum position. Therefore, our proposed method is able to recognize the object more accurately than conventional methods which use a given input image. Experimental results show the effectiveness of our approach.

Multimodal automatic indexing for broadcast soccer video

In this paper, we propose a novel method for estimating major soccer scenes from cameraworks and players trajectories based on probabilistic inference, and annotating scene indexes to broadcast soccer videos automatically. In our method, we define relations between cameraworks and scenes, and between players trajectories and scenes by conditional probabilities. Moreover defining temporal relations of scenes by transition probabilities, we represent those relations as dynamic bayesian networks (DBNs). And those probabilities are evaluated by learning parameters of the networks. After extracting the cameraworks and the players trajectories, we compute the posterior probability distribution of scenes, and give the computed results to the soccer video as the scene index. Finally, we discuss the extendibility of the proposal indexing technique in the case of adding ball trajectories and audios.

Supporting System for Network Server Monitoring

Networks have complex systems comprised of subsystem components like servers for various kinds of service and DCE (Data Communication Equipment) such as routers and switches. As each system operates constantly, it is hard to know exactly how the whole network operates. The network monitoring system, therefore, is required, in addition to monitoring and checking the network under steady operation, to detect system failures, weigh the situation and investigate into the cause. In this paper, we propose a method that we convert the three kinds of data, data by polling the server externally, data from computer resource of the server, and data from log information of the network server, into an integrated format, and then assemble them as generalized log format, so that we can extract the abnormal events without providing the monitoring system with any data such as key words, in advance. Firstly, we identify the patterns of frequency of the words by text-mining of the log information. Secondly, by means of signal processing of the frequency, periodically appeared patterns are figured out. Lastly, these patterns are associated with each other to presume the cause of the abnormal events. Our proposed method should support network administrators greatly in detecting system failures and investigating into the cause, and also reduce a lot of workload in monitoring log information.

Model-based object recognition using basic probability assignment

A method of object recognition is proposed based on Dempster-Shafer theory (DS theory), which can treat the ambiguity of image data. A model based object recognition technique based on an aspect imaging method will also be proposed. The aspect imaging method recognizes a similar aspect object by comparing an input image with an aspect image of the model database. Our proposed method calculates the degree of similarity based on the DS theory instead of Bayes theory, represents an uncertainty factor of similarity as a basic probability of DS theory, and assigns a basic probability of information to be decided. The experimental results show that the proposed method is both more robust and more reliable.

Model-based object recognition using a hierarchical node structure

Object recognition is discussed using a hierarchical node structure based on an aspect graph approach. Recognition is carried out by comparing hierarchically an input image with aspect images of the model database represented by the three levels, ‘the fine level’, ‘the intermediate level’ and ‘the coarse level’. Matching at the coarse level and intermediate level is used for narrowing down the candidates of the object appearing in the input image, then matching at the fine level is used to discriminate the object.