Kyota Higa

Local Feature Based Multiple Object Instance Identification Using Scale and Rotation Invariant Implicit Shape Model

In this paper, we propose a Scale and Rotation Invariant Implicit Shape Model (SRIISM), and develop a local feature matching based system using the model to accurately locate and identify large numbers of object instances in an image. Due to repeated instances and cluttered background, conventional methods for multiple object instance identification suffer from poor identification results. In the proposed SRIISM, we model the joint distribution of object centers, scale, and orientation computed from local feature matches in Hough voting, which is not only invariant to scale changes and rotation of objects, but also robust to false feature matches. In the multiple object instance identification system using SRIISM, we apply a fast 4D bin search method in Hough space with complexity \(O(n)\), where \(n\) is the number of feature matches, in order to segment and locate each instance. Furthermore, we apply maximum likelihood estimation (MLE) for accurate object pose detection. In the evaluation, we created datasets simulating various industrial applications such as pick-and-place and inventory management. Experiment results on the datasets show that our method outperforms conventional methods in both accuracy (5 %–30 % gain) and speed (2x speed up).

Multiple object identification using grid voting of object center estimated from keypoint matches

This paper proposes a method to detect and identify multiple objects in an image using grid voting of object center positions estimated from local descriptor keypoint matches. For each keypoint match, the proposed method estimates the object center position using scale and orientation associated with the keypoints. Then, it casts a vote for an image grid where the estimated object center is located. For the grids with high number of votes, geometric verification of the keypoint matches is carried out to accurately localize multiple objects in the image. Since the computational complexity of the grid voting is O(n), where n is the number of estimated object centers, the proposed method runs faster than a conventional method using mean shift clustering with O(n2) complexity. Experimental results using images with 52 objects show that the proposed method reduces the computational time by approximately 60% compared to the conventional method, while identification accuracy is comparable. With the reduced computational complexity, industrial applications such as an efficient inventory management in retail using images are enabled in practical computational time.

Onomatopoeia Expressions for Intuitive Understanding of Remote Office Situation

This paper proposes a system for intuitive understanding of remote office situation using onomatopoeia expressions. Onomatopoeia (imitative word) is a word that imitates sound or movement. This system detects office events such as “conversation” or “human movement” from audio and video signals of remote office, and converts them to onomatopoeia texts. Onomatopoeia texts are superimposed on the office image, and sent to the remote office. By using onomatopoeia expressions, the office event such as “conversation” and “human movement” can be compactly expressed as just one word. Thus, people can instantly understand remote office situation without watching the video for a while. Subjective experimental results show that easiness of event understanding is statistically significantly improved by the onomatopoeia expressions compared to the video at 99% confidence level. We have developed a prototype system with two cameras and eight microphones, and then have exhibited it at ultra-realistic communications forum in Japan. In the exhibition, the concept of this system was favorably accepted by visitors.

Music Part Segmentation in Music TV Programs Based on Chroma Vector Analysis

This paper presents a music part detection method incorporating chroma vector analysis for use with music TV programs. Results show that envelopes of chroma components of music signals tend to have horizontal (i.e. temporal) correlation in time-frequency representation because music signals have a periodic chord sequences. Based on this fact, we analyze time series of chroma components and attempt to segment music parts in music TV programs from other parts. Experimental results show an F-measure of 0.78, which is better than that obtained using the previous method.