Hidehisa Nagano

BM25 With Exponential IDF for Instance Search

This paper deals with a novel concept of an exponential IDF in the BM25 formulation and compares the search accuracy with that of the BM25 with the original IDF in a content-based video retrieval (CBVR) task. Our video retrieval method is based on a bag of keypoints (local visual features) and the exponential IDF estimates the keypoint importance weights more accurately than the original IDF. The exponential IDF is capable of suppressing the keypoints from frequently occurring background objects in videos, and we found that this effect is essential for achieving improved search accuracy in CBVR. Our proposed method is especially designed to tackle instance video search, one of the CBVR tasks, and we demonstrate its effectiveness in significantly enhancing the instance search accuracy using the TRECVID2012 video retrieval dataset.

Robust Search Methods for Music Signals Based on Simple Representation

Signal similarity search is an important technique for music information retrieval. A basic task is finding identical signal segments on unlabeled music-signal archives, given a short music signal fragment as a query. In such a task, the search must be fast and sufficiently robust against possible signal fluctuations due to noise and distortions. In this special session paper, we describe a search method designed to cope with additive interfering sounds by spectral partitioning. Then, we introduce another method designed to be robust under multiplicative noise or distortion based on binary area representation.

A fast search algorithm for background music signals based on the search for numerous small signal components

This paper proposes a method for detecting and locating a known music signal in a long audio stream. Unlike existing methods, ours assumes that the music is used as background music (BGM) and overlapped by another sound such as speech and that the interfering sound is typically louder than the target music. The proposed method is based on time-series active search, which is a quick signal search method reported earlier. To realize the BGM search, however, a novel extension is introduced. That is, the music signal is firstly decomposed into a number of small time-frequency regions, and the search is carried out for each of those components. The results of the search are then integrated based on a voting scheme to find the target music locations. Experiments show that accurate search is possible when SNR is -5 dB and that the search completes in about 8 s for a 30-m stored signal.

Visualizing Video Sounds With Sound Word Animation to Enrich User Experience

Sound information in videos plays an important role in shaping the user experience. When sound is not accessible in videos, text captions can provide sound information. However, conventional text captions are not very expressive for nonverbal sounds because they are designed to visualize speech sounds. Here, we present a framework to automatically transform nonverbal video sounds into animated sound words and position them near the sound source objects in the video for visualization. This provides natural visual representation of nonverbal sounds with rich information about the sound category and dynamics. To evaluate how the animated sound words generated by our framework affect the user experience, we implemented an experimental system and conducted a user study involving over 300 participants from an online crowdsourcing service. The results of the user study show that the animated sound words can effectively and naturally visualize the dynamics of sound while clarifying the position of the sound source as well as contribute to making video-watching more enjoyable and increasing the visual impact of videos.

Unscented statistical linearization and robustified Kalman filter for nonlinear systems with parameter uncertainties

Kalman filter (KF) design is well established for perfectly known linear system and observation models. Real-world phenomena, however, have significant associated uncertainties, and the tuning of the KF is not so straightforward for tackling them. In this paper, we present a method of designing a robust filter for nonlinear systems with model parameter uncertainties. The uncertainties are imposed on the temporal changes in system parameters, which corresponds to the conditions that most real-world problems exhibit. Our proposed filter is based on a robustified KF, which assumes Gaussian distributed states and is designed to be robust to significant changes in the system parameters. The uncertain nonlinear systems are handled by using the linearized approximation models to guarantee the Gaussianity of states. This is achieved by using a statistical linearization in conjunction with unscented transformations and we thus call the linearization technique unscented statistical linearization (USL). The USL is employed for the prediction step of nonlinearly transformed state and the subsequent filtering is executed by using the robustified KF to make the filter robust to upcoming observations. We call our proposed filter for the uncertain nonlinear systems a robustified nonlinear KF (robustified NKF) and confirm the effectiveness by experiments using artificially generated data.

A fast audio search method based on skipping irrelevant signals by similarity upper-bound calculation

In this paper, we describe an approach to accelerate fingerprint techniques by skipping the search for irrelevant sections of the signal and demonstrate its application to the divide and locate (DAL) audio fingerprint method. The search result for the applied method, DAL3, is the same as that of DAL mathematically. Experimental results show that DAL3 can reduce the computational cost of DAL to approximately 25% for the task of music signal retrieval.

Visualizing video sounds with sound word animation

Text captions are important means to provide sound information in videos when the sound is not accessible. However, conventional text captions are far less expressive for non-verbal sounds since they are designed to visualize speech sound. To address this problem, we propose a method for automatically transforming non-verbal video sounds to animated sound words, and positioning them near the sound source objects in the video for visualization. This provides natural visual representation of non-verbal sounds with rich information about the sound category and dynamics. We conducted a user study with over 300 participants using an online crowdsourcing service. The results showed that animated sound words could not only effectively and naturally visualize the dynamics of sound while clarify the position of the sound source, but also contribute to making video watching more enjoyable and increasing the visual impact of the video.

Monte Carlo filter particle filter

We propose a new realization method of the sequential importance sampling (SIS) algorithm to derive a new particle filter. The new filter constructs the importance distribution by the Monte Carlo filter (MCF) using sub-particles, therefore, its non-Gaussianity nature can be adequately considered while the other type of particle filter such as unscented Kalman filter particle filter (UKF-PF) assumes a Gaussianity on the importance distribution. Since the state estimation accuracy of the SIS algorithm theoretically improves as the estimated importance distribution becomes closer to the true posterior probability density function of state, the new filter is expected to outperform the existing, state-of-the-art particle filters. We call the new filter Monte Carlo filter particle filter (MCF-PF) and confirm its effectiveness through the numerical simulations.

Video Content Detection with Single Frame Level Accuracy Using Dynamic Thresholding Technique

This paper proposes a video retrieval method that detects frame sections that correspond to shots in a query (video segment) with single frame level accuracy. The method adopts the coarse-to-fine strategy to decrease the processing time and the memory consumption, dynamic threshold with initial ranges for small segments is proposed to detect the exact beginning and end of each corresponding frame section to each shot in a query. Experiments on real videos show that our method can achieve accurate video detection with exact frame position while reducing processing time and memory consumption.

Robustifying Kalman filter to rapidly adapt to significant changes in system model parameters of state-space models

A Kalman filter (KF) is state-of-the-art for estimating states of linear-Gaussian state-space models. The KF selects an expectation of a posterior probability density function of state and the expectation is an analytic solution for minimizing the square estimation error. The estimate of KF is therefore optimal, however, simultaneously inherits the problem of the variance/covariance matrix of the estimation error becoming too small as the filtering proceeds to some extent. In this paper, we tackle this problem by deliberately making a KF suboptimal in case of detecting a significantly large prediction error, which implies that the state estimate at this moment is no longer an expectation of the posterior probability density function. By this suboptimization, the resulting square estimation error becomes larger than that of the KF and we make the KF more responsive to upcoming observations. We call the new filter a robustified Kalman filter and demonstrate the revived ability to adapt to significant changes in system model parameters in a series of numerical experiments.