Yoonseob Lim

Sparse Contour Representations of Sound

Many signals are naturally described by continuous contours in the time-frequency plane, but standard time-frequency methods disassociate continuous structures into isolated “atoms” of energy. Here we propose a method that represents any discrete time-series as a set of time-frequency contours. The edges of the contours are defined by fixed points of a generalized reassignment algorithm. These edges are linked together by continuity such that each contour represents a single phase-coherent region of the time-frequency plane. By analyzing the signal across many time-scales, an over-complete set of contours is generated, and from this redundant set of shapes the simplest, most parsimonious forms may be selected. The result is an adaptive time-frequency analysis that can emphasize the continuity of long-range structure. The proposed method is demonstrated with a few examples.

Probabilistic sound source localization

Sound localization is one of ways that keep the relation with robot. Time delay of arrival (TDOA) between two microphones through cross-correlation method has been used for sound localization in our robot platforms. However, since cross-correlation values are highly dependent on the upcoming sound signal and acoustic environment, time delay values and localization result are easily perturbed. Probabilistic method for sound source localization is presented in this paper. Markovian process and post filtering steps are applied for calculating time delay value and a reliable sound localization.

Modiolus-Hugging Intracochlear Electrode Array with Shape Memory Alloy

In the cochlear implant system, the distance between spiral ganglia and the electrodes within the volume of the scala tympani cavity significantly affects the efficiency of the electrical stimulation in terms of the threshold current level and spatial selectivity. Because the spiral ganglia are situated inside the modiolus, the central axis of the cochlea, it is desirable that the electrode array hugs the modiolus to minimize the distance between the electrodes and the ganglia. In the present study, we propose a shape-memory-alloy-(SMA-) embedded intracochlear electrode which gives a straight electrode a curved modiolus-hugging shape using the restoration force of the SMA as triggered by resistive heating after insertion into the cochlea. An eight-channel ball-type electrode array is fabricated with an embedded titanium-nickel SMA backbone wire. It is demonstrated that the electrode array changes its shape in a transparent plastic human cochlear model. To verify the safe insertion of the electrode array into the human cochlea, the contact pressures during insertion at the electrode tip and the contact pressures over the electrode length after insertion were calculated using a 3D finite element analysis. The results indicate that the SMA-embedded electrode is functionally and mechanically feasible for clinical applications.

DSP implementation of sound source localization with gain control

This paper describes a DSP implementation of sound source localization algorithm with optimal gain control. This speech processing system is supposed to be mounted on the home service robot or various prototype robot systems which are being developed at KIST (Korea Institute of Science and Technology).

An Approach to Time-Frequency Analysis With Ridges of the Continuous Chirplet Transform

We propose a time-frequency representation based on the ridges of the continuous chirplet transform to identify both fast transients and components with well-defined instantaneous frequency in noisy data. At each chirplet modulation rate, every ridge corresponds to a territory in the time-frequency plane such that the territories form a partition of the time-frequency plane. For many signals containing sparse signal components, ridge length and ridge stability are maximized when the analysis kernel is adapted to the signal content. These properties provide opportunities for enhancing signal in noise and for elementary stream segregation by combining information across multiple analysis chirp rates.

Speaker selection and tracking in a cluttered environment with audio and visual information

Presented in this paper is a data association method using audio and visual data which localizes targets in a cluttered environment and detects who is speaking to a robot. A particle filter is applied to efficiently select the optimal association between the target and the measurements. State variables are composed of target positions and speaking states. To update the speaking state, we first evaluate the incoming sound signal based on cross-correlation and then calculate a likelihood from the audio information. The visual measurement is used to find an optimal association between the target and the observed objects. The number of targets that the robot should interact with is updated from the existence probabilities and associations. Experimental data were collected beforehand and simulated on a computer to verify the performance of the proposed method applied to the speaker selection problem in a cluttered environment. The algorithm was also implemented in a robotic system to demonstrate reliable interactions between the robot and speaking targets.

DSP implementation of probabilistic sound source localization

This paper describes a DSP implementation of probabilistic sound source localization algorithm. Time delay of arrival (TDOA) between two microphones through cross-correlation method has been used for sound localization in our robot platforms. However, since cross-correlation values are highly dependent on the upcoming sound signal and acoustic environment, time delay values and localization results are easily perturbed. Here, probabilistic method for sound source localization is presented. And its DSP implementation is applied to the pan-tilt robot that we developed.

Particle Filter Algorithm for Single Speaker Tracking with Audio-Video Data Fusion

In this paper, a single speaker tracking mechanism with particle filter is presented for a reliable interaction with robot. To have a smooth tracking result, we use not only audio data but also video data. For sound observation, we use Time Delay Of Arrival (TDOA) between two microphones, and in order to expect high reliability in the likelihood of localization, we apply variable sensitivity values according to the angle of arrival for a triangle arrangement of microphones to calculate the likelihood for audio observation. For video observation, we use face detection routine of OpenCV which is widely available through the Internet. Together with audio and video data, we found out that more reliable source tracking is possible if we apply data fusion mechanism at every possible environment on a robot platform than single information.

Intelligent and active system for human-robot interaction based on sound source localization

Today, we can see many kinds of robots anywhere. Robots are inseparable from human-beings. In result, human-robot interaction is getting more important. This paper describes a system for human-robot interaction which is one of the biggest issues recently. We approach the interaction from a localization point of view. The aim of the system for human-robot interaction is that the robot finds a person speaking in an area and goes to the speaker while the robot avoids static obstacles using mapping data obtained before. As well as this, the system acts like human especially when turning its head after someone calls.

A 3D simulation for social human-robot interaction with ontology-based robot knowledge

This paper introduces a preliminary result for social human-robot interaction architecture and its application. System includes human perception module that receives visual data from 3D human simulation and knowledge module that contains various concepts for social interaction situation. The proposed system has been tested for inferring social interaction type based on a simple social interaction scenario.