Antony Lam

Robust Heart Rate Measurement from Video Using Select Random Patches

The ability to remotely measure heart rate from videos without requiring any special setup is beneficial to many applications. In recent years, a number of papers on heart rate (HR) measurement from videos have been proposed. However, these methods typically require the human subject to be stationary and for the illumination to be controlled. For methods that do take into account motion and illumination changes, strong assumptions are still made about the environment (e.g. background can be used for illumination rectification). In this paper, we propose an HR measurement method that is robust to motion, illumination changes, and does not require use of an environments background. We present conditions under which cardiac activity extraction from local regions of the face can be treated as a linear Blind Source Separation problem and propose a simple but robust algorithm for selecting good local regions. The independent HR estimates from multiple local regions are then combined in a majority voting scheme that robustly recovers the HR. We validate our algorithm on a large database of challenging videos.

Spectral Modeling and Relighting of Reflective-Fluorescent Scenes

Hyper spectral reflectance data allows for highly accurate spectral relighting under arbitrary illumination, which is invaluable to applications ranging from archiving cultural e-heritage to consumer product design. Past methods for capturing the spectral reflectance of scenes has proven successful in relighting but they all share a common assumption. All the methods do not consider the effects of fluorescence despite fluorescence being found in many everyday objects. In this paper, we describe the very different ways that reflectance and fluorescence interact with illuminants and show the need to explicitly consider fluorescence in the relighting problem. We then propose a robust method based on well established theories of reflectance and fluorescence for imaging each of these components. Finally, we show that we can relight real scenes of reflective-fluorescent surfaces with much higher accuracy in comparison to only considering the reflective component.

Interactive event search through transfer learning

Activity videos are widespread on the Internet but current video search is limited to text tags due to limitations in recognition systems. One of the main reasons for this limitation is the wide variety of activities users could query. Thus codifying knowledge for all queries becomes problematic. Relevance Feedback (RF) is a retrieval framework that addresses this issue via interactive feedback with the user during the search session. An added benefit is that RF can also learn the subjective component of a users search preferences. However for good retrieval performance, RF may require a large amount of user feedback for activity search. We address this issue by introducing Transfer Learning (TL) into RF. With TL, we can use auxiliary data from known classification problems different from the users target query to decrease the needed amount of user feedback. We address key issues in integrating RF and TL and demonstrate improved performance on the challenging YouTube Action Dataset

Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Restoration

Hyperspectral imaging is beneficial in a diverse range of applications from diagnostic medicine, to agriculture, to surveillance to name a few. However, hyperspectral images often suffer from degradation such as noise and low resolution. In this paper, we propose an effective model for hyperspectral image (HSI) restoration, specifically image denoising and super-resolution. Our model considers three underlying characteristics of HSIs: sparsity across the spatial-spectral domain, high correlation across spectra, and non-local self-similarity over space. We first exploit high correlation across spectra and non-local self-similarity over space in the degraded HSI to learn an adaptive spatial-spectral dictionary. Then, we employ the local and non-local sparsity of the HSI under the learned spatial-spectral dictionary to design an HSI restoration model, which can be effectively solved by an iterative numerical algorithm with parameters that are adaptively adjusted for different clusters and different noise levels. In experiments on HSI denoising, we show that the proposed method outperforms many state-of-the-art methods under several comprehensive quantitative assessments. We also show that our method performs well on HSI super-resolution.

Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Denoising

Hyperspectral imaging is beneficial in a diverse range of applications from diagnostic medicine, to agriculture, to surveillance to name a few. However, hyperspectral images often times suffer from degradation due to the limited light, which introduces noise into the imaging process. In this paper, we propose an effective model for hyperspectral image (HSI) denoising that considers underlying characteristics of HSIs: sparsity across the spatial-spectral domain, high correlation across spectra, and non-local self-similarity over space. We first exploit high correlation across spectra and non-local self-similarity over space in the noisy HSI to learn an adaptive spatial-spectral dictionary. Then, we employ the local and non-local sparsity of the HSI under the learned spatial-spectral dictionary to design an HSI denoising model, which can be effectively solved by an iterative numerical algorithm with parameters that are adaptively adjusted for different clusters and different noise levels. Experimental results on HSI denoising show that the proposed method can provide substantial improvements over the current state-of-the-art HSI denoising methods in terms of both objective metric and subjective visual quality.

Toward Museum Guide Robots Proactively Initiating Interaction with Humans

This paper describes current work toward the design of a guide robot system. We present a method to recognize peoples interest and intention from their walking trajectories in indoor environments, which enables a service robot to proactively approach people to provide services to them. We conducted observational experiments in a museum as a target test environment where participants were asked to visit that museum. From these experiments, we have found mainly three kinds of walking trajectory patterns of the participants inside the museum that depend on their interest in the exhibits. Based on these findings, we developed a method to identify participants that may need guidance.We confirm the effectiveness of our method by experiments.

Interreflection Removal Using Fluorescence

Interreflections exhibit a number of challenges for existing shape-from-intensity methods that only assume a direct lighting model. Removing the interreflections from scene observations is of broad interest since it enhances the accuracy of those methods. In this paper, we propose a method for removing interreflections from a single image using fluorescence. From a bispectral observation of reflective and fluorescent components recorded in distinct color channels, our method separates direct lighting from interreflections. Experimental results demonstrate the effectiveness of the proposed method on complex and dynamic scenes. In addition, we show how our method improves an existing photometric stereo method in shape recovery.

Spectral Imaging Using Basis Lights.

Antony Lam1 http://research.nii.ac.jp/~antony Art Subpa-Asa2 art@set.or.th Imari Sato1 http://research.nii.ac.jp/~imarik Takahiro Okabe3 http://www.pluto.ai.kyutech.ac.jp/~okabe Yoichi Sato4 http://www.hci.iis.u-tokyo.ac.jp/~ysato 1 National Institute of Informatics Tokyo, Japan 2 The Stock Exchange of Thailand Bangkok, Thailand 3 Kyushu Institute of Technology Fukuoka, Japan 4 The University of Tokyo Tokyo, Japan

Face recognition and alignment using support vector machines

Face recognition in the presence of pose changes remains a largely unsolved problem. Severe pose changes, resulting in dramatically different appearances, is one of the main difficulties. We present a support vector machine (SVM) based system that learns the relations between corresponding local regions of the face in different poses as well as a simple SVM based system for automatic alignment of faces in differing poses. We then present experimental results from multiple random splits of the CMU PIE Database to verify the strength of our approach.

Separating Reflective and Fluorescent Components Using High Frequency Illumination in the Spectral Domain

Hyperspectral imaging is beneficial to many applications but most traditional methods do not consider fluorescent effects which are present in everyday items ranging from paper to even our food. Furthermore, everyday fluorescent items exhibit a mix of reflection and fluorescence so proper separation of these components is necessary for analyzing them. In recent years, effective imaging methods have been proposed but most require capturing the scene under multiple illuminants. In this paper, we demonstrate efficient separation and recovery of reflectance and fluorescence emission spectra through the use of two high frequency illuminations in the spectral domain. With the obtained fluorescence emission spectra from our high frequency illuminants, we then describe how to estimate the fluorescence absorption spectrum of a material given its emission spectrum. In addition, we provide an in depth analysis of our method and also show that filters can be used in conjunction with standard light sources to generate the required high frequency illuminants. We also test our method under ambient light and demonstrate an application of our method to synthetic relighting of real scenes.