Guoyu Lu

Where am I in the dark

Indoor localization is one of the key problems in robotics research. Most current localization systems use cellular base stations and Wifi signals, whose localization accuracy is largely dependent on the signal strength and is sensitive to environmental changes. With the development of camera-based technologies, image-based localization may be employed in an indoor environment where the GPS signal is weak. Most of the existing image-based localization systems are based on color images captured by cameras, but this is only feasible in environments with adequate lighting conditions. In this paper, we introduce an image-based localization system based on thermal imaging to make the system independent of light sources, which are especially useful during emergencies such as a sudden power outage in a building. As thermal images are not obtained as easily as color images, we apply active transfer learning to enrich the thermal image classification learning, where normal RGB images are treated as the source domain, and thermal images are the target domain. The application of active transfer learning avoids random target training sample selection and chooses the most informative samples in the learning process. Through the proposed active transfer learning, the query thermal images can be accurately used to indicate the location. Experiments show that our system can be efficiently deployed to perform indoor localization in a dark environment.

Can We Minimize the Influence Due to Gender and Race in Age Estimation

Automatic human age estimation has attracted a great deal of interest in the past few years. Although many advancements have been made by researchers, there are still many challenges: such as age estimation across different image acquisition methods, different expressions, gender and races. The influence due to race and gender seems to be the most common issue, because collecting a large amount of face images with comprehensive racial diversities seems impractical. The performance will degrade when estimating face images of races that differ from the training set. In this work, we present a new scheme to mitigate the influences of race and gender in the problem of age estimation. Our system will contribute a robust solution to solve the problem of age estimation across races and genders. This study is essential for developing a practical age estimation system (with mixture of races and gender.) To evaluate the performance of the proposed algorithm, we run comprehensive experiments on one widely used big database - MORPH-II, which contains more than 55, 000 images. On an average, an improvement of more than 20% has been achieved using the proposed scheme.

Localize Me Anywhere, Anytime: A Multi-task Point-Retrieval Approach

Image-based localization is an essential complement to GPS localization. Current image-based localization methods are based on either 2D-to-3D or 3D-to-2D to find the correspondences, which ignore the real scene geometric attributes. The main contribution of our paper is that we use a 3D model reconstructed by a short video as the query to realize 3D-to-3D localization under a multi-task point retrieval framework. Firstly, the use of a 3D model as the query enables us to efficiently select location candidates. Furthermore, the reconstruction of 3D model exploits the correlations among different images, based on the fact that images captured from different views for SfM share information through matching features. By exploring shared information (matching features) across multiple related tasks (images of the same scene captured from different views), the visual features view-invariance property can be improved in order to get to a higher point retrieval accuracy. More specifically, we use multi-task point retrieval framework to explore the relationship between descriptors and the 3D points, which extracts the discriminant points for more accurate 3D-to-3D correspondences retrieval. We further apply multi-task learning (MTL) retrieval approach on thermal images to prove that our MTL retrieval framework also provides superior performance for the thermal domain. This application is exceptionally helpful to cope with the localization problem in an environment with limited light sources.

Knowing Where I Am: Exploiting Multi-Task Learning for Multi-view Indoor Image-based Localization.

Indoor localization has attracted a large amount of applications in mobile and robotics area, especially in vast and sophisticated environments. Most indoor localization methods are based on cellular base stations and WiFi signals. Such methods require users to carry additional equipment. Localization accuracy is largely based on the beacon distribution. Image-based localization is mainly applied for outdoor environments to overcome the problem caused by weak GPS signals in large building areas. In this paper, we propose to localize images in indoor environments from multi-view settings. We use Structure-from-Motion to reconstruct the 3D environment of our indoor buildings to provide users a clear view of the whole building’s indoor structure. Since the orientation information is also quite essential for indoor navigation, images are localized based on a multi-task learning method, which treats each view direction classification as a task. We perform image retrieval based on the trained multi-task classifiers. Thus the orientation of the image together with the location information is achieved. We assign the pose of the retrieved image to the query image calculated from SfM reconstruction with the use of bundle adjustment to refine the pose estimation.

Memory efficient large-scale image-based localization

Local features have been widely used in the area of image-based localization. However, large-scale 2D-to-3D matching problems still involve massive memory consumption, which is mainly caused by the high dimensionality of the features (e.g. 128 dimensions of SIFT feature). This paper introduces a new method that decreases local features’ high dimensionality for reducing memory capacity and accelerating the descriptor matching process. With this new method, all descriptors are projected into a lower dimensional space through the new learned matrices that are able to reduce the curse of dimensionality in the large scale image-based localization. The low dimensional descriptors are then mapped into a Hamming space for further reducing the memory requirement. This study also proposes an image-based localization pipeline based on the new learned Hamming descriptors. The new learned descriptor and the localization pipeline are applied to two challenging datasets. The experimental results show that the proposed method achieves extraordinary image registration performance compared with the published results from state-of-the-art methods.

Improving Image-Based Localization through Increasing Correct Feature Correspondences

Image-based localization is to provide contextual information based on a query image. Current state-of-the-art methods use 3D Structure-from-Motion reconstruction model to aid in localizing the query image, either by 2D-to-3D matching or by 3D-to-2D matching. By adding camera pose estimation, the system can perform image localization more accurately. However, incorrect feature correspondences between the 2D image and 3D reconstruction remains the main reason for failures in image localization. In our paper, we introduce a new method, which adds features embedding, to reduce the incorrect feature correspondences. We do the query expansion to add correspondences, where the associated 3d point has a high probability to be found in the same camera as the seed set. Using the techniques described, the registration accuracy can be significantly improved. Experiments on several large image datasets have shown our methods to outperform most state-of-the-art methods.

A Fast 3D Indoor-Localization Approach Based on Video Queries

Image-based localization systems are typically used in outdoor environments, as high localization accuracy can be achieved particularly near tall buildings where the GPS signal is weak. Weak GPS signal is also a critical issue for indoor environments. In this paper, we introduce a novel fast 3D Structure-from-Motion (SfM) model based indoor localization approach using videos as queries. Different from an outdoor environment, the captured images in an indoor environment usually contain people, which often leads to an incorrect camera pose estimation. In our approach, we segment out people in the videos by means of an optical flow technique. This way, we filter people in video and complete the captured video for localization. A graph matching based verification is adopted to enhance both the number of images that are registered and the camera pose estimation. Furthermore, we propose an initial correspondence selection method based on local feature ratio test instead of traditional RANSAC which leads to a much faster image registration speed. The extensive experimental results show that our proposed approach has multiple advantages over existing indoor localization systems in terms of accuracy and speed.

Image-based indoor localization system based on 3D SfM model

Indoor localization is an important research topic for both of the robot and signal processing communities. In recent years, image-based localization is also employed in indoor environment for the easy availability of the necessary equipment. After capturing an image and sending it to an image database, the best matching image is returned with the navigation information. By allowing further camera pose estimation, the image-based localization system with the use of Structure-from-Motion reconstruction model can achieve higher accuracy than the methods of searching through a 2D image database. However, this emerging technique is still only on the use of outdoor environment. In this paper, we introduce the 3D SfM model based image-based localization system into the indoor localization task. We capture images of the indoor environment and reconstruct the 3D model. On the localization task, we simply use the images captured by a mobile to match the 3D reconstructed model to localize the image. In this process, we use the visual words and the approximate nearest neighbor methods to accelerate the process of nding the query features correspondences. Within the visual words, we conduct linear search in detecting the correspondences. From the experiments, we nd that the image-based localization method based on 3D SfM model gives good localization result based on both accuracy and speed.

Active domain adaptation with noisy labels for multimedia analysis

Supervised learning methods require sufficient labeled examples to learn a good model for classification or regression. However, available labeled data are insufficient in many applications. Active learning (AL) and domain adaptation (DA) are two strategies to minimize the required amount of labeled data for model training. AL requires the domain expert to label a small number of highly informative examples to facilitate classification, while DA involves tuning the source domain knowledge for classification on the target domain. In this paper, we demonstrate how AL can efficiently minimize the required amount of labeled data for DA. Since the source and target domains usually have different distributions, it is possible that the domain expert may not have sufficient knowledge to answer each query correctly. We exploit our active DA framework to handle incorrect labels provided by domain experts. Experiments with multimedia data demonstrate the efficiency of our proposed framework for active DA with noisy labels.

Structure-from-Motion reconstruction based on weighted Hamming descriptors

We propose a pipelined methods to reduce memory consumption of large-scale Structure-from-Motion reconstruction with the use of unsorted images extracted from photo collection websites. Recent research is able to reconstruct cities based on extracted images from photo collection websites. SIFT feature is used to find the correspondences between two images. For the large-scale reconstruction with unsorted images, the system needs to store all the descriptors and feature points information in memory to search for correspondences. As each SIFT descriptor is a 128 dimensional real-value vector, storing all the descriptors would consume a significant amount of memory. Based on this limitation, we project the high dimensional features into a low-dimensional space using a learned projection matrix. After projection, the distance of the descriptors belonging to the same point in 3D space is decreased; the distance of the descriptors belonging to the different points is increased. Furthermore, we learn a mapping function, which maps the real-value descriptor into binary code. As Hamming descriptors contain only two value options per bit and the length of the descriptor is limited, there are usually multiple descriptors having the same Hamming distance to the query descriptor. In dealing with this problem, we give different weights to each dimension and rank each bit of the Hamming descriptor based on each dimensions discriminant power; this contributes to reduce the ambiguity in matching the descriptors. The experiments show that our method achieves dense reconstruction results with less than 10 percent of the original memory consumption.