Truong Q. Nguyen

Adaptive Image Denoising by Mixture Adaptation

We propose an adaptive learning procedure to learn patch-based image priors for image denoising. The new algorithm, called the expectation-maximization (EM) adaptation, takes a generic prior learned from a generic external database and adapts it to the noisy image to generate a specific prior. Different from existing methods that combine internal and external statistics in ad hoc ways, the proposed algorithm is rigorously derived from a Bayesian hyper-prior perspective. There are two contributions of this paper. First, we provide full derivation of the EM adaptation algorithm and demonstrate methods to improve the computational complexity. Second, in the absence of the latent clean image, we show how EM adaptation can be modified based on pre-filtering. The experimental results show that the proposed adaptation algorithm yields consistently better denoising results than the one without adaptation and is superior to several state-of-the-art algorithms.

Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature

Single image super-resolution (SR) is very important in many computer vision systems. However, as a highly ill-posed problem, its performance mainly relies on the prior knowledge. Among these priors, the non-local total variation (NLTV) prior is very popular and has been thoroughly studied in recent years. Nevertheless, technical challenges remain. Because NLTV only exploits a fixed non-shifted target patch in the patch search process, a lack of similar patches is inevitable in some cases. Thus, the non-local similarity cannot be fully characterized, and the effectiveness of NLTV cannot be ensured. Based on the motivation that more accurate non-local similar patches can be found by using shifted target patches, a novel multishifted similar-patch search (MSPS) strategy is proposed. With this strategy, NLTV is extended as a newly proposed super-high-dimensional NLTV (SHNLTV) prior to fully exploit the underlying non-local similarity. However, as SHNLTV is very high-dimensional, applying it directly to SR is very difficult. To solve this problem, a novel statistics-based dimension reduction strategy is proposed and then applied to SHNLTV. Thus, SHNLTV becomes a more computationally effective prior that we call adaptive high-dimensional non-local total variation (AHNLTV). In AHNLTV, a novel joint weight strategy that fully exploits the potential of the MSPS-based non-local similarity is proposed. To further boost the performance of AHNLTV, the adaptive geometric duality (AGD) prior is also incorporated. Finally, an efficient split Bregman iteration-based algorithm is developed to solve the AHNLTV-AGD-driven minimization problem. Extensive experiments validate the proposed method achieves better results than many state-of-the-art SR methods in terms of both objective and subjective qualities.

Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity

Super-resolution (SR) from a single image plays an important role in many computer vision applications. It aims to estimate a high-resolution (HR) image from an input low- resolution (LR) image. To ensure a reliable and robust estimation of the HR image, we propose a novel single image SR method that exploits both the local geometric duality (GD) and the non-local similarity of images. The main principle is to formulate these two typically existing features of images as effective priors to constrain the super-resolved results. In consideration of this principle, the robust soft-decision interpolation method is generalized as an outstanding adaptive GD (AGD)-based local prior. To adaptively design weights for the AGD prior, a local non-smoothness detection method and a directional standard-deviation-based weights selection method are proposed. After that, the AGD prior is combined with a variational-framework-based non-local prior. Furthermore, the proposed algorithm is speeded up by a fast GD matrices construction method, which primarily relies on the selective pixel processing. The extensive experimental results verify the effectiveness of the proposed method compared with several state-of-the-art SR algorithms.

Context Matters: Refining Object Detection in Video with Recurrent Neural Networks.

Given the vast amounts of video available online, and recent breakthroughs in object detection with static images, object detection in video offers a promising new frontier. However, motion blur and compression artifacts cause substantial frame-level variability, even in videos that appear smooth to the eye. Additionally, video datasets tend to have sparsely annotated frames. We present a new framework for improving object detection in videos that captures temporal context and encourages consistency of predictions. First, we train a pseudo-labeler, that is, a domain-adapted convolutional neural network for object detection. The pseudo-labeler is first trained individually on the subset of labeled frames, and then subsequently applied to all frames. Then we train a recurrent neural network that takes as input sequences of pseudo-labeled frames and optimizes an objective that encourages both accuracy on the target frame and consistency across consecutive frames. The approach incorporates strong supervision of target frames, weak-supervision on context frames, and regularization via a smoothness penalty. Our approach achieves mean Average Precision (mAP) of 68.73, an improvement of 7.1 over the strongest image-based baselines for the Youtube-Video Objects dataset. Our experiments demonstrate that neighboring frames can provide valuable information, even absent labels.

Robust Bayesian method for simultaneous block sparse signal recovery with applications to face recognition

In this paper, we present a novel Bayesian approach to recover simultaneously block sparse signals in the presence of outliers. The key advantage of our proposed method is the ability to handle non-stationary outliers, i.e. outliers which have time varying support. We validate our approach with empirical results showing the superiority of the proposed method over competing approaches in synthetic data experiments as well as the multiple measurement face recognition problem.

Realistic surface geometry reconstruction using a hand-held RGB-D camera

In this paper, we have proposed a novel approach for the reconstruction of real object/scene with realistic surface geometry using a hand-held, low-cost, RGB-D camera. To achieve accurate reconstruction, the most important issues to consider are the quality of the geometry information provided and the global alignment method between frames. In our approach, new surface geometry refinement is used to recover finer scale surface geometry from depth data by utilizing high-quality RGB images. In addition, a weighted multi-scale iterative closest point method is exploited to align each scan to the global model accurately. We show the effectiveness of the proposed surface geometry refinement method by comparing it with other depth refinement methods. We also show both the qualitative and quantitative results of reconstructed models by comparing it with other reconstruction methods.

Analysis of Crosstalk in 3D Circularly Polarized LCDs Depending on the Vertical Viewing Location

Crosstalk in circularly polarized (CP) liquid crystal display (LCD) with polarized glasses (passive 3D glasses) is mainly caused by two factors: the polarizing system including wave retarders and the vertical misalignment (VM) of light between the LC module and the patterned retarder. We show that the latter, which is highly dependent on the vertical viewing location, is a much more significant factor of crosstalk in CP LCD than the former. There are three contributions in this paper. Initially, a display model for CP LCD, which accurately characterizes VM, is proposed. A novel display calibration method for the VM characterization that only requires pictures of the screen taken at four viewing locations. In addition, we prove that the VM-based crosstalk cannot be efficiently reduced by either preprocessing the input images or optimizing the polarizing system. Furthermore, we derive the analytic solution for the viewing zone, where the entire screen does not have the VM-based crosstalk.

Long-Range Motion Trajectories Extraction of Articulated Human Using Mesh Evolution

This letter presents a novel approach to extract reliable dense and long-range motion trajectories of articulated human in a video sequence. Compared with existing approaches that emphasize temporal consistency of each tracked point, we also consider the spatial structure of tracked points on the articulated human. We treat points as a set of vertices, and build a triangle mesh to join them in image space. The problem of extracting long-range motion trajectories is changed to the issue of consistency of mesh evolution over time. First, self-occlusion is detected by a novel mesh-based method and an adaptive motion estimation method is proposed to initialize mesh between successive frames. Furthermore, we propose an iterative algorithm to efficiently adjust vertices of mesh for a physically plausible deformation, which can meet the local rigidity of mesh and silhouette constraints. Finally, we compare the proposed method with the state-of-the-art methods on a set of challenging sequences. Evaluations demonstrate that our method achieves favorable performance in terms of both accuracy and integrity of extracted trajectories.

An instrumented glove for improving spasticity assessment

An instrumented glove worn by caregivers that can augment subjective assessments of spasticity with an objective, repeatable metric with reduced inter- and intra-rater variability and improved resolution over existing standards is highly desirable. We present the design and preliminary results of such a system using commercial, off the shelf (COTS) components. The glove includes spatially-resolved, force-dependent resistive sensor elements and an inertial measurement unit. We developed a mock patient that is equipped with a mechanism to adjust the arm stiffness, a load-cell and a potentiometer to measure the work done to move the arm. The mock patient provides ground truth to validate the proposed concept. We report the power measured by the sensors in the mock patient to move the arm and the power estimated by the glove in moving the arm and show Pearson correlation coefficient of 0.64. We observe that raw sensor data and instrumentation errors contributed to significant outliers in these experiments. Initial assessments by clinician show promise of the proposed approach to improve spasticity assessment. Future work includes improvements to instrumentation and further clinical evaluations.

Moving objects detection using classifying object proposals for driver assistance system

We present a new framework for driver assistance system, detecting moving objects in the street scene. Our algorithm supports a wide range of objects including vehicles, cyclists, pedestrian etc. Based on candidate bounding boxes detected by object proposals, our classifier only responds to the objects truly moving, which is more practical for real applications. Using unified features of color, structure and motion information, our system runs in real time with 66% detection rate in CamVid dataset. In addition, our method can be implemented efficiently with pipelined function blocks.