Jiandong Tian

Tricolor Attenuation Model for Shadow Detection

Shadows, the common phenomena in most outdoor scenes, bring many problems in image processing and computer vision. In this paper, we present a novel method focusing on extracting shadows from a single outdoor image. The proposed tricolor attenuation model (TAM) that describe the attenuation relationship between shadow and its nonshadow background is derived based on image formation theory. The parameters of the TAM are fixed by using the spectral power distribution (SPD) of daylight and skylight, which are estimated according to Plancks blackbody irradiance law. Based on the TAM, a multistep shadow detection algorithm is proposed to extract shadows. Compared with previous methods, the algorithm can be applied to process single images gotten in real complex scenes without prior knowledge. The experimental results validate the performance of the model.

RGBD Salient Object Detection via Deep Fusion

Numerous efforts have been made to design various low-level saliency cues for RGBD saliency detection, such as color and depth contrast features as well as background and color compactness priors. However, how these low-level saliency cues interact with each other and how they can be effectively incorporated to generate a master saliency map remain challenging problems. In this paper, we design a new convolutional neural network (CNN) to automatically learn the interaction mechanism for RGBD salient object detection. In contrast to existing works, in which raw image pixels are fed directly to the CNN, the proposed method takes advantage of the knowledge obtained in traditional saliency detection by adopting various flexible and interpretable saliency feature vectors as inputs. This guides the CNN to learn a combination of existing features to predict saliency more effectively, which presents a less complex problem than operating on the pixels directly. We then integrate a superpixel-based Laplacian propagation framework with the trained CNN to extract a spatially consistent saliency map by exploiting the intrinsic structure of the input image. Extensive quantitative and qualitative experimental evaluations on three data sets demonstrate that the proposed method consistently outperforms the state-of-the-art methods.

Outdoor shadow detection by combining tricolor attenuation and intensity

Shadow detection is of broad interest in computer vision. In this article, a new shadow detection method for single color images in outdoor scenes is proposed. Shadows attenuate pixel intensity, and the degrees of attenuation are different in the three RGB color channels. Previously, we proposed the Tricolor Attenuation Model (TAM) that describes the attenuation relationship between shadows and their non-shadow backgrounds in the three color channels. TAM can provide strong information on shadow detection; however, our previous study needs a rough segmentation as the pre-processing step and requires four thresholds. These shortcomings can be overcome by adding intensity information. This article addresses the problem of how to combine TAM and intensity and meanwhile to obtain a threshold for shadow segmentation. Simple and complicated shadow images are used to test the proposed method. The experimental results and comparisons validate its effectiveness.

New spectrum ratio properties and features for shadow detection

Successfully detecting shadows in still images is challenging yet has wide applications. Shadow properties and features are very important for shadow detection and processing. The aim of this work is to find some new physical properties of shadows and use them as shadow features to design an effective shadow detection method for outdoor color images. We observe that although the spectral power distribution (SPD) of daylight and that of skylight are quite different, in each channel, the spectrum ratio of the point-wise product of daylight SPD with sRGB color matching functions (CMFs) to the point-wise product of skylight SPD with sRGB CMFs roughly approximates a constant. This further leads to that the ratios of linear sRGB pixel values of surfaces illuminated by daylight (in non-shadow regions) to those illuminated by skylight (in shadow regions) equal to a constant in each channel. Following this observation, we calculated the spectrum ratios under various Sun angles and further found out four new shadow properties. With these properties as shadow features, we developed a simple shadow detection method to quickly locate shadows in single still images. In our method, we classify an edge as a shadow or non-shadow edge by verifying whether the pixel values on both sides of the Canny edges satisfy the three shadow verification criteria derived from the shadow properties. Extensive experiments and comparison show that our method outperforms state-of-the-art shadow detection methods. HighlightsWe found the ratios of regions lit by daylight vs. by skylight equal to a constant.We calculated spectrum ratios and found out four new shadow properties.Following the new shadow properties, we proposed a simple shadow detection method.We conducted most extensive experiments and comparison.We also tested our method on images without shadows.

Linearity of each channel pixel values from a surface in and out of shadows and its applications

Shadows, the common phenomena in most outdoor scenes, are illuminated by diffuse skylight whereas shaded from direct sunlight. Generally shadows take place in sunny weather when the spectral power distributions (SPD) of sunlight, skylight, and daylight show strong regularity: they principally vary with sun angles. In this paper, we first deduce that the pixel values of a surface illuminated by skylight (in shadow region) and by daylight (in non-shadow region) have a linear relationship, and the linearity is independent of surface reflectance and holds in each color channel. We then use six simulated images that contain 1995 surfaces and two real captured images to test the linearity. The results validate the linearity. Based on the deduced linear relationship, we develop three shadow processing applications include intrinsic image deriving, shadow verification, and shadow removal. The results of the applications demonstrate that the linear relationship have practical values.

Single image dehazing by latent region-segmentation based transmission estimation and weighted L 1-norm regularisation

Image dehazing is a useful technique which can eliminate the bad effect of haze on images and enhance the performances of image/video processing algorithms in the hazy weather. In this study, a single image dehazing method is proposed. The authors estimate the initial transmission properly based on latent region-segmentation and refine the estimated initial transmission by an objective function with a novel weighted L 1-norm regularisation term. The half-quadratic splitting minimisation method is employed to solve this optimisation problem. They also define an evaluation function to estimate the reliable global atmospheric light. With the refined transmission map and atmospheric light they recover the haze-free image by the haze imaging model. The authors’ method is compared with three state-of-the-art methods and is also validated by two image quality assessment methods. The comparative experimental results and evaluations demonstrate that their method can recover comparable and even better results with clear details, low contrast loss and high contrast in most cases.

Pixel-wise orthogonal decomposition for color illumination invariant and shadow-free image

In this paper, we propose a novel, effective and fast method to obtain a color illumination invariant and shadow-free image from a single outdoor image. Different from state-of-the-art methods for shadow-free image that either need shadow detection or statistical learning, we set up a linear equation set for each pixel value vector based on physically-based shadow invariants, deduce a pixel-wise orthogonal decomposition for its solutions, and then get an illumination invariant vector for each pixel value vector on an image. The illumination invariant vector is the unique particular solution of the linear equation set, which is orthogonal to its free solutions. With this illumination invariant vector and Lab color space, we propose an algorithm to generate a shadow-free image which well preserves the texture and color information of the original image. A series of experiments on a diverse set of outdoor images and the comparisons with the state-of-the-art methods validate our method.

DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal

Shadow removal is a challenging task as it requires the detection/annotation of shadows as well as semantic understanding of the scene. In this paper, we propose an automatic and end-to-end deep neural network (DeshadowNet) to tackle these problems in a unified manner. DeshadowNet is designed with a multi-context architecture, where the output shadow matte is predicted by embedding information from three different perspectives. The first global network extracts shadow features from a global view. Two levels of features are derived from the global network and transferred to two parallel networks. While one extracts the appearance of the input image, the other one involves semantic understanding for final prediction. These two complementary networks generate multi-context features to obtain the shadow matte with fine local details. To evaluate the performance of the proposed method, we construct the first large scale benchmark with 3088 image pairs. Extensive experiments on two publicly available benchmarks and our large-scale benchmark show that the proposed method performs favorably against several state-of-the-art methods.

A simple and efficient approach to barcode localization

In this paper, we propose a simple and efficient approach to localizing the barcode regions in an image. We first apply the multichannel Gabor filtering technique to extract eight directional texture features. We then apply a randomized hierarchical search strategy to quickly find a sufficient number of pairs of line segments, which have high frequency and high similarity measures. We finally employ the histogram analysis technique on the start and end points of each qualified pair of line segments to localize the barcode regions. Our extensive experimental results show that the proposed scheme outperforms the two peer systems and can successfully localize the barcode regions in an image with a precision of 96% and a recall of 86%. In addition, the proposed system can be easily ported to a cell phone to improve the ShopTalk system to aid the blind to successfully retrieve common grocery products.

Video Desnowing and Deraining Based on Matrix Decomposition

The existing snow/rain removal methods often fail for heavy snow/rain and dynamic scene. One reason for the failure is due to the assumption that all the snowflakes/rain streaks are sparse in snow/rain scenes. The other is that the existing methods often can not differentiate moving objects and snowflakes/rain streaks. In this paper, we propose a model based on matrix decomposition for video desnowing and deraining to solve the problems mentioned above. We divide snowflakes/rain streaks into two categories: sparse ones and dense ones. With background fluctuations and optical flow information, the detection of moving objects and sparse snowflakes/rain streaks is formulated as a multi-label Markov Random Fields (MRFs). As for dense snowflakes/rain streaks, they are considered to obey Gaussian distribution. The snowflakes/rain streaks, including sparse ones and dense ones, in scene backgrounds are removed by low-rank representation of the backgrounds. Meanwhile, a group sparsity term in our model is designed to filter snow/rain pixels within the moving objects. Experimental results show that our proposed model performs better than the state-of-the-art methods for snow and rain removal.