Tomas F. Yago Vicente

Leave-One-Out Kernel Optimization for Shadow Detection

The objective of this work is to detect shadows in images. We pose this as the problem of labeling image regions, where each region corresponds to a group of superpixels. To predict the label of each region, we train a kernel Least-Squares Support Vector Machine (LSSVM) for separating shadow and non-shadow regions. The parameters of the kernel and the classifier are jointly learned to minimize the leave-one-out cross validation error. Optimizing the leave-one-out cross validation error is typically difficult, but it can be done efficiently in our framework. Experiments on two challenging shadow datasets, UCF and UIUC, show that our region classifier outperforms more complex methods. We further enhance the performance of the region classifier by embedding it in a Markov Random Field (MRF) framework and adding pairwise contextual cues. This leads to a method that outperforms the state-of-the-art for shadow detection. In addition we propose a new method for shadow removal based on region relighting. For each shadow region we use a trained classifier to identify a neighboring lit region of the same material. Given a pair of lit-shadow regions we perform a region relighting transformation based on histogram matching of luminance values between the shadow region and the lit region. Once a shadow is detected, we demonstrate that our shadow removal approach produces results that outperform the state of the art by evaluating our method using a publicly available benchmark dataset.

Large-Scale Training of Shadow Detectors with Noisily-Annotated Shadow Examples

This paper introduces training of shadow detectors under the large-scale dataset paradigm. This was previously impossible due to the high cost of precise shadow annotation. Instead, we advocate the use of quickly but imperfectly labeled images. Our novel label recovery method automatically corrects a portion of the erroneous annotations such that the trained classifiers perform at state-of-the-art level. We apply our method to improve the accuracy of the labels of a new dataset that is 20 times larger than existing datasets and contains a large variety of scenes and image types. Naturally, such a large dataset is appropriate for training deep learning methods. Thus, we propose a semantic-aware patch level Convolutional Neural Network architecture that efficiently trains on patch level shadow examples while incorporating image level semantic information. This means that the detected shadow patches are refined based on image semantics. Our proposed pipeline can be a useful baseline for future advances in shadow detection.

Single Image Shadow Detection Using Multiple Cues in a Supermodular MRF.

We propose a single region shadow classifier based on a multikernel SVM. Our multikernel model is a linear combination of χ2 and Earth Mover’s Distance(EMD)[5] kernels that operate on texture and color histograms disjointly. This single region classifier already outperforms the more complex state of art methods, without performing MRF/CRF optimization. The local appearance of a single region is often ambiguous. Even for a human observer it can be hard to discern if a region is in shadow or not, without considering its context. Hence, it is sensible to look beyond the boundaries of a single region to decide its shadow label [1] [6]. In contrast to previous work we strive to use such contextual information sparingly. For MRF optimization reasons we prefer that most of the work is handled by the single region classifier (unary MRF potentials), with sparse pairwise connections that smooth the label changes across regions. We build on the work of [1] to propose our own improved pairwise classifiers but constrained to adjacent regions: for pairs of regions sharing the same material and same illumination condition, and for same material pairs viewed under different illumination (first lit, second in shadow). We also propose a shadow boundary classifier. Since shadow boundaries often overlap with reflectance changes confounding the effects of the illumination change, our classifier focuses on boundaries of shadows cast over surfaces with the same underlying material. We integrate our single region classifier, our pairwise classifiers, and our boundary classifier using an MRF. Confident positive predictions of the pairwise and boundary classifiers are used to define the pairwise potentials and the graph topology of the MRF. The unary potentials are defined based on the single region classifier. We want to minimize the following functional:

Leave-One-Out Kernel Optimization for Shadow Detection and Removal

The objective of this work is to detect shadows in images. We pose this as the problem of labeling image regions, where each region corresponds to a group of superpixels. To predict the label of each region, we train a kernel Least-Squares SVM for separating shadow and non-shadow regions. The parameters of the kernel and the classifier are jointly learned to minimize the leave-one-out cross validation error. Optimizing the leave-one-out cross validation error is typically difficult, but it can be done efficiently in our framework. Experiments on two challenging shadow datasets, UCF and UIUC, show that our region classifier outperforms more complex methods. We further enhance the performance of the region classifier by embedding it in an MRF framework and adding pairwise contextual cues. This leads to a method that significantly outperforms the state-of-the-art.

Illumination estimation from shadow borders

In this paper we discuss illumination estimation from a single image in general scenes and associate it with the existence of shadow edges, avoiding several pitfalls that burden previous illumination estimation approaches, which rely on associating a parametrization of illumination with the per pixel intensity of shadows or shading. We show a way to couple shadow and illumination estimation, relying only on the subset of shadow edges that is relevant to the provided geometry. In our approach, illumination estimation is posed as the minimization of an energy function that penalizes the matching between the expected shadow outline and observed image edges. Minimizing this energy function is strongly tied to selecting the appropriate set of potential shadow edges in the image. Our approach leads to an illumination estimation algorithm that performs on par with or better than the state of the art, even when scene geometry knowledge is limited, while having much lower computational complexity than state-of-the-art methods. We demonstrate the effectiveness of this approach both with quantitative results on synthetic data and qualitative evaluation on real images.

Single Image Shadow Removal via Neighbor-Based Region Relighting

In this paper we present a novel method for shadow removal in single images. For each shadow region we use a trained classifier to identify a neighboring lit region of the same material. Given a pair of lit-shadow regions we perform a region relighting transformation based on histogram matching of luminance values between the shadow region and the lit region. Then, we adjust the CIELAB \(a\) and \(b\) channels of the shadow region by adding constant offsets based on the difference of the median shadow and lit pixel values. We demonstrate that our approach produces results that outperform the state of the art by evaluating our method using a publicly available benchmark dataset.

Texture classification for rail surface condition evaluation

Rail surface defects threaten train and passenger safety. Hence rail surfaces must be restored using different processes depending on measurement of the severity of the defects. In this paper, we propose a new method for automatic classification of rail surface defect severity from images collected by rail inspection vehicles. It contains 2 components: a rail surface segmentation module, which utilizes structured random forests to generate an edge map and a Generalized Hough Transform to locate the boundaries of the rail surface; and a defect severity classification module, which combines multiple classifiers through a stacked ensemble model. The first-level learners are trained using descriptors of the rail surface images extracted by texton forests and texton dictionaries, with x2-kernel SVM classifiers. The probability estimation output of the first-level learners is the input to a second level linear-kernel SVM. Our experiments on a dataset of 939 images categorized into 8 severity levels achieved 82% accuracy.