Jonathan Weber

Tree Leaves Extraction in Natural Images: Comparative Study of Preprocessing Tools and Segmentation Methods

In this paper, we propose a comparative study of various segmentation methods applied to the extraction of tree leaves from natural images. This study follows the design of a mobile application, developed by Cerutti et al. (published in ReVeS Participation-Tree Species Classification Using Random Forests and Botanical Features. CLEF 2012), to highlight the impact of the choices made for segmentation aspects. All the tests are based on a database of 232 images of tree leaves depicted on natural background from smartphones acquisitions. We also propose to study the improvements, in terms of performance, using preprocessing tools, such as the interaction between the user and the application through an input stroke, as well as the use of color distance maps. The results presented in this paper shows that the method developed by Cerutti et al. (denoted Guided Active Contour), obtains the best score for almost all observation criteria. Finally, we detail our online benchmark composed of 14 unsupervised methods and 6 supervised ones.

Spatial and spectral morphological template matching

Template matching is a very topical issue in a wide range of imaging applications. Mathematical morphology offers the hit-or-miss transform, an operator which has been successfully applied for template matching in binary images. More recently, it has been extended to grayscale images and even to multivariate images. Nevertheless, these extensions, despite being relevant from a theoretical point-of-view, might lack practical interest due to the inherent difficulty to set up correctly the transform and its parameters (e.g. the structuring functions). In this paper, we propose a new and more intuitive operator which allows for morphological template matching in multivariate images from both a spatial and spectral point of view. We illustrate the potential of this operator in the context of remote sensing.

Efficient Satellite Image Time Series Analysis Under Time Warping

Satellite Image Time Series are becoming increasingly available and will continue to do so in the coming years thanks to the launch of space missions which aim at providing a coverage of the Earth every few days with high spatial resolution. In the case of optical imagery, it will be possible to produce land use and cover change maps with detailed nomenclatures. However, due to meteorological phenomena, such as clouds, these time series will become irregular in terms of temporal sampling, and one will need to compare time series with different lengths. In this paper, we present an approach to image time series analysis which is able to deal with irregularly sampled series and which also allows the comparison of pairs of time series where each element of the pair has a different number of samples. We present the dynamic time warping from a theoretical point of view and illustrate its capabilities with two applications to real-time series.

Vectorial Quasi-flat Zones for Color Image Simplification

Quasi-flat zones enable the computation of homogeneous image regions with respect to one or more arbitrary criteria, such as pixel intensity. They are most often employed in simplification and segmentation, while multiple strategies exist for their application to color data as well. In this paper we explore a vector ordering based alternative method for computing color quasi-flat zones, which enables the use of vectorial α and ω parameters. The interest of this vectorial strategy w.r.t marginal quasi-flat zones is illustrated both qualitatively and quantitatively by means of color simplification and segmentation experiments.

Fast quasi-flat zones filtering using area threshold and region merging

Quasi-flat zones are morphological operators which segment the image into homogeneous regions according to certain criteria. They are used as an image simplification tool or an image segmentation pre-processing, but they induced a very important oversegmentation. Several filtering methods have been proposed to deal with this issue but they suffer from different drawbacks, e.g., loss of quality or edge deformation. In this article, we propose a new method based on existing approaches which achieves better or similar results than existing approaches, does not suffer from their drawbacks and requires less computation time. It consists of two successive steps. First, small quasi-flat zones are removed according to a minimal area threshold. They are then filled through the growth of remaining zones.

Morphological Template Matching in Color Images

Template matching is a fundamental problem in image analysis and computer vision. It has been addressed very early by Mathematical Morphology, through the well-known Hit-or-Miss Transform. In this chapter, we review most of the existing works on this morphological template matching operator, from the standard case of binary images to the (not so standard) case of grayscale images and the very recent extensions to color and multivariate data. We also discuss the issues raised by the application of the HMT operator to the context of template matching and provide guidelines to the interested reader. Various use cases in different application domains have been provided to illustrate the potential impact of this operator.

Towards efficient satellite image time series analysis: Combination of dynamic time warping and quasi-flat zones

Satellite Image Time Series (SITS, for short) are useful resources for Earth monitoring. Upcoming satellites will provide a global coverage of the Earths surface with a short revisit time (five days); a huge amount of data to analyze will be produced. In order to be able to analyze efficiently and accurately these images, new methods have to be designed. In this article, we propose to combine a spatio-temporal segmentation pre-processing method - quasi-flat zones, which have been recently extended to video analysis - and the distortion power of DTW to simplify the representation of the SITS, in order to reduce both the time and the memory consumption. Experiments carried out on a series of 46 images show that the memory consumption can be reduced by an order of magnitude without reducing the relevance of the analysis.

La morphologie mathématique binaire pour l'extraction automatique des bâtiments dans les images THRS

This paper presents a new method for building extraction in Very High Resolution remotely sensed images in urban areas. The approach proposed is based on the use binary mathematical morphology operators. The method is composed of several steps: (1) conversion of grey level images to binary images, (2) smoothing by means of morphological filtering, (3) building detection with an adaptive hit-or-miss transform, (4) shape restoration. Two strategies of binarization are proposed. The first one consists in performing an interactive or automatic thresholding. The second one is based on an unsupervised classification. The method has been applied on a Quickbird panchromatic image. Results show the interest of the approach. MOTS-CLES : morphologie mathematique, segmentation, transformee en « tout ou rien ».

Evaluating Surgical Skills from Kinematic Data Using Convolutional Neural Networks

The need for automatic surgical skills assessment is increasing, especially because manual feedback from senior surgeons observing junior surgeons is prone to subjectivity and time consuming. Thus, automating surgical skills evaluation is a very important step towards improving surgical practice. In this paper, we designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by extracting patterns in the surgeon motions performed in robotic surgery. The proposed method is validated on the JIGSAWS dataset and achieved very competitive results with 100% accuracy on the suturing and needle passing tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate its black-box effect using class activation map. This feature allows our method to automatically highlight which parts of the surgical task influenced the skill prediction and can be used to explain the classification and to provide personalized feedback to the trainee.

Surgical motion analysis using discriminative interpretable patterns

OBJECTIVE The analysis of surgical motion has received a growing interest with the development of devices allowing their automatic capture. In this context, the use of advanced surgical training systems makes an automated assessment of surgical trainee possible. Automatic and quantitative evaluation of surgical skills is a very important step in improving surgical patient care. MATERIAL AND METHOD In this paper, we present an approach for the discovery and ranking of discriminative and interpretable patterns of surgical practice from recordings of surgical motions. A pattern is defined as a series of actions or events in the kinematic data that together are distinctive of a specific gesture or skill level. Our approach is based on the decomposition of continuous kinematic data into a set of overlapping gestures represented by strings (bag of words) for which we compute comparative numerical statistic (tf-idf) enabling the discriminative gesture discovery via its relative occurrence frequency. RESULTS We carried out experiments on three surgical motion datasets. The results show that the patterns identified by the proposed method can be used to accurately classify individual gestures, skill levels and surgical interfaces. We also present how the patterns provide a detailed feedback on the trainee skill assessment. CONCLUSIONS The proposed approach is an interesting addition to existing learning tools for surgery as it provides a way to obtain a feedback on which parts of an exercise have been used to classify the attempt as correct or incorrect.