Donatello Conte

A survey on image-based insect classification

Entomology has had many applications in many biological domains (i.e insect counting as a biodiversity index). To meet a growing biological demand and to compensate a decreasing workforce amount, automated entomology has been around for decades. This challenge has been tackled by computer scientists as well as by biologists themselves. This survey investigates fourty-four studies on this topic and tries to give a global picture on what are the scientific locks and how the problem was addressed. Views are adopted on image capture, feature extraction, classification methods and the tested datasets. A general discussion is finally given on the questions that might still remain unsolved such as: the image capture conditions mandatory to good recognition performance, the definition of the problem and whether computer scientist should consider it as a problem in its own or just as an instance of a wider image recognition problem. Graphical abstractDisplay Omitted HighlightsFourty-four about image-based insect recognition are scrutinized.Each paper is qualified from three perspectives: image capture, feature extraction and classification.Datasets used in the literature are investigated.A discussion is given in which several questions about the problem are raised.

Image quality assessment based on regions of interest

Most methods in the literature of image quality assessment (IQA) use whole image information for measuring image quality. However, human perception does not always use this criterion to assess the quality of images. Individuals usually provide their opinions by considering only some parts of an image, called regions of interest. Based on this hypothesis, in this research work, a segmentation technique is initially employed to obtain a bi-level image map composed of the foreground and background information. A patch selection strategy is then proposed to choose some particular patches based on the foreground information as the regions of interest for IQA. Three recent IQA methods in the literature are considered to demonstrate the improvement in IQA when using only the extracted regions of interest. To evaluate the impact of the proposed patch selection strategy in various IQA metrics, three publicly available datasets were used for experiments. Experimental results have revealed that our proposal, based on the regions of interest, can improve quality measures of three IQA methods.

Document Image Quality Assessment Based on Texture Similarity Index

In this paper, a full reference document image quality assessment (FR DIQA) method using texture features is proposed. Local binary patterns (LBP) as texture features are extracted at the local and global levels for each image. For each extracted LBP feature set, a similarity measure called the LBP similarity index (LBPSI) is computed. A weighting strategy is further proposed to improve the LBPSI obtained based on local LBP features. The LBPSIs computed for both local and global features are then combined to get the final LBPSI, which also provides the best performance for DIQA. To evaluate the proposed method, two different datasets were used. The first dataset is composed of document images, whereas the second one includes natural scene images. The mean human opinion scores (MHOS) were considered as ground truth for performance evaluation. The results obtained from the proposed LBPSI method indicate a significant improvement in automatically/accurately predicting image quality, especially on the document image-based dataset.

Document image quality assessment based on improved gradient magnitude similarity deviation

Digitization of business processes and the use of mobile devices as portable scanner lead to a massive production of document images that is beyond manual handling. In such a scenario, automatic estimation of document image quality is a concern in order to adapt as early as possible document image analysis methods. In this paper, a method for full reference document image quality assessment (DIQA) using mainly foreground information is proposed. In the proposed method, a segmentation technique is employed on a reference document image to approximately separate foreground and background information. Foreground information of the document image are then considered in the form of foreground patches for computing image quality. For each foreground patch, corresponding gradient maps, obtained from the reference and distorted gradient magnitude maps, are used to compute a gradient magnitude similarity map of the patch. Gradient magnitude similarity deviation of the patch is then calculated by the means of standard deviation over all the values in the gradient magnitude similarity map obtained for the patch. An average pooling is finally performed on all the standard deviations obtained for all the foreground patches to obtain the final image quality metric of the distorted document image. To evaluate the proposed method, we used 3 different datasets. The first dataset was a dataset composed of 377 document images of which 29 were reference images and 348 were distorted images. The other datasets were LIVE and CSIQ datasets composed of scene images with MHOS as ground truth. The results obtained from the proposed system are encouraging.

Learning Graph Matching with a Graph-Based Perceptron in a Classification Context

Many tasks in computer vision and pattern recognition are formulated as graph matching problems. Despite the NP-hard nature of the problem, fast and accurate approximations have led to significant progress in a wide range of applications. Learning graph matching functions from observed data, however, still remains a challenging issue. This paper presents an effective scheme to parametrize a graph model for object matching in a classification context. For this, we propose a representation based on a parametrized model graph, and optimize it to increase a classification rate. Experimental evaluations on real datasets demonstrate the effectiveness (in terms of accuracy and speed) of our approach against graph classification with hand-crafted cost functions.

AMHUSE: a multimodal dataset for HUmour SEnsing

We present AMHUSE (A Multimodal dataset for HUmour SEnsing) along with a novel web-based annotation tool named DANTE (Dimensional ANnotation Tool for Emotions). The dataset is the result of an experiment concerning amusement elicitation, involving 36 subjects in order to record the reactions in presence of 3 amusing and 1 neutral video stimuli. Gathered data include RGB video and depth sequences along with physiological responses (electrodermal activity, blood volume pulse, temperature). The videos were later annotated by 4 experts in terms of valence and arousal continuous dimensions. Both the dataset and the annotation tool are made publicly available for research purposes.

Remove Noise in Video with 3D Topological Maps

In this paper we present a new method for foreground masks denoising in videos. Our main idea is to consider videos as 3D images and to deal with regions in these images. Denoising is thus simply achieved by merging foreground regions corresponding to noise with background regions. In this framework, the main question is the definition of a criterion allowing to decide if a region corresponds to noise or not. Thanks to our complete cellular description of 3D images, we can propose an advanced criterion based on Betti numbers, a topological invariant. Our results show the interest of our approach which gives better results than previous methods.

Effective Training of Convolutional Neural Networks for Insect Image Recognition

Insects are living beings whose utility is critical in life sciences. They enable biologists obtaining knowledge on natural landscapes (for example on their health). Nevertheless, insect identification is time-consuming and requires experienced workforce. To ease this task, we propose to turn it into an image-based pattern recognition problem by recognizing the insect from a photo. In this paper state-of-art deep convolutional architectures are used to tackle this problem. However, a limitation to the use of deep CNNs is the lack of data and the discrepancies in classes cardinality. To deal with such limitations, transfer learning is used to apply knowledge learnt from ImageNet-1000 recognition task to insect image recognition task. A question arises from transfer-learning: is it relevant to retrain the entire network or is it better not to modify some layers weights? The hypothesis behind this question is that there must be part of the network which contains generic (problem-independent) knowledge and the other one contains problem-specific knowledge. Tests have been conducted on two different insect image datasets. VGG-16 models were adapted to be more easily learnt. VGG-16 models were trained (a) from scratch (b) from ImageNet-1000. An advanced study was led on one of the datasets in which the influences on performance of two parameters were investigated: (1) The amount of learning data (2) The number of layers to be finetuned. It was determined VGG-16 last block is enough to be relearnt. We have made the code of our experiment as well as the script for generating an annotated insect dataset from ImageNet publicly available.

Graph Edit Distance in the Exact Context.

This paper presents a new Mixed Integer Linear Program (MILP) formulation for the Graph Edit Distance (GED) problem. The contribution is an exact method that solves the GED problem for attributed graphs. It has an advantage over the best existing one when dealing with the case of dense of graphs, because all its constraints are independent from the number of edges in the graphs. The experiments have shown the efficiency of the new formulation in the exact context.

A Deep Neural Network Architecture to Estimate Node Assignment Costs for the Graph Edit Distance

The problem of finding a distance and a correspondence between a pair of graphs is commonly referred to as the Error-tolerant Graph matching problem. The Graph Edit Distance is one of the most popular approaches to solve this problem. This method needs to define a set of parameters and the cost functions aprioristically. On the other hand, in recent years, Deep Neural Networks have shown very good performance in a wide variety of domains due to their robustness and ability to solve non-linear problems. The aim of this paper is to present a model to compute the assignments costs for the Graph Edit Distance by means of a Deep Neural Network previously trained with a set of pairs of graphs properly matched. We empirically show a major improvement using our method with respect to the state-of-the-art results.