Eduardo Fidalgo

Compass radius estimation for improved image classification using Edge-SIFT

The combination of SIFT descriptors with other features usually improves image classification, like Edge-SIFT, which extracts keypoints from an edge image obtained after applying the compass operator to a colour image. We evaluate for the first time, how the use of different radii in the compass operator affects the classification performance. We demonstrate that the value proposed in the literature, radius=4.00, is not the optimum from an image classification point of view. We also put in evidence that in ideal conditions, choosing an appropriate radius for each image yields accuracy values even higher than 95%. Finally, we propose a new method to estimate the best radius for the compass operator in each dataset. Using a training subset selected on the basis of a minimum dispersion criterion of edges density, we construct a richer dictionary for each dataset in our Bag of Words pipeline. From that dictionary it is selected a radius for the whole dataset that yields higher accuracy than using the value proposed in the literature. Using this method, we obtained improvements in the accuracy up to 24.4% in Soccer, 6.77% in COIL-RWTH-2, 4.46% in Birds, 3.82% in ImageNet_Dogs, 2.75% in ImageNet_Birds, 2.02% in Flowers and 1.75% in Caltech101 datasets. It was demonstrated that compass radius in Edge-SIFT affects to classification.The classification performance of different radii was evaluated on eight datasets.It is shown that selecting a radius for each image results in better classification.A method to automatically estimate a better radius for each dataset is proposed.The estimated radius guarantees better results than the state-of-the-art.

Query Based Object Retrieval Using Neural Codes

The task of retrieving a specific object from an image, which is similar to a query object is one of the critical applications in the computer vision domain. The existing methods fail to return similar objects when the region of interest is not specified correctly in a query image. Furthermore, when the feature vector is large, the retrieval from big collections is usually computationally expensive. In this paper, we propose an object retrieval method, which is based on the neural codes (activations) generated by the last inner-product layer of the Faster R-CNN network demonstrating that it can be used not only for object detection but for retrieval too. To evaluate the method, we have used a subset of ImageNet comprising of images related to indoor scenes, and to speed-up the retrieval, we first process all the images from the dataset and we save information (i.e. neural codes, objects present in the image, confidence scores and bounding box coordinates) corresponding to each detected object. Then, given a query image, the system detects the object present and retrieves its neural codes, which are then used to compute the cosine similarity against saved neural codes. We retrieved objects with high cosine similarity scores, and then we compared it with the results obtained using confidence scores. We showed that our approach takes only 0.534 s to retrieve all the 1454 objects in our test set.

Illegal Activity Categorisation in DarkNet Based on Image Classification Using CREIC Method

The TOR Project allows the publication of content anonymously, which cause the proliferation of illegal material whose authorship is almost impossible to identify. In this paper, we present and make publicly available TOIC (TOr Image Categories), an image dataset which comprises five different illegal classes based on crawled TOR addresses. To classify those images we used Edge-SIFT features jointly with dense SIFT descriptors obtained from an “edge image” calculated with the Compass Operator. We demonstrate how a Bag of Visual Words model trained with the early fusion of dense SIFT and Edge-SIFT features can create an efficient model to detect and categorise illegal content in TOR network. Then, we estimated the radius for a complete dataset before the Edge-SIFT calculation, and we demonstrate that the classification performance is higher when the most salient edge information is extracted from the edges. We tested our proposal in both TOIC and in the public dataset Butterflies to prove the consistency of the method, obtaining an accuracy increase of 2.32 and 7.00 points respectively. We obtained with the Ideal Radius Selection an accuracy of 92.49% on TOIC dataset which makes this approach an attractive tool to detect and categorise illegal content in TOR network.

Textile Retrieval Based on Image Content from CDC and Webcam Cameras in Indoor Environments

Textile based image retrieval for indoor environments can be used to retrieve images that contain the same textile, which may indicate that scenes are related. This makes up a useful approach for law enforcement agencies who want to find evidence based on matching between textiles. In this paper, we propose a novel pipeline that allows searching and retrieving textiles that appear in pictures of real scenes. Our approach is based on first obtaining regions containing textiles by using MSER on high pass filtered images of the RGB, HSV and Hue channels of the original photo. To describe the textile regions, we demonstrated that the combination of HOG and HCLOSIB is the best option for our proposal when using the correlation distance to match the query textile patch with the candidate regions. Furthermore, we introduce a new dataset, TextilTube, which comprises a total of 1913 textile regions labelled within 67 classes. We yielded 84.94% of success in the 40 nearest coincidences and 37.44% of precision taking into account just the first coincidence, which outperforms the current deep learning methods evaluated. Experimental results show that this pipeline can be used to set up an effective textile based image retrieval system in indoor environments.

Boosting image classification through semantic attention filtering strategies

Abstract Saliency Maps, frequently used to highlight significant information, can be combined with other paradigms, such as Bag of Visual Words (BoVW), to improve image description when the saliency regions correspond closely with the objects of interest. In this paper, we present three attention filtering strategies based on their saliency map that improve image classification using the BoVW framework, Spatial Pyramid Matching (SPM) and Convolutional Neural Networks (CNN) features. Firstly, we demonstrate how the blurring factor used in the Hou’s image signature algorithm determines what information remains and impacts to the obtained accuracy in image classification. Next, we propose AutoBlur, a simple but effective approach to automatically select this factor. Then, based on AutoBlur, we introduce two variants of our approach SARF (Semantic Attention Region Filtering), to semantically remove non-relevant regions through a Mean Shift segmentation. The first one is based on the intersection of the Hou’s image attention areas with its Mean Shift segmentation, while the second one discards regions using a key point voting system that relies on the Euclidean distance. The experiments carried out showed that the methods of Semantic Attention Filtering that we are proposing could be successfully used with both BoVW, SPM and CNN’s in most of the evaluated situations. In the five datasets assessed, all the three proposed methods outperform the baseline when using BoVWs in almost every case. For Spatial Pyramid Matching, the behaviour is similar, finding that the baseline is superior to our proposals in only one of the datasets used. In the case of CNN’s, our filtering proposal outperforms the baseline in two datasets, being very similar to it in the other cases.

Object Detection for Crime Scene Evidence Analysis Using Deep Learning

Object detection is the key module in most visual-based surveillance applications and security systems. In crime scene analysis, the images and videos play a significant role in providing visual documentation of a scene. It allows police officers to recreate a scene for later analysis by detecting objects related to a specific crime. However, due to the presence of a large volume of data, the task of detecting objects of interest is very tedious for law enforcement agencies. In this work, we present a Faster R-CNN (Region-based Convolutional Neural Network) based real-time system, which automatically detects objects which might be found in an indoor environment. To test the effectiveness of the proposed system, we applied it to a subset of ImageNet containing 12 object classes and Karina dataset. We achieved an average accuracy of 74.33%, and the mean time taken to detect objects per image was 0.12 s in Nvidia-TitanX GPU.

Evaluation of the State of Cutting Tools According to Its Texture Using LOSIB and LBP Variants

The FRESVIDA project deals with the life assessment of cutting tools working under severe conditions using digital image processing techniques. The description of texture in materials through artificial vision techniques is very useful for this goal. There are several works based on Local Binary Patterns (LBP) and many variants such as Local Binary Pattern Variance (LBPV) or Diamond-LBP Code (DLBPCS) that have proved to be effective when distinguishing materials according to their texture. The Outex dataset comprises images from 24 materials acquired under different illumination conditions, becoming the present reference dataset for texture evaluation. In this work, several descriptors have been extracted based on the LBP from the Outex dataset, as well as their combination with LOSIB (Local Oriented Statistical Information Booster). All of them have been classified with Support Vector Machine (SVM) to assess which one is more useful for the above-mentioned task. In this case, all descriptors achieve a lower performance level combined with LOSIB because Outex is a data set that studies rotation invariances.

Surface finish control in machining processes using haralick descriptors and neuronal networks

This paper presents a method to perform a surface finish control using a computer vision system. The goal pursued was to design an acceptance criterion for the control of surface roughness of steel parts, dividing them in those with low roughness - acceptable class - and those with high roughness - defective class. We have used 143 images obtained from AISI 303 stainless steel machining. Images were described using three different methods - texture local filters, the first four Haralick descriptors from the gray-level co-occurrence matrix and a 20 features vector obtained from the first subband of a wavelet transform of the original image and also the gray-level original image. Classification was conducted using K-nn and Neuronal Networks. The best error rate - 4.0% - with k-nn was achieved using texture descriptors. With the neuronal network, an eight node hidden layer network using Haralick descriptors leads to the optimal configuration - 0.0% error rate.