Erhan Gundogdu

Comparison of infrared and visible imagery for object tracking: Toward trackers with superior IR performance

The subject of this paper is the visual object tracking in infrared (IR) videos. Our contribution is twofold. First, the performance behaviour of the state-of-the-art trackers is investigated via a comparative study using IR-visible band video conjugates, i.e., video pairs captured observing the same scene simultaneously, to identify the IR specific challenges. Second, we propose a novel ensemble based tracking method that is tuned to IR data. The proposed algorithm sequentially constructs and maintains a dynamical ensemble of simple correlators and produces tracking decisions by switching among the ensemble correlators depending on the target appearance in a computationally highly efficient manner. We empirically show that our algorithm significantly outperforms the state-of-the-art trackers in our extensive set of experiments with IR imagery.

Sparse representation of two- and three-dimensional images with fractional Fourier, Hartley, linear canonical, and Haar wavelet transforms

Fractional Fourier Transform are introduced as sparsifying transforms.Linear Canonical Transforms are introduced as sparsifying transforms.Various approaches for compressing three-dimensional images are suggested. Display Omitted Sparse recovery aims to reconstruct signals that are sparse in a linear transform domain from a heavily underdetermined set of measurements. The success of sparse recovery relies critically on the knowledge of transform domains that give compressible representations of the signal of interest. Here we consider two- and three-dimensional images, and investigate various multi-dimensional transforms in terms of the compressibility of the resultant coefficients. Specifically, we compare the fractional Fourier (FRT) and linear canonical transforms (LCT), which are generalized versions of the Fourier transform (FT), as well as Hartley and simplified fractional Hartley transforms, which differ from corresponding Fourier transforms in that they produce real outputs for real inputs. We also examine a cascade approach to improve transform-domain sparsity, where the Haar wavelet transform is applied following an initial Hartley transform. To compare the various methods, images are recovered from a subset of coefficients in the respective transform domains. The number of coefficients that are retained in the subset are varied systematically to examine the level of signal sparsity in each transform domain. Recovery performance is assessed via the structural similarity index (SSIM) and mean squared error (MSE) in reference to original images. Our analyses show that FRT and LCT transform yield the most sparse representations among the tested transforms as dictated by the improved quality of the recovered images. Furthermore, the cascade approach improves transform-domain sparsity among techniques applied on small image patches.

Evaluation of Feature Channels for Correlation-Filter-Based Visual Object Tracking in Infrared Spectrum

Correlation filters for visual object tracking in visible imagery has been well-studied. Most of the correlation-filterbased methods use either raw image intensities or feature maps of gradient orientations or color channels. However, well-known features designed for visible spectrum may not be ideal for infrared object tracking, since infrared and visible spectra have dissimilar characteristics in general. We assess the performance of two state-of-the-art correlationfilter-based object tracking methods on Linköping Thermal InfraRed (LTIR) dataset of medium wave and longwave infrared videos, using deep convolutional neural networks (CNN) features as well as other traditional hand-crafted descriptors. The deep CNN features are trained on an infrared dataset consisting of 16K objects for a supervised classification task. The highest performance in terms of the overlap metric is achieved when these deep CNN features are utilized in a correlation-filter-based tracker.

Spatial windowing for correlation filter based visual tracking

Correlation filters have been extensively studied to address online visual object tracking task, while achieving favourable performance against the-state-of-the-art methods in various benchmark datasets. Nevertheless, undesired conditions, i.e. partial occlusions or abrupt deformations of the object appearance, severely degrade the performance of correlation filter based tracking methods. To this end, we propose a method for estimating a spatial window for the object observation such that the correlation output of the correlation filter and the windowed observation (i.e. element-wise multiplication of the window and the observation) is improved, especially in these adverse conditions. This approach leads to a performance uplift in the tracking result compared to the classical windowing operation. Moreover, the estimated spatial window of the object patch indicates the object regions that are useful for correlation. We observe a considerable amount of performance increase in the benchmark video sequences by using the proposed visual tracking method.

MARVEL: A Large-Scale Image Dataset for Maritime Vessels

Fine-grained visual categorization has recently received great attention as the volumes of the labelled datasets for classification of specific objects, such as cars, bird species, and aircrafts, have been increasing. The collection of large datasets has helped vision based classification approaches and led to significant improvements in performances of the state-of-the-art methods. Visual classification of maritime vessels is another important task assisting naval security and surveillance applications. In this work, we introduce a large-scale image dataset for maritime vessels, consisting of 2 million user uploaded images and their attributes including vessel identity, type, photograph category and year of built, collected from a community website. We categorize the images into 109 vessel type classes and construct 26 superclasses by combining heavily populated classes with a semi-automatic clustering scheme. For the analysis of our dataset, extensive experiments have been performed, involving four potentially useful applications; vessel classification, verification, retrieval, and recognition. We report encouraging results for each application. The introduced dataset is publicly available.

Ensemble Of adaptive correlation filters for robust visual tracking

Correlation filters have recently been popular due to their success in short-term single-object tracking as well as their computational efficiency. Nevertheless, the appearance model of a single correlation filter based tracking algorithm quickly forgets the past poses of the target object due to the updates over time. To overcome this undesired forgetting, our approach is to run trackers with separate models simultaneously. Hence, we propose a novel tracker relying on an ensemble of correlation filters, where the ensemble is obtained via a decision tree partitioning in the object appearance space. Our technique efficiently searches among the ensemble trackers and activates the ones which are most specialized on the current object appearance. Our tracking method is capable of switching frequently in the ensemble. Thus, an inherently adaptive and non-linear learning rate is achieved. Moreover, we demonstrate the superior performance of our method in benchmark video sequences.

Feature detection and matching towards augmented reality applications on mobile devices

In this work, a novel feature detection algorithm, a new local binary pattern for local binary description and a tree-based descriptor indexing for descriptor matching are proposed. Similar to well-known FAST detector, proposed feature detector performs detection via pixel intensity comparisons in nested circles. Interest point description is achieved by a novel comparison pattern, whereas matching is performed by a fuzzy decision tree. Based on simulations, it is observed that the proposed system performs competitive or better than the state-of-the-art similar techniques. Moreover, the overall system is capable of running in real-time in a 2.8GHz PC with this promising performance.

Fine-grained recognition of maritime vessels and land vehicles by deep feature embedding

Recent advances in large-scale image and video analysis have empowered the potential capabilities of visual surveillance systems. In particular, deep learning-based approaches bring in substantial benefits in solving certain computer vision problems such as fine-grained object recognition. Here, the authors mainly concentrate on classification and identification of maritime vessels and land vehicles, which are the key constituents of visual surveillance systems. Employing publicly available data sets for maritime vessels and land vehicles, the authors aim to improve visual recognition. Specifically, the authors focus on five tasks regarding visual recognition; coarse-grained classification, fine-grained classification, coarse-grained retrieval, fine-grained retrieval, and verification. To increase the performance in these tasks, the authors utilise a multi-task learning framework and present a novel loss function which simultaneously considers deep feature learning and classification by exploiting the available hierarchical labels of individual samples and the global statistics of distances between the data pairs. The authors observe that the proposed multi-task learning model improves the fine-grained recognition performance on MARVEL and Stanford Cars data sets, compared to training of a model targeting a single recognition task.

Generic and attribute-specific deep representations for maritime vessels

Fine-grained visual categorization has recently received great attention as the volumes of labeled datasets for classification of specific objects, such as cars, bird species, and air-crafts, have been increasing. The availability of large datasets led to significant performance improvements in several vision-based classification tasks. Visual classification of maritime vessels is another important task, assisting naval security and surveillance applications. We introduced, MARVEL, a large-scale image dataset for maritime vessels, consisting of 2 million user-uploaded images and their various attributes, including vessel identity, type, category, year built, length, and tonnage, collected from a community website. The images were categorized into vessel type classes and also into superclasses defined by combining semantically similar classes, following a semi-automatic clustering scheme. For the analysis of the presented dataset, extensive experiments have been performed, involving several potentially useful applications: vessel type classification, identity verification, retrieval, and identity recognition with and without prior vessel type knowledge. Furthermore, we attempted interesting problems of visual marine surveillance such as predicting and classifying maritime vessel attributes such as length, summer deadweight, draught, and gross tonnage by solely interpreting the visual content in the wild, where no additional cues such as scale, orientation, or location are provided. By utilizing generic and attribute-specific deep representations for maritime vessels, we obtained promising results for the aforementioned applications.

Fine-grained visual marine vessel classification for coastal surveillance and defense applications

The need for capabilities of automated visual content analysis has substantially increased due to presence of large number of images captured by surveillance cameras. With a focus on development of practical methods for extracting effective visual data representations, deep neural network based representations have received great attention due to their success in visual categorization of generic images. For fine-grained image categorization, a closely related yet a more challenging research problem compared to generic image categorization due to high visual similarities within subgroups, diverse applications were developed such as classifying images of vehicles, birds, food and plants. Here, we propose the use of deep neural network based representations for categorizing and identifying marine vessels for defense and security applications. First, we gather a large number of marine vessel images via online sources grouping them into four coarse categories; naval, civil, commercial and service vessels. Next, we subgroup naval vessels into fine categories such as corvettes, frigates and submarines. For distinguishing images, we extract state-of-the-art deep visual representations and train support-vector-machines. Furthermore, we fine tune deep representations for marine vessel images. Experiments address two scenarios, classification and verification of naval marine vessels. Classification experiment aims coarse categorization, as well as learning models of fine categories. Verification experiment embroils identification of specific naval vessels by revealing if a pair of images belongs to identical marine vessels by the help of learnt deep representations. Obtaining promising performance, we believe these presented capabilities would be essential components of future coastal and on-board surveillance systems.