Sebastien Wong

Understanding Data Augmentation for Classification: When to Warp?

In this paper we investigate the benefit of augmenting data with synthetically created samples when training a machine learning classifier. Two approaches for creating additional training samples are data warping, which generates additional samples through transformations applied in the data-space, and synthetic over-sampling, which creates additional samples in feature-space. We experimentally evaluate the benefits of data augmentation for a convolutional backpropagation-trained neural network, a convolutional support vector machine and a convolutional extreme learning machine classifier, using the standard MNIST handwritten digit dataset. We found that while it is possible to perform generic augmentation in feature-space, if plausible transforms for the data are known then augmentation in data-space provides a greater benefit for improving performance and reducing overfitting.

Fast 2D convolution using reconfigurable computing

Convolution and its related counterpart correlation are two commonly used operations in image processing. However these operations are computationally expensive, and perform sluggishly when implemented on microprocessors. Part of the poor performance is due to the serial nature of microprocessors, while the operations of convolution and correlation are inherently parallel. One approach to implementing these operations in parallel is to build them in hardware using application specific integrated circuits (ASICs). Another approach is to use Field Programmable Gate Arrays (FPGAs) and reconfigurable computing. Reconfigurable computing offers a trade-off between the flexibility of software running on a microprocessor and the speed of custom designed hardware. This paper describes a parallel pipelined architecture for 2D convolution written in Handel-C for a reconfigurable computing platform. The resulting system offered a 400 times increase in speed over software written in C.

Relating image, shape, position, and velocity in visual tracking

This paper solves four problems associated with typical correlation tracking systems. The first problem is that uncertainty in the position observation of an object is not propagated from the detection stage to the tracking stage. The second problem is that the shape of the reference template always lags the actual shape of the object. The third problem is the need for a separate acquisition process to generate the initial reference template. The fourth problem is the inability to track multiple objects. To overcome these problems we developed the Shape Estimating Filter (SEF), a homogeneous extension of the basic correlation tracker; and its multi-target counterpart the Competitive Attentional Correlation Tracker using Shape (CACTuS).

Subset selection of training data for machine learning: a situational awareness system case study

Recent advances in machine learning with big data sets has allowed for significant advances in the optimisation of classification and recognition systems. However, for applications such as situational awareness systems, the entirety of the available data dwarfs the amount permissible for a training set with tractable machine learning optimization times. Furthermore, the performance of any optimized system is highly dependent of the training set correctly and completely representing the entire data space of scenarios. In this paper we present a technique to characterize the entire data space to ascertain the key factors for representation and subsequently select a subset that statistically represents the correct mix of scenarios. We demonstrate the effectiveness of these characterization and subset selection techniques by using a genetic algorithm to optimize the performance of a gunfire recognition system.

An empirical evaluation of infrared clutter for point target detection algorithms

This paper describes a study into the impact of local environmental conditions on the detection of point targets using wide field of view infrared sensors on airborne platforms. A survey of the common complexity metrics for measuring IR clutter, and common point target detection algorithms was conducted. A quantitative evaluation was performed using 20 hours of infrared imagery collected over a three month period from helicopter flights in a variety of clutter environments. The research method, samples of the IR data sets, and results of the correlation between environmental conditions, scene complexity metrics and point target detection algorithms are presented. The key findings of this work are that variations in IR detection performance can be attributed to a combination of environmental factors (but no single factor is sufficient to describe performance variations), and that historical clutter metrics are insufficient to describe the performance of modern detection algorithms.

A comparative evaluation of visual tracking systems

This paper provides comparative evaluations of two visual object tracking algorithms - the Shape Estimating Filter (SEF), a homogeneous extension of the basic correlation tracker; and its multi-object counterpart the Competitive Attentional Correlation Tracker using Shape (CACTuS). The CACTuS is evaluated comparatively against its predecessor to show direct improvement in tracking effectiveness. Our approach will involve an evaluation framework consisting of a range of modern, peer reviewed tracking performance metrics, allowing for a detailed multi-faceted analysis of tracking results. As such we provide an overview of current performance evaluation methods, including techniques for multi-object tracker evaluation.

A data set for evaluating the performance of multi-class multi-object video tracking

One of the challenges in evaluating multi-object video detection, tracking and classification systems is having publically available data sets with which to compare different systems. However, the measures of performance for tracking and classification are different. Data sets that are suitable for evaluating tracking systems may not be appropriate for classification. Tracking video data sets typically only have ground truth track IDs, while classification video data sets only have ground truth class-label IDs. The former identifies the same object over multiple frames, while the latter identifies the type of object in individual frames. This paper describes an advancement of the ground truth meta-data for the DARPA Neovision2 Tower data set to allow both the evaluation of tracking and classification. The ground truth data sets presented in this paper contain unique object IDs across 5 different classes of object (Car, Bus, Truck, Person, Cyclist) for 24 videos of 871 image frames each. In addition to the object IDs and class labels, the ground truth data also contains the original bounding box coordinates together with new bounding boxes in instances where un-annotated objects were present. The unique IDs are maintained during occlusions between multiple objects or when objects re-enter the field of view. This will provide: a solid foundation for evaluating the performance of multi-object tracking of different types of objects, a straightforward comparison of tracking system performance using the standard Multi Object Tracking (MOT) framework, and classification performance using the Neovision2 metrics. These data have been hosted publically.

Track Everything: Limiting Prior Knowledge in Online Multi-Object Recognition

This paper addresses the problem of online tracking and classification of multiple objects in an image sequence. Our proposed solution is to first track all objects in the scene without relying on object-specific prior knowledge, which in other systems can take the form of hand-crafted features or user-based track initialization. We then classify the tracked objects with a fast-learning image classifier, that is based on a shallow convolutional neural network architecture and demonstrate that object recognition improves when this is combined with object state information from the tracking algorithm. We argue that by transferring the use of prior knowledge from the detection and tracking stages to the classification stage, we can design a robust, general purpose object recognition system with the ability to detect and track a variety of object types. We describe our biologically inspired implementation, which adaptively learns the shape and motion of tracked objects, and apply it to the Neovision2 Tower benchmark data set, which contains multiple object types. An experimental evaluation demonstrates that our approach is competitive with the state-of-the-art video object recognition systems that do make use of object-specific prior knowledge in detection and tracking, while providing additional practical advantages by virtue of its generality.

Learned filters for object detection in multi-object visual tracking

We investigate the application of learned convolutional filters in multi-object visual tracking. The filters were learned in both a supervised and unsupervised manner from image data using artificial neural networks. This work follows recent results in the field of machine learning that demonstrate the use learned filters for enhanced object detection and classification. Here we employ a track-before-detect approach to multi-object tracking, where tracking guides the detection process. The object detection provides a probabilistic input image calculated by selecting from features obtained using banks of generative or discriminative learned filters. We present a systematic evaluation of these convolutional filters using a real-world data set that examines their performance as generic object detectors.

Mutual information for enhanced feature selection in visual tracking

In this paper we investigate the problem of fusing a set of features for a discriminative visual tracking algorithm, where good features are those that best discriminate an object from the local background. Using a principled Mutual Information approach, we introduce a novel online feature selection algorithm that preserves discriminative features while reducing redundant information. Applying this algorithm to a discriminative visual tracking system, we experimentally demonstrate improved tracking performance on standard data sets.