Cigdem Beyan

Classifying imbalanced data sets using similarity based hierarchical decomposition

Classification of data is difficult if the data is imbalanced and classes are overlapping. In recent years, more research has started to focus on classification of imbalanced data since real world data is often skewed. Traditional methods are more successful with classifying the class that has the most samples (majority class) compared to the other classes (minority classes). For the classification of imbalanced data sets, different methods are available, although each has some advantages and shortcomings. In this study, we propose a new hierarchical decomposition method for imbalanced data sets which is different from previously proposed solutions to the class imbalance problem. Additionally, it does not require any data pre-processing step as many other solutions need. The new method is based on clustering and outlier detection. The hierarchy is constructed using the similarity of labeled data subsets at each level of the hierarchy with different levels being built by different data and feature subsets. Clustering is used to partition the data while outlier detection is utilized to detect minority class samples. The comparison of the proposed method with state of art the methods using 20 public imbalanced data sets and 181 synthetic data sets showed that the proposed method?s classification performance is better than the state of art methods. It is especially successful if the minority class is sparser than the majority class. It has accurate performance even when classes have sub-varieties and minority and majority classes are overlapping. Moreover, its performance is also good when the class imbalance ratio is low, i.e. classes are more imbalanced. A novel method for imbalanced dataset classification.A new hierarchical classifier which does not use a fixed feature/class hierarchy.Uses clustering and outlier detection to construct the hierarchy.Shows that different feature spaces can be used to build a hierarchy.Successful when the class imbalanced ratio is low, classes are highly overlapping.

A rule-based event detection system for real-life underwater domain

Understanding and analyzing fish behaviour is a fundamental task for biologists that study marine ecosystems because the changes in animal behaviour reflect environmental conditions such as pollution and climate change. To support investigators in addressing these complex questions, underwater cameras have been recently used. They can continuously monitor marine life while having almost no influence on the environment under observation, which is not the case with observations made by divers for instance. However, the huge quantity of recorded data make the manual video analysis practically impossible. Thus machine vision approaches are needed to distill the information to be investigated. In this paper, we propose an automatic event detection system able to identify solitary and pairing behaviours of the most common fish species of the Taiwanese coral reef. More specifically, the proposed system employs robust low-level processing modules for fish detection, tracking and recognition that extract the raw data used in the event detection process. Then each fish trajectory is modeled and classified using hidden Markov models. The events of interest are detected by integrating end-user rules, specified through an ad hoc user interface, and the analysis of fish trajectories. The system was tested on 499 events of interest, divided into solitary and pairing events for each fish species. It achieved an average accuracy of 0.105, expressed in terms of normalized detection cost. The obtained results are promising, especially given the difficulties occurring in underwater environments. And moreover, it allows marine biologists to speed up the behaviour analysis process, and to reliably carry on their investigations.

Detection of Abnormal Fish Trajectories Using a Clustering Based Hierarchical Classifier

We address the analysis of fish trajectories in unconstrained underwater videos to help marine biologist to detect new/rare fish behaviours and to detect environmental changes which can be observed from the abnormal behaviour of fish. The fish trajectories are separated into normal and abnormal classes which indicate the common behaviour of fish and the behaviours that are rare/ unusual respectively. The proposed solution is based on a novel type of hierarchical classifier which builds the tree using clustered and labelled data based on similarity of data while using different feature sets at different levels of hierarchy. The paper presents a new method for fish trajectory analysis which has better performance compared to state-of-the-art techniques while the results are significant considering the challenges of underwater environments, low video quality, erratic movement of fish and highly imbalanced trajectory data that we used. Moreover, the proposed method is also powerful enough to classify highly imbalanced real-world datasets.

Fusion of thermal- and visible-band video for abandoned object detection

Timely detection of packages that are left unattended in public spaces is a security concern, and rapid detection is important for prevention of potential threats. Because constant surveillance of such places is challenging and labor intensive, automated abandoned-object-detection systems aiding operators have started to be widely used. In many studies, stationary objects, such as people sitting on a bench, are also detected as suspicious objects due to abandoned items being defined as items newly added to the scene and remained stationary for a predefined time. Therefore, any stationary object results in an alarm causing a high number of false alarms. These false alarms could be prevented by classifying suspicious items as living and nonliving objects. In this study, a system for abandoned object detection that aids operators surveilling indoor environments such as airports, railway or metro stations, is proposed. By analysis of information from a thermal- and visible-band camera, people and the objects left behind can be detected and discriminated as living and nonliving, reducing the false-alarm rate. Experiments demonstrate that using data obtained from a thermal camera in addition to a visible-band camera also increases the true detection rate of abandoned objects.

Detecting abnormal fish trajectories using clustered and labeled data

We propose an approach for the analysis of fish trajectories in unconstrained underwater videos. Trajectories are classified into two classes: normal trajectories which contain the usual behavior of fish and abnormal trajectories which indicate the behaviors that are not as common as the normal class. The paper presents two innovations: 1) a novel approach to abnormal trajectory detection and 2) improved performance on video based abnormal trajectory analysis of fish in unconstrained conditions. First we extract a set of features from trajectories and apply PCA. We then perform clustering on a subset of features. Based on the clustering, outlier detection is applied to each cluster. Improved results are obtained which is significant considering the challenges of underwater environments, low video quality, and erratic movement of fish.

Event detection in underwater domain by exploiting fish trajectory clustering

In this paper we propose a clustering-based approach for the analysis of fish trajectories in real-life unconstrained underwater videos, with the purpose of detecting behavioural events; in such a context, both video quality limitations and the motion properties of the targets make the trajectory analysis task for event detection extremely difficult. Our approach is based on the k-means clustering algorithm and allows to group similar trajectories together, thus providing a simple way to detect the most used paths and the most visited areas, and, by contrast, to identify trajectories which do not fall into any common clusters, therefore representing unusual behaviours. Our results show that the proposed approach is able to separate trajectory patterns and to identify those matching predefined behaviours or which are more likely to be associated to new/anomalous behaviours.

A multimodal approach for individual tracking of people and their belongings

Abstract In this study, a fully automatic surveillance system for indoor environments which is capable of tracking multiple objects using both visible and thermal band images is proposed. These two modalities are fused to track people and the objects they carry separately using their heat signatures and the owners of the belongings are determined. Fusion of complementary information from different modalities (for example, thermal images are not affected by shadows and there is no thermal reflection or halo effect in visible images) is shown to result in better object detection performance. We use adaptive background modeling and local intensity operation for object detection and the mean-shift tracking algorithm for fully automatic tracking. Trackers are refreshed to resolve potential problems which may occur due to the changes in object’s size, shape and to handle occlusion-split and to detect newly emerging objects as well as objects that leave the scene. The proposed scheme is applied to the abandoned object detection problem and the results are compared with the state of art methods. The results show that the proposed method facilitate individual tracking of objects for various applications, and provide lower false alarm rates compared to the state of art methods when applied to the abandoned object detection problem.

Applying semi-synchronised task farming to large-scale computer vision problems

Distributed compute clusters allow the computing power of heterogeneous (and homogeneous) resources to be utilised to solve large-scale science and engineering problems. One class of problem that has attractive scalability properties, and is therefore often implemented using compute clusters, is task farming (or parameter sweep) applications. A typical characteristic of such applications is that no communication is needed between distributed subtasks during the overall computation. However, interesting large-scale task farming problem instances that do require global communication between subtask sets also exist. We propose a framework called semi-synchronised task farming in order to address problems requiring distributed formulations containing subtasks that alternate between independence and synchronisation. We apply this framework to several large-scale contemporary computer vision problems and present a detailed performance analysis to demonstrate framework scalability. Semi-synchronised task farming splits a given problem into a number of stages. Each stage involves firstly distributing independent subtasks to be completed in parallel. Following subtask set completion, a set of synchronised global decisions, based on information retrieved from the distributed results, is made. The results influence the following subtask distribution stage. This subtask distribution followed by result collation process is iterated until overall problem solutions are obtained. We construct a simplified Bulk Synchronous Parallel (BSP) model to formalise this framework and with this formalisation, we develop a predictive model for overall task completion time. We present experimental benchmark results comparing the performance observed by applying our framework to solve real-world problems on compute clusters with that of solving the tasks in a serial fashion. Furthermore by assessing the predicted time savings that our framework provides in simulation and validating these predictions on a range of complex problems drawn from real-world computer vision tasks, we are able to reliably predict the performance gain obtained when using a compute cluster to tackle resource intensive computer vision tasks.

Detection of abandoned objects using thermal and visible band tracking

In this study, an abandoned object detection algorithm which is based on individual tracking of multiple objects such as people and their belongings is presented. To track people and their belongings individually; in addition to the visible band data, thermal band data is used and these objects are tracked using an improved, adaptive mean shift algorithm. By using the information coming from fusion of different modalities and using the heat signatures, objects are discriminated as people and belongings, trajectories of these objects are found, owners of belongings are determined and abandoned objects are detected. In association with mean shift tracking, adaptive background modeling and local intensity operation are used for fully automatic tracking. The results show that our method is robust, comparable with other methods by low false alarm rates and could be used to assist surveillance operators in public indoor environments.

Mean-shift tracking for surveillance applications using thermal and visible band data fusion

Separate tracking of objects such as people and the luggages they carry is important for video surveillance applications as it would allow making higher level inferences and timely detection of potential threats. However, this is a challenging problem and in the literature, people and objects they carry are tracked as a single object. In this study, we propose using thermal imagery in addition to the visible band imagery for tracking in indoor applications (such as airports, metro or railway stations). We use adaptive background modeling in association with mean-shift tracking for fully automatic tracking. Trackers are refreshed using the background model to handle occlusion and split and to detect newly emerging objects as well as objects that leave the scene. Visible and thermal domain tracking information are fused to allow tracking of people and the objects they carry separately using their heat signatures. By using the trajectories of these objects, interactions between them could be deduced and potential threats such as abandoning of an object by a person could be detected in real-time. Better tracking performance is also achieved compared to using a single modality as thermal reflection and halo effect which adversely affect tracking are eliminated by the complementing visible band data. The proposed method has been tested on videos containing various scenarios. The experimental results show that the presented method is effective for separate tracking of objects such as people and their belongings and for detecting the interactions in the presence of occlusions.