Pouria Sadeghi-Tehran

An approach to automatic real-time novelty detection, object identification, and tracking in video streams based on recursive density estimation and evolving Takagi–Sugeno fuzzy systems

Recently, surveillance, security, patrol, search, and rescue applications increasingly require algorithms and methods that can work automatically in real time. This paper reports a new real‐time approach based on three novel techniques for automatic detection, object identification, and tracking in video streams, respectively. The novelty detection and object identification are based on the newly proposed recursive density estimation (RDE) method. RDE is using a Cauchy‐type of kernel, which is calculated recursively as opposed to the widely used (in particular in the kernel density estimation (KDE) approach) Gaussian one. The key difference is that the proposed approach works on a per frame basis and does not require a window (usually of size of several dozen) of frames to be stored in the memory and processed. It should be noted that the new RDE approach is free from user‐ or problem‐specific thresholds by differ from the other state‐of‐the‐art approaches. Finally, an evolving Takagi–Sugeno (eTS)‐type fuzzy system is proposed for tracking. The proposed approach has been compared with KDE and Kalman filter (KF) and has proven to be significantly (in an order of magnitude) faster and computationally more efficient than RDE and more precise than KF.

Self-evolving parameter-free Rule-based Controller

In this paper, a new approach for Self-evolving PArameter-free fuzzy Rule-based Controller (SPARC) is proposed. Two illustrative examples are provided aiming a proof of concept. The proposed controller can start with no pre-defined fuzzy rules, and does not need to pre-define the range of the output or control variables. This SPARC learns autonomously from its own actions while performing the control of the plant. It does not use any parameters, explicit membership functions, any off-line pre-training nor the explicit model (e.g. in a form of differential equations) of the plant. It combines the relative older concept of indirect adaptive control with the newer concepts of (self-)evolving fuzzy rule-based systems (and controllers, in particular) and with the very recent concept of parameter-free, data cloud and data density based fuzzy rule based systems (and controllers in particular). It has been demonstrated that a fully autonomously and in an unsupervised manner (based only on the data density and selecting representative prototypes/focal points from the control hyper-surface acting as a data space) it is possible generate a parameter-free control structure and evolve it in on-line mode. Moreover, the results demonstrate that this autonomous controller is effective (has comparative error and performance characteristics) to other known controllers, including self-learning ones, but surpasses them with its flexibility and extremely lean structure (small number of prototypes/focal points which serve as seeds to form parameter-free and membership function-free fuzzy rules based on them). The illustrative examples aim primarily proof of concept.

Online self-evolving fuzzy controller for autonomous mobile robots

In this paper, an online self-evolving fuzzy controller is proposed for an autonomous leader/follower. The self-evolving controller starts with a simple configuration and learns from its own actions while controlling the mobile robot during the leader following behaviour. A traditional Takagi-Sugeno type fuzzy controller is also implemented and compared with the proposed controller to verify the reliability and performance of the self-evolving controller. Experiments are carried out with a real mobile robot Pioneer 3DX at Lancaster University.

A Fast Recursive Approach to Autonomous Detection, Identification and Tracking of Multiple Objects in Video Streams under Uncertainties

Real-time processing the information coming form video, infra-red or electro-optical sources is a challenging task due the uncertainties such as noise and clutter, but also due to the large dimensionalities of the problem and the demand for fast and efficient algorithms. This paper details an approach for automatic detection, single and multiple objects identification and tracking in video streams with applications to surveillance, security and autonomous systems. It is based on a method that provides recursive density estimation (RDE) using a Cauchy type of kernel. The main advantage of the RDE approach as compared to other traditional methods (e.g. KDE) is the low computational and memory storage cost since it works on a frame-by-frame basis; the lack of thresholds, and applicability to multiple objects identification and tracking. A robust to noise and clutter technique based on spatial density is also proposed to autonomously identify the targets location in the frame.

ARTOT: Autonomous real-time object detection and tracking by a moving camera

A new approach to autonomously detect and track a moving object in a video captured by a moving camera (possibly mounted on a unmanned vehicle, UxV) is proposed in this paper. It is based on a combination of the recently introduced recursive density estimation (RDE) approach and the well-known scale invariant feature transformation (SIFT). The new approach involves building a model of the background using RDE in video sequences captured by a moving camera. RDE was robust in many videos with moving background in the absence of image registration (pixel position alignment). The output of RDE is a cluster of foreground pixels which can be associated with the object of interest. After the moving object is detected, the foreground pixels are enclosed in a rectangular region of interest (ROI). The approximate size and location of the rectangular region is then sent to the object tracking algorithm. The tracking algorithm uses the rectangular search area to detect and match SIFT keypoints across successive video frames. If and when the tracking fails, the RDE algorithm is started again to detect the moving object. The proposed algorithm does not require any human involvement and it operates in real-time. The tracking algorithm is also computationally efficient because only a small ROI is processed in each video frame. In the future we aim to substitute the SIFT approach with speeded-up robust features (SURF) for higher accuracy in tracking and for faster processing speed. Additionally, the case of multiple objects can be addressed using clustering in the spatial domain and is a subject of current research.

A real-time approach for autonomous detection and tracking of moving objects from UAV

A new approach to autonomously detect and track moving objects in a video captured by a moving camera from a UAV in real-time is proposed in this paper. The introduced approach replaces the need for a human operator to perform video analytics by autonomously detecting moving objects and clustering them for tracking purposes. The effectiveness of the introduced approach is tested on the footage taken from a real UAV and the evaluation results are demonstrated in this paper.

A real-time approach for novelty detection and trajectories analysis for anomaly recognition in video surveillance systems

In this paper, we present a novel approach for automatic object detection and also using on-line trajectory clustering for RT anomaly detection in video streams. The proposed approach is based on two main steps. In the first step, a recently introduced approach called Recursive Density Estimation (RDE) is used for novelty detection. This method is using a Cauchy type of kernel which works on a frame-by-frame basis and does not require a pre-defined threshold to identify objects. In the second step, multifeature object trajectory is clustered on-line to identify anomalies in video streams. To identify an anomaly, first the trajectories are transformed into a set of features in a space to which eClustering approach identifies the modes and the corresponding clusters. At the end, by using cluster fusion the final common pattern is estimated and any sparse trajectories are considered as anomalous.

Look-a-Like: A Fast Content-Based Image Retrieval Approach Using a Hierarchically Nested Dynamically Evolving Image Clouds and Recursive Local Data Density

The need to find related images from big data streams is shared by many professionals, such as architects, engineers, designers, journalist, and ordinary people. Users need to quickly find the relevant images from data streams generated from a variety of domains. The challenges in image retrieval are widely recognized, and the research aiming to address them led to the area of content‐based image retrieval becoming a “hot” area. In this paper, we propose a novel computationally efficient approach, which provides a high visual quality result based on the use of local recursive density estimation between a given query image of interest and data clouds/clusters which have hierarchical dynamically nested evolving structure. The proposed approach makes use of a combination of multiple features. The results on a data set of 65,000 images organized in two layers of a hierarchy demonstrate its computational efficiency. Moreover, the proposed Look‐a‐like approach is self‐evolving and updating adding new images by crawling and from the queries made.

AURORA: autonomous real-time on-board video analytics

In this paper, we describe the design and implementation of a computationally efficient system for detecting moving objects on a moving platform which can be deployed on small, lightweight, low-cost and power-efficient hardware. The primary application of the payload system is that of performing real-time on-board autonomous object detection of moving objects from videos stream taken from a camera mounted to an unmanned aerial vehicle (UAV). The implemented object detection algorithms utilise recursive density estimation and evolving local means clustering to perform change and object detection of moving objects without prior knowledge. Furthermore, experiments are presented which demonstrate that the introduced system is able to detect, by on-board processing, any moving objects from a UAV in real time while at the same time sending only important data to a control station located on the ground with minimal time to set up and become operational.

ARTOD: Autonomous Real Time Objects Detection by a Moving Camera Using Recursive Density Estimation

A new approach to autonomously detect moving objects in a video captured by a moving camera is proposed in this chapter. The proposed method is separated in two modules. In the first part, the well-known scale invariant feature transformation (SIFT) and the RANSAC algorithm are used to estimate the camera movement. In the second part, recursive density estimation (RDE) is used to build a model of the background and detect moving objects in a scene. The results are presented for both indoor and outdoor video sequences taken from a UAV for outdoor scenario and handheld camera for indoor experiment.