Shoaib Ehsan

On-Board Vision Processing for Small UAVs: Time to Rethink Strategy

The ultimate research goal for unmanned aerial vehicles (UAVs) is to facilitate autonomy of operation.Research in the last decade has highlighted the potential of vision sensing in this regard. Although vital for accomplishment of missions assigned to any type of unmanned aerial vehicles, vision sensing is more critical for small aerial vehicles due to lack of high precision inertial sensors. In addition, uncertainty of GPS signal in indoor and urban environments calls for more reliance on vision sensing for such small vehicles. With off-line processing does not offer an attractive option in terms of autonomy, these vehicles have been challenging platforms to implement vision processing on-board due to their strict payload capacity and power budget. The strict constraints drive the need for new vision processing architectures for small unmanned aerial vehicles. Recent research has shown encouraging results with FPGA based hardware architectures. This paper reviews the bottle necks involved in implementing vision processing on-board,advocates the potential of hardware based solutions to tackle strict constraints of small unmanned aerial vehicles and finally analyzes feasibility of ASICs, Structured ASICs and FPGAs for use on future systems.

Multi-Scale Pixel-Based Image Fusion Using Multivariate Empirical Mode Decomposition

A novel scheme to perform the fusion of multiple images using the multivariate empirical mode decomposition (MEMD) algorithm is proposed. Standard multi-scale fusion techniques make a priori assumptions regarding input data, whereas standard univariate empirical mode decomposition (EMD)-based fusion techniques suffer from inherent mode mixing and mode misalignment issues, characterized respectively by either a single intrinsic mode function (IMF) containing multiple scales or the same indexed IMFs corresponding to multiple input images carrying different frequency information. We show that MEMD overcomes these problems by being fully data adaptive and by aligning common frequency scales from multiple channels, thus enabling their comparison at a pixel level and subsequent fusion at multiple data scales. We then demonstrate the potential of the proposed scheme on a large dataset of real-world multi-exposure and multi-focus images and compare the results against those obtained from standard fusion algorithms, including the principal component analysis (PCA), discrete wavelet transform (DWT) and non-subsampled contourlet transform (NCT). A variety of image fusion quality measures are employed for the objective evaluation of the proposed method. We also report the results of a hypothesis testing approach on our large image dataset to identify statistically-significant performance differences.

Measuring the coverage of interest point detectors

Repeatability is widely used as an indicator of the performance of an image feature detector but, although useful, it does not convey all the information that is required to describe performance. This paper explores the spatial distribution of interest points as an alternative indicator of performance, presenting a metric that is shown to concur with visual assessments. This metric is then extended to provide a measure of complementarity for pairs of detectors. Several state-of-the-art detectors are assessed, both individually and in combination. It is found that Scale Invariant Feature Operator (SFOP) is dominant, both when used alone and in combination with other detectors.

Rapid Online Analysis of Local Feature Detectors and Their Complementarity

A vision system that can assess its own performance and take appropriate actions online to maximize its effectiveness would be a step towards achieving the long-cherished goal of imitating humans. This paper proposes a method for performing an online performance analysis of local feature detectors, the primary stage of many practical vision systems. It advocates the spatial distribution of local image features as a good performance indicator and presents a metric that can be calculated rapidly, concurs with human visual assessments and is complementary to existing offline measures such as repeatability. The metric is shown to provide a measure of complementarity for combinations of detectors, correctly reflecting the underlying principles of individual detectors. Qualitative results on well-established datasets for several state-of-the-art detectors are presented based on the proposed measure. Using a hypothesis testing approach and a newly-acquired, larger image database, statistically-significant performance differences are identified. Different detector pairs and triplets are examined quantitatively and the results provide a useful guideline for combining detectors in applications that require a reasonable spatial distribution of image features. A principled framework for combining feature detectors in these applications is also presented. Timing results reveal the potential of the metric for online applications.

Exploring Integral Image Word Length Reduction Techniques for SURF Detector

Speeded Up Robust Features (SURF) is a state of the art computer vision algorithm that relies on integral image representation for performing fast detection and description of image features that are scale and rotation invariant. Integral image representation, however, has major draw back of large binary word length that leads to substantial increase in memory size. When designing a dedicated hardware to achieve real-time performance for the SURF algorithm, it is imperative to consider the adverse effects of integral image on memory size, bus width and computational resources. With the objective of minimizing hardware resources, this paper presents a novel implementation concept of a reduced word length integral image based SURF detector. It evaluates two existing word length reduction techniques for the particular case of SURF detector and extends one of these to achieve more reduction in word length. This paper also introduces a novel method to achieve integral image word length reduction for SURF detector.

Seizure detection from EEG signals using Multivariate Empirical Mode Decomposition

We present a data driven approach to classify ictal (epileptic seizure) and non-ictal EEG signals using the multivariate empirical mode decomposition (MEMD) algorithm. MEMD is a multivariate extension of empirical mode decomposition (EMD), which is an established method to perform the decomposition and time-frequency (T-F) analysis of non-stationary data sets. We select suitable feature sets based on the multiscale T-F representation of the EEG data via MEMD for the classification purposes. The classification is achieved using the artificial neural networks. The efficacy of the proposed method is verified on extensive publicly available EEG datasets.

A Method for Detecting Abnormal Program Behavior on Embedded Devices

A potential threat to embedded systems is the execution of unknown or malicious software capable of triggering harmful system behavior, aimed at theft of sensitive data or causing damage to the system. Commercial off-the-shelf embedded devices, such as embedded medical equipment, are more vulnerable as these type of products cannot be amended conventionally or have limited resources to implement protection mechanisms. In this paper, we present a self-organizing map (SOM)-based approach to enhance embedded system security by detecting abnormal program behavior. The proposed method extracts features derived from processors program counter and cycles per instruction, and then utilises the features to identify abnormal behavior using the SOM. Results achieved in our experiment show that the proposed method can identify unknown program behaviors not included in the training set with over 98.4% accuracy.

User Tracking Methods for Augmented Reality

Augmented reality has been an active area ofresearch for the last two decades or so. This paper presents acomprehensive review of the recent literature on trackingmethods used in Augmented Reality applications, both forindoor and outdoor environments. After critical discussion ofthe methods used for tracking, the paper identifies limitations ofthe state-of-the-art techniques and suggests potential futuredirections to overcome the bottlenecks.

Novel Hardware Algorithms for Row-Parallel Integral Image Calculation

The integral image is an intermediate image representation that allows rapid calculation of rectangular features at constant speed, irrespective of filter size, and is particularly useful for multi-scale computer vision algorithms like Speeded-Up Robust Features (SURF). Although calculation of the integral image involves simple addition operations, the total number of operations is significant due to the generally large size of image data. Recursive equations allow considerable reduction in the required number of addition operations but require calculation of the integral image in a serial fashion. This is generally not desirable for real-time embedded vision systems with strict time limitations and low-powered but parallel hardware resources. With the objective of minimizing the hardware resources involved, this paper proposes two novel hardware algorithms based on decomposition of these recursive equations, allowing calculation of up to four integral image values in a row-parallel way with out significantly increasing the number of addition operations.

An Assessment of Recent Attacks on Specific Embedded Systems

In this paper, we present an assessment of recent attacks on embedded systems, in particular mobile phones, wireless sensor networks, unmanned aerial vehicles and unmanned ground vehicles. As these systems become increasingly connected and networked, the number of attacks on them increases exposing them to real threats and risks, particularly when used in mission critical applications. It is necessary to investigate all aspects of the security systems associated with embedded systems in order to help protect these systems from attackers. In this we present a survey on a number of embedded systems to show system vulnerabilities, recent attacks and the security measurements undertaken to protect the embedded systems.