Tariq Abdullah

Self-Organizing Dynamic Ad Hoc Grids

Resource management for ad hoc grids is challenging due to the participation of heterogeneous, dynamic, autonomous and ephemeral nodes. Different underlying network infrastructures, and varying use and access policies make resource management even more complex. Therefore it is required to develop such a resource management mechanism that will enable the ad hoc grid to self-organize according to the workload of the resource manager. The proposed mechanism is based on the emergent behavior of the participating nodes and adapts with respect to changes in the ad hoc grid environment. Scalability and robustness of the proposed mechanism is tested by running the experiments on PlanetLab. Simulation results show that our mechanism performs better than previously proposed mechanisms.

Resource Discovery with Dynamic Matchmakers in Ad Hoc Grid

Nodes in an ad hoc grid are characterized by heterogeneity, autonomy, and volatility. These characteristics result in varying workload of the resource manager in the ad hoc grid. Therefore it is required to develop a resource al-location mechanism that can balance the workload of there source manager, hereafter referred to as matchmaker, and can enable the ad hoc grid to self-organize itself. In this paper, we define a mechanism that dynamically promotes and demotes nodes as matchmaker(s) and matchmakers back to the normal nodes in an ad hoc grid environment. The proposed mechanism uses the matchmaker workload as the basic criterion for promotion and demotion of the matchmaker(s). Simulation results show that our approach per-forms better than previously proposed solutions.

Video Stream Analysis in Clouds: An Object Detection and Classification Framework for High Performance Video Analytics

Object detection and classification are the basic tasks in video analytics and become the starting point for other complex applications. Traditional video analytics approaches are manual and time consuming. These are subjective due to the very involvement of human factor. We present a cloud based video analytics framework for scalable and robust analysis of video streams. The framework empowers an operator by automating the object detection and classification process from recorded video streams. An operator only specifies an analysis criteria and duration of video streams to analyse. The streams are then fetched from a cloud storage, decoded and analysed on the cloud. The framework executes compute intensive parts of the analysis to GPU powered servers in the cloud. Vehicle and face detection are presented as two case studies for evaluating the framework, with one month of data and a 15 node cloud. The framework reliably performed object detection and classification on the data, comprising of 21,600 video streams and 175 GB in size, in 6.52 hours. The GPU enabled deployment of the framework took 3 hours to perform analysis on the same number of video streams, thus making it at least twice as fast than the cloud deployment without GPUs.

Hybrid Resource Discovery Mechanism in Ad Hoc Grid Using Structured Overlay

Resource management has been an area of research in ad hoc grids for many years. Recently, different research projects have focused resource management in centralized, decentralized or in a hybrid manner. In this paper, we discuss a micro economic based, hybrid resource discovery mechanism. The proposed mechanism focuses on the extension of a structured overlay network to manage the (dis)appearance of matchmakers in the grid and to route the messages to the appropriate matchmaker in the ad hoc grid. The mechanism is based on the emergent behavior of the participating nodes and adapts with respect to changes in the ad hoc grid environment. Experiments are executed on PlanetLab to test the scalability and robustness of the proposed mechanism. Simulation results show that our mechanism performs better than previously proposed mechanisms.

Traffic Monitoring Using Video Analytics in Clouds

Traffic monitoring is a challenging task on crowded roads. Traditional traffic monitoring procedures are manual, expensive, time consuming and involve human operators. They are subjective due to the very involvement of human factor and sometimes provide inaccurate/incomplete monitoring results. Large scale storage and analysis of video streams were not possible due to limited availability of storage and compute resources in the past. Recent advances in data storage, processing and communications have made it possible to store and process huge volumes of video data and develop applications that are neither subjective nor limited in feature sets. It is now possible to implement object detection and tracking, behavioural analysis of traffic patterns, number plate recognition and automate security and surveillance on video streams produced by traffic monitoring and surveillance cameras. In this paper, we present a video stream acquisition, processing and analytics framework in the clouds to address some of the traffic monitoring challenges mentioned above. This framework provides an end-to-end solution for video stream capture, storage and analysis using a cloud based GPU cluster. The framework empowers traffic control room operators by automating the process of vehicle identification and finding events of interest from the recorded video streams. An operator only specifies the analysis criteria and the duration of video streams to analyse. The video streams are then automatically fetched from the cloud storage, decoded and analysed on a Hadoop based GPU cluster without operator intervention in our framework. It reduces the latencies in video analysis process by porting its compute intensive parts to the GPU cluster. The framework is evaluated with one month of recorded video streams data on a cloud based GPU cluster. The results show a speedup of 14 times on a GPU and 4 times on a CPU when compared with one human operator analysing the same amount of video streams data.

Ant Colony Inspired Microeconomic Based Resource Management in Ad Hoc Grids

Ad hoc grids are inherently complex and are dynamic systems. This is due to decentralized control, heterogeneity in resources of the participating nodes, variations in resource availability and user defined access and use polices for the resources. On the other hand, the universe is full of complex adaptive systems such as the immune system, sand dune ripples, and ant foraging etc. The participants in these systems apply simple local rules, resulting in robustness and self-organization. In this paper, we present an ant colony inspired, microeconomic based resource management system for ad hoc grids. The mechanism is based on the emergent behavior of the participating nodes and adapts itself to changes in the ad hoc grid environment. The mechanism enables the ad hoc grid to self-organize itself under varying workload of the participating nodes. Experiments are executed on PlanetLab to test the scalability and robustness of the proposed mechanism.

Nature Inspired Self Organization for Adhoc Grids

Ant Colony Optimization (ACO) and other similar nature inspired mechanisms like artificial neural networks, swarm intelligence and evolutionary algorithms are based on naturally existing Complex Adaptive Systems (CAS). Human immune system, sand dune ripples, and ant foraging are some examples of the naturally existing CAS. Participating agents in these systems interact according to simple local rules which result in complex behavior and self-organization at system level. Adhoc grids are dynamic in nature and participating nodes show intermittent and volatile participation. Resource availability fluctuates over time inadhoc grids and results in a new adhoc grid state. These changes require adoption of the adhoc grid to anew state by applying some self organizing mechanism. In this paper, we present nature-inspired (ACO), micro-economic based mechanisms for infrastructure level self-organization in adhoc grids. These mechanisms help in achieving a scalable, dynamic and a self-organizing adhoc grid infrastructure. These mechanisms are evaluated with varying workloads in different network conditions. Study of these mechanisms helped in understanding the effect of ACO based self-organization mechanism on the infrastructural spectrum, ranging from completely centralized to fully decentralized.

Genomics Analyser: A Big Data Framework for Analysing Genomics Data

Abstract Genomics data is unstructured and mostly stored on hard disks. It is both technically and culturally residing in big data domain due to the challenges of volume, velocity and variety. Huge volumes of data are generated from diverse sources in different formats and at a high frequency. Appropriate data models are required to accommodate these data formats for analysing and producing required results with a quick response time. Genomics data can be analysed for a variety of purposes. Existing genomics data analysis pipelines are disk I/O intensive and focus on optimizing data processing for individual analysis tasks. Intensive disk I/O operations and focus on optimizing individual analysis tasks are the biggest bottleneck of existing genomics analysis pipelines. Making any updates in genomics data require reading the whole data set again. In this paper, we present a genomics data analysis framework that addresses both the issues of existing genomics analysis pipelines. It reads unstructured genomics data from sources, transforms it in a structured format and stores this data into a NoSQL database. In this way, genomics data can be queried like any other data and an update in the genomics data does not require reading the whole data set. The framework also presents an efficient analysis pipeline for analysing the genomics data for a variety of purposes like genotype clustering, gene expression microarrays, chromosome variations or gene linkage analysis. A case study of genotype clustering is presented to demonstrate and evaluate the effectiveness of the presented framework. Our results show that the framework improves overall performance of the genomics data analysis pipeline by 49% from existing genomics data analysis pipelines. Furthermore, our approach is robust and is able sustain high performance with high system workloads.

Big Data Analytics in Healthcare: A Cloud-Based Framework for Generating Insights

With exabytes of data being generated from genome sequencing, a whole new science behind genomics big data has emerged. As technology improves, the cost of sequencing a human genome has gone down considerably increasing the number of genomes being sequenced. Huge amounts of genomics data along with a vast variety of clinical data cannot be handled using existing frameworks and techniques. It is to be efficiently stored in a warehouse where a number of things have to be taken into account. Firstly, the genome data is to be integrated effectively and correctly with clinical data. The other data sources along with their formats have to be identified. Required data is then to be extracted from these other sources (such as clinical data sets) and integrated with the genome. The main challenge here is to be able to handle the integration complexity as a large number of data sets are being integrated with huge amounts of genome. Secondly, since the data is captured at disparate locations individually by clinicians and scientists, it brings the challenge of data consistency. It has to be made sure that the data consistency is not compromised as it is passed along the warehouse. Checks have to be put in place to make sure the data remains consistent from start to finish. Thirdly, to carry this out effectively, the data infrastructure has to be in the correct order. How frequently the data is accessed plays a crucial role here. Data in frequent use will be handled differently than data which is not in frequent use. Lastly, efficient browsing mechanisms have to be put in place to allow the data to be quickly retrieved. The data is then iteratively analyzed to get meaningful insights. The challenge here is to perform analysis very quickly. Cloud computing plays an important role as it is used to provide scalability.

Effect of the degree of neighborhood on resource discovery in ad hoc grids

Resource management is one of the important issues in the efficient use of grid computing, in general, and poses specific challenges in the context of ad hoc grids due to the heterogeneity, dynamism, and intermittent participation of participating nodes in the ad hoc grid. In this paper, we consider three different kinds of organizations in an ad hoc grid ranging from completely centralized to completely decentralized (P2P). On the basis of self organization mechanisms, we study the effect of the neighborhood degree of a node for finding resources on the efficiency of resource allocation. We investigate the message complexity of each organization and its corresponding efficiency in terms of task/resource matching and the response time. We show that the intermediate state of the ad hoc grid with multiple adaptive matchmakers outperforms both a completely centralized and a completely decentralized (P2P) infrastructure.