Swarnava Dey

ANGELS for distributed analytics in IoT

The current global emphasis on “Internet of Things (IoT)” have highlighted the extreme importance of sensor-based intelligent and ubiquitous systems which are more commonly known as “cyber-physical systems.” The technology has the potential to create a network of smart devices and things to an extent that has never been envisaged before, far outnumbering the number of devices connected in the Internet as we know today. The sheer number of such connected ubiquitous devices is likely to give rise to a hitherto unforeseen volume of data of different types with a demand for execution of analytical algorithms over the data. On the success of these analytic processes will depend the actual “smartness” of the “Intelligent Infrastructures” which now form the crux of the IoT paradigm. We have seen the advent of cloud-based paradigms to analyse the data in a data-parallel fashion within large data centres which now form the basis of the “big-data” problem. But apart from the servers in the data centres, we potentially have a huge pool of compute resources if we think about the smart devices in and around our homes collectively, which remain relatively idle. In this paper, we present a proposal with some emulated experimental results where we claim that in an IoT framework, the smart devices such as mobile phones, home gateways etc. can be utilised for execution of dataparallel analytic jobs. This is effectively a work-in-progress and it is acknowledged that there will be further challenges for real devices. Future research will attempt to consider these challenges.

Smart city surveillance: Leveraging benefits of cloud data stores

The smart cities of future need to have a robust and scalable video surveillance infrastructure. In addition it may also make use of citizen contributed video feeds, images and sound clips for surveillance purposes. Multimedia data from various sources need to be stored in large scalable data stores for compulsory retention period, on-line, off-line analytics and archival. Multimedia feeds related to surveillance are voluminous and varied in nature. Apart from large multimedia files, events detected using video analytics and associated metadata needs to be stored. The underlying data storage infrastructure therefore needs to be designed for mainly continuous streaming writes from video cameras and some variety in terms of I/O sizes, read-write mix, random vs. sequential access. As of now, the video surveillance storage domain is mostly dominated by iSCSI based storage systems. Cloud based storage is also provided by some vendors. Taking in account the need for scalability, reliability and data center cost minimization, it is worth investigating if large scale video surveillance backend can be integrated to the open source cloud based data stores available in the “big data” trend. We developed a multimedia surveillance backend system architecture based on the Sensor Web Enablement framework and cloud based “key-value” stores. Our framework gets data from camera/ edge device simulators, splits media files and metadata and stores those in a segregated way in cloud based data stores hosted on Amazons EC2. We have benchmarked performances of a few cloud based key-value stores under large scale video surveillance workload and demonstrated that those perform satisfactorily, bringing in inherent scalability and reliability of a cloud based storage system to a video surveillance system for a smart safe city. With a case study of the storage of video surveillance system, we show in this paper that with the availability of several cloud based distributed data stores and benchmarking tools, an applications data management needs can be served using hybrid cloud based data stores and selection of such stores can be facilitated using benchmark tools if the application workload characteristics are known.

Challenges of Using Edge Devices in IoT Computation Grids

Internet of Things (IoT) has the potential to become a technology revolution with a vision of creating very large scale network, comprising of unprecedented number of connected devices. These devices, often referred to as smart items or intelligent things can be home appliances, healthcare devices, vehicles, buildings, factories and almost anything networked and fitted with sensors, actuators, embedded computers. There has been sustained research work and standardization effort from different IoT perspectives like integration of sensor and RFID devices to the Internet. With the increasing trend of gathering business insights from unstructured data, the high volume of data generated by such devices is also of interest. Cloud based data mining platforms are suitable for analyses of such data and researchers have proposed architectures where personal mobile phones can act as Edge Gateway between the sensor network and cloud analytics platform. It seems that the surge in the volume of data generated by huge number of Smart Items can only be matched if a large percentage of mobile users start sharing the computation capability of their personal devices and work together towards true Participatory Computing in the IoT systems. In this work we try to understand the challenges associated with running computation jobs on the mobile devices using different types of workload often observed in IoT applications. Based on the insights gained from experiments performed by us, we propose a scheme where mobile phones, residential gateways and other edge devices offer free slots to servers in a cloud based data analytics system. Based on the free time slots offered by the mobile phones, if commensurately sized computational jobs can be scheduled, the unpredictability associated with using mobile phones as grid resources can be solved.

Utilising condor for data parallel analytics in an IoT context — An experience report

The current emphasis on sensor-based intelligent and ubiquitous systems, more commonly known as “cyber-physical systems”, has the potential to give rise to a new generation of systems and services encompassing several domains such as e-Governance, healthcare, transportation, waste management, energy & utilities, insurance, etc., resulting in the metamorphosis of the Internet as we see it, into the Internet of Things (IoT). One probable commonality in each of these services will be the abundance of different types of data from different sources with the success of the systems depending on real-time or near real-time analysis of data. Such analyses are normally performed via well-known algorithms with a time-constraint on the execution, thus creating a requirement for parallel execution techniques. Some of these analyses may have a higher frequency of execution on a relatively small set of data, in which case the current big-data frameworks may actually add an overhead. Further, the frameworks like Hadoop demand the algorithms to be mapped onto a particular paradigm, which may not always be a suitable option. This paper, which is a work-in-progress, provides an experience report on the use of Condor, a well known Grid framework, for data-parallel “black-box” style execution of analysis algorithms in the context of Internet of Things. We concentrate on algorithms which are already in use, and can be partitioned into data-parallel subtasks without any modification and use Condor, which has traditionally been used for high-performance or high-throughput computing, as the execution framework.

ANGELS: A framework for mobile grids

The current emphasis on Internet of Things (IoT) across the globe highlights the extreme importance of sensor-based intelligent and ubiquitous systems which are collectively known as cyber-physical systems. It is envisaged that the technology will create a network of smart devices and things to an unprecedented extent, far outnumbering the number of devices connected in the Internet as we know today. This unprecedented number of interconnected smart devices is likely to generate an unforeseen volume of data of different types, ready for analysis. The smartness of the Intelligent Infrastructures will depend on effective execution of various analytical algorithms on this accumulated dataset which forms the crux of the IoT paradigm. As of now, researchers rely on cloud-based paradigms to analyse the data in a data-parallel fashion, where analyses take place in large data centres comprised of large number of servers. Apart from the servers, there is a huge pool of relatively unused pool of computing resources in form of smart devices in and around our homes, such as mobile phones, home energy gateways, play-stations etc., which we collectively call edge devices. It is our belief that this pool of resources, if used in the right manner, can contribute to a large extent to effective and successful analysis of the data-set generated by the ubiquitous sensors, leading to a proper realization of the Intelligent Infrastructure paradigm, and we call this approach ANGELS computing. In this paper, we describe a framework which allows remote execution of programs within mobile devices and thus can act as one of the building blocks for the IoT paradigm. This is effectively a work-in-progress and it is acknowledged that there will be further challenges in such a framework and we will attempt to solve them during our course of research.

Offloading work to mobile devices: an availability-aware data partitioning approach

In the context of Internet of Things (IoT), data acquisition, management, and analysis for knowledge extraction has given rise to a new generation of services. The heterogeneity of services being offered today on the enormous expanse of data as available from sensors and smart devices, needs a distributed infrastructure for analysis and computation. We consider a scenario where the computing infrastructure of an IoT framework is distributed and is capable of harnessing the computing power of mobile devices connected to the network in a heterogeneous grid. Edge devices like smart-phones, gateways, etc. can potentially contribute to the infrastructure. However, such devices and communication channels to such devices are intermittently unavailable, which may be due to network failure, planned shut-down, high workload on the devices, etc. In addition to this, we consider a task offloading context where the computing infrastructure inside the IoT invites bids from connected and available devices from the network to offload a part of the computation on part of the input data-set. We expect the overall completion time to improve in this setting. We investigate the data partitioning problem under the scenario where unavailability of communication and computation are advertised a priori and we pose the problem as a scheduling problem where cost of execution of an analysis job is to be minimized. We present a constraint-based model, evaluate the same on various scenarios and present some of the results.

CAD Patient Classification Using MIMIC-II

With availability of large volume of collected data from healthcare centers and significant improvement in computation power, evidence based learning is helping in building robust disease diagnostic models.

A Framework for Speculative Scheduling and Device Selection for Task Execution on a Mobile Cloud

In this paper, we study the problem of opportunistic task scheduling and workload management in a mobile cloud setting considering computation power variation. We gathered mobile usage data for a number of persons and applied supervised clustering to show that a pattern of usage exists and that follows a state-based model. Based on this model, we present a strategy to choose and offload work on a mobile device. We present a framework and experimental results showing the efficacy of our proposed approach.

An Erasure Coded Archival Storage System

There is an ever increasing need of storage capacity for storage of digital archives and historical data-digital preservation, because of regulatory and compliance requirements. There is an increasing interest in disk based archival system. Major technical challenges in creating large disk based storage archive are - providing large capacity at low costs, large read and write throughput, data integrity and sustaining hardware and operating system refresh. In this paper we present the architecture and working principle of an archival storage system that uses an erasure-coded redundancy scheme. We present the design of a Quality of Service (QoS) framework that tries to achieve an optimum balance between file availability, performance and system availability. The design includes a file encoding and placement scheme that allows files to be read from the archive without the need to access any metadata. Finally, we present the results obtained from running an experimental setup on Amazon Web Services.

Partitioning of CNN Models for Execution on Fog Devices

Fog Computing has in recent times captured the imagination of industrial and research organizations working on various aspects of connected livelihood and governance of smart cities. Improvements in deep neural networks imply extensive use of such models for analytics and inferencing on large volume of data, including sensor observations, images, speech. A growing need for such inferencing to be run on devices closer to the data sources, i.e. devices which reside at the edge of the network, popularly known as fog devices exists, in order to reduce the upstream network traffic. However, being computationally constrained in nature, executing complex deep inferencing models on such devices has been proved difficult. This has led to several new approaches to partition/distribute the computation and/or data over multiple fog devices. In this paper we propose a novel depth-wise input partitioning scheme for CNN models and experimentally prove that it achieves better performance compared to row/column or grid based schemes.