Elias De Coninck

Vision: smart home control with head-mounted sensors for vision and brain activity

Today, an increasing number of household appliances is being connected to the Internet to form a smart home. Intelligent control algorithms in the cloud adapt the configuration of this Internet-of-Things to our daily routines and personal preferences. Frequently, there are unforeseen situations where the control algorithms will not capture the actual desired configuration. In these cases, the user must intervene in the control algorithms and manually adjust the connected objects setting. Browsing to the appropriate web service or launching the vendor-specific companion app for even a simple interaction like lowering the temperature setting is a tedious process. In this paper, we report on our early insights in building a mobile system that provides a common, intuitive interface to all actuators in the smart home. Using a head-mounted camera and a commercial Emotiv EEG neuro-headset, we let the user configure the IoT by merely looking at an object and performing a related facial expression. This way, users only need to look at an object and think about the desired action. We leverage on the home cloudlet for the compute-intensive signal processing for object detection.

Enabling component-based mobile cloud computing with the AIOLOS middleware

Currently, mobile and wearable devices (such as smartphones and tablets) and cloud computing are converging in the new, rapidly growing field of mobile cloud computing. Emerging distributed cloud architectures such as edge clouds can be used to support and scale out resource-intensive, low-latency mobile applications. However, at the moment, a lot of burden is put on the application developer in order to develop and deploy distributed cloud-enabled mobile applications. Therefore, we present AIOLOS: an integrated middleware platform that supports transparent distributed deployment and scaling among mobile devices and cloud infrastructures. To evaluate the middleware, we show experimental results of AIOLOS using a complex 3D mapping use case.

Distributed Neural Networks for Internet of Things: The Big-Little Approach

Nowadays deep neural networks are widely used to accurately classify input data. An interesting application area is the Internet of Things (IoT), where a massive amount of sensor data has to be classified. The processing power of the cloud is attractive, however the variable latency imposes a major drawback in situations where near real-time classification is required. In order to exploit the apparent trade-off between utilizing the stable but limited embedded computing power of IoT devices and the seemingly unlimited computing power of Cloud computing at the cost of higher and variable latency, we propose a Big-Little architecture for deep neural networks. A small neural network trained to a subset of prioritized output classes is running on the embedded device, while a more specific classification is calculated when required by a large neural network in the cloud. We show the applicability of this concept in the IoT domain by evaluating our approach for state of the art neural network classification problems on popular embedded devices such as the Raspberry Pi and Intel Edison.

DIANNE: a modular framework for designing, training and deploying deep neural networks on heterogeneous distributed infrastructure

Deep learning has shown tremendous results on various machine learning tasks, but the nature of the problems being tackled and the size of state-of-the-art deep neural networks often require training and deploying models on distributed infrastructure. DIANNE is a modular framework designed for dynamic (re)distribution of deep learning models and procedures. Besides providing elementary network building blocks as well as various training and evaluation routines, DIANNE focuses on dynamic deployment on heterogeneous distributed infrastructure, abstraction of Internet of Things (loT) sensors, integration with external systems and graphical user interfaces to build and deploy networks, while retaining the performance of similar deep learning frameworks. In this paper the DIANNE framework is proposed as an all-in-one solution for deep learning, enabling data and model parallelism though a modular design, offloading to local compute power, and the ability to abstract between simulation and real environment

The cascading neural network: building the Internet of Smart Things

Most of the research on deep neural networks so far has been focused on obtaining higher accuracy levels by building increasingly large and deep architectures. Training and evaluating these models is only feasible when large amounts of resources such as processing power and memory are available. Typical applications that could benefit from these models are, however, executed on resource-constrained devices. Mobile devices such as smartphones already use deep learning techniques, but they often have to perform all processing on a remote cloud. We propose a new architecture called a cascading network that is capable of distributing a deep neural network between a local device and the cloud while keeping the required communication network traffic to a minimum. The network begins processing on the constrained device, and only relies on the remote part when the local part does not provide an accurate enough result. The cascading network allows for an early-stopping mechanism during the recall phase of the network. We evaluated our approach in an Internet of Things context where a deep neural network adds intelligence to a large amount of heterogeneous connected devices. This technique enables a whole variety of autonomous systems where sensors, actuators and computing nodes can work together. We show that the cascading architecture allows for a substantial improvement in evaluation speed on constrained devices while the loss in accuracy is kept to a minimum.

Multi-fidelity matryoshka neural networks for constrained IoT devices.

Using deep neural networks on resource constrained devices is a trending topic in neural network research. Various techniques for compressing neural networks have been proposed that allow evaluating a large neural network on a device with limited memory and processing power. These approaches usually generate a single compressed student network based on a larger teacher network. In some cases a more dynamic trade-off may be desired. In this paper we trained a sequence of increasingly large networks where each network is constrained to contain the unmodified features of all smaller networks. The weight matrix of the largest network has submatrices that correspond to the weight matrices of each of the smaller networks. This technique allows us to keep the parameters of several networks in memory while having the same memory footprint as the single largest network. A trade-off between accuracy and speed can be made at runtime. The proposed approach is validated on two image classification tasks running on a real-world Internet-of-Things (IoT) device.

Middleware Platform for Distributed Applications Incorporating Robots, Sensors and the Cloud

Cyber-physical systems (CPS) in the factory of thefuture will consist of cloud-hosted software governing an agileproduction process that is executed by mobile robots and thatis controlled by analyzing the data from a vast number ofsensors. CPSs thus operate on a distributed production floorinfrastructure and the set-up continuously changes with eachnew manufacturing task. In this paper, we present our OSGibasedmiddleware that abstracts the deployment of servicebasedCPS software components on a distributed platformcomprising robots, actuators, sensors and the cloud. Moreover, our middleware provides specific support to develop componentsbased on artificial neural networks, a technique that recentlybecame very popular for sensor data analytics and robot control. We demonstrate a system where a robot takes actions based onthe input from sensors in its vicinity.

Androsgi: bringing the power of OSGi to Android

Nowadays, Android is by far the most popular operating system on a myriad of mobile devices. Although based on the Java programming language, Android does not offer the same benefits as a dynamic software module system such as OSGi. Nevertheless OSGi would have many advantages on mobile environments. Software modularity allows you to only deploy the required pieces of software at runtime. Moreover, many of the enterprise OSGi specifications facilitate the development of distributed applications connecting to the Cloud. In order to exploit these advantages on Android, we present Androsgi, an Eclipse IDE plugin that allows you to easily run OSGi on top of Android. The Androsgi plugin facilitates deploying OSGi bundles, and calling both local and remote OSGi services from within your Android application.