Fahim Kawsar

Smart objects as building blocks for the Internet of things

The combination of the Internet and emerging technologies such as nearfield communications, real-time localization, and embedded sensors lets us transform everyday objects into smart objects that can understand and react to their environment. Such objects are building blocks for the Internet of Things and enable novel computing applications. As a step toward design and architectural principles for smart objects, the authors introduce a hierarchy of architectures with increasing levels of real-world awareness and interactivity. In particular, they describe activity-, policy-, and process-aware smart objects and demonstrate how the respective architectural abstractions support increasingly complex application.

The Internet of Things: The Next Technological Revolution

A wide range of researchers from academia and industry, as well as businesses, government agencies, and cities, are exploring the technologies comprising the Internet of Things from three main perspectives: scientific theory, engineering design, and the user experience.

AwareMirror: a personalized display using a mirror

In this paper, we propose a personalized display, “AwareMirror: an augmented mirror”. AwareMirror presents information relevant to a person in front of it by super-imposing his/her image. A toothbrush has been chosen as an identification tool while proximity sensors have been utilized to detect a persons position (in front of the mirror). Also, three types of information that can affect a users decision have been selected. The mirror has been constructed using an acrylic magic mirror board and ordinal computer monitor. The acrylic board has been attached in front of the monitor, and only bright color from the display can penetrate the board. As a result of preliminary evaluation, we found that the mirror is useful to offer information in an unobtrusive manner while preserving its metaphor.

DeepX: a software accelerator for low-power deep learning inference on mobile devices

Breakthroughs from the field of deep learning are radically changing how sensor data are interpreted to extract the high-level information needed by mobile apps. It is critical that the gains in inference accuracy that deep models afford become embedded in future generations of mobile apps. In this work, we present the design and implementation of DeepX, a software accelerator for deep learning execution. DeepX signif- icantly lowers the device resources (viz. memory, computation, energy) required by deep learning that currently act as a severe bottleneck to mobile adoption. The foundation of DeepX is a pair of resource control algorithms, designed for the inference stage of deep learning, that: (1) decompose monolithic deep model network architectures into unit- blocks of various types, that are then more efficiently executed by heterogeneous local device processors (e.g., GPUs, CPUs); and (2), perform principled resource scaling that adjusts the architecture of deep models to shape the overhead each unit-blocks introduces. Experiments show, DeepX can allow even large-scale deep learning models to execute efficently on modern mobile processors and significantly outperform existing solutions, such as cloud-based offloading.

Deploy spontaneously: supporting end-users in building and enhancing a smart home

This paper explores system issues for involving end users in constructing and enhancing a smart home. In support of this involvement we present an infrastructure and a tangible deployment tool. Active participation of users is essential in a domestic environment as it offers simplicity, greater usercentric control, lower deployment costs and better support for personalization. Our proposed infrastructure provides the foundation for end user deployment utilizing a loosely coupled framework to represent an artefact and its augmented functionalities. Pervasive applications are built independently and are expressed as a collection of functional tasks. A runtime component, FedNet maps these tasks to corresponding service provider artefacts. The tangible deployment tool uses FedNet and allows end users to deploy and control artefacts and applications only by manipulating RFID cards. Primary advantages of our approach are two-fold. Firstly, it allows end users to deploy ubicomp systems easily in a Do-it-Yourself fashion. Secondly, it allows developers to write applications and to build augmented artefacts in a generic way regardless of the constraints of the target environment. We describe an implemented prototype and illustrate its feasibility in a real life deployment session by the end users. Our study shows that the end users might be involved in deploying future ubicomp systems if appropriate tools and supporting infrastructure are provided.

Augmenting everyday life with sentient artefacts

The paper introduces sentient artefacts, our everyday life objects augmented with sensors to provide value added services. Such artefacts can be used to capture users context in an intuitive way, as they do not require any explicit interactions. These artefacts enable us to develop context aware application by capturing everyday scenarios effectively. In the paper we present a daily life scenario, and then demonstrate how such scenarios can be implemented effectively using applications that integrate multiple sentient artefacts.

An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices

Detecting and reacting to user behavior and ambient context are core elements of many emerging mobile sensing and Internet-of-Things (IoT) applications. However, extracting accurate inferences from raw sensor data is challenging within the noisy and complex environments where these systems are deployed. Deep Learning -- is one of the most promising approaches for overcoming this challenge, and achieving more robust and reliable inference. Techniques developed within this rapidly evolving area of machine learning are now state-of-the-art for many inference tasks (such as, audio sensing and computer vision) commonly needed by IoT and wearable applications. But currently deep learning algorithms are seldom used in mobile/IoT class hardware because they often impose debilitating levels of system overhead (e.g., memory, computation and energy). Efforts to address this barrier to deep learning adoption are slowed by our lack of a systematic understanding of how these algorithms behave at inference time on resource constrained hardware. In this paper, we present the first -- albeit preliminary -- measurement study of common deep learning models (such as Convolutional Neural Networks and Deep Neural Networks) on representative mobile and embedded platforms. The aim of this investigation is to begin to build knowledge of the performance characteristics, resource requirements and the execution bottlenecks for deep learning models when being used to recognize categories of behavior and context. The results and insights of this study, lay an empirical foundation for the development of optimization methods and execution environments that enable deep learning to be more readily integrated into next-generation IoT, smartphones and wearable systems.

Home computing unplugged: why, where and when people use different connected devices at home

We investigate how technology usage in homes has changed with the increasing prevalence of mobile devices including Tablets and Smart Phones. We logged Internet usage from 86 Belgium households to determine their six most common Internet Activities. Next, we surveyed households about what devices they own, how they share those devices, and which device they use for different Internet activities. We then conducted semi-structured interviews with 18 of 55 households that responded to the survey in which participants explained their device usage patterns and where they use technology in their home. Our findings suggest that the nature of online activity and social context influence device preference. Many participants reported that their Desktop PC is now a special purpose device, which they use only for specific activities such as working from home or online gaming. Compared to past studies, we observed technology use in many more locations in the home, most notably kitchens and bathrooms.

Design and implementation of a framework for building distributed smart object systems

A smart object system encompasses the synergy between computationally augmented everyday objects and external applications. This paper presents a software framework for building smart object systems following a declarative programming approach centered around custom written documents that glue the smart objects together. More specifically, in the proposed framework, applications’ requirements and smart objects’ services are objectified through structured documents. A runtime infrastructure provides the spontaneous federation between smart objects and applications through structural type matching of these documents. There are three primary advantages of our approach: firstly, it allows developers to write applications in a generic way without prior knowledge of the smart objects that could be used by the applications. Secondly, smart object management (locating, accessing, etc.) issues are completely handled by the infrastructure; thus application development becomes rapid and simple. Finally, the programming abstraction used in the framework allows extension of functionalities of smart objects and applications very easily. We describe an implemented prototype of our framework and show examples of its use in a real life scenario to illustrate its feasibility.

An explorative comparison of magic lens and personal projection for interacting with smart objects

One shortcoming of self-describing smart objects augmented with digital resources is the limitation of output modalities due to their long established physical appearances. To overcome this drawback intangible representations e.g., sound, video projection etc. are usually coupled with the tangible representations of smart objects that enable access and interaction with their value added features. In this paper, we explore two mobile interaction techniques that associate such intangible representation to smart objects using a pico projector augmented camera phone. The first technique utilizes a Magic Lens metaphor applying mobile augmented reality (contextual information is overlaid while looking at a smart object through camera) to uncover and interact with smart objects. The second technique, Personal Projection follows similar mechanisms in discovery and interaction, except information is projected onto the nearest surface. We report the implementation of these two techniques and a comparative qualitative study with three prototype smart object applications. The findings give us deeper insights on the positive and negative aspects of these two techniques and open up a range of stimulating research issues that we discuss in the paper.