Patrick Lazik

Visual light landmarks for mobile devices

The omnipresence of indoor lighting makes it an ideal vehicle for pervasive communication with mobile devices. In this paper, we present a communication scheme that enables interior ambient LED lighting systems to send data to mobile devices using either cameras or light sensors. By exploiting rolling shutter camera sensors that are common on tablets, laptops and smartphones, it is possible to detect high-frequency changes in light intensity reflected off of surfaces and in direct line-of-sight of the camera. We present a demodulation approach that allows smartphones to accurately detect frequencies as high as 8kHz with 0.2kHz channel separation. In order to avoid humanly perceivable flicker in the lighting, our system operates at frequencies above 2kHz and compensates for the non-ideal frequency response of standard LED drivers by adjusting the lights duty-cycle. By modulating the PWM signal commonly used to drive LED lighting systems, we are able to encode data that can be used as localization landmarks. We show through experiments how a binary frequency shift keying modulation scheme can be used to transmit data at 1.25 bytes per second (fast enough to send an ID code) from up to 29 unique light sources simultaneously in a single collision domain. We also show how tags can demodulate the same signals using a light sensor instead of a camera for low-power applications.

Indoor pseudo-ranging of mobile devices using ultrasonic chirps

In this paper, we present an indoor ultrasonic location tracking system that can utilize off-the-shelf audio speakers (potentially already in place) to provide fine-grained indoor position data to modern mobile devices like smartphones and tablets. We design and evaluate a communication primitive based on rate-adaptive wide-band linear frequency modulated chirp pulses that utilizes the audio bandwidth just above the human hearing frequency range where mobile devices are still sensitive. Typically transmitting data, even outside of this range, introduces broadband human audible noises (clicks) due to the non-ideal impulse response of speakers. Unlike existing audio modulation schemes, our scheme is optimized based on psychoacoustic properties. For example, all tones exhibit slowly changing power-levels and gradual frequency changes so as to minimize human perceivable artifacts. Chirps also bring the benefit of Pulse Compression, which greatly improves ranging resolution and makes them resilient to both Doppler shifts as well as multi-path propagation that typically plague indoor environments. The scheme also supports the decoding of multiple unique identifier packets being transmitted simultaneously. By applying a Time-Difference-of-Arrival (TDOA) pseudo-ranging technique the mobile devices can localize themselves without tight out-of-band synchronization with the broadcasting infrastructure. This design is not only scalable with respect to the number of transmitters and tracked devices, but also improves user privacy since the mobile devices compute their positions locally. We show through user studies and experimentation on smartphones that we are able to provide sub-meter (95% < 10cm) accurate indoor positioning in a manner that is imperceptible to humans.

ALPS: A Bluetooth and Ultrasound Platform for Mapping and Localization

The proliferation of Bluetooth Low-Energy (BLE) chipsets on mobile devices has lead to a wide variety of user-installable tags and beacons designed for location-aware applications. In this paper, we present the Acoustic Location Processing System (ALPS), a platform that augments BLE transmitters with ultrasound in a manner that improves ranging accuracy and can help users configure indoor localization systems with minimal effort. A user places three or more beacons in an environment and then walks through a calibration sequence with their mobile device where they touch key points in the environment like the floor and the corners of the room. This process automatically computes the room geometry as well as the precise beacon locations without needing auxiliary measurements. Once configured, the system can track a users location referenced to a map. The platform consists of time-synchronized ultrasonic transmitters that utilize the bandwidth just above the human hearing limit, where mobile devices are still sensitive and can detect ranging signals. To aid in the mapping process, the beacons perform inter-beacon ranging during setup. Each beacon includes a BLE radio that can identify and trigger the ultrasonic signals. By using differences in propagation characteristics between ultrasound and radio, the system can classify if beacons are within Line-Of-Sight (LOS) to the mobile phone. In cases where beacons are blocked, we show how the phones inertial measurement sensors can be used to supplement localization data. We experimentally evaluate that our system can estimate three-dimensional beacon location with a Euclidean distance error of 16.1cm, and can generate maps with room measurements with a two-dimensional Euclidean distance error of 19.8cm. When tested in six different environments, we saw that the system can identify Non-Line-Of-Sight (NLOS) signals with over 80% accuracy and track a users location to within less than 100cm.

Hybrid visible light communication for cameras and low-power embedded devices

Visible light communication (VLC) between LED light bulbs and smart-phone cameras has already begun to gain traction for identification and indoor localization applications. To support detection by cameras, the frequencies and data rates are typically limited to below 1kHz and tens of bytes per second (Bps). In this paper, we present a technique for transmitting data from solid-state luminaries, used for interior ambient lighting, simultaneously to both cameras and low-power embedded devices in a manner that is imperceptible to occupants. This allows the camera communication VLC channel to also act as a higher speed downstream link and low-power wakeup mechanism for energy-constrained devices. Our approach uses Manchester encoding and Binary Frequency Shift Keying (BFSK) to modulate the high-speed data stream and applies duty-cycle adjustment to generate the slower camera communication signal. We explore the trade-off between the performance of the two communication channels. Our hybrid communication protocol is also compatible with existing IR receivers. This allows lights to communicate with low-cost commodity chipsets and control home appliances such as TVs, AV receivers, AC window units, etc. We show that we are able to reliably simultaneously transmit low-speed data at 1.3 Bps to camera enabled devices and higher-speed data at 104 Bps to low-power embedded devices. Since the majority of energy in many RF communication protocols often goes towards media access and receiving, VLC-triggered wakeup can significantly decrease system energy consumption. We also demonstrate a proof-of-concept wakeup circuit that consumes less then 204uA and can be triggered in less then 10ms.

Ultrasonic time synchronization and ranging on smartphones

In this paper, we present the design and evaluation of a platform that can be used for time synchronization and indoor positioning of mobile devices. The platform uses the Time-Difference-Of-Arrival (TDOA) of multiple ultrasonic chirps broadcast from a network of beacons placed throughout the environment to find an initial location as well as synchronize a receivers clock with the infrastructure. These chirps encode identification data and ranging information that can be used to compute the receivers location. Once the clocks have been synchronized, the system can continue performing localization directly using Time-of-Flight (TOF) ranging as opposed to TDOA. This provides similar position accuracy with fewer beacons (for tens of minutes) until the mobile device clock drifts enough that a TDOA signal is once again required. Our hardware platform uses RF-based time synchronization to distribute clock synchronization from a subset of infrastructure beacons connected to a GPS source. Mobile devices use a novel time synchronization technique leverages the continuously free-running audio sampling subsystem of a smartphone to synchronize with global time. Once synchronized, each device can determine an accurate proximity from as little as one beacon using TOF measurements. This significantly decreases the number of beacons required to cover an indoor space and improves performance in the face of obstructions. We show through experiments that this approach outperforms the Network Time Protocol (NTP) on smartphones by an order of magnitude, providing an average 720μs synchronization accuracy with clock drift rates as low as 2ppm.

Distributed control of a swarm of buildings connected to a smart grid: demo abstract

Energy-efficient control mechanisms are necessary to manage the ever increasing energy demand. Recently several tools for building energy consumption control have been proposed for small (e.g. homes) [8] and large (e.g. offices) buildings [3][6][1]. The mechanism each tool uses is different, e.g. HVAC control [3] and appliance rescheduling [8], but they share the goal of improving consumption of the buildings with respect to a given cost function. Some examples of cost functions are reduced energy consumption, reduced electricity bill, lower peak power, and increased ancillary service participation. The tools however do not capture the impacts of their control actions on the grid. These actions can lead to supply/demand imbalance and voltage/frequency deviation and thus, threaten grid stability. Utilities can take protective actions against those who cause instability by increasing electricity price or even momentarily disconnecting them from the grid. The effects of these protective actions can be so severe that the savings obtained by building management tools might disappear.

AURES: A Wide-Band Ultrasonic Occupancy Sensing Platform

In this paper, we present a platform designed for low-power real-time sensing of the number of occupants in indoor spaces. The system transmits a wide-band ultrasonic signal into a room and then processes the superposition of the reflections recorded by a microphone. The system has two modes of operation, one for presence detection and one for estimating the number of occupants in a region. The presence detection uses the difference between multiple transmissions in succession with a set of general classifiers that make a binary decision about if the room contains occupants. We then use a semi-supervised learning approach based on Weighted Principal Component Analysis (WPCA) that requires minimal training data to estimate the number of occupants. We also present the design of an energy harvesting embedded platform and demonstrate that our algorithm can continuously execute using energy harvested from indoor solar panels. The platform has a dual Bluetooth Low-Energy and 802.15.4 interface to communicate with a gateway or nearby mobile phone that runs an interface that aids in collecting training data. We evaluate our algorithm on a wide-variety of indoor spaces as well as benchmark the hardware in terms of sampling rate given an energy budget. On more than three weeks of data, we see that we can detect motions with an average of 85% recall rate and perform occupancy counting with an average error of 10% in terms of maximum occupancy.

Mortar.io: a concrete building automation system: demo abstract

The commoditization of wireless sensing systems makes it feasible to include BAS functionality in small and medium-sized buildings. The configuration complexity and cost of installation is now the dominant barrier to adoption. In this demo we introduce a platform called Mortar.io, which focuses on ease-of-installation, secure configuration, and management of BAS sub-systems in a manner that can scale from small to large installations. Unlike cloud-reliant systems, Mortar.io distributes storage and control functionality across end devices making it robust to network and internet outages. The system, once initialized, can run autonomously on a low-cost controller within a building or connect to the cloud for remote monitoring and configuration. We will also show our efficient multi-resolution data store that buffers data locally and replicates aggregate data across devices for reliability. A publish-subscribe model built on top of XMPP is used for messaging with per-device access control and a transducer schema. Finally, a web portal provides an interface to monitor and schedule lighting, plug-loads, environmental sensors and HVAC from a single uniform interface.

Demonstration abstract: how many lights do you see?

We demonstrate a Visual Light Communication (VLC) system [1] that enables LED lighting luminaires to communicate with cameras on mobile devices. Each LED pulses at a frequency above the humanly perceivable flicker threshold where cameras and photodiodes can still detect changes in light intensity. Our modulation scheme supports multiple light sources in a single collision domain, and works for both, line-of-sight (LOS) operation as well as from reflected surfaces like those found in architectural lighting. The spatial confinement of light makes this system ideal for use as localization landmarks. Our demonstration includes four LED ambient lights acting as location landmarks transmitting modulated data. A mobile device receiving and processing the signal displays the ID and RSSI of the closest landmark. Interacting with the system will allow users to see the practical effects of multiple-access, frequency of operation, distance from the lights, camera parameters and camera motion.

Enhancing indoor smartphone location acquisition using floor plans

Indoor localization systems typically determine a position using either ranging measurements, inertial sensors, environmental-specific signatures or some combination of all of these methods. Given a floor plan, inertial and signature-based systems can converge on accurate locations by slowly pruning away inconsistent states as a user walks through the space. In contrast, range-based systems are capable of instantly acquiring locations, but they rely on densely deployed beacons and suffer from inaccurate range measurements given non-line-of-sight (NLOS) signals. In order to get the best of both worlds, we present an approach that systematically exploits the geometry information derived from building floor plans to directly improve location acquisition in range-based systems. Our solving approach can disambiguate multiple feasible locations taking into account a mix of LOS and NLOS hypotheses to accurately localize with significantly fewer beacons. We demonstrate our geometry-aware solving approach using a new ultrasonic beacon platform that is able to perform direct time-of-flight ranges on commodity smartphones. The platform uses Bluetooth Low Energy (BLE) for time synchronization and ultrasound for measuring propagation distance. We evaluate our systems accuracy with multiple deployments in a university campus and show that our approach shifts the 80% accuracy point from 4-8m to 1m as compared to solvers that do not use the floor plan information. We are able to detect and remove NLOS signals with 91.5% accuracy.