Henry Holtzman

BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields

We transform an LCD into a display that supports both 2D multi-touch and unencumbered 3D gestures. Our BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect multiple points of contact. Our key contribution is to exploit the spatial light modulation capability of LCDs to allow lensless imaging without interfering with display functionality. We switch between a display mode showing traditional graphics and a capture mode in which the backlight is disabled and the LCD displays a pinhole array or an equivalent tiled-broadband code. A large-format image sensor is placed slightly behind the liquid crystal layer. Together, the image sensor and LCD form a mask-based light field camera, capturing an array of images equivalent to that produced by a camera array spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points. Two motivating applications are described: a hybrid touch plus gesture interaction and a light-gun mode for interacting with external light-emitting widgets. We show a working prototype that simulates the image sensor with a camera and diffuser, allowing interaction up to 50 cm in front of a modified 20.1 inch LCD.

neXtream: A Multi-Device, Social Approach to Video Content Consumption

Video content has become available on an increasingly diverse set of devices and from an ever growing number of sources, creating a vast amount of choice for viewers. At the same time, the varying methods of viewing, interacting with, and sharing content have diverged. This paper introduces neXtream, a new approach to delivering video by integrating multiple devices, content sources, and social networks. This concept is developed following research in social television and converged applications, providing both personalization features and social interaction. NeXtream delivers video by dynamically generating streams of video customized to a viewer, while facilitating a common dialog between users around the content, creating both a userand community-centric viewing experience. NeXtream integrates smartphones, PCs, and TVs to deliver video content to viewers. The paper presents the system concept, theory, and architecture, and describes the developed prototype.

Near-invariant blur for depth and 2D motion via time-varying light field analysis

Recently, several camera designs have been proposed for either making defocus blur invariant to scene depth or making motion blur invariant to object motion. The benefit of such invariant capture is that no depth or motion estimation is required to remove the resultant spatially uniform blur. So far, the techniques have been studied separately for defocus and motion blur, and object motion has been assumed 1D (e.g., horizontal). This article explores a more general capture method that makes both defocus blur and motion blur nearly invariant to scene depth and in-plane 2D object motion. We formulate the problem as capturing a time-varying light field through a time-varying light field modulator at the lens aperture, and perform 5D (4D light field + 1D time) analysis of all the existing computational cameras for defocus/motion-only deblurring and their hybrids. This leads to a surprising conclusion that focus sweep, previously known as a depth-invariant capture method that moves the plane of focus through a range of scene depth during exposure, is near-optimal both in terms of depth and 2D motion invariance and in terms of high-frequency preservation for certain combinations of depth and motion ranges. Using our prototype camera, we demonstrate joint defocus and motion deblurring for moving scenes with depth variation.

ShAir: extensible middleware for mobile peer-to-peer resource sharing

ShAir is a middleware infrastructure that allows mobile applications to share resources of their devices (e.g., data, storage, connectivity, computation) in a transparent way. The goals of ShAir are: (i) abstracting the creation and maintenance of opportunistic delay-tolerant peer-to-peer networks; (ii) being decoupled from the actual hardware and network platform; (iii) extensibility in terms of supported hardware, protocols, and on the type of resources that can be shared; (iv) being capable of self-adapting at run-time; (v) enabling the development of applications that are easier to design, test, and simulate. In this paper we discuss the design, extensibility, and maintainability of the ShAir middleware, and how to use it as a platform for collaborative resource-sharing applications. Finally we show our experience in designing and testing a file-sharing application.

Lightweight Self-organizing Reconfiguration of Opportunistic Infrastructure-mode WiFi Networks

The purpose of this work is to provide a method for exploiting pervasive wireless communication capabilities that are often underutilized on smart devices (e.g., phones, tables, cameras, TVs, etc.) in an opportunistic and collaborative way. This goal can be accomplished by sharing device resources using their built-in WiFi adapter. In this paper we explain why the standard ad-hoc mode for building mobile peer-to-peer networks is not always the best choice and we propose an alternative self-organizing approach in which an opportunistic infrastructure-mode WiFi network is built. The particularity of this network is that each device can either be an access point or a client and change its role and wireless channel over time. This contribution advances the state of the art by using a context-aware approach that considers actual frequency allocation to other devices and monitored traffic. We finally show that our approach increases the average speed for delivering messages to a level that in several situations outperforms previous work in the area, as well as a simple single-channel ad-hoc WiFi network.

Newstream: a multi-device, cross-medium, and socially aware approach to news content

News content extends well beyond the scope of a single medium: video, audio, text, interactive experiences, and social sharing all contribute to an individuals understanding and experience of content. While some content that exists across media can supplement other forms, such as an interactive feature accompanying a text-based article, other content might replicate information and therefore be redundant to a consumer. This paper introduces Newstream, a novel approach to navigating news-based content across a variety of media sources. Building off of prior work, neXtream, Newstream can craft specialized experiences that both facilitate common dialogs within social networks and allow users to dynamically move across various mediums, controlling levels of depth based on interest while avoiding repetition. Newstream integrates multiple devices and networks to maintain unified back-end knowledge about each user.

Kairoscope: managing time perception and scheduling through social event coordination

If everyone says time is relative, why is it still so rigidly defined? There have been many attempts to address the issue of coordinating schedules, but each of these attempts runs into an issue of rigidity: in order to negotiate an event, a specific time must be designated in advance. This model is inherently poor at accommodating lifes unpredictability. Kairoscope looks at time from a human perspective, focusing on time as made up of a series of events, rather than simply a series of events in time. By removing our reliance on a fixed time system, events are coordinated socially and on the fly, without worrying about precision. This paper explores the creation of Kairoscope, rooted in ideas of time perception and aiming to reduce time-related stress, optimize time usage, and increase social interaction. The result is a socially-coordinated, constantly adapting, and highly malleable guide through time.

OpenTag: Privacy Protection for RFID

Radio frequency identifications use in retail is good for pervasive computing, but raises considerable privacy issues. OpenTag programmable tags address privacy issues while remaining fully compatible with the supply-chain RFID standard.

8D: interacting with a relightable glasses-free 3D display

We present an 8-dimensional (8D) display that allows glasses-free viewing of 3D imagery, whist capturing and reacting to incident environmental and user controlled light sources. We demonstrate two interactive possibilities enabled by our lens-array-based hardware prototype, and realtime GPU-accelerated software pipeline. Additionally, we describe a path to deploying such displays in the future, using current Sensor-in-Pixel (SIP) LCD panels, which physically collocate sensing and display elements.

CommenTV: A time-sensitive social commenting system for audiovisual content

CommenTV enables users to share their comments in parallel with time-based content. CommenTV utilizes existing web technologies to enhance current TV or video player solutions with a new type of comment system. CommenTV is able to take and display texts, images, and related videos as social comments. In this way, the CommenTV users can experience audiovisual content that blends with their social networks by sharing comments. Ultimately, CommenTV can be a social bookmarking system for audiovisual content that informs content creators about how viewers perceive their content and lets viewers participate in creating a more social content.