Jaron Lanier

A hand gesture interface device

This paper reports on the development of a hand to machine interface device that provides real-time gesture, position and orientation information. The key element is a glove and the device as a whole incorporates a collection of technologies. Analog flex sensors on the glove measure finger bending. Hand position and orientation are measured either by ultrasonics, providing five degrees of freedom, or magnetic flux sensors, which provide six degrees of freedom. Piezoceramic benders provide the wearer of the glove with tactile feedback. These sensors are mounted on the light-weight glove and connected to the driving hardware via a small cable. Applications of the glove and its component technologies include its use in conjunction with a host computer which drives a real-time 3-dimensional model of the hand allowing the glove wearer to manipulate computer-generated objects as if they were real, interpretation of finger-spelling, evaluation of hand impairment in addition to providing an interface to a visual programming language.

Reality built for two: a virtual reality tool

Researchers have been working with head mounted displays and virtual reality (VR) since 1965 when Ivan Sutherland published his first paper on the subject1. This work has centered on a single user within virtual space. The literature has covered applications such as telerobotics, virtual control panels, architectural simulation and scientific visualization.Several factors now militate toward broadening the range of VR applications: developments in hardware and software, as well as a growing readiness in many fields to incorporate VR. User interface constraints of VR systems need to change in response to this changing user profile. Design of environments and behaviors in virtual worlds should be simple and accessible to experts in many fields. With the advent of multi-user systems, communications will become a major application of virtual reality.VPL has recently developed a system that allows more than one user to share a virtual space. The forms and behaviors of virtual worlds are specified graphically, so that non-programmers can design them. The system, called Reality Built for Two, will be demonstrated at the Symposium on Interactive 3D Graphics.

Homuncular Flexibility in Virtual Reality

Immersive virtual reality allows people to inhabit avatar bodies that differ from their own, and this can produce significant psychological and physiological effects. The concept of homuncular flexibility Lanier, 2006 proposes that users can learn to control bodies different from their own by changing the relationship between tracked and rendered motion. We examine the effects of remapping movements in the real world onto an avatar that moves in novel ways. In Experiment 1, participants moved their legs more than their arms in conditions where leg movements were more effective for the task. In Experiment 2, participants controlling 3-armed avatars learned to hit more targets than participants in 2-armed avatars. We discuss the implications of embodiment in novel bodies.

Model of Illusions and Virtual Reality

In Virtual Reality (VR) it is possible to induce illusions in which users report and behave as if they have entered into altered situations and identities. The effect can be robust enough for participants to respond “realistically,” meaning behaviors are altered as if subjects had been exposed to the scenarios in reality. The circumstances in which such VR illusions take place were first introduced in the 80s. Since then, rigorous empirical evidence has explored a wide set of illusory experiences in VR. Here, we compile this research and propose a neuroscientific model explaining the underlying perceptual and cognitive mechanisms that enable illusions in VR. Furthermore, we describe the minimum instrumentation requirements to support illusory experiences in VR, and discuss the importance and shortcomings of the generic model.

Appearance and task success in novel avatars

Novel avatar bodies are ones that are not controlled in a one-to-one relationship between the user’s body and the avatar body, for example, when the avatar’s arms are controlled by the user’s legs, or, when the avatar has a third arm. People have been shown to complete tasks more successfully when controlling novel avatar bodies than when controlling avatars that conform to the normal human configurations, when those novel avatars are better suited to the task (Won, Bailenson, Lee, & Lanier, 2015). However, the novel avatars in such studies tend to follow two conventions. First, the novel avatars still resemble biological forms, and second, the novel extensions of the avatar are connected to the avatar body. In the following study, participants operated bodies with three arms. We examined the interaction between biological appearance of the third arm and whether it was attached to the body. There was a significant effect of biological appearance on performance, such that participants inhabiting an avatar with a biological appearance did worse overall. There was also an interaction with biological appearance and an extension that appeared detached from the participant’s body such that participants in this condition performed most poorly overall. We propose a relationship between self-reported presence and task success, and discuss the implications of these findings for the design, implementation, and use of novel avatars.

HoloARt: Painting with Holograms in Mixed Reality

We propose HoloARt, a new media art that explores the use of holograms in mixed reality for creative self expression. We designed a system that allows the user to turn their physical environment into a canvas where digital holograms and physical objects co-exist in the real and virtual world. Users are able to virtually spray and splatter hologram paint on top of physical objects and surfaces as well as painting in the air by only using their hands. The content grows dynamically, following the natural movements of the user. The system is self-contained and does not require hand controllers nor positional tracking sensors on the space.

The renaissance of VR: are we going to do it right this time?

After being dormant in the public eye for more than 10 years, VR is now back to the front pages re-discovered as the portal to push the limits of our imagination towards compelling immersive experiences that can also blend with the physical world. Well-established companies and strongly backed new companies are committing serious investments into VR and its applications. This second VR wave is coming strong and fast in making VR accessible for the general public, not as the specialized technology for a few elite research labs of the 90s and early 2000s.

Virtual image display as a backlight for 3D.

We describe a device which has the potential to be used both as a virtual image display and as a backlight. The pupil of the emitted light fills the device approximately to its periphery and the collimated emission can be scanned both horizontally and vertically in the manner needed to illuminate an eye in any position. The aim is to reduce the power needed to illuminate a liquid crystal panel but also to enable a smooth transition from 3D to a virtual image as the user nears the screen.

Future tense Confusions of the hive mind

Future Tense, one of the revolving features on this page, presents stories and essays from the intersection of computational science and technological speculation, their boundaries limited only by our ability to imagine what will and could be.Cherish the individual.

HoloARt Video Showcase: Painting with Holograms in Mixed Reality

We propose HoloARt, a new media art that explores the use of holograms in mixed reality for creative self expression. We designed a system that allows the user to turn their physical environment into a canvas where digital holograms and physical objects co-exist in the real and virtual world. Users are able to virtually spray and splatter hologram paint on top of physical objects and surfaces as well as painting in the air by only using their hands. The content grows dynamically, following the natural movements of the user. The system is self-contained and does not require hand controllers or positional tracking sensors on the space. This video accompanies our CHI paper and Art Installation [1].