Klen Čopič Pucihar

Evaluating dual-view perceptual issues in handheld augmented reality: device vs. user perspective rendering

In handheld Augmented Reality (AR) the magic-lens paradigm is typically implemented by rendering the video stream captured by the back-facing camera onto the devices screen. Unfortunately, such implementations show the real world from the devices perspective rather than the users perspective. This dual-perspective results in misaligned and incorrectly scaled imagery, a predominate cause for the dual-view problem with potential to distort users spatial perception. This paper presents a user study that analyzes users expectations, spatial-perception, and their ability to deal with the dual-view problem, by comparing device-perspective and fixed Point-of-View (POV) user-perspective rendering. The results confirm the existence of the dual-view perceptual issue and that the majority of participants expect user-perspective rendering irrespective of their previous AR experience. Participants also demonstrated significantly better spatial perception and preference of the user-perspective view.

The use of surrounding visual context in handheld AR: device vs. user perspective rendering

The magic lens paradigm, a commonly used descriptor for handheld Augmented Reality (AR), presents the user with dual views: the augmented view (magic lens) that appears on the device, and the real view of the surroundings (what the user can see around the perimeter of the device). The augmented view is typically implemented by rendering the video captured by the rear-facing camera directly onto the devices screen. This results in dual perspectives - the real world being captured from the devices perspective rather than the users perspective (what an observer would see looking through a transparent glass pane). These differences manifest themselves in misaligned and/or incorrectly scaled transparency resulting in the dual-view problem. This paper presents two user studies comparing (a) device-perspective and (b) fixed Point-of-View (POV) user-perspective magic lenses to analyze the effect of the dual-view problem on the use of the surrounding visual context. The results confirm that the dual-view problem, a result of dual perspective, has a significant effect on the use of information from the surrounding visual context. The study also highlights that magnification and not the dual-view problem is the key factor explaining the correlation between magic lens size and the increased intensity of the magic lens type effect. From the results, we derive design guidelines for future magic lenses.

Exploring the Evolution of Mobile Augmented Reality for Future Entertainment Systems

Despite considerable progress in mobile augmented reality (AR) over recent years, there are few commercial entertainment systems utilizing this exciting technology. To help understand why, we shall review the state of the art in mobile AR solutions, in particular sensor-based, marker-based, and markerless solutions through a design lens of existing and future entertainment services. The majority of mobile AR that users are currently likely to encounter principally utilize sensor-based or marker-based solutions. In sensor-based systems, the poor accuracy of the sensor measurements results in relatively crude augmentation, whereas in marker-based systems, the requirement to physically augment our environment with fiducial markers limits the opportunity for wide-scale deployment. While the alternative online markerless systems overcome these limitations, they are sensitive to environmental conditions (i.e. light conditions), are computationally more expensive, and present greater complexity of implementation, particularly in terms of their system-initialization requirements. To simplify the operation of online markerless systems, a novel, fully autonomous map initialization method based on accelerometer data is also presented; when compared with alternative move-matching techniques, it is simpler to implement, more robust, faster, and less computationally expensive. Finally, we highlight that while there are many technical challenges remaining to make mobile AR development easier, we also acknowledge that because of the nature of AR, it is often difficult to assess the experience that mobile AR will provide to users without resorting to complex system implementations. We address this by presenting a method of creating low-fidelity prototypes for mobile AR entertainment systems.

Creating a stereoscopic magic-lens to improve depth perception in handheld augmented reality

Handheld Augmented Reality (AR) is often presented using the magic-lens paradigm where the handheld device is portrayed as if it was transparent. Such a virtual transparency is usually implemented using video captured by a single camera rendered on the devices screen. This removes binocular-disparity, which may undermine users ability to correctly estimate depth when seeing the world through the magic-lens. To confirm such an assumption this paper presents a qualitative user study that compares a magic-lens implemented on a mobile phone and a transparent glass replica. Observational results and questionnaire analysis indicate that binocular-disparity may play a significant role in participants depth perception. These promising results led to the subsequent implementation of a stereoscopic magic-lens prototype on a commercially available mobile device.

Designing Seamless Mobile Augmented Reality Location Based Game Interfaces

The majority of mobile devices nowadays harness location and orientation-sensing capabilities, permitting for sensor based Mixed Reality (MR) gaming experiences such as Augmented Reality (AR) games. Nonetheless, only few entertainment implementations of sensor based AR systems emerged, predominantly due to low precision of sensory information causing crude and jerky augmentation, significantly impacting meaningful augmentation that inevitably affects gameplay experience. In this paper, we present a novel pervasive mobile AR Location Based Game (LBG) named Time-wARpXplorer (TARX) where a seamless design approach is used in order to address the quality of augmentation. TARX comprises of a mobile client and an online authoring tool, created to encourage visitors and locals to explore the city of Lancaster by travelling back in time and space to discover their immediate and distant surroundings. Within the game, time travel is linked to present day through the implementation of Foursquares checkin platform to further raise awareness of historic sites. This paper focuses on the design considerations implemented through an iterative design process and prototyping of the mobile client, of which a novel interface emerged.

[Poster] Contact-view: A magic-lens paradigm designed to solve the dual-view problem

Typically handheld AR systems utilize a single back-facing camera and the screen in order to implement device transparency. This creates the dual-view problem a consequence of virtual transparency which does not match true transparency - what the user would see looking through a transparent glass pane. The dual-view problem affects usability of handheld AR systems and is commonly addressed though user-perspective rendering solutions. Whilst such approach produces promising results, the complexity of implementing user-perspective rendering and the fact it does not solve all sources that produce the dual-view problem, means it only ever addresses part of the problem. This paper seeks to create a more complete solution for the dual-view problem that will be applicable to readily available handheld-device. We pursue this goal by designing, implementing and evaluating a novel interaction paradigm we call `contact-view. By utilizing the back and front-facing camera and the environment base-plane texture - predefined or incrementally created on the fly, we enable placing the device directly on top of the base-plane. As long as the position of the phone in relation to the base-plane is known, appropriate segment of the occluded base-plane can be rendered on the device screen, result of which is transparency in which dual-view problem is eliminated.

Utilizing sensor fusion in markerless mobile augmented reality

One of the key challenges of markerless Augmented Reality (AR) systems, where no a priori information of the environment is available, is map and scale initialization. In such systems, the scale is unknown as it is impossible to determine the scale from a sequence of images alone. Implementing scale is vital for ensuring that augmented objects are contextually sensitive to the environment they are projected upon. In this paper we demonstrate a sensor and vision fusion approach for robust and user-friendly initialization of map and scale. The map is initialized, using inbuilt accelerometers, whilst scale is initialized by the camera auto-focusing capability. The later is possible by applying the Depth From Focus (DFF) method, which was, till now, limited to high precision camera systems. The demonstrated illustrates benefits of such a system, which is running on a commercially available mobile phone Nokia N900.

Estimating scale using depth from focus for mobile augmented reality

Whilst there has been a considerable progress in augmented reality (AR) over recent years, it has principally been related to either marker based or apriori mapped systems, which limits its opportunity for wide scale deployment. Recent advances in marker-less systems that have no apriori information, using techniques borrowed from robotic vision, are now finding their way into mobile augmented reality and are producing exciting results. However, unlike marker based and apriori tracking systems these techniques are independent of scale which is a vital component in ensuring that augmented objects are contextually sensitive to the environment they are projected upon. In this paper we address the problem of scale by adapting a Depth From Focus (DFF) technique, which has previously been limited to high-end cameras to a commercial mobile phone. The results clearly show that the technique is viable and adds considerably to the enhancement of mobile augmented reality. As the solution only requires an auto focusing camera, it is also applicable to other AR platforms.

User curated augmented reality art exhibitions

Creating mobile augmented reality applications to display gallery artworks or museum content is a well-established concept within the research community. However, the focus of these systems is generally technologically driven and primarily addresses the end user and not the views of the gallery or the original artist. In this paper we present the design and development of the mobile application Taking the Artwork Home, which allows people to digitally curate augmented reality art exhibitions in their own homes. A research through design methodology was adopted so that we could more fully understand how the views of the gallery and artists impacted on the artifact design and therefore the user experience.

Designing mobile augmented reality art applications: addressing the views of the galleries and the artists

The utilization of mobile augmented reality to display gallery artworks or museum content in novel ways is a well-established concept in the augmented reality research community. However, the focus of these systems is generally technologically driven or only addresses the end user and not the views of the gallery or the original artist. In this paper we discuss the design and development of the mobile application Taking the Artwork Home, which allows people to digitally curate their own augmented reality art exhibitions in their own homes by digitally replacing the pictures they have on their walls with content from the Peter Scott Gallery in Lancaster. In particular, we present the insights gained from a research through design methodology that allowed us to consider how the views of the gallery and artists impacted on the system design and therefore the user experience. Thus the final artifact is the result of an iterative evaluation process with over 100 users representing a broad range of demographics and continues to be evaluated/enhanced by observing its operation in the wild. Further, we consider the effect the project has had on gallery practices to enable both augmented reality designers, and galleries and museums to maximize the potential application of the technology when working together on such projects.