Stephan Lukosch

Collaboration in Augmented Reality

Augmented Reality (AR) is a technology that allows users to view and interact in real time with virtual images seamlessly superimposed over the real world. AR systems can be used to create unique collaborative experiences. For example, co-located users can see shared 3D virtual objects that they interact with, or a user can annotate the live video view of a remote worker, enabling them to collaborate at a distance. The overall goal is to augment the face-to-face collaborative experience, or to enable remote people to feel that they are virtually co-located. In this special issue on collaboration in augmented reality, we begin with the visions of science fiction authors of future technologies that might significantly improve collaboration, then introduce research articles which describe progress towards these visions, finally we outline a research agenda discussing the work still to be done.

Serious Gaming in Augmented Reality using HMDs for Assessment of Upper Extremity Motor Dysfunctions

Abstract For a better understanding of how different disorders affect motor function, a uniform, standardized and objective evaluation is a desirable goal for the clinical community. We explore the potential of Augmented Reality (AR) combined with serious gaming and free hand tracking to facilitate objective, cost-effective and patient-friendly methods for evaluation of upper extremity motor dysfunction in different patient groups. In this paper, we describe the design process of the game and the system architecture of the AR framework to meet these requirements. Furthermore, we report our findings from two pilot studies we conducted with healthy people aged over 50. First, we present a usability study (nu2009=u20095) on three different modalities of visual feedback for natural hand interaction with AR objects (i.u2009e., no augmented hand, partial augmented hand and a full augmented hand model). The results show that a virtual representation of the fingertips or hand improves the usability of natural hand interaction. Secondly, a study about game engagement is presented. The results of this experiment (nu2009=u20098) show that there might be potential for engagement, but usability needs to be improved before it can emerge.

A Scientific Foundation of Simulation Games for the Analysis and Design of Complex Systems

Background. The use of simulation games for complex systems analysis and design has been acknowledged about 50 years ago. However, articles do not combine all salient factors for successful simulation games, and often stem from a clear view of one particular field of science only. With combining multiple disciplines, connect analysis and design as well as research and practice, we provide deep insights in design and use of simulation games. Aim. This article analyzes the design and evaluation process of a variety of game-based projects and activities, using existing scientific concepts and approaches, in order to establish games as a valid research tool. Our focus lies on the approach towards the use of games as design instrument; using them as an intervention in a larger, complex context, in order to design this context. With our contribution, we aim at providing insights and recommendations on the design and use of games as valid research tools, the limitations of this use, possible pitfalls, but also best practices. Method. We carried out a literature review of related work to identify the most important scientific concepts related to our approach of game design. Further use of combined quantitative and qualitative case study analyses highlights the design process and results of our own game studies. Results. The analyses yielded a consolidated conceptualization of simulation games as research instruments in complex systems analysis and design. The results also include methods for the evaluation of simulation games, additional evaluation methods, and limitations to use simulation games as research instruments. Conclusions. We propose guidelines for using simulation games as research instruments that may be of value to practitioners and scientists alike. Recommendation. We recommend practitioners and scientists to apply the guidelines presented here in their efforts to analyze and design complex systems.

On the Spot Information in Augmented Reality for Teams in the Security Domain

For operational teams in the security domain it is important to quickly and adequately exchange context-related information. This is necessary to develop distributed situational awareness and facilitate collaboration. Currently, information exchange is mainly based on oral communication. Oral communication can be misunderstood or ambiguous. This paper reports on different scenarios from the security domain in which augmented reality (AR) techniques are used to support information exchange. A combination of quantitative and qualitative evaluation showed that AR can improve the distributed situational awareness of a team.

A Haptic Serious Augmented Reality Game for Motor Assessment of Parkinson's Disease Patients

In the clinical community there is a need for assessment tools that allow for objective, quantitative and valid measures of motor dysfunction. In this paper, we report on the design and evaluation of a serious game that engages patients with Parkinsons disease in upper extremity (hand/arm) movements. The game employs augmented reality to show virtual movement targets, i.e. candies falling from a conveyor belt, and a haptic game controller to catch the candies, that is able to acquire quantitative data about the patients movement. This paper first describes the design process of the game and the system components. Secondly, we present results of our small quantitative evaluation study (N11, age: 26–60, healthy persons) regarding the usability of the system, the task load and user experience of the game. Our findings show that the system has a relatively good usability and the game is engaging, but there is still need for technical improvement with regard to tracking the controller in 3D space.

Workspace Awareness in Collaborative AR using HMDs: A User Study Comparing Audio and Visual Notifications

For most professional tasks nowadays, it is necessary to work in teams. Such collaboration often requires the exchange of visual context-related information among the team members. For so-called shared workspace collaboration, awareness of other peoples activities is of utmost importance. We have developed an augmented reality (AR) framework in order to support visual communication between a team of two people who are virtually co-located. We address these people as the remote user, who uses a laptop and the local user, who wears a head-mounted display (HMD) with an RGB camera. The remote user can support the local user in solving a spatial problem by providing instructions as virtual objects in the view of the local user. For placing virtual objects in the shared workspace, we use a state-of-the-art algorithm for localization and mapping without markers. In this paper, we report on a user study that explores on how automatic audio and visual notifications about the remote users activities affect the collaboration. The results show that in our current implementation, visual notifications are preferred over audio or no notifications independent from the level of difficulty of the task.

Designing for Engagement in Augmented Reality Games to Assess Upper Extremity Motor Dysfunctions

We present a Markerless 3D Augmented Reality Application for virtual accessory try-on applications around human arm. The system is based on a Kinect sensor and a multi-layer rendering framework to render RGB, depth data and 3D model of accessories simultaneously. The aim is to support realistic visualization of virtual objects around human arm, by detecting wrist pose and handling occlusion for various interactive marketing and retail applications, such as virtual watch try-on.My research focus is on posthumanism: future biomorphic transformations and sculptural reconstruction of the human body. I work with digital—physical sculpture, data art, performance and contemporary technology and materials, such as 3D printing, attempting to bridge the gap between art, science, design and technology. My work investigates a presently topical discourse on the aspects of posthumanism theories, such as bioengineering, gender issues and outer space exploration. Philosophically, I lean on posthumanism theories, uncovered by Cary Wolfe, Daryl Wennemann, N. Katherine Hayles, et al. I further these ideas into sculptural surfaces, enmeshing future skin concept and biomorphic transformations of the skin based on the data from the human body. Current research eventually will result in a multi sensorial experience with mixed reality scenarios — exhibition and performances with augmented to body 3D printed sculptures — a visualisation of humans personal data. The work contributes articulating future possibilities of the relationship of technology and the body. The contribution can be important to the postgender related discussion in relation to digital technologies, data and bodies; to the augmented and mixed reality issues relating to the body and also the themes of embodiment, bodies of matter and data bodies. The objects encapsulate different layers of information and might encourage different perspective on posthumanism.

User-Centered Design of an Online Mobile Game Suite to Affect Well-Being of Older Adults

This paper describes the ongoing design process of Pocket Odyssey, an online multi-player game suite designed for mobile phones that supports older adults in maintaining their physical, social, and mental well-being. It has been designed to provide engaging gameplay that supports positive emotion, fosters online social connections between players, enables play with others in a player’s physical surroundings, and integrates tasks that exercise physical and cognitive skills. This paper discusses the user-centered design approach that takes entertainment preferences, views on technology, and aspects of life that older adults consider meaningful as a basis. Based on the results of an exploratory survey and a series of workshops with the user group, this paper presents the initial game concept and the rationale behind its design. Finally, it describes early prototyping efforts and future activities.

[POSTER] ChiroChroma: An Augmented Reality Game for the Assessment of Hand Motor Functionality

For a better understanding of how different diseases (e.g. neurovascular diseases, neurodegenerative diseases, and musculoskeletal pain conditions) affect human motor function, a uniform, standardized and objective evaluation is a desirable goal expressed within the clinical community. We explore the capabilities of an augmented reality (AR) game that uses free hand interaction to facilitate an objective assessment of the upper extremity motor dysfunction. First, the design process of the game and the system architecture are described. Second, a study about usability of the AR framework and game engagement is presented based on an experiment we conducted with five patients and ten healthy people. Lastly, a short analysis of the accuracy of the hand data when participants performed “fingers tapping” gesture is done. The results of the study show that even though users experienced the system as physically and mentally demanding, it was engaging enough to make them complete the game. The study also shows that hand data captured is accurate enough to allow a high degree (95%) of pinching gesture recognition.

Towards Engaging Upper Extremity Motor Dysfunction Assessment Using Augmented Reality Games

Advances in technology offer new opportunities for a better understanding of how different disorders affect motor function. Our aim is to explore the potential of augmented reality (AR) using free hand and body tracking to develop engaging games for a uniform, cost-effective and objective evaluation of upper extremity motor dysfunction in different patient groups. Based on the insights from a study with 20 patients (10 Parkinson’s Disease patients and 10 stroke patients) who performed hand/arm movement tasks in AR, we created a set of different augmented reality games for upper extremity motor dysfunction assessment.