Goshiro Yamamoto

Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation

Augmented reality (AR) technology is mature for creating learning experiences for K-12 (pre-school, grade school, and high school) educational settings. We reviewed the applications intended to complement traditional curriculum materials for K-12. We found 87 research articles on augmented reality learning experiences (ARLEs) in the IEEE Xplore Digital Library and other learning technology publications. Forty-three of these articles conducted user studies, and seven allowed the computation of an effect size to the performance of students in a test. In our meta-analysis, research show that ARLEs achieved a widely variable effect on student performance from a small negative effect to a large effect, with a mean effect size of 0.56 or moderate effect. To complement this finding, we performed a qualitative analysis on the design aspects for ARLEs: display hardware, software libraries, content authoring solutions, and evaluation techniques. We explain that AR incur three inherent advantages: real world annotation, contextual visualization, and vision-haptic visualization. We illustrate these advantages through the exemplifying prototypes, and ground these advantages to multimedia learning theory, experiential learning theory, and animate vision theory. Insights from this review are aimed to inform the design of future ARLEs.

Augmented reality as multimedia: the case for situated vocabulary learning

Augmented reality (AR) has the potential to create compelling learning experiences. However, there are few research works exploring the design and evaluation of AR for educational settings. In our research, we treat AR as a type of multimedia that is situated in authentic environments and apply multimedia learning theory as a framework for developing our educational applications. We share our experiences in developing a handheld AR system and one specific use case, namely, situated vocabulary learning. Results of our evaluations show that we are able to create AR applications with good system usability. More importantly, our preliminary evaluations show that AR may lead to better retention of words and improve student attention and satisfaction.

Camera pose estimation under dynamic intrinsic parameter change for augmented reality

In this paper, we propose a method for estimating the camera pose for an environment in which the intrinsic camera parameters change dynamically. In video see-through augmented reality (AR) technology, image-based methods for estimating the camera pose are used to superimpose virtual objects onto the real environment. In general, video see-through-based AR cannot change the image magnification that results from a change in the cameras field-of-view because of the difficulty of dealing with changes in the intrinsic camera parameters. To remove this limitation, we propose a novel method for simultaneously estimating the intrinsic and extrinsic camera parameters based on an energy minimization framework. Our method is composed of both online and offline stages. An intrinsic camera parameter change depending on the zoom values is calibrated in the offline stage. Intrinsic and extrinsic camera parameters are then estimated based on the energy minimization framework in the online stage. In our method, two energy terms are added to the conventional marker-based method to estimate the camera parameters: reprojection errors based on the epipolar constraint and the constraint of the continuity of zoom values. By using a novel energy function, our method can accurately estimate intrinsic and extrinsic camera parameters. We confirmed experimentally that the proposed method can achieve accurate camera parameter estimation during camera zooming. Graphical abstractDisplay Omitted HighlightsWe propose an intrinsic and extrinsic camera parameter estimation method.We extended a conventional marker-based method by adding two energy terms.Camera parameters are estimated accurately using new energy function.The effectiveness of our method is shown in simulated and real environments.

A usability scale for handheld augmented reality

Handheld augmented reality (HAR) applications must be carefully designed and improved based on user feedback to sustain commercial use. However, no standard questionnaire considers perceptual and ergonomic issues found in HAR. We address this issue by creating a HAR Usability Scale (HARUS). To create HARUS, we performed a systematic literature review to enumerate user-reported issues in HAR applications. Based on these issues, we created a questionnaire measuring manipulability -- the ease of handling the HAR system, and comprehensibility -- the ease of understanding the information presented by HAR. We then provide evidences of validity and reliability of the HARUS questionnaire by applying it to three experiments. The results show that HARUS consistently correlates with other subjective and objective measures of usability, thereby supporting its concurrent validity. Moreover, HARUS obtained a good Cronbachs alpha in all three experiments, thereby demonstrating internally consistency. HARUS, as well as its decomposition into individual manipulability and comprehensibility scores, are evaluation tools that researchers and professionals can use to analyze their HAR applications. By providing such a tool, they can gain quality feedback from users to improve their HAR applications towards commercial success.

Geometrically-Correct Projection-Based Texture Mapping onto a Deformable Object

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an objects surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

Relation between location of information displayed by augmented reality and user's memorization

This study aims to investigate the effectiveness of Augmented Reality (AR) on users memory skills when it is used as an information display method. By definition, AR is a technology which displays virtual images on the real world. These computer generated images naturally contain location information on the real world. It is also known that humans can easily memorize and remember information if this information is retained along with some locations on the real world. Thus, we hypothesize that displaying annotations by using AR may have better effects on the users memory skill, if they are associated with the location of the target object on the real world rather than when connected with an unrelated location. A user study was conducted with 30 participants in order to verify our hypothesis. As a result, a significant difference was found between the situation when information was associated with the location of the target object on the real world and when it was connected with an unrelated location. In this paper, we present the test results and explain the verification based on the results.

PiTaSu: wearable interface for assisting senior citizens with memory problems

Abstract Little research has been carried out on specialized wearable input interfaced designs to assist memory impaired senior citizens. This paper proposes and implements PiTaSu (Picture based Tapping on wall Surfaces) to realize a direct input interface system to offer visual feedback and tactile feedback. PiTaSu is based on a pictorial based Augmentative and Alternative Communication (AAC) system. PiTaSu consists of a body-worn or shoulder-attached mobile projector, a camera and an accelerometer wrist band. The projector shows information that will help assist the memory impaired senior citizen in their daily task. The camera and the accelerometer detect a tapping position and tapping trigger. Experimental results have demonstrated that a senior citizen can use PiTaSu without learning special skills, and the projection based user interface has potential. Therefore, PiTaSu can assist memory-impaired senior citizens as a daily task reminder.

SlidAR: A 3D positioning method for SLAM-based handheld augmented reality

Abstract Handheld Augmented Reality (HAR) has the potential to introduce Augmented Reality (AR) to large audiences due to the widespread use of suitable handheld devices. However, many of the current HAR systems are not considered very practical and they do not fully answer to the needs of the users. One of the challenging areas in HAR is the in-situ AR content creation where the correct and accurate positioning of virtual objects to the real world is fundamental. Due to the hardware limitations of handheld devices and possible restrictions in the environment, the correct 3D positioning of objects can be difficult to achieve we are unable to use AR markers or correctly map the 3D structure of the environment. We present SlidAR, a 3D positioning for Simultaneous Localization And Mapping (SLAM) based HAR systems. SlidAR utilizes 3D ray-casting and epipolar geometry for virtual object positioning. It does not require a perfect 3D reconstruction of the environment nor any virtual depth cues. We have conducted a user experiment to evaluate the efficiency of SlidAR method against an existing device-centric positioning method that we call HoldAR. Results showed that SlidAR was significantly faster, required significantly less device movement, and also got significantly better subjective evaluation from the test participants. SlidAR also had higher positioning accuracy, although not significantly.

Authoring Augmented Reality Learning Experiences as Learning Objects

Engineers and educators alike have prototyped a variety of augmented reality learning experiences (ARLEs). However, adapting ARLEs in educational practice would require an interdisciplinary approach that considers learning theory, pedagogy and instructional design. To address this requirement, we model ARLEs as learning objects by outlining the necessary components, and we propose a participatory design to demonstrate the authoring process of an augmented reality learning object (ARLO). ARLOs can be made useful in many scenarios if teachers are empowered to edit its context elements, content and instructional activity. Lastly, we point to the research questions entailed in modeling ARLEs as ARLOs.

User interaction in smart ambient environment targeted for senior citizen

Many countries are facing a problem when the age-structure of the society is changing. The numbers of senior citizen are rising rapidly, and caretaking personnel numbers cannot match the problems and needs of these citizens. Using smart, ubiquitous technologies can offer ways in coping with the need of more nursing staff and the rising costs of taking care of senior citizens for the society. Helping senior citizens with a novel, easy to use interface that guides and helps, could improve their quality of living and make them participate more in daily activities. This paper presents a projection-based display system for elderly people with memory impairments and the proposed user interface for the system. The user’s process recognition based on a sensor network is also described. Elderly people wearing the system can interact the projected user interface by tapping physical surfaces (such as walls, tables, or doors) using them as a natural, haptic feedback input surface.