Arjun Nagendran

A unified framework for individualized avatar-based interactions

This paper presents a framework to interactively control avatars in remote environments. The system, called AMITIES, serves as the central component that connects people controlling avatars (inhabiters), various manifestations of these avatars (surrogates), and people interacting with these avatars (participants). A multiserver–client architecture, based on a low-demand network protocol, connects the participant environment(s), the inhabiter station(s), and the avatars. A human-in-the-loop metaphor provides an interface for remote operation, with support for multiple inhabiters, multiple avatars, and multiple participants. Custom animation blending routines and a gesture-based interface provide inhabiters with an intuitive avatar control paradigm. This gesture control is enhanced by genres of program-controlled behaviors that can be triggered by events or inhabiter choices for individual or groups of avatars. This mixed (agency and gesture-based) control paradigm reduces the cognitive and physical loads on the inhabiter while supporting natural bidirectional conversation between participants and the virtual characters or avatar counterparts, including ones with physical manifestations, for example, robotic surrogates. The associated system affords the delivery of personalized experiences that adapt to the actions and interactions of individual users, while staying true to each virtual characters personality and backstory. In addition to its avatar control paradigm, AMITIES provides processes for character and scenario development, testing, and refinement. It also has integrated capabilities for session recording and event tagging, along with automated tools for reflection and after-action review. We demonstrate effectiveness by describing an instantiation of AMITIES, called TeachLivE, that is widely used by colleges of education to prepare new teachers and provide continuing professional development to existing teachers. Finally, we show the systems flexibility by describing a number of other diverse applications, and presenting plans to enhance capabilities and application areas.

AMITIES: avatar-mediated interactive training and individualized experience system

This paper presents an architecture to control avatars and virtual characters in remote interaction environments. A human-in-the-loop (interactor) metaphor provides remote control of multiple virtual characters, with support for multiple interactors and multiple observers. Custom animation blending routines and a gesture-based interface provide interactors with an intuitive digital puppetry paradigm. This paradigm reduces the cognitive and physical loads on the interactor while supporting natural bi-directional conversation between a user and the virtual characters or avatar counterparts. A multi-server-client architecture, based on a low-demand network protocol, connects the user environment, interactor station(s) and observer station(s). The associated system affords the delivery of personalized experiences that adapt to the actions and interactions of individual users, while staying true to each virtual characters personality and backstory. This approach has been used to create experiences designed for training, education, rehabilitation, remote presence and other-related applications.

Symmetric telepresence using robotic humanoid surrogates

Telepresence involves the use of virtual reality technology to facilitate apparent physical participation in distant events, including potentially performing tasks, while creating a sense of being in that location. Traditionally, such systems are asymmetric in nature where only one side (participant) is “teleported” to the remote location. In this manuscript, the authors explore the possibility of symmetric three‐dimensional telepresence where both sides (participants) are “teleported” simultaneously to each others location; the overarching concept of symmetric telepresence in virtual environments is extended to telepresence robots in physical environments. Two identical physical humanoid robots located in UK and the USA serve as surrogates while performing a transcontinental shared collaborative task. The actions of these surrogate robots are driven by capturing the intent of the participants controlling them in either location. Participants could communicate verbally but could not see the other person or the remote location while performing the task. The effectiveness of gesturing along with other observations during this preliminary experiment is presented. Results reveal that the symmetric robotic telepresence allowed participants to use and understand gestures in cases where they would otherwise have to describe their actions verbally. Copyright

Biologically inspired legs for UAV perched landing

The ability of UAVs to perform a “perch and stare” manoeuvre could significantly enhance their effectiveness during surveillance and reconnaissance missions. Herein, the concept of a biologically inspired leg-based landing system has been presented. The controllers have been tested in simulation and designed to minimize the forces experienced by UAVs during the perched landing manoeuvre, which is significant considering the sensitive payloads carried by them. With accurate position control, velocity matching using a polynomial spline trajectory can be used to minimize impact with the perch. Cushioning the inside of the gripper mechanism with a material that has spring-damper properties can help reduce the impact and prevent slip and bounce on contact with the target. Estimating the kinetic energy of UAVs during the capture provides the controller with sufficient information to vary the amount of force being applied in real-time to decelerate it. By combining an adaptive bell-shaped stiffness controller and velocity matching scheme, a safe and successful perched landing manoeuvre can be accomplished. The required accuracy and high speed of operation required for this manoeuvre are limited by the performance of available actuators and the control bandwidth of the hardware. A video render of the perched landing manoeuvre is included as a part of this submission and can be found at [26].

The “Djedi” Robot Exploration of the Southern Shaft of the Queen's Chamber in the Great Pyramid of Giza, Egypt

There are many unanswered questions regarding the construction and purpose of the Great Pyramid of Giza, Egypt. A climbing robot called “Djedi” has been designed, constructed, and deployed to explore shafts of the queens chamber within the Great Pyramid. The Djedi robot is based on the concept of inchworm motion and is capable of carrying a long reach drill or snake camera. The robot successfully climbed the southern shaft of the Great Pyramid, deployed its snake camera, and revealed writing not seen for thousands of years. This paper details the design of the robot, including climbing steps in the shaft and lessons learned from experimental deployment.

Applications of Avatar Mediated Interaction to Teaching, Training, Job Skills and Wellness

The focus of this chapter is on the application of a framework for remotely delivering role-playing experiences that afford users the opportunity to practice real-world skills in a safe virtual setting. The framework, AMITIES, provides a single individual the capabilities to remotely orchestrate the performances of multiple virtual characters. We illustrate this by introducing avatar–enabled scenarios that range from teacher preparation to effectively dealing with complex interpersonal situations such as resistance to peer pressure and participation in job interviews (either as the interviewer or the interviewee).

Virtual Learning Environments for Students with Disabilities: A Review and Analysis of the Empirical Literature and Two Case Studies.

Students with autism spectrum disorder (ASD) show varying levels of impairment in social skills situations. Interventions have been developed utilizing virtual environments (VEs) to teach and improve social skills. This article presents a systematic literature review of peer-reviewed journal articles focusing on social interventions in VEs involving K-12th grade students with ASD. This exhaustive analysis across four major online databases was guided by operational terms related to intervention type and K-12 students with ASD. The empirical search yielded a very narrow body of literature (n=19) on the use of VEs as social skill interventions for students with ASD. Two case study examples of experiments exploring the use of VEs and students with ASD are presented to illustrate possible applications of this technology.

Continuum of virtual-human space: towards improved interaction strategies for physical-virtual avatars

In this work, a broad continuum of 3D space that encapsulates avatars, ranging from artificial to real in shape, appearance and intelligence is defined. The research focuses on the control of physical-virtual avatars that occupy a specific region in this space that may be suitable for interacting with elements in the environment. To facilitate this control, a paradigm called microposes is developed that overcomes the need for high network bandwidth during remote tele-operation of avatars. The avatar itself uses a control strategy that interprets the received microposes data and executes motions that appear natural and human-like in the presence of data loss and noise. The physical-virtual avatar is used in several training and learning scenarios. Results during testing reveal a reduced bandwidth requirement during remote tele-operation of physical virtual avatars and a good motion tracking performance with respect to a commanded pose from the inhabiter.

Experimental Tests of ‘Bidi-bot’: A Mechanism Designed for Clearing Loose Debris from the Path of Mobile Search and Rescue Robots

Abstract Urban search and rescue robots have the potential of identifying the location of trapped people following a disaster. The majority of survivors in open spaces will be rapidly located and extracted by rescue personnel. Therefore, the greatest challenge for rescue robotics is to penetrate deep within collapsed buildings to search for survivors. In this paper, several robotic challenges are presented to represent some of the challenges faced within a collapsed building. A robotic mechanism, termed the sweep-extend mechanism is proposed as a means for mobile search and rescue robots to clear a path through loose debris. The mechanism has been mounted on a mobile platform and tested against the proposed scenarios. The mechanism was demonstrated to move debris, such as bricks, away from the path of the robot. The work also highlights limitations in the mechanism’s ability to deal with densely packed debris, collections of large debris, and the need for robust dust shielding.

Tether monitoring for entanglement detection, disentanglement and localisation of autonomous robots

Tethered mobile robots are ideal for electrically noisy environments and for time-consuming tasks that require robust data communication and uninterrupted power delivery. However, tethers may become entangled in cluttered environments, leading to immobilisation and consequent mission failure. This work addresses real-time monitoring of tethers to detect tether entanglement, perform disentanglement through tether following and localise within line of sight. Experimental hardware is proposed to implement the tether monitoring techniques. Experiments are performed for single and dual mobile robots to search a target environment and entanglement detection is shown to be successful using quantitative metrics such as mean localization error. Copyright