Ping-Hsuan Han

Visualizing the keyboard in virtual reality for enhancing immersive experience

Recently, virtual reality (VR) becomes more and more popular and provides users an immersive experience with a head-mounted display (HMD). However, in some applications, users have to interact with physical objects while immersed in VR. With a non-see-through HMD, it is difficult to perceive visual information from the real world. Users must recall the spatial layout of the real surroundings and grope around to find the physical objects. After locating the objects, it is still inconvenient to use them without any visual feedback, which would detract the immersive experience.

OoEs: playing in the immersive game with augmented haptics

With the recent advances of head-mounted displays (HMD) for immersive virtual reality, game designers are able to provide players with many different gameplay to manipulate in the virtual environment. Like most of the console platforms and portable devices, tactile feedback is an important aspect in games and for notification. For example, Po2 [Israr et al. 2015] adopts two vibrating actuators on two hands and renders illusory tactile motion on and across the hands for interactive gameplay. NotiRing [Roumen et al. 2015] are wearable interactive rings with five different kinds of notification channels that can provide the user a visual, auditory or tactile feedback for receiving the notification from the wearable device. Although the use of HMD has been quite popular recently, the tactile feedback in the immersive environment is less than visual or auditory feedback in the virtual space, mainly because haptic feedback is more complicated and various, it includes kinesthesia and cutaneous feedback. Our goal is to provide the user the first-person perspective and experience of playing elements in an immersive game like a wizard, elves, mage or bender - bending water, fire, earth, and air, combing with different visual, auditory and haptic feedback for each element.

AR-Arm: Augmented Visualization for Guiding Arm Movement in the First-Person Perspective

In many activities, such as martial arts, physical exercise, and physiotherapy, the users are asked to perform a sequence of body movements with highly accurate arm positions. Sometimes, the movements are too complicated for users to learn, even by imitating the action of the coach directly. This paper presents a fully immersive augmented reality (AR) system, which provides egocentric hints to guide the arm movement of the user via a video see-through head-mounted display (HMD). By using this system, the user can perform the exactitude of arm movement simply by moving his arms to follow and match the virtual arms, rendered from coachs movement of database, in the first-person view. To ensure the rendered virtual arms correctly aligned with the users real shoulders, a calibration method is proposed to estimate the length of the users arms and the positions of his head and shoulders in advance. In addition, we apply the system to Tai-Chi-Chuan practicing, our preliminary study has shown that the proposed egocentric hints can provide intuitive guidance for users to follow the arm movement of the coach with exactitude.

Moving around in virtual space with spider silk

With the recent advances of wearable I/O devices, designers of immersive VR systems are able to provide users with many different ways to explore the virtual space. For example, Birdly [Rheiner 2014] is a flying simulator composed of visual, auditory, and smell feedback that can provide the user a compelling experience of flying in the sky. SpiderVision adopts a non-see-through head-mounted display (HMD) and two cameras with opposite directions to provide the user a front-and-back vision [Fan et al. 2014]. Although the use of HMD is quite popular recently, moving around in a virtual space is not as easy as looking around in a virtual space, mainly because position tracking is more complicated than orientation tracking with state-of-the-art technologies. Our goal is to provide the user the first-person perspective and experience of moving around in 3D space like a super human -- jump high, glide off, fly with rope, teleport, etc., even without the position tracking technologies.

SoEs: Attachable Augmented Haptic on Gaming Controller for Immersive Interaction

We present SoEs (Sword of Elements), an attachable augmented haptic device for enhancing gaming controller in the immersive first-person game. Generally, Player can easily receive visual and auditory feedback through head-mounted displays (HMD) and headphones from first-person perspective in virtual world. However, the tactile feedback is less than those feedbacks in immersive environment. Although gaming controller, i.e. VIVE or Oculus controller, can provide tactile feedback by some vibration sensors, the haptic feedback is more complicated and various, it includes kinesthesia and cutaneous feedback. Our key idea is to provide a low-cost approach to simulate the haptic feedback of player manipulation in the immersive environment such as striking while the iron is hot which the player could feel the heat and reaction force. Eventually, the game makers could utilize the attachable device into their games for providing haptic feedback.

Exploring Manipulation Behavior on Video See-Through Head-Mounted Display with View Interpolation

Video see-through HMD mixes the real and virtual world, and users can have a good experience on virtual part, but the real part captured by cameras still have some problem, especially the distance perception. In this paper, we try to remove the error due to the distance between cameras and users. We use depth image-based rendering algorithm to re-compute the true distance of the scene, and render the correct image to the user. And we use multiple cameras with different viewpoints to reduce the occlusion areas.

AoEs: enhancing teleportation experience in immersive environment with mid-air haptics

To alleviate cybersickness in the immersive virtual reality (VR), teleportation is a common method of moving around in virtual spaces. Although users can receive the visual and auditory feedbacks from their first-person perspective with the advances of immersive head-mounted displays (HMD), they do not have the haptic experience when they teleport to another environment. Based on the immersive HMD, many research groups have shown that haptic feedback is one of the important key to enhance the immersive experience in the virtual reality. However, to simulate the haptic feedback from different environments e.g. desert and snow, it require many devices in the real environment to simulate the sun, airflow, humidity and temperature. In this work, our main concept is to provide a haptic tower in the room-scale VR, which allow the game designers to enhance the player experience in the immersive environments. We present Area of Elements (AoEs), an integration device for simulating immersive environments with haptics, which can provide two kinds of virtual environments simultaneously for teleportation experience.

View interpolation for video see-through head-mounted display

By using the head-mounted display (HMD), we can have an immersive virtual reality experience. But the user cannot see any information from the real world. To solve the problem, video seethrough HMD can acquire images from real environment, and present into the HMD, then, we could build a mixed reality (MR) or augmented reality (AR) system. However, how to append and calibrate cameras on HMD for recovering real environment is still a research issue. HTC VIVE has a single camera in front of its device. [Steptoe et al. 2014] and OVRVISION Pro proposed to append dual cameras to capture left and right images. Due to the difference of viewpoint, images captured by cameras are different to what human eyes see (figure 2). Although we could recover true 3D information with a depth map, there are still some occlusion areas that we cannot recover by single camera. Therefore, multiple cameras with different positions could complement each other for reducing occlusion areas. In this work, four configurations are simulated with a synthesized scene.

tARget: limbs movement guidance for learning physical activities with a video see-through head-mounted display

In the aging society, people are paying more attention to having good exercise habits. The advancement of technology grants the possibility of learning various kinds of exercises using multi-media equipment, for example, watching instruction videos. However, it is difficult for users to learn accurate movements due to lack of feedback information.

Estimating the simulator sickness in immersive virtual reality with optical flow analysis

Recently, Virtual Reality (VR) has shown the ability of providing immersive experience to users and taking them to travel around the virtual environment. Sometimes when traveling in a large-scale virtual environment, the tracking space may not be large enough to fit the virtual environment. Several moving approaches such as virtual jumping and flying are proposed to overcome this limitation. Nevertheless, inappropriate design of moving approaches will cause serious simulator sickness. Intensive visual flow can be one of the major factors that cause the simulator sickness. According to Oculus Best Practices [Oculus 2017], it is necessary to avoid an intensive display of visual flow while developing VR application. There are several researches trying to combat simulator sickness through reducing optical flow. FOV restrictor [Fernandes and Feiner 2016] is one of the successful ways. By decreasing the FOV of the HMD, it can effectively reduce optical flow from peripheral vision and make users feel more comfortable. However, the immersiveness in VR will be declined due to the limited size of FOV.