Kazuyo Iwamoto

Development of an eye movement tracking type head mounted display: capturing and displaying real environment images with high reality

This paper describes the head mounted display (HMD) system which can capture and display images with high reality. Conventional HMD has the disadvantages of small screen size and low resolution. Therefore, the resolution is not high enough for telerobotics applications which need real environment images with high reality. In order to address these problems, an eye movement tracking type head mounted display (EMT-HMD) is being developed. The EMT-HMD can display wide viewing angle and high resolution images of the environment. The theory is that a small, high resolution image is displayed at view point of the operator and a wide, low resolution image is displayed at the periphery. Human can see an object as a high resolution image only near the view point. Therefore, the operator who wears the EMT-HMD can feel that the displayed image is wide view angle and high resolution. In this paper, an image capturing system of real environment images is discussed. Next, some evaluation experiments for displaying real environment images are introduced using an EMT-HMD system and an image capturing system.

A head-mounted eye movement tracking display and its image display method

As a visual image presentation device in virtual reality and telerobotics, the head-mounted display (HMD) is attracting attention. The basic principle of HMD is as follows. Two small displays are arranged in front of the two eyes, and present images with parallax, conveying a stereo feeling to the operator. The number of scan lines of the displays used in these conventional HMDs is generally limited. For that reason, it is difficult to construct a stereo feeling which will present sufficient images to cover the human view field while maintaining resolution, and this is an obstacle in presenting highly realistic images. For solving this problem, an eye-movement tracking type HMD which can present images with a wide view field and high resolution has been proposed. Human vision can observe objects with high resolution only near the gaze point. Paying attention to this visual characteristic, the eye-movement tracking type HMD is developed as a system which presents images of high resolution in a small region in the view-line direction and presents images of low resolution but wide view field in its surroundings, and thus equivalently presents images of wide view field and high resolution. In this paper, we will propose an image presentation scheme for the embodiment of an eye-movement tracking type HMD and will also describe efforts to confirm its appropriateness by experiments with simple images.

Plate bending by line heating with interactive support through a monocular video see-through head mounted display

This paper describes the line heating experiment performed. Operating instructions are presented to a worker interactively through a monocular video see-through head mounted display (HMD). We are advancing research of interactive work support technology that unskilled person can perform a task that needs advanced skill by presenting the operating instructions through a wearable device. In order to present operating instructions, a monocular video see-through HMD was developed as a human interface. Thereby, operating instructions are superimposed on a work object and a processing point and a tool position posture, etc. are presented. Although this technology is applicable to various applications, here it is applied to the plate bending by line heating. The plate bending by line heating is a task that makes an iron plate transformation into a target shape by heating and cooling. It is well used for ship shell fabrication. In the experiment, a worker performs a line heating task according to the operating instructions presented by monocular video see-through HMD. As a comparison experiment, the torch fixed to the motorized linear slide was moved at fixed speed, and the iron plate was heated. In this paper, the comparison results of these experiments are reported.

A monocular video see-through head mounted display for interactive support system — Instruction of 3-dimensional position, posture and speed

We are advancing to research an interactive support technology that assists a worker to do a task by presenting the working procedure with Image Capture Display Device. It is expected that an unskilled worker can do various tasks using this technology. We work on applying this technology to the plate bending by line heating. In order to realize it, we have developed an Image Capture Display Device, which consists of a monocular head mounted display and a camera as a human interface. In this paper, the interactive support technology and plate bending by line heating as an industrial application are explained. A method of instructing 3-dimensional position, posture and speed of burner by monocular video see-through head mounted display is described and the evaluation experiments are reported.

Interactive support using a video see-through Head Mounted Display for plate bending by line heating - Investigation of image capture and display -

The interactive support system interactively supports the worker by presenting the working process with head mounted display (HMD). It is expected that an unskilled worker can do advanced work using this technology. We work on applying this technology in the plate bending by line heating. In this paper, a method of presenting computer graphics (CG) with sufficient accuracy using a HMD and a camera is examined. Furthermore, a prototype of image capture / display system is examined and constructed, and the evaluation experiments are reported.

Binocular head mounted display with eye movement tracking function: development of the prototype and its application

This paper describes a head mounted display that can present images with high reality. The head mounted display (HMD) is often used on tele-robotics systems as visual presentation equipment. In such a tele-robotics system, an operator works on the basis of visual information presented by the HMD. In order to work precisely, high reality images are demanded. In order to present high reality images, a wide view angle and high resolution images need to be presented. But in conventional HMDs the field of view is narrow or the resolution is low, because of the restriction of the scanning line of a display. For that reason, the usage is limited. In order to solve those problems, we are developing the eye movement tracking type HMD (EMT-HMD) that can present the wide view angle and high resolution images of the environment although conventional displays are used. The principle is that a small image of high resolution is present in a narrow range of view point and present a wide image of low resolution in the perimeter by using two displays. In this paper, we examine a system which tracks the movement of eyes in a two dimensional range. The structure of a trial production system is explained. Also some evaluation experiments are introduced.

Development and evaluation of high resolution monocular video see-through display

The interactive working support technology can help to perform tasks requiring advanced skills by giving appropriate working instructions interactively depending on the working situation even if a worker is not an expert. In this technology, an image presentation device is an important human interface to present the working instructions efficiently to the worker, because the image can provide a lot of information at one time. Therefore, the video see-through display is being developed as a human interface which presents the working instructions to the worker. The video see-through display consists of a camera and a display. At first, the image of a working object is captured by the camera. The working instructions will be drown by computer graphics (CG) on the captured real image. Then, the synthetic image is presented to the worker through the display. By following the presented working instructions, the worker will be able to perform the tasks efficiently. So far, we applied this technology to the manufacturing tasks, using a video see-through display we have developed, and confirmed the effectiveness of the image presentation technique we have proposed. However, there is still a problem with a low resolution when a commercially available small camera and a small display are used. In addition, the image quality is not wholly satisfactory with problems such as the distortion of the image. For this reason, it was difficult to be applied to detailed assembly tasks. That is, in order to perform more precise tasks on manufacturing site, it is desirable to develop the display system which can present higher resolution image. In this paper, the relation between the display resolution and the equivalent eyesight of presented image is discussed. And a system construction that achieves the presentation of the high-resolution image is examined. As a result of the examination, a high-resolution monocular video see-through display using full high-definition video camera and display, is developed. Furthermore, in order to confirm the basic effectiveness of the system, some results of the evaluation experiments using some operators are reported.

Evaluation of depth perception of monocular video see-through head mounted display

In order to support the manufacturing operations interactively, we develop monocular video see-through head mounted display (HMD) as a human interface. The operating instructions are programmed according to the state of the object and the work situation, and presented to a worker through HMD. The presentation image is a synthetic image of the operating instructions drawn by the computer graphics (CG) on the real image captured by the camera. The worker can perform the task while watching those operating instructions. At that time, because of the worker equips the monocular video see-through HMD, she/he watches the object directly by one eye, and watches the image of object through HMD by another eye. In order to work efficiently in such a situation, it is preferable that the worker has similar depth perception compared with the state that HMD is not equipped. Therefore, it is necessary to evaluate the depth perception of the worker equipped with the monocular video see-through HMD. In this paper, monocular video see-through HMD for manufacturing support is examined, and, as a result, two prototype devices are proposed. The experimental evaluation results of the depth perception using those devices are reported.

Training support for pouring task in casting process using stereoscopic video see-through display - Presentation of molten metal flow simulation based on captured task motion

The work support technology using augmented reality has been researched in the field of manufacturing. It is the technology that can present appropriate instructions to a worker. Thereby, the worker performs a task requiring an advanced skill even if she/he is not an expert. In this technology, most of work instructions are presented while the worker is working. However, there are many manufacturing tasks that is difficult to be retried when it fails. In order to deal with such tasks, a training support technology that a worker can train tasks beforehand using augmented reality is proposed. This technology can present simulated task results according to the motion of the worker. As a simulation result, for example, the worker can see internal state of the work object. And she/he can train as many times as she/he wants. Therefore, her/his skills can be improved through trial and error. In this paper, a concept of task training support technology is introduced. To realize the concept, the pouring task in casting process is selected as a target task and the prototype system is constructed. Then, the evaluation results of pouring task training using the prototype system are reported.

An instruction method of 3D task motion with stereoscopic video see-through display and its application to pouring task

Research of manufacturing support using augmented reality has been advanced. It is a technology to present the appropriate work instructions to a worker through a display device. By using the technology, the worker can perform a task that requires skill, even if a worker is not an expert. Previously, in this technology, static work instructions, such as the component mounting position and assembly procedure have been mainly presented. However, the presentations of dynamic work instructions are required in some manufacturing tasks. For example, some tasks are required the operations of the work tools including the translation and rotation in three dimensions. If the dynamic work instructions of such a complicated task can be presented, it will be possible to support more varieties of manufacturing tasks. In order to realize it, an instruction method of the three-dimensional task motion of the work tool is examined using a stereoscopic video see-through display in this paper. Then, in order to evaluate the instruction method, the pouring task in casting process is selected as an example application. Furthermore, the results of evaluation experiments are reported.