Makoto Iwahara

High-quality integral videography using a multiprojector

Integral videography (IV) is an animated extension of integral photography. Despite IVs many advantages, the quality of its spatial images has thus far been poor; the pixel pitch of the display and the lens pitch are considered to be the main factors affecting the IV image format. Our solution for increasing pixel density is to use multiple projectors to create a high-resolution image and project the resultant image onto a small screen by using long-zoom-lens projection optics. We manufactured a lens array for the display device, and here we present experimental results on using two SXGA projectors. The pixel pitch and lens pitch of the new display are 85 mum and 1.016 mm, respectively. The multiprojector IV display device has a spatial resolution of approximately 1, 2, and 3 mm for image depths of 10, 35, and 60 mm, respectively, in front of and behind the lens array.

Surgical navigation by autostereoscopic image overlay of integral videography

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patients body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.

Scalable high-resolution integral videography autostereoscopic display with a seamless multiprojection system

We propose a scalable high-resolution autostereoscopic display that uses integral videography (IV) and a seamless multiprojection system. IV is an animated extension of integral photography (IP). Although IP and IV are ideal ways to display three-dimensional images, their spatial viewing resolution needs improvement; the pixel pitch of the display and the lens pitch are the main factors affecting IV image quality. We improved the quality by increasing the number and density of the pixels. Using multiple projectors, we create a scalable high-resolution image and project it onto a small screen using long-focal-length projection optics. To generate seamless IV images, we developed an image calibration method for geometric correction and color modulation. We also fabricated a lens array especially for the display device. Experiments were conducted with nine XGA projectors and nine PCs for parallel image rendering and displaying. A total of 2868 x 2150 pixels were displayed on a 241 mm x 181 mm (302.4 dots/in.) rear-projection screen. The lens pitch was 1.016 mm, corresponding to 12 pixels of the projected image. Measurement of the geometric accuracy of the reproduced IV images demonstrated that the spatial resolution of the display system matched that of the theoretical analysis.

Improved viewing resolution of integral videography by use of rotated prism sheets

We demonstrated that placing a pair of prism sheets in front of a display and rotating them overcomes the upper resolution limit of Integral Photograpy (IP) / Integral Videography (IV) imposed by the Nyquist sampling theorem. A pair of prism sheet with the same pitch placed in front of an IP or IV display parallel-shifts the light rays in the 3D space. Rotating the pair shifts the light rays, causing them to appear to rotate around their original positions. Changing the gap between the sheets changes the diameter of the apparent rotation. Changing the speed at which the sheets are rotated changes the speed of the image movement.. Experimental results showed that the quality of the IP and IV images is improved by using this technique. It is a simple and effective way to improve the viewing resolution of IP and IV images without reducing their 3D aspects, such as image depth. It also eliminates the need to move the lenslet array.

Three-dimensional display with a long viewing distance by use of integral photography.

We developed a technique of three-dimensional (3-D) display for distant viewing of a 3-D image without the need for special glasses. The photobased integral photography (IP) method allows precise 3-D images to be displayed at long viewing distances without any influence from deviated or distorted lenses in a lens array. We calculate elemental images from a referential viewing area for each lens and project the corresponding result images to photographic film through each lens. We succeed in creating an image display that appears to have three dimensionality even when viewed from a distance, with an image depth of 5.7 m or more in front of the display and 3.5 m or more behind the display. To the best of our knowledge, the long-distance IP display presented here is technically unique because it is the first report of generation of an image with such a long viewing distance.

High Quality Autostereoscopic Surgical Display Using Anti-aliased Integral Videography Imaging

This paper presents an autostereoscopic three-dimensional (3-D) surgical display with high quality integral videography (IV) rendering algorithm. IV is an animated extension of integral photography, which provides 3-D images without using any supplementary glasses or tracking devices. Despite IV’s many advantages, the quality of its spatial image has thus far been poor. We developed a high quality image rendering method with oversampling technique for enhancing the resolution of elemental IV image and low-pass-filter for smoothing the image. Furthermore, we manufactured a high-resolution IV display for evaluating the feasibility of proposed method. The experimental results show the quality of anti-aliased IV image is improved. We also integrated the developed IV image into image-guided surgery display system. This approach will allow us to acquire the optimum process to produce high quality 3-D image for planning and guidance of minimally invasive surgery.

An Autostereoscopic Display System for Image-Guided Surgery Using High-Quality Integral Videography with High Performance Computing

This paper presents a high-quality glass-less autostereoscopic display system that is integrated into image-guided surgery. The glass-less autostereoscopic image was created by employing a modification of Integral Videography (IV), which is an animated extension of Integral Photography. IV records and reproduces 3-D images using a micro convex lens array and plat display, which can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without any need for special viewing devices. This paper reports the use of high-resolution IV display and high performance computing for producing medical 3-D image. We evaluate the feasibility of this display by using a set of medical image with segmentation and reproducing 3-D CT and MRI autostereoscopic image for surgical planning and intra-operative guidance. The main contribution of this paper is application and modification of medical 3-D display technique originally developed in high-resolution autostereoscopic display and high performance computing.

Real-time 3D-image-guided navigation system based on integral videography

A surgical navigation system that utilizes real-time three-dimensional (3D) image was developed. It superimposes the real, intuitive 3D image for medical diagnosis and operation. This system creates 3D image based on the principle of integral photography (IP), which can display geometrically accurate 3D autostereoscopic images and reproduce motion parallax without any need of special devices. We developed a new method for creating 3D autostereoscopic image, named Integral Videography (IV), which can also display moving 3D object. Thus the displayed image can be updated following the changes in surgeons field of vision during the operation. 3D image was superimposed on the surgical fields in the patient via a half-silvered mirror as if they could be seen through the body. In addition, a real-time Integral Videography algorithm for calculating the 3D image of surgical instruments was used for registration between the location of surgical instruments and the organ during the operation. The experimental results of targeting point location and avoiding critical area showed the errors of this navigation system were in the range of 2-3mm. By introducing a display device with higher pixel density, accuracy of the system can be improved. Because of the simplicity and the accuracy of real-time projected point location, this system will be practically usable in the medical field.

Three-Dimensional Image-Guided Navigation with Overlaid Three-Dimensional Image (Volumegraph) and Volumetric Ultrasonogram (V-US)

Augmented reality is necessary for minimally invasive surgery. Recent medical imaging technology developments, such as X-ray, computed tomography (CT), and magnetic resonance imaging (MRI), have changed the methods and the dimensions of observation of the human body. Such image data are required for preoperative investigation to recognize the structure of organs and tumors three-dimensionally. Three-dimensional images seen in the air by means of a beamed light have been reported previously. The images are superimposed on the patient’s head and body via a semitransparent mirror. We have applied these techniques to the navigation system for neurosurgical operations. The three-dimensional data obtained from CT and MRI before the operation were processed by a computer. This reconstructed three-dimensional image was superimposed and registered at the patient’s head according to fiduciary markers (registration). The application of augmented reality in the surgical field makes it possible to do a neurosurgical intervention more easily. The surgeon can operate by this three-dimensional image-guided navigation system easily and accurately. In the development of volumetric ultrasonogram (VUS) navigation, the distortion of the brain during surgery is the key problem for truly accurate and effective guidance.

Long viewing distance autostereoscopic display

This paper presents a technique of autostereoscopic display for distant viewing of a three-dimensional (3-D) image without the need for special glasses. Most of the previous studies have focused on improving the viewing resolution of integral photography (IP) image and widening the viewing angle. There has been no report about producing an IP image with a depth of several meters for viewing with the naked eye as we known. Our photo-based integral photography (IP) method enables precise 3-D images to be displayed at long viewing distances without any influence from deviated or distorted lenses in a lens array. We calculate the elemental images from a referential viewing area for each lens and project the corresponding result images to photographic film through each lens. We created an image display that appears to have three-dimensionality even when viewed from a distance, with an image depth of 5.7 m or more in front of the display, and 3.5 m or more behind the display. To the best of our knowledge, the presented long-distance IP display is technically unique as it is the first report of generating an image with such a long viewing distance.