Ng Wan Sing

Photorealistic rendering for augmented reality using environment illumination

Mixing 3D computer-generated images with real-scene images seamlessly in augmented reality has many desirable and wide areas of applications such as entertainment, cinematography, design visualization and medical trainings. The challenging task is to make virtual objects blend harmoniously into the real scene and appear as if they are like real. Apart from constructing detailed geometric 3D model representation and obtaining accurate surface properties for virtual objects, adopting real scene lighting information to render virtual objects is another important factor to achieve photorealistic rendering. Such a factor not only improves visual complexity of virtual objects, but also determines the consistency of illumination between the virtual objects and the surrounding real objects in the scene. Conventional rendering techniques such as ray tracing, and radiosity require intensive computation and data preparation to solve the lighting transport equation. Hence, they are less practical for rendering virtual objects in augmented reality, which demands a real-time performance. This work explores an image-based and hardware-based approach to improve photorealism for rendering synthetic objects in augmented reality. It uses a recent technique of image-based lighting, environment illumination maps, and a simple yet practical multi-pass rendering framework for augmented reality.

A HIFU robot for transperineal treatment of prostate cancer

Prostate cancer accounts for 29% of all cancers diagnosed in males in the United States. Present therapeutic procedures for prostate cancer are either invasive or minimally invasive. However, recent advances in technology (e.g. High Intensity Focused Ultrasound (HIFU)) allow the possibility of also non-invasive therapeutic procedures. This paper describes localizing the position of the prostate using an ultrasound probe. With this information, the robot can accurately manipulate a non-invasive therapeutic tool, such as HIFU, to an area of interest in the prostate via the transperineal route.

MRI head segmentation for object based volume visualization

In this paper, we present a new image segmentation approach for MRI of the head, which is a semi-automatic process. Unlike automatic segmentation or manual segmentation, the semi-automatic segmentation approach is a robust and interactive segmentation process. This approach carries out 3D volume data segmentation based on 2D image slices. By utilising the user-provided image mask, including areas of interest or structural information, the semi-automatic segmentation process can generate a new segmented volume dataset and structural information. The object based volume visualization method can use this segmented dataset and structural information to perform structure based manipulation and visualization, which cannot be achieved using a normal volume rendering method.

Photorealistic rendering for augmented reality

This paper explores an image-based approach to improve photorealism for rendering synthetic objects in augmented reality. It uses pre-filtered high-dynamic range environment maps of the real scene, containing real lighting information, to render diffuse and glossy virtual objects with a multi-pass technique via texture mapping operation. In addition, the light sources, required for soft shadows generation, can be estimated from the maps. Soft shadows are added to the virtual objects using shadow buffer technique, which can utilize graphics hardware. This work can be beneficial for ARToolkit development in improving photorealism of virtual objects.

Convex object based volume visualization: a formal proof and example

Abstract In this paper, we provide formal definitions for object space, space subdivision and ray tracing. We also introduce concept models such as ordered space subdivisions and ordered visualization. From these works, we can evaluate the correctness and performance of our space subdivision scheme for volume visualization. With these definitions, we also analyze some established space subdivision approaches such as uniform space subdivision and octree based space subdivision. In this paper, we suggest a new subdivision strategy, the convex object based subdivision scheme, which can be more efficient in generating the convex objects for ordered visualization.

Flexible Needle-tissue Interaction Modelling Using Depth-varying Mean Parameter

Needle insertion in soft tissue has raised a lot of research interests lately. A spring-beam-damper model has been developed for needle-tissue interaction. Considering the tissue inhomogeneity, depth-varying mean parameters are proposed to calculate the spring/damper reaction forces. Based on this approach, an online parameter estimator has been designed using modified least square method with forgetting factor. In this paper, mathematical relationships are developed for efficient and robust needle tip trajectory prediction in an inhomogeneous and noisy environment. Extensive experiments have been carried out in various phantoms to validate the improved needle-tissue interaction model based on the online experimental data. Results reveal that the model can track the needle tip trajectory with satisfactory accuracy even in the presence of large disturbances and noises, but sometimes at the cost of the parameter estimation. Experiments also show that the convergent rate and estimation accuracy can be greatly improved when the needle gets supported.

Tracked arm manipulator for lumpectomy

In this paper, we present a tracked and semimotorized arm manipulator system aiming to provide accurate positioning information for breast biopsy tool to be used as a treatment device. The system is integrated with augmented reality (AR) and visual reality (VR) visualization. Two tracking systems were used on the arm, camera tracking and encoder tracking to simultaneously track the ultrasound probe and surgical tool. Stereo-camera is used to track an optical marker, which is temporary mounted. The marker is removed once the initial location of the arms is computed and encoder-based tracking takes over. On each passive joint of the arm system, a rotary encoder is installed which tells the angle of each joint. Therefore the positions of both ultrasound probe and surgical tool are known. The surgical tool is fitted onto a motorized 2 degree of freedom (DOF) with motorized cutting sequence, which enables cutting planning, probe positioning and controlled tissue cutting procedure. The prototype of the system proved the concept and future work is expected to improve the accuracy.

Interactive surgical planning using context based volume visualization techniques

We present a new volume visualization scheme, context based volume visualization to assist the surgeon in surgical planning. This visualization scheme differs from ordinary surface based rendering and volume rendering by providing a framework to combine surface based rendering and volume rendering. We can achieve the powerful manipulating capability of surface based rendering and as good a rendering effect as volume rendering at the same time. Using a special data structured-segment tree to manage the visualization scene, which includes all the volume objects and graphics objects (the surgical tools) needed to be visualized, this visualization scheme provides a common context based interface for both graphics objects and volume objects, through which, we can control graphics and volume objects easily in the same way. The context based visualization scheme can greatly increase the performance of volume visualization by generating the scene much faster through selectively revisualizing the affected objects. Based on these ideas, we implement an interactive surgical planning system, Virtual Doctor based on OpenGL 1.1 on WinNT platform and VolumePro vg500 card. This system is a good 3D volume visualization tool and augmented reality system for interactive surgical planning and further research in this area.