Boris Lenseigne

Accurate Gaze Direction Measurements With Free Head Movement for Strabismus Angle Estimation

We present the Delft Assessment Instrument for Strabismus in Young children (DAISY) a device designed to measure angles of strabismus in young children fast and accurately. DAISY allows for unrestrained head movements by the mean of a triple camera vision system that simultaneously estimates the head rotation and the eye pose. The device combines two different methods to record bilateral eye position: corneal reflections (Purkinje images) and pupillary images. Detailed results are provided on three orthotropic subjects (age 25-27). Three different conditions were tested: 1) gaze ahead, 2) gaze ahead with different head rotations, and 3) fixed head with different eye positions. Systematic errors occurred between subjects that need further study. The system reached sufficient accuracy to be applied for the measurement of angles of strabismus, almost independent from the head pose.

Simultaneous capture of the color and topography of paintings using fringe encoded stereo vision

IntroductionPaintings are versatile near-planar objects with material characteristics that vary widely. The fact that paint has a material presence is often overlooked, mostly due to the fact that we encounter many of these artworks through two dimensional reproductions. The capture of paintings in the third dimension is not only interesting for study, restoration and conservation, but it also facilitates making three dimensional reproductions through novel 3-D printing methods. No single imaging method is ideally suited to capture the painting’s color and topography and each of them have specific drawbacks. We have therefore designed an efficient hybrid imaging system dedicated to capturing paintings in both color and topography with a high resolution.ResultsA hybrid solution between fringe projection and stereo imaging is proposed involving two cameras and a projector. Fringe projection is aided by sparse stereo matching to serve as an image encoder. These encoded images processed by the stereo cameras then help solve the correspondence problem in stereo matching, leading to a dense and accurate topographical map, while simultaneously capturing its color. Through high-end cameras, special lenses and filters we capture a surface area of 170 square centimeter with an in-plane effective resolution of 50 micron and a depth precision of 9 micron. Semi-automated positioning of the system and data stitching consequently allows for the capture of larger surfaces. The capture of the 2 square meter big Jewish Bride by Rembrandt yielded 1 billion 3-D points.ConclusionThe reproductive properties of the imaging system are conform the digitization guidelines for cultural heritage. The data has enabled us to make high resolution 3-D prints of the works by Rembrandt and Van Gogh we have captured, and confirms that the system performs well in capturing both the color and depth information.

Fetoscopic Panorama Reconstruction: Moving from Ex-vivo to In-vivo

Twin-to-Twin Transfusion Syndrome (TTTS) is a condition that occurs in about 10% of pregnancies involving monochorionic twins. This complication can be treated with fetoscopic laser coagulation. The procedure could greatly benefit from panorama reconstruction to gain an overview of the placenta. Current state-of-the-art methods focus on panorama reconstruction in an ex-vivo setting. However, these methods fail in the in-vivo surgical setting. This paper describes the panorama reconstruction approach, the challenges posed by the in-vivo setting and the influence of these challenges on the panorama reconstruction. With experiments we show that the viewing quality is greatly reduced and that the limited motion of the fetoscope complicates and limits the precision of the image registration. We also identify the aspect necessary to shift from ex-vivo to in-vivo panorama reconstruction. Following our recommendations it should be possible to develop an approach that can be applied to TTTS surgery.

Empirical mode decomposition for saliency detection

We propose a novel method for saliency detection and attention selection inspired by processes in the human visual cortex. To mimic the varying spatial resolution of the human eye as well as the constant eye movements (saccades) and to model the effect of temporal adaptiveness, we use empirical mode decomposition and corresponding intrinsic mode functions (IMFs), instead of applying standard multi-scale framework as suggested in the state of the art. We derive IMFs between scales to calculate data driven center surround maps which locally reflect amount of information in the scene and we combine opposition color channels, luminosity information and orientation maps into a single saliency map calculated on IMFs. To equalize influence of different components contributing to the final saliency map, normalization steps are proposed. Finally, the MSER regions are calculated directly on the saliency map in order to obtain the most dominant points. We present results on both artificially generated images used in psychological experiments, natural images and application of our method for unknown object detection in robotics.

Stable Image Registration for In-Vivo Fetoscopic Panorama Reconstruction

A Twin-to-Twin Transfusion Syndrome (TTTS) is a condition that occurs in about 10% of pregnancies involving monochorionic twins. This complication can be treated with fetoscopic laser coagulation. The procedure could greatly benefit from panorama reconstruction to gain an overview of the placenta. In previous work we investigated which steps could improve the reconstruction performance for an in-vivo setting. In this work we improved this registration by proposing a stable region detection method as well as extracting matchable features based on a deep-learning approach. Finally, we extracted a measure for the image registration quality and the visibility condition. With experiments we show that the image registration performance is increased and more constant. Using these methods a system can be developed that supports the surgeon during the surgery, by giving feedback and providing a more complete overview of the placenta.

Reproducing oil paint gloss in print for the purpose of creating reproductions of Old Masters

In the field of Fine Art reproduction, 3D scanning plus 3D printing, combined with dedicated software, now allows to capture and reproduce the color and texture of oil paintings. However, for life-like reproduction of the material appearance of such paintings, the typical gloss and translucency must also be included, which is currently not the case. The aim of this paper is to elaborate on the challenges and results of capturing and reproducing oil paint gloss (next to texture and color) using a scanning and printing system. A sample was hand-made using oil paint and acrylic varnish, and its gloss was then reproduced. A gloss map of the painted sample was acquired using a high end DLSR camera and a simple acquisition protocol. Next, Océ High Resolution 3D printing technology was used to create samples with spatially varying gloss. For this, two different strategies were combined: (1) multilevel half-toning of the colors was used to reproduce matte color layers, and (2) varnish was half-toned on top in increasing coverage to recreate increasing gloss levels. This paper presents an overview of the state-of-the-art literature in gloss reproduction and perception, our process of reproduction as well as the visual evaluation of the quality of the created reproduction.

A Novel Method for Simultaneous Acquisition of Visible and Near-Infrared Light Using a Coded Infrared-Cut Filter

This paper presents a novel image sensing method to enhance the sensitivity of a camera. Most image sensors used in commercial digital cameras are sensitive for both visible and infrared light. An IR-cut filter, that obstructs the infrared component of natural light, is used in such cameras to realize a similar color reproduction as for the human visual system. However, recent studies have shown that the near infrared light contains useful information to further enhance the visible image. This paper introduces a new sensing method by using a coded IR-cut filter to enable simultaneous capturing of NIR and visible light on a single image sensor. The coded IR-cut filter lets a fraction of the near infrared light pass and blocks out the rest. Both visible and near infrared light images can be separated from the sensor output when taking the diffraction of the NIR light into account. Experiments, using a synthesized image sensor output, demonstrate the validity of the method.

Color sensitivity of the multi-exposure HDR imaging process

Abstract Multi-exposure high dynamic range(HDR) imaging builds HDR radiance maps by stitching together different views of a same scene with varying exposures. Practically, this process involves converting raw sensor data into low dynamic range (LDR) images, estimate the camera response curves, and use them in order to recover the irradiance for every pixel. During the export, applying white balance settings and image stitching, which both have an influence on the color balance in the final image. In this paper, we use a calibrated quasi-monochromatic light source, an integrating sphere, and a spectrograph in order to evaluate and compare the average spectral response of the image sensor. We finally draw some conclusion about the color consistency of HDR imaging and the additional steps necessary to use multi-exposure HDR imaging as a tool to measure the physical quantities such as radiance and luminance.

Depth from vergence and active calibration for humanoid robots

In human eyes, many clues are used to perceive depth. For nearby tasks involving eye-hand coordination, depth from vergence is a strong cue. In our research on humanoid robots we study binocular robotic eyes that can pan and tilt and perceive depth from stereo, as well as depth from vergence by fixing both eyes on a nearby object. In this paper, we report on a convergent robot vision set-up: Firstly, we describe the mathematical model for convergent vision system. Secondly, we introduce an algorithm to estimate the depth of an object under focus. Thirdly, as the centers of rotation of the eye motors do not align with the center of image planes, we develop an active calibration algorithm to overcome this problem. Finally, we examine the factors that have impact on the depth error. The results of experiments and tests show the good performance of our system and provide insight into depth from vergence.