Franck Marzani

Reconstruction of hyperspectral cutaneous data from an artificial neural network-based multispectral imaging system

The development of an integrated MultiSpectral Imaging (MSI) system yielding hyperspectral cubes by means of artificial neural networks is described. The MSI system is based on a CCD camera, a rotating wheel bearing a set of seven interference filters, a light source and a computer. The resulting device has been elaborated for in vivo imaging of skin lesions. It provides multispectral images and is coupled with a software reconstructing hyperspectral cubes from multispectral images. Reconstruction is performed by a neural network-based algorithm using heteroassociative memories. The resulting hyperspectral cube provides skin optical reflectance spectral data combined with bidimensional spatial information. This combined information will hopefully improve diagnosis and follow-up in a range of skin disorders from skin cancer to inflammatory diseases.

Development of a Protocol for CCD Calibration: Application to a Multispectral Imaging System

In this paper we describe in detail a method for calibrating a CCD-based camera. The calibration aims to remove both temporal and systematic noises introduced by the sensor, electronics, and optics after which we can correct the non-linearity of its response. For the non-linearity correction we use a simple and powerful approach consisting on a complementary approach between a polynomial fitting and an LUT based algorithm. The proposed methodology is accurate in the sense that it takes into account individual characteristics of each pixel. In each pixel, systematic noises are measured through acquiring offset images, thermal images, and FlatField images. A rigorous protocol for acquiring these images based on experimentation is established. The method to acquire Flat-Field image is novel and is particularly efficient in that it can correct all defects due to non-uniform pixel responses, vignettage, blemishes on optic and/or filters, and perhaps even illumination nonuniformity. We notice that such a methodology of calibration is particularly efficient in the case of an optical filter based multispectral imaging system, although it remains valid for any imaging system based on a CCD sensor.

A 3-D marker-free system for the analysis of movement disabilities - an application to the legs

Describes an approach allowing the analysis of human motion in 3D space. The system that we developed is composed of three CCD (charge-coupled device) cameras that capture synchronized image sequences of a human body in motion without the use of markers. Characteristic points belonging to the boundaries of the body in motion are first extracted from the initial images. 2D superquadrics are then adjusted on these points by a fuzzy clustering process. After that, the position of a 3D model based on a set of articulated superquadrics, each of them describing a part of the human body, is reconstructed. An optical flow process allows the prediction of the position of the model from its position at a previous time, and gives initial values for the fuzzy classification. The results that we present more specifically concern the analysis of movement disabilities of a human leg during gait. They are improved by using articulation-based constraints. The methodology can be used in human motion analysis for clinical applications.

Optical calibration of a multispectral imaging system based on interference filters

We present a new approach to optically calibrate a multispectral imaging system based on interference filters. Such a system typically suffers from some blurring of its channel images. Because the effectiveness of spectrum reconstruction depends heavily on the quality of the acquired channel images, and because this blurring negatively affects them, a method for deblurring and denoising them is required. The blur is modeled as a uniform intensity distribution within a circular disk. It allows us to characterize, quantitatively, the degradation for each channel image. In terms of global reduction of the blur, it consists of the choice of the best channel for the focus adjustment according to minimal corrections applied to the other channels. Then, for a given acquisition, the restoration can be performed with the computed parameters using adapted Wiener filtering. This process of optical calibration is evaluated on real images and shows large improvements, especially when the scene is detailed.

Integration of 3D and multispectral data for cultural heritage applications: Survey and perspectives

Cultural heritage is increasingly put through imaging systems such as multispectral cameras and 3D scanners. Though these acquisition systems are often used independently, they collect complementary information (spectral vs. spatial) used for the study, archiving and visualization of cultural heritage. Recording 3D and multispectral data in a single coordinate system enhances the potential insights in data analysis. We present the state of the art of such acquisition systems and their applications for the study of cultural heritage. We also describe existing registration techniques that can be used to obtain 3D models with multispectral texture and explore the idea of optically tracking acquisition systems to ensure an easy and precise registration.

3-D shape reconstruction in an active stereo vision system using genetic algorithms

Abstract The recovery of 3-D shape information (depth) using stereo vision analysis is one of the major areas in computer vision and has given rise to a great deal of literature in the recent past. The widely known stereo vision methods are the passive stereo vision approaches that use two cameras. Obtaining 3-D information involves the identification of the corresponding 2-D points between left and right images. Most existing methods tackle this matching task from singular points, i.e. finding points in both image planes with more or less the same neighborhood characteristics. One key problem we have to solve is that we are on the first instance unable to know a priori whether a point in the first image has a correspondence or not due to surface occlusion or simply because it has been projected out of the scope of the second camera. This makes the matching process very difficult and imposes a need of an a posteriori stage to remove false matching. In this paper we are concerned with the active stereo vision systems which offer an alternative to the passive stereo vision systems. In our system, a light projector that illuminates objects to be analyzed by a pyramid-shaped laser beam replaces one of the two cameras. The projections of laser rays on the objects are detected as spots in the image. In this particular case, only one image needs to be treated, and the stereo matching problem boils down to associating the laser rays and their corresponding real spots in the 2-D image. We have expressed this problem as a minimization of a global function that we propose to perform using Genetic Algorithms (GAs). We have implemented two different algorithms: in the first, GAs are performed after a deterministic search. In the second, data is partitioned into clusters and GAs are independently applied in each cluster. In our second contribution in this paper, we have described an efficient system calibration method. Experimental results are presented to illustrate the feasibility of our approach. The proposed method yields high accuracy 3-D reconstruction even for complex objects. We conclude that GAs can effectively be applied to this matching problem.

Skin parameter map retrieval from a dedicated multispectral imaging system applied to dermatology/cosmetology

In vivo quantitative assessment of skin lesions is an important step in the evaluation of skin condition. An objective measurement device can help as a valuable tool for skin analysis. We propose an explorative new multispectral camera specifically developed for dermatology/cosmetology applications. The multispectral imaging system provides images of skin reflectance at different wavebands covering visible and near-infrared domain. It is coupled with a neural network-based algorithm for the reconstruction of reflectance cube of cutaneous data. This cube contains only skin optical reflectance spectrum in each pixel of the bidimensional spatial information. The reflectance cube is analyzed by an algorithm based on a Kubelka-Munk model combined with evolutionary algorithm. The technique allows quantitative measure of cutaneous tissue and retrieves five skin parameter maps: melanin concentration, epidermis/dermis thickness, haemoglobin concentration, and the oxygenated hemoglobin. The results retrieved on healthy participants by the algorithm are in good accordance with the data from the literature. The usefulness of the developed technique was proved during two experiments: a clinical study based on vitiligo and melasma skin lesions and a skin oxygenation experiment (induced ischemia) with healthy participant where normal tissues are recorded at normal state and when temporary ischemia is induced.

Toward a 3D Multispectral Scanner: An Application to Multimedia

A stereoscopic system based on a multispectral camera and an LCD projector uses multispectral information for 3D object reconstruction. By linking 3D points to a curve representing the spectral reflectance, the system gives a physical representation of the matter thats independent from illuminant, observer, and acquisition devices

Neural networks in two cascade algorithms for spectral reflectance reconstruction

In this paper, we deal with the problem of the spectral reflectance curves reconstruction. Because of the reconstruction of such curves is an inverse problem, slight variations in input data completely skew the expected results. So, finding a robust reconstruction operator is highly required. We present a robust method based upon neural networks. This method takes advantage of that neural networks are generally robust to the noise. Furthermore, we propose two cascade algorithms of using these neural networks. The first algorithm allows faithful reconstruction of spectra that are previously learned. The second algorithm allows good generalization allowing for reconstructing a wide range of reflectance that are not learned in the training stage. The results confirm the robustness and the reliability of the proposed method compared to some classical ones.

Calibration of a three-dimensional reconstruction system using a structured light source

We present a method for calibrating a range finder system composed of a camera and a structured light source. The system is used to reconstruct the three-dimensional (3-D) surface of an object. This is achieved by projecting a pattern, represented by a set of regularly spaced spots, on the surface of the object using the structured light source. An image of the illuminated object is next taken and by analyzing the distortion of the projected pattern, the 3-D surface of the object can be reconstructed. This reconstruction operation can be envisaged only if the system is calibrated. Instead of using a classical calibration method, which is based on the determination of the matrices that characterize the intrinsic and extrinsic parameters of the system, we propose a fast and easy to set up methodology, consisting of taking a sequence of images of a plane in translation on which a set of regularly spaced spots is projected using the structured light projection system. Next, a relation- ship between the position of the plane and the coordinates of the spots in the image is established. Using this relationship, we are able to deter- mine the 3-D coordinates of a set of points on the objects surface know- ing the 2-D coordinates of the spots in the image of the object taken by the range finder system. Finally, from the 3-D coordinates of the set of points, the 3-D surface of the object is reconstructed.