Patrick Verlinde

Multi-modal identity verification using expert fusion

The contribution of this paper is to compare paradigms coming from the classes of parametric, and non-parametric techniques to solve the decision fusion problem encountered in the design of a multi-modal biometrical identity verification system. The multimodal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called score, stating how well the claimed identity is verified. A decision fusion module receiving as input the d scores has to take a binary decision: accept or reject the claimed identity. We have solved this fusion problem using parametric and non-parametric classifiers. The performances of all these fusion modules have been evaluated and compared with other approaches on a multi-modal database, containing both vocal and visual biometric modalities. ” 2000 Elsevier Science B.V. All rights reserved.

Multi-modal identity verification using support vector machines (SVM)

The contribution of this paper is twofold: (1) to formulate a decision fusion problem that is encountered in the design of a multi-modal identity verification system as a particular classification problem, and (2) to solve this problem by using a support vector machine (SVM). The multi-modal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called a score, stating how well the claimed identity is verified. A fusion module receiving the d scores as input has to take a binary decision: to accept or reject the identity. This fusion problem has been solved using SVMs. The performance of this fusion module has been evaluated and compared with other proposed methods on a multi-modal database containing both vocal and visual modalities.

Multilevel data fusion for the detection of targets using multispectral image sequences

An approach is presented to the long range automatic detec- tion of vehicles, using multisensor image sequences. The method is tested on a database of multispectral image sequences, acquired under diverse operational conditions. The approach consists of two parts. The first part uses a semisupervised approach, based on texture parameters, for detecting stationary targets. For each type of sensor one learning image is chosen. Texture parameters are calculated at each pixel of the learning images and are combined using logistic regression into a value that represents the conditional probability that the pixel belongs to a target given the texture parameters. The actual detection algorithm ap- plies the same combination to the texture features calculated on the remainder of the database (test images). When the results of this feature-level fusion are stored as an image, the local maxima correspond to likely target positions. These feature-level-fused images are calcu- lated for each sensor. In a sensor fusion step, the results obtained per sensor are then combined again. Region growing around the local maxima is then used to detect the targets. The second part of the algo- rithm searches for moving targets. To detect moving vehicles, any mo- tion of the sensor must be detected first. If sensor motion is detected, it is estimated using a Markov random field model. Available prior knowl- edge about the sensor motion is used to simplify the motion estimation. The estimate is used to warp past images onto the current image in a temporal fusion approach and moving targets are detected by threshold- ing the difference between the original and warped images. Decision level fusion combines the results from both parts of the algorithm.

Multilevel data fusion approach for gradually upgrading the performances of identity verification systems

The aim of this paper is to propose a strategy that uses data fusion at three different levels to gradually improve the performance of an identity verification system. In a first step temporal data fusion can be used to combine multiple instances of a single (mono-modal) expert to reduce its measurement variance. If system performance after this first step is not good enough to satisfy the end-users needs, one can improve it by fusing in a second step result of multiple experts working on the same modality. For this approach to work, it is supposed that the respective classification errors of the different experts are de-correlated. Finally, if the verification systems performance after this second step is still not good enough, one will be forced to move onto the third step in which performance can be improved by using multiple experts working on different (biometric) modalities. To be useful however, these experts have to be chosen in such a way that adding the extra modalities increases the separation in the multi-dimensional modality-space between the distributions of the different populations that have to be classified by the system. This kind of level-based strategy allow to gradually tune the performance of an identity verification system to the end-users requirements while controlling the increase of investment costs. In this paper results of several fusion modules will be shown at each level. All experiments have been performed on the same multi-modal database to be able to compare the gain in performance each time one goes up a level.

Combining vocal and visual cues in an identity verification system using K-NN based classifiers

The contribution of this paper is twofold: (1) to formulate a fusion problem encountered in the design of a multi-modal identity verification system as a particular classification problem, (2) to propose a simple classifier to solve this problem. The multi-modal identity verification system under consideration is built of d modalities in parallel, each one delivering as output a scalar number, called the score, stating how well the claimed identity is verified. A fusion module receiving as input the d scores has to take a binary decision: accept or reject identity. We have solved this fusion problem using a classic k-nearest-neighbor (k-NN) classifier. The most important problem encountered with this simple classifier is the unbalance between the number of reference points in either class. Adapting the classic k-NN classifier using distance weighting and vector quantization principles enables to reduce the influence and the number of impostor reference points respectively. This constitutes the originality of this paper. The performances of these different fusion modules have been evaluated on a multi-modal database, containing both vocal and visual modalities.

Long-range target detection in a cluttered environment using multisensor image sequences

An approach to the long range automatic detection of vehicles, using multi-sensor image sequences, is presented. The algorithm was tested on a database of six sequences, acquired under diverse operational conditions. The vehicles in the sequences can be either moving or stationary. The sensors also can be moving. The presented approach consists of two parts. The first part detects targets in single images using seven texture measurements. The values of some of the textural features at a target position will differ from those found in the background. To perform a first classification between target- and non-target pixels, linear discriminant analysis is used on one test image for each type of sensor. Because the features are closely linked to the physical properties of the sensors, the discriminant function also gives good results to the remainder of the database sequences. By applying the discriminant function to the feature space of textural parameters, a new image is created. The local maxima of this image correspond to probably target positions. To reduce the false alarm rate, any available prior knowledge about possible target size and aspect ratio is incorporated using a region growing procedure around the local maxima. The second part of the algorithm detects moving targets. First any motion of the sensor itself need to be detected. The detection is based on a comparison of the spatial cooccurrence matrix within one image and the temporal cooccurrence matrix between successive images. If sensor motion is detected, it is estimated using a multi-resolution Markov Random Field model. Available prior knowledge about the sensor motion is used to simplify the motion estimation. The motion estimate is used to warp past images onto the current one. Moving targets are detected by thresholding the difference between the original and warped images. Temporal and spatial consistency are used to reduce false alarm rate.

The joint multi-sensor mine-signatures project: an opportunity for testing data fusion algorithms!

The Multi-sensor Mine-signatures (MsMs) project is a major campaign to collect calibrated and well- documented data, suitable for use by workers developing advanced multi-sensor algorithms for antipersonnel mine detection. The data, together with a full description of the site layout and measurement protocols, are publicly avail- able via the website http://demining.jrc. itlmsms. Measurements are made on a test lane consisting of 7 plots of different soils, each 6m by 6m, populated with sur- rogate mines, calibration objects, simulated clutter and po- sition markers. There are 48 targets in each plot, conjgured identically for all plots. Many diferent sensors have been used to collect the data including pulsed and continuous wave metal detec- tors, ground penetrating radars, a microwave radiometer; thermal infrared cameras, a magnetometer; and a scanning laser doppler vibrometel: The database is in an ongoing process of being updated and it is also being reformatted to make it more uniform and user-fnendly. The test site remains essentially unchanged, apart from some equipment upgrades, and is still available for further data collection. In particular, the targets have not been moved, so as to provide stable surrounding soil conditions representative of mines le

Recent progress in the joint multisensor mine-signatures database project

undisturbed for long periods post-conjict.

Preparing the Joint Multi-sensor Mine-signatures project database for data fusion

The MsMs project is a major campaign to collect calibrated and well-documented data, suitable for use by workers developing advanced multisensor algorithms for antipersonnel mine detection. The data, together with a full description of the site layout and measurement protocols, are publicly available via the internet site http://demining.jrc.it/msms. Measurements are made on a test lane consisting of 7 plots of different soils, each 6m by 6m, populated with surrogate mines, calibration objects, simulated clutter and position markers. There are 48 targets in each plot, configured identically for all plots. A first report was presented last year. Since then, laser acoustic vibrometer and magnetometer data have been added and the metal detector and thermal infrared data have been augmented. The database has been reformatted to make it more uniform and user-friendly and to remove typographic mistakes. The test site remains essentially unchanged, apart from some equipment upgrades, and is available for further data collection. In particular, the targets have not been moved, so as to provide stable surrounding soil conditions representative of mines left undisturbed for long periods post-conflict. This presentation will describe the new data and data format, the status of the upgrades and the outlook for the future.

First results of the joint multisensor mine-signatures measurement campaign (MsMs Project)

The Joint Multi-sensor Mine-signatures (MsMs) project was started in the year 2000 and has as its main goal to organize and execute an experimental campaign for collecting data of buried land-mines with multiple sensors. These data sets are being made widely available to researchers and developers working amongst else on sensor fusion, and signal processing for improved detection and identification of land-mines. The. outdoor test facility (6/spl times/80 m) of the Joint Research Centre (JRC) of the European Commission, located at Ispra (Italy), houses the test minefield. Six test strips of 6/spl times/6 m consisting of different soil types (cluttered grassy terrain, loamy soil, sandy soil, clay soil, soil with high content of organic matter and ferromagnetic soil) are complemented with one reference test strip of 6/spl times/6 m consisting of pure sand. The list of objects buried in the minefield includes mine simulants of three different dimensions with either a low or a high metal content, reference targets for position referencing and calibration checking, and clutter objects including empty bullet cartridges, metal cans, barbed wire, stones, wood, plastic boxes, etc. This test minefield is going to be left intact for a long period, in order to be able to perform multiple runs on it. For the test campaign of the year 2000, the core sensors were a metal detector a ground penetrating radar, a microwave radiometer and several thermal infrared imagers. The first data sets are in the process of being released right now. This paper aims to prepare the first results to be obtained by fusing the data coming from these different sensors, by presenting the available datasets.