Emile A. Hendriks

Video segmentation by MAP labeling of watershed segments

This paper addresses the problem of spatio-temporal segmentation of video sequences. An initial intensity segmentation method (watershed segmentation) provides a number of initial segments which are subsequently labeled, with a known number of labels, according to motion information. The label field is modeled as a Markov random field where the statistical spatial and and temporal interactions are expressed on the basis of the initial watershed segments. The labeling criterion is the maximization of the conditional a posteriori probability of the label field given the motion hypotheses, the estimate of the label field of the previous frame, and the image intensities. For the optimization, an iterative motion estimation-labeling algorithm is proposed and experimental results are presented.

Sign Language Recognition by Combining Statistical DTW and Independent Classification

To recognize speech, handwriting, or sign language, many hybrid approaches have been proposed that combine dynamic time warping (DTW) or hidden Markov models (HMMs) with discriminative classifiers. However, all methods rely directly on the likelihood models of DTW/HMM. We hypothesize that time warping and classification should be separated because of conflicting likelihood modeling demands. To overcome these restrictions, we propose using statistical DTW (SDTW) only for time warping, while classifying the warped features with a different method. Two novel statistical classifiers are proposed - combined discriminative feature detectors (CDFDs) and quadratic classification on DF Fisher mapping (Q-DFFM) - both using a selection of discriminative features (DFs), and are shown to outperform HMM and SDTW. However, we have found that combining likelihoods of multiple models in a second classification stage degrades performance of the proposed classifiers, while improving performance with HMM and SDTW. A proof-of-concept experiment, combining DFFM mappings of multiple SDTW models with SDTW likelihoods, shows that, also for model-combining, hybrid classification can provide significant improvement over SDTW. Although recognition is mainly based on 3D hand motion features, these results can be expected to generalize to recognition with more detailed measurements such as hand/body pose and facial expression.

A realtime hardware system for stereoscopic videoconferencing with viewpoint adaptation

This paper describes a hardware system and the underlying algorithms that were developed for realtime stereoscopic videoconferencing with viewpoint adaptation within the European PANORAMA project. The goal was to achieve a true telepresence illusion for the remote partners. For this purpose, intermediate views at arbitrary positions must be synthesized from the views of a stereoscopic camera system with rather large baseline. The actual viewpoint is adapted according to the head position of the viewer, such that the impression of motion parallax is produced. The whole system consists of a disparity estimator, stereoscopic MPEG-2 encoder, disparity encoder and multiplexer at the transmitter side, and a demultiplexer, disparity decoder, MPEG-2 decoder and interpolator with viewpoint adaptation at the receiver side. For transmission of the encoded signals, an ATM network is provided. In the final system, autostereoscopic displays will be used. The algorithms for disparity estimation, disparity encoding and disparity-driven intermediate viewpoint synthesis were specifically developed under the constraint of hardware feasibility.

Towards a Robust Solution to People Counting

Estimating the number of persons in a public place provides useful information for video-based surveillance and monitoring applications. In the case of oblique camera setup, counting is either achieved by detecting individuals or by statistically establishing relations between values of simple image features (e.g. amount of moving pixels, edge density, etc.) to the number of people. While the methods of the first category exhibit poor accuracy in cases of occlusions, the second category of methods are sensitive to perspective distortions, and require people to move in order to be counted. In this paper we investigate the possibilities of developing a robust statistical method for people counting. To maximize its applicability scope, we choose-in contrast to the majority of existing methods from this category-not to require prior learning of categories corresponding to different number of people. Second, we search for a suitable way of correcting the perspective distortion. Finally, we link the estimation to a confidence value that takes into account the known factors being of influence on the result. The confidence is then used to refine final results.

Isophote properties as features for object detection

Usually, object detection is performed directly on (normalized) gray values or gray primitives like gradients or Haar-like features. In that case the learning of relationships between gray primitives, that describe the structure of the object, is the complete responsibility of the classifier. We propose to apply more knowledge about the image structure in the preprocessing step, by computing local isophote directions and curvatures, in order to supply the classifier with much more informative image structure features. However, a periodic feature space, like orientation, is unsuited for common classification methods. Therefore, we split orientation into two more suitable components. Experiments show that the isophote features result in better detection performance than intensities, gradients or Haar-like features.

Influence of the observation likelihood function on particle filtering performance in tracking applications

Since the introduction of particle filtering for object tracking, a lot of improvements have been suggested. However, the definition of the observation likelihood function, needed for determining the particle weights, has received little attention. Because particle weights determine how the particles are re-sampled, the likelihood function has a strong influence on the tracking performance. We show experimental results for three different tracking tasks for different parameter values of the assumed observation model. The results show a large influence of the model parameters on the tracking performance. Optimizing the likelihood function can give significant tracking improvement. Different optimal parameter settings are observed for the three different tracking tasks. Consequently, when performing multiple tasks a trade-off must be made for the parameter setting. In practical situations where robust tracking must be achieved with a limited amount of particles, the true observation probability is not always the optimal likelihood function.

A fast and robust point tracking algorithm

We present an algorithm that efficiently tracks a predefined set of landmark points in a time sequence of images. The algorithm iteratively optimizes the correspondences between the point measurements in the images, while allowing for spurious and missing point measurements. This trajectory based approach hypothesizes missing points by interpolation. Spurious measurements are either left our because they do not form the optimal correspondences or are removed afterwards if they have the smoothness or other constraint exceed its predetermined maximum.

Detection of pollen grains in multifocal optical microscopy images of air samples.

Pollen is a major cause of allergy and monitoring pollen in the air is relevant for diagnostic purposes, development of pollen forecasts, and for biomedical and biological researches. Since counting airborne pollen is a time‐consuming task and requires specialized personnel, an automated pollen counting system is desirable. In this article, we present a method for detecting pollen in multifocal optical microscopy images of air samples collected by a Burkard pollen sampler, as a first step in an automated pollen counting procedure. Both color and shape information was used to discriminate pollen grains from other airborne material in the images, such as fungal spores and dirt. A training set of 44 images from successive focal planes (stacks) was used to train the system in recognizing pollen color and for optimization. The performance of the system has been evaluated using a separate set of 17 image stacks containing 65 pollen grains, of which 86% was detected. The obtained precision of 61% can still be increased in the next step of classifying the different pollen in such a counting system. These results show that the detection of pollen is feasible in images from a pollen sampler collecting ambient air. This first step in automated pollen detection may form a reliable basis for an automated pollen counting system. Microsc. Res. Tech., 2009.

Synthesis of multi viewpoint images at non-intermediate positions

We present an algorithm for the synthesis of multi viewpoint images at non-intermediate positions, based on stereoscopic images. We consider the synthesis of images from virtual camera positions and the synthesis of images for scene reconstruction using stereo displays. The algorithm provides scene reconstruction without geometric distortion and without any restriction to the position of the viewer. All synthesized images are based on extrapolation of a single source image and a single disparity field. This provides low use of bandwidth and compatibility with mono video systems. With teleconferencing images, the generated views were subjectively evaluated as good for viewing positions not more than one half camera baseline from the centre position. Objectively, reconstructed left and right images have PSNR values of 41 dB.

Articulated Planar Reformation for Change Visualization in Small Animal Imaging

The analysis of multi-timepoint whole-body small animal CT data is greatly complicated by the varying posture of the subject at different timepoints. Due to these variations, correctly relating and comparing corresponding regions of interest is challenging.In addition, occlusion may prevent effective visualization of these regions of interest. To address these problems, we have developed a method that fully automatically maps the data to a standardized layout of sub-volumes, based on an articulated atlas registration.We have dubbed this process articulated planar reformation, or APR. A sub-volume can be interactively selected for closer inspection and can be compared with the corresponding sub-volume at the other timepoints, employing a number of different comparative visualization approaches. We provide an additional tool that highlights possibly interesting areas based on the change of bone density between timepoints. Furthermore we allow visualization of the local registration error, to give an indication of the accuracy of the registration. We have evaluated our approach on a case that exhibits cancer-induced bone resorption.