Hagen Spies

Range Flow Estimation

Abstract We discuss the computation of the instantaneous 3D displacement vector fields of deformable surfaces from sequences of range data. We give a novel version of the basic motion constraint equation that can be evaluated directly on the sensor grid. The various forms of the aperture problem encountered are investigated and the derived constraint solutions are solved in a total least squares (TLS) framework. We propose a regularization scheme to compute dense full flow fields from the sparse TLS solutions. The performance of the algorithm is analyzed quantitatively for both synthetic and real data. Finally we apply the method to compute the 3D motion field of living plant leaves.

Study of Dynamical Processes with Tensor-Based Spatiotemporal Image Processing Techniques

Image sequence processing techniques are used to study exchange, growth, and transport processes and to tackle key questions in environmental physics and biology. These applications require high accuracy for the estimation of the motion field since the most interesting parameters of the dynamical processes studied are contained in first-order derivatives of the motion field or in dynamical changes of the moving objects. Therefore the performance and optimization of low-level motion estimators is discussed. A tensor method tuned with carefully optimized derivative filters yields reliable and dense displacement vector fields (DVF) with an accuracy of up to a few hundredth pixels/frame for real-world images. The accuracy of the tensor method is verified with computer-generated sequences and a calibrated image sequence. With the improvements in accuracy the motion estimation is now rather limited by imperfections in the CCD sensors, especially the spatial nonuniformity in the responsivity. With a simple two-point calibration, these effects can efficiently be suppressed. The application of the techniques to the analysis of plant growth, to ocean surface microturbulence in IR image sequences, and to sediment transport is demonstrated.

Accurate optical flow in noisy image sequences

Optical flow estimation in noisy image sequences requires a special denoising strategy. Towards this end we introduce a new tensor-driven anisotropic diffusion scheme which is designed to enhance optical-flow-like spatio-temporal structures. This is achieved by selecting diffusivities in a special manner depending on the eigenvalues of the well known structure tensor. We illustrate how the proposed choice differs from edge- and coherence-enhancing anisotropic diffusion. Furthermore we extend a recently discovered discretization scheme for anisotropic diffusion to 3D data. An automatic stop criterion to terminate the diffusion after a suitable time is given. The performance of the introduced method is examined quantitatively using image sequences with a substantial amount of noise added.

Estimation of Surface Flow and Net Heat Flux from Infrared Image Sequences

The study of dynamical processes at the sea surface interface using infrared image sequence analysis has gained tremendous popularity in recent years. Heat is transferred by similar transport mechanisms as gases relevant to global climatic changes. These similarities lead to the use of infrared cameras to remotely visualize and quantitatively estimate parameters of the underlying processes. Relevant parameters that provide important evidence about the models of air-sea gas transfer are the temperature difference across the thermal sub layer, the probability density function of surface renewal and the flow field at the surface. Being a driving force in air sea interactions, it is of equal importance to measure heat fluxes. In this paper we will present algorithms to measure the above parameters of air-sea gas transfer during night-time and show how to combine physical modeling and quantitative digital image processing algorithms to identify transport models. The image processing routines rely on an extension of optical flow computations to incorporate brightness changes in a total least squares (TLS) framework. Statistical methods are employed to support a model of gas transfer and estimate its parameters. Measurements in a laboratory environment were conducted and results verified with ground truth data gained from traditional measurement techniques.

Dense range flow from depth and intensity data

The combined use of intensity and depth information greatly helps in the estimation of the local 3D movements (range flow) of moving surfaces. We demonstrate how the two can be combined in both: a local total least squares algorithm, and an iterative global variational technique. While the former assumes locally constant flow, the latter relies on a smoothly varying flow field. The improvement achieved through incorporating intensity is illustrated qualitatively and quantitatively on synthetic and real test data.

Regularised Range Flow

Extending a differential total least squares method for range flow estimation we present an iterative regularisation approach to compute dense range flow fields. We demonstrate how this algorithm can be used to detect motion discontinuities. This can be used to segment the data into independently moving regions. The different types of aperture problem encountered are discussed. Our regularisation scheme then takes the various types of flow vectors and combines them into a smooth flow field within the previously segmented regions. A quantitative performance analysis is presented on both synthetic and real data. The proposed algorithm is also applied to range data from castor oil plants obtained with the Biris laser range sensor to study the 3-D motion of plant leaves.

Differential Range Flow Estimation

We present a total least squares based differential method for the estimation of 3D range flow from a sequence of range images. We address the various manifestations of the aperture problem encountered with this type of data. It is described how they can be detected and how the appropriate normal flow can be computed. The performance of the proposed method is assessed on both synthetic and real data.

Mixed OLS-TLS for the Estimation of Dynamic Processes with a Linear Source Term

We present a novel technique to eliminate strong biases in parameter estimation were part of the data matrix is not corrupted by errors. Problems of this type occur in the simultaneous estimation of optical flow and the parameter of linear brightness change as well as in range flow estimation. For attaining highly accurate optical flow estimations under real world situations as required by a number of scientific applications, the standard brightness change constraint equation is violated. Very often the brightness change has to be modelled by a linear source term. In this problem as well as in range flow estimation, part of the data term consists of an exactly known constant. Total least squares (TLS) assumes the error in the data terms to be identically distributed, thus leading to strong biases in the equations at hand. The approach presented in this paper is based on a mixture of ordinary least squares (OLS) and total least squares, thus resolving the bias encountered in TLS alone. Apart from a thorough performance analysis of the novel estimator, a number of applications are presented.

Estimation of complex motion from thermographic image sequences

In this contribution a novel technique for computing complex motion involving heat transport processes will be presented. The proposed technique is a local gradient based approach, combining transport models with motion analysis. It allows for the simultaneous estimation of both motion and parameter of an underlying transport model. Since the analysis is based on thermal image sequences, estimates are computed to a high temporal and spatial resolution, limited only by the resolution and frame rate of the employed IR camera. This novel technique was utilized on exchange processes at the atmosphere/ocean boundary, where significant parameters of heat transfer could be measured and a transport model verified. Using the presented algorithms, surface flows as well as convergences and divergences on air-water interfaces can be measured accurately. Apart from applications in oceanography and botany, relevant benefits of the proposed technique to NDT will be presented. It is possible to compensate for motion to reach accuracies much better than 1/10th of a pixel. Through the direct estimation of locally resolved diffusivities in materials, insights can be gained about defects present. By estimating not only isotropic diffusion but also the whole matrix of anisotropic diffusion, the technique is highly relevant to measurements of composite materials.

A Total Least Squares Framework for Low-Level Analysis of Dynamic Scenes and Processes

We present a new method to simultaneously estimate optical flow fields and parameters of dynamic processes, violating the standard brightness change constraint equation. This technique constitutes a straightforward generalization of the standard brightness constancy assumption. Using TLS estimation the spatiotemporal brightness structure is analyzed in an entirely symmetric way with respect to the spatial and temporal coordinates. We directly incorporate nonlinear brightness changes based upon differential equations of the underlying processes.