Benjamin Berkels

A Variational Framework for Simultaneous Motion Estimation and Restoration of Motion-Blurred Video

The problem of motion estimation and restoration of objects in a blurred video sequence is addressed in this paper. Fast movement of the objects, together with the aperture time of the camera, result in a motion-blurred image. The direct velocity estimation from this blurred video is inaccurate. On the other hand, an accurate estimation of the velocity of the moving objects is critical for restoration of motion-blurred video. Therefore, restoration needs accurate motion estimation and vice versa, and a joint process is called for. To address this problem we derive a novel model of the blurring process and propose a Mumford-Shah type of variational framework, acting on consecutive frames, for joint object deblurring and velocity estimation. The proposed procedure distinguishes between the moving object and the background and is accurate also close to the boundary of the moving object. Experimental results both on simulated and real data show the importance of this joint estimation and its superior performance when compared to the independent estimation of motion and restoration.

Optimized imaging using non-rigid registration.

The extraordinary improvements of modern imaging devices offer access to data with unprecedented information content. However, widely used image processing methodologies fall far short of exploiting the full breadth of information offered by numerous types of scanning probe, optical, and electron microscopies. In many applications, it is necessary to keep measurement intensities below a desired threshold. We propose a methodology for extracting an increased level of information by processing a series of data sets suffering, in particular, from high degree of spatial uncertainty caused by complex multiscale motion during the acquisition process. An important role is played by a non-rigid pixel-wise registration method that can cope with low signal-to-noise ratios. This is accompanied by formulating objective quality measures which replace human intervention and visual inspection in the processing chain. Scanning transmission electron microscopy of siliceous zeolite material exhibits the above-mentioned obstructions and therefore serves as orientation and a test of our procedures.

Extracting Grain Boundaries and Macroscopic Deformations from Images on Atomic Scale

Abstract Nowadays image acquisition in materials science allows the resolution of grains at atomic scale. Grains are material regions with different lattice orientation which are frequently in addition elastically stressed. At the same time, new microscopic simulation tools allow to study the dynamics of such grain structures. Single atoms are resolved experimentally as well as in simulation results on the data microscale, whereas lattice orientation and elastic deformation describe corresponding physical structures mesoscopically. A qualitative study of experimental images and simulation results and the comparison of simulation and experiment requires the robust and reliable extraction of mesoscopic properties from the microscopic image data. Based on a Mumford–Shah type functional, grain boundaries are described as free discontinuity sets at which the orientation parameter for the lattice jumps. The lattice structure itself is encoded in a suitable integrand depending on a local lattice orientation and one global elastic displacement. For each grain a lattice orientation and an elastic displacement function are considered as unknowns implicitly described by the image microstructure. In addition the approach incorporates solid–liquid interfaces. The resulting Mumford–Shah functional is approximated with a level set active contour model following the approach by Chan and Vese. The implementation is based on a finite element discretization in space and a step size controlled, regularized gradient descent algorithm.

Joint tof image denoising and registration with a CT surface in radiation therapy

The management of intra-fractional respiratory motion is becoming increasingly important in radiation therapy. Based on in advance acquired accurate 3D CT data and intra-fractionally recorded noisy time-of-flight (ToF) range data an improved treatment can be achieved. In this paper, a variational approach for the joint registration of the thorax surface extracted from a CT and a ToF image and the denoising of the ToF image is proposed. This enables a robust intra-fractional full torso surface acquisition and deformation tracking to cope with variations in patient pose and respiratory motion. Thereby, the aim is to improve radiotherapy for patients with thoracic, abdominal and pelvic tumors. The approach combines a Huber norm type regularization of the ToF data and a geometrically consistent treatment of the shape mismatch. The algorithm is tested and validated on synthetic and real ToF/CT data and then evaluated on real ToF data and 4D CT phantom experiments.

Mumford–Shah Model for One-to-One Edge Matching

This paper presents a new algorithm based on the Mumford-Shah model for simultaneously detecting the edge features of two images and jointly estimating a consistent set of transformations to match them. Compared to the current asymmetric methods in the literature, this fully symmetric method allows one to determine one-to-one correspondences between the edge features of two images. The entire variational model is realized in a multiscale framework of the finite element approximation. The optimization process is guided by an estimation minimization-type algorithm and an adaptive generalized gradient flow to guarantee a fast and smooth relaxation. The algorithm is tested on T1 and T2 magnetic resonance image data to study the parameter setting. We also present promising results of four applications of the proposed algorithm: interobject monomodal registration, retinal image registration, matching digital photographs of neurosurgery with its volume data, and motion estimation for frame interpolation.

Poisson noise removal from high-resolution STEM images based on periodic block matching

Scanning transmission electron microscopy (STEM) provides sub-ångstrom, atomic resolution images of crystalline structures. However, in many applications, the ability to extract information such as atom positions, from such electron micrographs, is severely obstructed by low signal-to-noise ratios of the acquired images resulting from necessary limitations to the electron dose. We present a denoising strategy tailored to the special features of atomic-resolution electron micrographs of crystals limited by Poisson noise based on the block-matching and 3D-filtering (BM3D) algorithm by Dabov et al. We also present an economized block-matching strategy that exploits the periodic structure of the observed crystals. On simulated single-shot STEM images of inorganic materials, with incident electron doses below 4 C/cm 2, our new method achieves precisions of 7 to 15 pm and an increase in peak signal-to-noise ratio (PSNR) of 15 to 20 dB compared to noisy images and 2 to 4 dB compared to images denoised with the original BM3D.

High-precision scanning transmission electron microscopy at coarse pixel sampling for reduced electron dose

Determining the precise atomic structure of materials’ surfaces, defects, and interfaces is important to help provide the connection between structure and important materials’ properties. Modern scanning transmission electron microscopy (STEM) techniques now allow for atomic resolution STEM images to have down to sub-picometer precision in locating positions of atoms, but these high-precision techniques generally require large electron doses, making them less useful for beam-sensitive materials. Here, we show that 1- to 2-pm image precision is possible by non-rigidly registering and averaging a high-angle dark field image series of a 5- to 6-nm Au nanoparticle even though a very coarsely sampled image and decreased exposure time was used to minimize the electron dose. These imaging conditions minimize the damage to the nanoparticle and capture the whole nanoparticle in the same image. The high-precision STEM image reveals bond length contraction around the entire nanoparticle surface, and no bond length variation along a twin boundary that separates the nanoparticle into two grains. Surface atoms at the edges and corners exhibit larger bond length contraction than atoms near the center of surface facets.

Time Discrete Geodesic Paths in the Space of Images

In this paper the space of images is considered as a Riemannian manifold using the metamorphosis approach, where the underlying Riemannian metric simultaneously measures the cost of image transport and intensity variation. A robust and effective variational time discretization of geodesics paths is proposed. This requires to minimize a discrete path energy consisting of a sum of consecutive image matching functionals over a set of image intensity maps and pairwise matching deformations. For square-integrable input images the existence of discrete, connecting geodesic paths defined as minimizers of this variational problem is shown. Furthermore,

Convex Relaxation for Grain Segmentation at Atomic Scale

\Gamma

Reconstructing Optical Flow Fields by Motion Inpainting

-convergence of the underlying discrete path energy to the continuous path energy is proved. This includes a diffeomorphism property for the induced transport and the existence of a square-integrable weak material derivative in space and time. A spatial discretization via finite elements combined with an alternating descent scheme in the set of image intensity maps and the set of matching deformations is presented to approximate discrete geodesic paths numerically. Computational results underline the efficiency of the proposed approach and demonstrate important qualitative properties.