Jan Henrik Fitschen

A Variational Model for Color Assignment

Color image enhancement is a challenging task in digital imaging with many applications. This paper contributes to image enhancement methods. We propose a new variational model for color improvement in the RGB space based on a desired target intensity image. Our model improves the visual quality of the color image while it preserves the range and takes the hue of the original, badly exposed image into account without amplifying its color artifacts. To approximate the hue of the original image we use the fact that affine transforms are hue preserving. To cope with the noise in the color channels we design a particular coupled TV regularization term. Since the target intensity of the image is unaltered our model respects important image structures. Numerical results demonstrate the very good performance of our method.

Disparity and Optical Flow Partitioning Using Extended Potts Priors

This paper addresses the problems of disparity and optical flow partitioning based on the brightness invariance assumption. We investigate new variational approaches to these problems with Potts priors and possibly box constraints. For the optical flow partitioning, our model includes vector-valued data and an adapted Potts regularizer. Using the notion of asymptotically level stable (als) functions, we prove the existence of global minimizers of our functionals. We propose a modified alternating direction method of multipliers. This iterative algorithm requires the computation of global minimizers of classical univariate Potts problems which can be done efficiently by dynamic programming. We prove that the algorithm converges both for the constrained and unconstrained problems. Numerical examples demonstrate the very good performance of our partitioning method.

Transport Between RGB Images Motivated by Dynamic Optimal Transport

We propose two models for the interpolation between RGB images based on the dynamic optimal transport model of Benamou and Brenier (Numer Math 84:375–393, 2000). While the application of dynamic optimal transport and its extensions to unbalanced transform were examined for gray-value images in various papers, this is the first attempt to generalize the idea to color images. The non-trivial task to incorporate color into the model is tackled by considering RGB images as three-dimensional arrays, where the transport in the RGB direction is performed in a periodic way. Following the approach of Papadakis et al. (SIAM J Imaging Sci 7:212–238, 2014) for gray-value images we propose two discrete variational models, a constrained and a penalized one which can also handle unbalanced transport. We show that a minimizer of our discrete model exists, but it is not unique for some special initial/final images. For minimizing the resulting functionals we apply a primal-dual algorithm. One step of this algorithm requires the solution of a four-dimensional discretized Poisson equation with various boundary conditions in each dimension. For instance, for the penalized approach we have simultaneously zero, mirror, and periodic boundary conditions. The solution can be computed efficiently using fast Sin-I, Cos-II, and Fourier transforms. Numerical examples demonstrate the meaningfulness of our model.

Optimal Transport for Manifold-Valued Images

We introduce optimal transport-type distances for manifold-valued images. To do so we lift the initial data to measures on the product space of image domain and signal space, where they can then be compared by optimal transport with a transport cost that combines spatial distance and signal discrepancy. Applying recently introduced ‘unbalanced’ optimal transport models leads to more natural results. We illustrate the benefit of the lifting with numerical examples for interpolation of color images and classification of handwritten digits.

Computation and Visualization of Local Deformation for Multiphase Metallic Materials by Infimal Convolution of TV-Type Functionals

Estimating the local strain tensor from a sequence of microstructural images, realized during a tensile test of an engineering material, is a challenging problem. In this paper we propose to compute the strain tensor from image sequences acquired during tensile tests with increasing forces in horizontal direction by a variational optical flow model. To separate the global displacement during insitu tensile testing, which can be roughly approximated by a plane, from the local displacement we use an infimal convolution regularization consisting of first and second order terms. We apply a primal-dual method to find a minimizer of the energy function. This approach has the advantage that the strain tensor is directly computed within the algorithm and no additional derivative of the displacement must be computed. The algorithm is equipped with a coarse-to-fine strategy to cope with larger displacements and an adaptive parameter choice. Numerical examples with simulated and experimental data demonstrate the advantageous performance of our algorithm.

Strain analysis by a total generalized variation regularized optical flow model

ABSTRACT In this paper, we deal with the problem of estimating the local strain tensor from a sequence of micro-structural images realized during deformation tests of engineering materials. Since the strain tensor is defined via the Jacobian of the displacement field, we propose to compute the displacement field by a variational model which takes care of properties of the Jacobian of the displacement. In particular, we are interested in areas of high strain. The data term of our variational model relies on the brightness invariance property of the image sequence. As prior we choose the second order total generalized variation of the displacement field. This splits the Jacobian into a smooth and a non-smooth part. The latter reflects the material cracks. An additional constraint is incorporated to handle physical properties of the non-smooth part for tensile tests. We prove that the resulting convex model has a minimizer and show how a primal-dual method can be applied to find a minimizer. The corresponding algorithm has the advantage that the strain tensor is directly computed within the iteration process. It is further equipped with a coarse-to-fine strategy to cope with larger displacements. Numerical examples with simulated and experimental data demonstrate the very good performance of our algorithm. In comparison to state-of-the-art engineering software, our method can resolve local phenomena much better.

Removal of curtaining effects by a variational model with directional forward differences

Focused ion beam (FIB) tomography provides high resolution volumetric images on a micro scale. However, due to the physical acquisition process the resulting images are often corrupted by a so-called curtaining or waterfall effect. In this paper, a new convex variational model for removing such effects is proposed. More precisely, an infimal convolution model is applied to split the corrupted 3D image into the clean image and two types of corruptions, namely a striped part and a laminar one. As regularizing terms different direction dependent first and second order differences are used to cope with the specific structure of the corruptions. This generalizes discrete unidirectional total variational (TV) approaches. A minimizer of the model is computed by well-known primal dual techniques. Numerical examples show the very good performance of our new method for artificial and real-world data. Besides FIB tomography, we have also successfully applied our technique for the removal of pure stripes in Moderate Resolution Imaging Spectroradiometer (MODIS) data.

Dynamic optimal transport with mixed boundary condition for color image processing

Recently, Papadakis et al. [11] proposed an efficient primal-dual algorithm for solving the dynamic optimal transport problem with quadratic ground cost and measures having densities with respect to the Lebesgue measure. It is based on the fluid mechanics formulation by Benamou and Brenier [1] and proximal splitting schemes. In this paper we extend the framework to color image processing. We show how the transportation problem for RGB color images can be tackled by prescribing periodic boundary conditions in the color dimension. This requires the solution of a 4D Poisson equation with mixed Neumann and periodic boundary conditions in each iteration step of the algorithm. This 4D Poisson equation can be efficiently handled by fast Fourier and Cosine transforms. Furthermore, we sketch how the same idea can be used in a modified way to transport periodic 1D data such as the histogram of cyclic hue components of images. We discuss the existence and uniqueness of a minimizer of the associated energy functional. Numerical examples illustrate the meaningfulness of our approach.

Infimal Convolution Coupling of First and Second Order Differences on Manifold-Valued Images

Recently infimal convolution type functions were used in regularization terms of variational models for restoring and decomposing images. This is the first attempt to generalize the infimal convolution of first and second order differences to manifold-valued images. We propose both an extrinsic and an intrinsic approach. Our focus is on the second one since the summands arising in the infimal convolution lie on the manifold themselves and not in the higher dimensional embedding space. We demonstrate by numerical examples that the approach works well on the circle, the 2-sphere, the rotation group, and the manifold of positive definite matrices with the affine invariant metric.

Priors with Coupled First and Second Order Differences for Manifold-Valued Image Processing

We generalize discrete variational models involving the infimal convolution (IC) of first and second order differences and the total generalized variation (TGV) to manifold-valued images. We propose both extrinsic and intrinsic approaches. The extrinsic models are based on embedding the manifold into an Euclidean space of higher dimension with manifold constraints. An alternating direction methods of multipliers can be employed for finding the minimizers. However, the components within the extrinsic IC or TGV decompositions live in the embedding space which makes their interpretation difficult. Therefore, we investigate two intrinsic approaches: for Lie groups, we employ the group action within the models; for more general manifolds, our IC model is based on recently developed absolute second order differences on manifolds, while our TGV approach uses an approximation of the parallel transport by the pole ladder. For computing the minimizers of the intrinsic models, we apply gradient descent algorithms. Numerical examples demonstrate that our approaches work well for certain manifolds.