Margret Keuper

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a subnetwork specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

Motion Trajectory Segmentation via Minimum Cost Multicuts

For the segmentation of moving objects in videos, the analysis of long-term point trajectories has been very popular recently. In this paper, we formulate the segmentation of a video sequence based on point trajectories as a minimum cost multicut problem. Unlike the commonly used spectral clustering formulation, the minimum cost multicut formulation gives natural rise to optimize not only for a cluster assignment but also for the number of clusters while allowing for varying cluster sizes. In this setup, we provide a method to create a long-term point trajectory graph with attractive and repulsive binary terms and outperform state-of-the-art methods based on spectral clustering on the FBMS-59 dataset and on the motion subtask of the VSB100 dataset.

Spectral Graph Reduction for Efficient Image and Streaming Video Segmentation

Computational and memory costs restrict spectral techniques to rather small graphs, which is a serious limitation especially in video segmentation. In this paper, we propose the use of a reduced graph based on superpixels. In contrast to previous work, the reduced graph is reweighted such that the resulting segmentation is equivalent, under certain assumptions, to that of the full graph. We consider equivalence in terms of the normalized cut and of its spectral clustering relaxation. The proposed method reduces runtime and memory consumption and yields on par results in image and video segmentation. Further, it enables an efficient data representation and update for a new streaming video segmentation approach that also achieves state-of-the-art performance.

Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts

Formulations of the Image Decomposition Problem [18] as a Multicut Problem (MP) w.r.t. a superpixel graph have received considerable attention. In contrast, instances of the MP w.r.t. a pixel grid graph have received little attention, firstly, because the MP is NP-hard and instances w.r.t. a pixel grid graph are hard to solve in practice, and, secondly, due to the lack of long-range terms in the objective function of the MP. We propose a generalization of the MP with long-range terms (LMP). We design and implement two efficient algorithms (primal feasible heuristics) for the MP and LMP which allow us to study instances of both problems w.r.t. the pixel grid graphs of the images in the BSDS-500 benchmark. The decompositions we obtain do not differ significantly from the state of the art, suggesting that the LMP is a competitive formulation of the Image Decomposition Problem. To demonstrate the generality of the LMP, we apply it also to the Mesh Decomposition Problem posed by the Princeton benchmark [16], obtaining state-of-the-art decompositions.

STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling

We propose a novel superpixel-based multi-view convolutional neural network for semantic image segmentation. The proposed network produces a high quality segmentation of a single image by leveraging information from additional views of the same scene. Particularly in indoor videos such as captured by robotic platforms or handheld and bodyworn RGBD cameras, nearby video frames provide diverse viewpoints and additional context of objects and scenes. To leverage such information, we first compute region correspondences by optical flow and image boundary-based superpixels. Given these region correspondences, we propose a novel spatio-temporal pooling layer to aggregate information over space and time. We evaluate our approach on the NYU-Depth-V2 and the SUN3D datasets and compare it to various state-of-the-art single-view and multi-view approaches. Besides a general improvement over the state-of-the-art, we also show the benefits of making use of unlabeled frames during training for multi-view as well as single-view prediction.

Hierarchical Markov Random Fields for mast cell segmentation in electron microscopic recordings

We present a hierarchical Markov Random Field (HMRF) for multi-label image segmentation. With such a hierarchical model, we can incorporate global knowledge into our segmentation algorithm. Solving the MRF is formulated as a MAX-SUM problem for which there exist efficient solvers based on linear programming. We show that our method allows for automatic segmentation of mast cells and their cell organelles from 2D electron microscopic recordings. The presented HMRF outperforms classical MRFs as well as local classification approaches wrt. pixelwise segmentation accuracy. Additionally, the resulting segmentations are much more consistent regarding the region compactness.

Blind Deconvolution of Widefield Fluorescence Microscopic Data by Regularization of the Optical Transfer Function (OTF)

With volumetric data from wide field fluorescence microscopy, many emerging questions in biological and biomedical research are being investigated. Data can be recorded with high temporal resolution while the specimen is only exposed to a low amount of photo toxicity. These advantages come at the cost of strong recording blur caused by the infinitely extended point spread function (PSF). For wide field microscopy, its magnitude only decays with the square of the distance to the focal point and consists of an airy bessel pattern which is intricate to describe in the spatial domain. However, the Fourier transform of the incoherent PSF (denoted as Optical Transfer Function (OTF)) is well localized and smooth. In this paper, we present a blind deconvolution method that improves results of state-of-the-art deconvolution methods on wide field data by exploiting the properties of the wide field OTF.

3D Deformable Surfaces with Locally Self-Adjusting Parameters - A Robust Method to Determine Cell Nucleus Shapes

When using deformable models for the segmentation of biological data, the choice of the best weighting parameters for the internal and external forces is crucial. Especially when dealing with 3D fluorescence microscopic data and cells within dense tissue, object boundaries are sometimes not visible. In these cases, one weighting parameter set for the whole contour is not desirable. We are presenting a method for the dynamic adjustment of the weighting parameters, that is only depending on the underlying data and does not need any prior information. The method is especially apt to handle blurred, noisy, and deficient data, as it is often the case in biological microscopy.

Evaluation of a New Point Clouds Registration Method Based on Group Averaging Features

Registration of point clouds is required in the processing of large biological data sets. The trade off between computation time and accuracy of the registration is the main challenge in this task. We present a novel method for registering point clouds in two and three dimensional space based on Group Averaging on the Euclidean transformation group. It is applied on a set of neighboring points whose size directly controls computing time and accuracy. The method is evaluated regarding dependencies of the computing time and the registration accuracy versus the point density assuming their random distribution. Results are verified in two biological applications on 2D and 3D images.

A 3D Active Surface Model for the Accurate Segmentation of Drosophila Schneider Cell Nuclei and Nucleoli

We present an active surface model designed for the segmentation of Drosophila Schneider cell nuclei and nucleoli from wide-field microscopic data. The imaging technique as well as the biological application impose some major challenges to the segmentation. On the one hand, we have to deal with strong blurring of the 3D data, especially in z-direction. On the other hand, concerning the biological application, we have to deal with non-closed object boundaries and touching objects. To cope with these problems, we have designed a fully 3D active surface model. Our model prefers roundish object shapes and especially imposes roughly spherical surfaces where there is little gradient information. We have adapted an external force field for this model, which is based on gradient vector flow (