Frank R. Schmidt

Shape priors in variational image segmentation: Convexity, Lipschitz continuity and globally optimal solutions

In this work, we introduce a novel implicit representation of shape which is based on assigning to each pixel a probability that this pixel is inside the shape. This probabilistic representation of shape resolves two important drawbacks of alternative implicit shape representations such as the level set method: Firstly, the space of shapes is convex in the sense that arbitrary convex combinations of a set of shapes again correspond to a valid shape. Secondly, we prove that the introduction of shape priors into variational image segmentation leads to functionals which are convex with respect to shape deformations. For a large class of commonly considered (spatially continuous) functionals, we prove that - under mild regularity assumptions - segmentation and tracking with statistical shape priors can be performed in a globally optimal manner. In experiments on tracking a walking person through a cluttered scene we demonstrate the advantage of global versus local optimality.

Efficient planar graph cuts with applications in Computer Vision

We present a fast graph cut algorithm for planar graphs. It is based on the graph theoretical work and leads to an efficient method that we apply on shape matching and image segmentation. In contrast to currently used methods in computer vision, the presented approach provides an upper bound for its runtime behavior that is almost linear. In particular, we are able to match two different planar shapes of N points in O(N2 log N) and segment a given image of N pixels in O(N log N). We present two experimental benchmark studies which demonstrate that the presented method is also in practice faster than previously proposed graph cut methods: On planar shape matching and image segmentation we observe a speed-up of an order of magnitude, depending on resolution.

Fast Matching of Planar Shapes in Sub-cubic Runtime

The matching of planar shapes can be cast as a problem of finding the shortest path through a graph spanned by the two shapes, where the nodes of the graph encode the local similarity of respective points on each contour. While this problem can be solved using dynamic time warping, the complete search over the initial correspondence leads to cubic runtime in the number of sample points. In this paper, we cast the shape matching problem as one of finding the shortest circular path on a torus. We propose an algorithm to determine this shortest cycle which has provably sub-cubic runtime. Numerical experiments demonstrate that the proposed algorithm provides faster shape matching than previous methods. As an application, we show that it allows to efficiently compute a clustering of a shape data base.

Video Segmentation with Just a Few Strokes

As the use of videos is becoming more popular in computer vision, the need for annotated video datasets increases. Such datasets are required either as training data or simply as ground truth for benchmark datasets. A particular challenge in video segmentation is due to disocclusions, which hamper frame-to-frame propagation, in conjunction with non-moving objects. We show that a combination of motion from point trajectories, as known from motion segmentation, along with minimal supervision can largely help solve this problem. Moreover, we integrate a new constraint that enforces consistency of the color distribution in successive frames. We quantify user interaction effort with respect to segmentation quality on challenging ego motion videos. We compare our approach to a diverse set of algorithms in terms of user effort and in terms of performance on common video segmentation benchmarks.

Shape matching by variational computation of geodesics on a manifold

Klassen et al. [9] recently developed a theoretical formulation to model shape dissimilarities by means of geodesics on appropriate spaces. They used the local geometry of an infinite dimensional manifold to measure the distance dist(A,B) between two given shapes A and B. A key limitation of their approach is that the computation of distances developed in the above work is inherently unstable, the computed distances are in general not symmetric, and the computation times are typically very large. In this paper, we revisit the shooting method of Klassen et al. for their angle-oriented representation. We revisit explicit expressions for the underlying space and we propose a gradient descent algorithm to compute geodesics. In contrast to the shooting method, the proposed variational method is numerically stable, it is by definition symmetric, and it is up to 1000 times faster.

Fast Trust Region for Segmentation

Trust region is a well-known general iterative approach to optimization which offers many advantages over standard gradient descent techniques. In particular, it allows more accurate nonlinear approximation models. In each iteration this approach computes a global optimum of a suitable approximation model within a fixed radius around the current solution, a.k.a. trust region. In general, this approach can be used only when some efficient constrained optimization algorithm is available for the selected non-linear (more accurate) approximation model. In this paper we propose a Fast Trust Region (FTR) approach for optimization of segmentation energies with non-linear regional terms, which are known to be challenging for existing algorithms. These energies include, but are not limited to, KL divergence and Bhattacharyya distance between the observed and the target appearance distributions, volume constraint on segment size, and shape prior constraint in a form of L2 distance from target shape moments. Our method is 1-2 orders of magnitude faster than the existing state-of-the-art methods while converging to comparable or better solutions.

Large‐Scale Integer Linear Programming for Orientation Preserving 3D Shape Matching

We study an algorithmic framework for computing an elastic orientation‐preserving matching of non‐rigid 3D shapes. We outline an Integer Linear Programming formulation whose relaxed version can be minimized globally in polynomial time. Because of the high number of optimization variables, the key algorithmic challenge lies in efficiently solving the linear program. We present a performance analysis of several Linear Programming algorithms on our problem. Furthermore, we introduce a multiresolution strategy which allows the matching of higher resolution models.

Efficient shape matching via graph cuts

Meaningful notions of distance between planar shapes typically involve the computation of a correspondence between points on one shape and points on the other. To determine an optimal correspondence is a computationally challenging combinatorial problem. Traditionally it has been formulated as a shortest path problem which can be solved efficiently by Dynamic Time Warping. In this paper, we show that shape matching can be cast as a problem of finding a minimum cut through a graph which can be solved efficiently by computing the maximum network flow. In particular, we show the equivalence of the minimum cut formulation and the shortest path formulation, i.e. we show that there exists a one-to-one correspondence of a shortest path and a graph cut and that the length of the path is identical to the cost of the cut. In addition, we provide and analyze some examples for which the proposed algorithm is faster resp. slower than the shortest path method.

Efficient Globally Optimal 2D-to-3D Deformable Shape Matching

We propose the first algorithm for non-rigid 2D-to-3D shape matching, where the input is a 2D query shape as well as a 3D target shape and the output is a continuous matching curve represented as a closed contour on the 3D shape. We cast the problem as finding the shortest circular path on the product 3-manifold of the two shapes. We prove that the optimal matching can be computed in polynomial time with a (worst-case) complexity of O(mn2 log(n)), wherem and n denote the number of vertices on the 2D and the 3D shape respectively. Quantitative evaluation confirms that the method provides excellent results for sketch-based deformable 3D shape retrieval.

Intrinsic mean for semi-metrical shape retrieval via graph cuts

We address the problem of describing the mean object for a set of planar shapes in the case that the considered dissimilarity measures are semi-metrics, i.e. in the case that the triangle inequality is generally not fulfilled. To this end, a matching of two planar shapes is computed by cutting an appropriately defined graph the edge weights of which encode the local similarity of respective contour parts on either shape. The cost of the minimum cut can be interpreted as a semi-metric on the space of planar shapes. Subsequently, we introduce the notion of a mean shape for the case of semi-metrics and show that this allows to perform a shape retrieval which mimics human notions of shape similarity.