Paulo A. V. Miranda

Fuzzy Connectedness Image Segmentation in Graph Cut Formulation: A Linear-Time Algorithm and a Comparative Analysis

A deep theoretical analysis of the graph cut image segmentation framework presented in this paper simultaneously translates into important contributions in several directions.The most important practical contribution of this work is a full theoretical description, and implementation, of a novel powerful segmentation algorithm, GCmax. The output of GCmax coincides with a version of a segmentation algorithm known as Iterative Relative Fuzzy Connectedness, IRFC. However, GCmax is considerably faster than the classic IRFC algorithm, which we prove theoretically and show experimentally. Specifically, we prove that, in the worst case scenario, the GCmax algorithm runs in linear time with respect to the variable M=|C|+|Z|, where |C| is the image scene size and |Z| is the size of the allowable range, Z, of the associated weight/affinity function. For most implementations, Z is identical to the set of allowable image intensity values, and its size can be treated as small with respect to |C|, meaning that O(M)=O(|C|). In such a situation, GCmax runs in linear time with respect to the image size |C|.We show that the output of GCmax constitutes a solution of a graph cut energy minimization problem, in which the energy is defined as the ℓ∞ norm ∥FP∥∞ of the map FP that associates, with every element e from the boundary of an object P, its weight w(e). This formulation brings IRFC algorithms to the realm of the graph cut energy minimizers, with energy functions ∥FP∥q for q∈[1,∞]. Of these, the best known minimization problem is for the energy ∥FP∥1, which is solved by the classic min-cut/max-flow algorithm, referred to often as the Graph Cut algorithm.We notice that a minimization problem for ∥FP∥q, q∈[1,∞), is identical to that for ∥FP∥1, when the original weight function w is replaced by wq. Thus, any algorithm GCsum solving the ∥FP∥1 minimization problem, solves also one for ∥FP∥q with q∈[1,∞), so just two algorithms, GCsum and GCmax, are enough to solve all ∥FP∥q-minimization problems. We also show that, for any fixed weight assignment, the solutions of the ∥FP∥q-minimization problems converge to a solution of the ∥FP∥∞-minimization problem (∥FP∥∞=limq→∞∥FP∥q is not enough to deduce that).An experimental comparison of the performance of GCmax and GCsum algorithms is included. This concentrates on comparing the actual (as opposed to provable worst scenario) algorithms’ running time, as well as the influence of the choice of the seeds on the output.

Riverbed: A Novel User-Steered Image Segmentation Method Based on Optimum Boundary Tracking

This paper presents an optimum user-steered boundary tracking approach for image segmentation, which simulates the behavior of water flowing through a riverbed. The riverbed approach was devised using the image foresting transform with a never-exploited connectivity function. We analyze its properties in the derived image graphs and discuss its theoretical relation with other popular methods such as live wire and graph cuts. Several experiments show that riverbed can significantly reduce the number of user interactions (anchor points), as compared to live wire for objects with complex shapes. This paper also includes a discussion about how to combine different methods in order to take advantage of their complementary strengths.

Oriented Image Foresting Transform Segmentation by Seed Competition

Seed-based methods for region-based image segmentation are known to provide satisfactory results for several applications, being usually easy to extend to multidimensional images. However, while boundary-based methods like live wire can easily incorporate a preferred boundary orientation, region-based methods are usually conceived for undirected graphs, and do not resolve well between boundaries with opposite orientations. This motivated researchers to investigate extensions for some region-based frameworks, seeking to better solve oriented transitions. In this same spirit, we discuss how to incorporate this orientation information in a region-based approach called “IFT segmentation by seed competition” by exploring digraphs. We give direct proof for the optimality of the proposed extensions in terms of energy functions associated with the cuts. To stress these theoretical results, we also present an experimental evaluation that shows the obtained gains in accuracy for some 2D and 3D data sets of medical images.

Hybrid Approaches for Interactive Image Segmentation Using the Live Markers Paradigm

Interactive image segmentation methods normally rely on cues about the foreground imposed by the user as region constraints (markers/brush strokes) or boundary constraints (anchor points). These paradigms often have complementary strengths and weaknesses, which can be addressed to improve the interactive experience by reducing the users effort. We propose a novel hybrid paradigm based on a new form of interaction called live markers, where optimum boundary-tracking segments are turned into internal and external markers for region-based delineation to effectively extract the object. We present four techniques within this paradigm: 1) LiveMarkers; 2) RiverCut; 3) LiveCut; and 4) RiverMarkers. The homonym LiveMarkers couples boundary-tracking via live-wire-on-the-fly (LWOF) with optimum seed competition by the image foresting transform (IFT-SC). The IFT-SC can cope with complex object silhouettes, but presents a leaking problem on weaker parts of the boundary that is solved by the effective live markers produced by LWOF. Conversely, in RiverCut, the long boundary segments computed by Riverbed around complex shapes provide markers for Graph Cuts by the Min-Cut/Max-Flow algorithm (GCMF) to complete segmentation on poorly defined sections of the objects border. LiveCut and RiverMarkers further demonstrate that live markers can improve segmentation even when the combined approaches are not complementary (e.g., GCMFs shrinking bias is also dramatically prevented when using it with LWOF). More- over, since delineation is always region based, our methodology subsumes both paradigms, representing a new way of extending boundary tracking to the 3D image domain, while speeding up the addition of markers close to the objects boundary— a necessary but time consuming task when done manually. We justify our claims through an extensive experimental evaluation on natural and medical images data sets, using recently proposed robot users for boundary-tracking methods.

Image segmentation by oriented image foresting transform: Handling ties and colored images

Anatomical structures and tissues are often hard to be segmented in medical images due to their poorly defined boundaries, i.e., low contrast in relation to other nearby false boundaries. The specification of the boundary polarity can help to alleviate part of this problem. Although this information is naturally exploited by boundary-based methods like live wire, region-based methods are usually conceived for undirected graphs. This motivated researchers to investigate extensions to better handle oriented transitions. Recently, an Oriented Image Foresting Transform has been proposed. In this work, we further generalize it by different non-smooth connectivity functions that allow a better handling of ties in its energy formulation. We give direct proof for the optimality of the proposed extensions in terms of a global maxima of an energy function, and show the obtained gains in accuracy over a more challenging 3D dataset of MR images of the foot. We also discuss the incorporation of the boundary polarity for the interactive segmentation of colored images.

A unifying graph-cut image segmentation framework: algorithms it encompasses and equivalences among them

We present a general graph-cut segmentation framework GGC, in which the delineated objects returned by the algorithms optimize the energy functions associated with the ℓp norm, 1 ≤ p ≤ ∞. Two classes of well known algorithms belong to GGC: the standard graph cut GC (such as the min-cut/max-flow algorithm) and the relative fuzzy connectedness algorithms RFC (including iterative RFC, IRFC). The norm-based description of GGC provides more elegant and mathematically better recognized framework of our earlier results from [18, 19]. Moreover, it allows precise theoretical comparison of GGC representable algorithms with the algorithms discussed in a recent paper [22] (min-cut/max-flow graph cut, random walker, shortest path/geodesic, Voronoi diagram, power watershed/shortest path forest), which optimize, via ℓp norms, the intermediate segmentation step, the labeling of scene voxels, but for which the final object need not optimize the used ℓp energy function. Actually, the comparison of the GGC representable algorithms with that encompassed in the framework described in [22] constitutes the main contribution of this work.

Elucidating the Relations among Seeded Image Segmentation Methods and their Possible Extensions

Many image segmentation algorithms have been proposed, specially for the case of binary segmentation (object/background) in which hard constraints (seeds) are provided interactively. Recently, several theoretical efforts were made in an attempt to unify their presentation and clarify their relations. These relations are usually pointed out textually or depicted in the form of a table of parameters of a general energy formulation. In this work we introduce a more general diagram representation which captures the connections among the methods, by means of conventional relations from set theory. We formally instantiate several methods under this diagram, including graph cuts, power watersheds, fuzzy connectedness, grow cut, distance cuts, and others, which are usually presented as unrelated methods. The proposed diagram representation leads to a more elucidated view of the methods, being less restrictive than the tabular representation. It includes new relations among methods, besides bringing together the connections gathered from different works. It also points out some promising unexplored intermediate regions, which can lead to possible extensions of the existing methods. We also demonstrate one of such possible extensions, which is used to effectively combine the strengths of region and local contrast features.

Image Segmentation by Oriented Image Foresting Transform with Geodesic Star Convexity

Anatomical structures and tissues are often hard to be segmented in medical images due to their poorly defined boundaries, i.e., low contrast in relation to other nearby false boundaries. The specification of the boundary polarity and the usage of shape constraints can help to alleviate part of this problem. Recently, an Oriented Image Foresting Transform OIFT has been proposed. In this work, we discuss how to incorporate Gulshans geodesic star convexity prior in the OIFT approach for interactive image segmentation, in order to simultaneously handle boundary polarity and shape constraints. This convexity constraint eliminates undesirable intricate shapes, improving the segmentation of objects with more regular shape. We include a theoretical proof of the optimality of the new algorithm in terms of a global maximum of an oriented energy function subject to the shape constraints, and show the obtained gains in accuracy using medical images of thoracic CT studies.

Detecting Contour Saliences using Tensor Scale

Tensor scale is a morphometric parameter that unifies the representation of local structure thickness, orientation, and anisotropy, which can be used in several image processing tasks. This paper introduces a new application for tensor scale, which is the detection of saliences on a given contour, based on the tensor scale orientations computed for the entire object and mapped to its contour. For validation purposes, we present a shape descriptor that uses the detected contour saliences. Experimental results are provided, comparing the proposed method with our previous contour salience descriptor (CS). We show that the proposed method can be not only faster and more robust in the detection of salience points than the CS method, but also more effective as a shape descriptor.

IFT-SLIC: A General Framework for Superpixel Generation Based on Simple Linear Iterative Clustering and Image Foresting Transform

Image representation based on super pixels has become indispensable for improving efficiency in Computer Vision systems. Object recognition, segmentation, depth estimation, and body model estimation are some important problems where super pixels can be applied. However, super pixels can influence the efficacy of the system in positive or negative manner, depending on how well they respect the object boundaries in the image. In this paper, we improve super pixel generation by extending a popular algorithm -- Simple Linear Iterative Clustering (SLIC) -- to consider minimum path costs between pixel and cluster centers rather than their direct distances. This creates a new Image Foresting Transform (IFT) operator that naturally defines super pixels as regions of strongly connected pixels by choice of the most suitable path-cost function for a given application. Non-smooth connectivity functions are also explored in our IFT-SLIC approach leading to improved performance. Experimental results indicate better super pixel extraction using the proposed approach as compared to that of SLIC.