Juyong Zhang

Robust Interactive Image Segmentation Using Convex Active Contours

The state-of-the-art interactive image segmentation algorithms are sensitive to the user inputs and often unable to produce an accurate boundary with a small amount of user interaction. They frequently rely on laborious user editing to refine the segmentation boundary. In this paper, we propose a robust and accurate interactive method based on the recently developed continuous-domain convex active contour model. The proposed method exhibits many desirable properties of an effective interactive image segmentation algorithm, including robustness to user inputs and different initializations, the ability to produce a smooth and accurate boundary contour, and the ability to handle topology changes. Experimental results on a benchmark data set show that the proposed tool is highly effective and outperforms the state-of-the-art interactive image segmentation algorithms.

A diffusion approach to seeded image segmentation

Seeded image segmentation is a popular type of supervised image segmentation in computer vision and image processing. Previous methods of seeded image segmentation treat the image as a weighted graph and minimize an energy function on the graph to produce a segmentation. In this paper, we propose to conduct the seeded image segmentation according to the result of a heat diffusion process in which the seeded pixels are considered to be the heat sources and the heat diffuses on the image starting from the sources. After the diffusion reaches a stable state, the image is segmented based on the pixel temperatures. It is also shown that our proposed framework includes the RandomWalk algorithm for image segmentation as a special case which diffuses only along the two coordinate axes. To better control diffusion, we propose to incorporate the attributes (such as the geometric structure) of the image into the diffusion process, yielding an anisotropic diffusion method for image segmentation. The experiments show that the proposed anisotropic diffusion method usually produces better segmentation results. In particular, when the method is tested using the groundtruth dataset of Microsoft Research Cambridge (MSRC), an error rate of 4.42% can be achieved, which is lower than the reported error rates of other state-of-the-art algorithms.

Variational mesh decomposition

The problem of decomposing a 3D mesh into meaningful segments (or parts) is of great practical importance in computer graphics. This article presents a variational mesh decomposition algorithm that can efficiently partition a mesh into a prescribed number of segments. The algorithm extends the Mumford-Shah model to 3D meshes that contains a data term measuring the variation within a segment using eigenvectors of a dual Laplacian matrix whose weights are related to the dihedral angle between adjacent triangles and a regularization term measuring the length of the boundary between segments. Such a formulation simultaneously handles segmentation and boundary smoothing, which are usually two separate processes in most previous work. The efficiency is achieved by solving the Mumford-Shah model through a saddle-point problem that is solved by a fast primal-dual method. A preprocess step is also proposed to determine the number of segments that the mesh should be decomposed into. By incorporating this preprocessing step, the proposed algorithm can automatically segment a mesh into meaningful parts. Furthermore, user interaction is allowed by incorporating the users inputs into the variational model to reflect the users special intention. Experimental results show that the proposed algorithm outperforms competitive segmentation methods when evaluated on the Princeton Segmentation Benchmark.

Guided Mesh Normal Filtering

The joint bilateral filter is a variant of the standard bilateral filter, where the range kernel is evaluated using a guidance signal instead of the original signal. It has been successfully applied to various image processing problems, where it provides more flexibility than the standard bilateral filter to achieve high quality results. On the other hand, its success is heavily dependent on the guidance signal, which should ideally provide a robust estimation about the features of the output signal. Such a guidance signal is not always easy to construct. In this paper, we propose a novel mesh normal filtering framework based on the joint bilateral filter, with applications in mesh denoising. Our framework is designed as a two‐stage process: first, we apply joint bilateral filtering to the face normals, using a properly constructed normal field as the guidance; afterwards, the vertex positions are updated according to the filtered face normals. We compute the guidance normal on a face using a neighboring patch with the most consistent normal orientations, which provides a reliable estimation of the true normal even with a high‐level of noise. The effectiveness of our approach is validated by extensive experimental results.

Augmented Lagrangian Method for Total Variation Based Image Restoration and Segmentation Over Triangulated Surfaces

Recently total variation (TV) regularization has been proven very successful in image restoration and segmentation. In image restoration, TV based models offer a good edge preservation property. In image segmentation, TV (or vectorial TV) helps to obtain convex formulations of the problems and thus provides global minimizations. Due to these advantages, TV based models have been extended to image restoration and data segmentation on manifolds. However, TV based restoration and segmentation models are difficult to solve, due to the nonlinearity and non-differentiability of the TV term. Inspired by the success of operator splitting and the augmented Lagrangian method (ALM) in 2D planar image processing, we extend the method to TV and vectorial TV based image restoration and segmentation on triangulated surfaces, which are widely used in computer graphics and computer vision. In particular, we will focus on the following problems. First, several Hilbert spaces will be given to describe TV and vectorial TV based variational models in the discrete setting. Second, we present ALM applied to TV and vectorial TV image restoration on mesh surfaces, leading to efficient algorithms for both gray and color image restoration. Third, we discuss ALM for vectorial TV based multi-region image segmentation, which also works for both gray and color images. The proposed method benefits from fast solvers for sparse linear systems and closed form solutions to subproblems. Experiments on both gray and color images demonstrate the efficiency of our algorithms.

Variational Mesh Denoising Using Total Variation and Piecewise Constant Function Space

Mesh surface denoising is a fundamental problem in geometry processing. The main challenge is to remove noise while preserving sharp features (such as edges and corners) and preventing generating false edges. We propose in this paper to combine total variation (TV) and piecewise constant function space for variational mesh denoising. We first give definitions of piecewise constant function spaces and associated operators. A variational mesh denoising method will then be presented by combining TV and piecewise constant function space. It is proved that, the solution of the variational problem (the key part of the method) is in some sense continuously dependent on its parameter, indicating that the solution is robust to small perturbations of this parameter. To solve the variational problem, we propose an efficient iterative algorithm (with an additional algorithmic parameter) based on variable splitting and augmented Lagrangian method, each step of which has closed form solution. Our denoising method is discussed and compared to several typical existing methods in various aspects. Experimental results show that our method outperforms all the compared methods for both CAD and non-CAD meshes at reasonable costs. It can preserve different levels of features well, and prevent generating false edges in most cases, even with the parameters evaluated by our estimation formulae.Mesh surface denoising is a fundamental problem in geometry processing. The main challenge is to remove noise while preserving sharp features (such as edges and corners) and preventing generating false edges. We propose in this paper to combine total variation (TV) and piecewise constant function space for variational mesh denoising. We first give definitions of piecewise constant function spaces and associated operators. A variational mesh denoising method will then be presented by combining TV and piecewise constant function space. It is proved that, the solution of the variational problem (the key part of the method) is in some sense continuously dependent on its parameter, indicating that the solution is robust to small perturbations of this parameter. To solve the variational problem, we propose an efficient iterative algorithm (with an additional algorithmic parameter) based on variable splitting and augmented Lagrangian method, each step of which has closed form solution. Our denoising method is discussed and compared to several typical existing methods in various aspects. Experimental results show that our method outperforms all the compared methods for both CAD and non-CAD meshes at reasonable costs. It can preserve different levels of features well, and prevent generating false edges in most cases, even with the parameters evaluated by our estimation formulae.

Robust surface reconstruction via dictionary learning

Surface reconstruction from point cloud is of great practical importance in computer graphics. Existing methods often realize reconstruction via a few phases with respective goals, whose integration may not give an optimal solution. In this paper, to avoid the inherent limitations of multi-phase processing in the prior art, we propose a unified framework that treats geometry and connectivity construction as one joint optimization problem. The framework is based on dictionary learning in which the dictionary consists of the vertices of the reconstructed triangular mesh and the sparse coding matrix encodes the connectivity of the mesh. The dictionary learning is formulated as a constrained ℓ2,q-optimization (0 < q < 1), aiming to find the vertex position and triangulation that minimize an energy function composed of point-to-mesh metric and regularization. Our formulation takes many factors into account within the same framework, including distance metric, noise/outlier resilience, sharp feature preservation, no need to estimate normal, etc., thus providing a global and robust algorithm that is able to efficiently recover a piecewise smooth surface from dense data points with imperfections. Extensive experiments using synthetic models, real world models, and publicly available benchmark show that our method outperforms the state-of-the-art in terms of accuracy, robustness to noise and outliers, geometric feature and detail preservation, and mesh connectivity.

Exploring Local Modifications for Constrained Meshes

Mesh editing under constraints is a challenging task with numerous applications in geometric modeling, industrial design, and architectural form finding. Recent methods support constraint‐based exploration of meshes with fixed connectivity, but commonly lack local control. Because constraints are often globally coupled, a local modification by the user can have global effects on the surface, making iterative design exploration and refinement difficult. Simply fixing a local region of interest a priori is problematic, as it is not clear in advance which parts of the mesh need to be modified to obtain an aesthetically pleasing solution that satisfies all constraints. We propose a novel framework for exploring local modifications of constrained meshes. Our solution consists of three steps. First, a user specifies target positions for one or more vertices. Our algorithm computes a sparse set of displacement vectors that satisfies the constraints and yields a smooth deformation. Then we build a linear subspace to allow realtime exploration of local variations that satisfy the constraints approximately. Finally, after interactive exploration, the result is optimized to fully satisfy the set of constraints. We evaluate our framework on meshes where each face is constrained to be planar.

Mesh Snapping: Robust Interactive Mesh Cutting Using Fast Geodesic Curvature Flow

This paper considers the problem of interactively finding the cutting contour to extract components from a given mesh. Some existing methods support cuts of arbitrary shape but require careful and tedious input from the user. Others need little user input however they are sensitive to user input and need a postprocessing step to smooth the generated jaggy cutting contours. The popular geometric snake can be used to optimize the cutting contour, but it cannot deal with the topology change. In this paper, we propose a geodesic curvature flow based framework to overcome all these problems. Since in many cases the meaningful cutting contour on a 3D mesh is locally shortest in the sense of some weighted curve length, the geodesic curvature flow is an ideal tool for our problem. It evolves the cutting contour to the nearby local minimum. We should mention that the previous numerical scheme, discretized geodesic curvature flow (dGCF) is too slow and has not been applied to mesh segmentation. With a careful observation to dGCF, we devise here a fast computation scheme called fast geodesic curvature flow (FGCF), which only needs to solve a smaller and easier problem. The initial cutting contour is generated by a variant of random walks algorithm, which is very fast and gives reasonable cutting result with little user input. Experiment results on the benchmark mesh segmentation data set show that our proposed framework is robust to user input and capable of producing good results reflecting geometric features and human shape perception.

Iso-level tool path planning for free-form surfaces

The aim of tool path planning is to maximize the efficiency against some given precision criteria. In practice, scallop height should be kept constant to avoid unnecessary cutting, while the tool path should be smooth enough to maintain a high feed rate. However, iso-scallop and smoothness often conflict with each other. Existing methods smooth iso-scallop paths one-by-one, which make the final tool path far from being globally optimal. This paper proposes a new framework for tool path optimization. It views a family of iso-level curves of a scalar function defined over the surface as tool path so that desired tool path can be generated by finding the function that minimizes certain energy functional and different objectives can be considered simultaneously. We use the framework to plan globally optimal tool path with respect to iso-scallop and smoothness. The energy functionals for planning iso-scallop, smoothness, and optimal tool path are respectively derived, and the path topology is studied too. Experimental results are given to show effectiveness of the proposed methods.