A Mutual Reference Shape for Segmentation Fusion and Evaluation
S. Jehan-Besson, R. Clouard, C. Tilmant, A. de Cesare, A. Lalande, J. Lebenberg, P. Clarysse, L. Sarry, F. Frouin, M. Garreau
aa r X i v : . [ ee ss . I V ] F e b A Mutual Reference Shape for Segmentation Fusion andEvaluation
S. Jehan-Besson (a), R. Clouard (b), C. Tilmant (c), A. de Cesare (d), A. Lalande (e), J.Lebenberg (f), P. Clarysse (g), L. Sarry (h), F. Frouin (i), M. Garreau (j) a CNRS now with LITO U1288 INSERM Institut Curie Paris,[email protected] b Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC, 14000 Caen, France c Institut Pascal, UMR 6602 UBP CNRS, Clermont-Ferrand, France d Sorbonne Universités, UPMC Univ. Paris 06, CNRS, INSERM, Laboratoire d’Imagerie Biomédicale(LIB), 75013, Paris, France e ImViA EA 7535 laboratoire, Université de Bourgogne, France f CEA NeuroSpin UNATI, France g Univ Lyon, INSA-Lyon, Université Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220,U1206, F-69621, LYON, France. h Clermont Auvergne Université, CNRS, Institut Pascal, Clermont-Ferrand, France i LITO INSERM U1288 Institut Curie, Paris, France j INSERM U1099, Université de Rennes 1, LTSI, Rennes, F-35000 FrancePreprint submitted to ArXiv February 18, 2021
Mutual Reference Shape for Segmentation Fusion andEvaluation
S. Jehan-Besson (a), R. Clouard (b), C. Tilmant (c), A. de Cesare (d), A. Lalande (e), J.Lebenberg (f), P. Clarysse (g), L. Sarry (h), F. Frouin (i), M. Garreau (j) k CNRS now with LITO U1288 INSERM Institut Curie Paris,[email protected] l Normandie Univ, UNICAEN, ENSICAEN, CNRS, GREYC, 14000 Caen, France m Institut Pascal, UMR 6602 UBP CNRS, Clermont-Ferrand, France n Sorbonne Universités, UPMC Univ. Paris 06, CNRS, INSERM, Laboratoire d’Imagerie Biomédicale(LIB), 75013, Paris, France o ImViA EA 7535 laboratoire, Université de Bourgogne, France p CEA NeuroSpin UNATI, France q Univ Lyon, INSA-Lyon, Université Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220,U1206, F-69621, LYON, France. r Clermont Auvergne Université, CNRS, Institut Pascal, Clermont-Ferrand, France s LITO INSERM U1288 Institut Curie, Paris, France t INSERM U1099, Université de Rennes 1, LTSI, Rennes, F-35000 France
Abstract
This paper proposes the estimation of a mutual shape from a set of different segmen-tation results using both active contours and information theory. The mutual shape ishere defined as a consensus shape estimated from a set of different segmentations ofthe same object. In an original manner, such a shape is defined as the minimum of acriterion that benefits from both the mutual information and the joint entropy of theinput segmentations. This energy criterion is justified using similarities between infor-mation theory quantities and area measures, and presented in a continuous variationalframework. In order to solve this shape optimization problem, shape derivatives arecomputed for each term of the criterion and interpreted as an evolution equation ofan active contour. A mutual shape is then estimated together with the sensitivity andspecificity of each segmentation. Some synthetic examples allow us to cast the lighton the difference between the mutual shape and an average shape. The applicability ofour framework has also been tested for segmentation evaluation and fusion of differenttypes of real images (natural color images, old manuscripts, medical images).
Keywords:
Mutual shape, segmentation, variational approaches, segmentation fusion,segmentation evaluation, active contours, shape gradients, shape optimization,
Preprint submitted to ArXiv February 18, 2021 verage shape, information theory
1. Introduction
Constructing a “consensus” shape from a set of different segmentation results isan important point when dealing with image segmentation evaluation when no expertreference is available (evaluation without gold standard). It is also a key point for anappropriate fusion of several segmentation results in a single shape. Such a shape mustideally take advantage of the information provided by each input shape while beingrobust to outliers. We propose to tackle the estimation of such a reference shape usinginformation theory (mutual information and joint entropy) through the definition of ashape optimization problem. The consensus shape will then be defined as the minimumof an original criterion based on information theory and area measures and computedwithin the framework of active contours and shape gradients. This shape is called “amutual shape” and its applicability is tested for image segmentation evaluation andfusion.In the context of segmentation evaluation, the estimation of such a reference shapemay be important especially when dealing with large databases of medical imageswhen the manual delineation of all the frames by an expert becomes a tedious and timeconsuming task. The obtained contour is also subject to inter- and intra-variability andbeing expert-dependent, it can then not be considered as an absolute reference. As faras the evaluation without gold standard is concerned, the STAPLE algorithm (Simulta-neous Truth and Performance Level Estimation) proposed by Warfield et al. [41] is nowclassically used in this difficult context. Their algorithm consists in one instance of theEM (Expectation Maximization) algorithm where the true segmentation is estimatedby maximizing the likelihood of the complete data. Their pixel-wise approach leads tothe estimation of a reference shape simultaneously with the sensitivity and specificityof each input segmentation. From these measures, the performance level of each inputsegmentation can be estimated and a classification of all the segmentation entries canbe performed. The most recent MAP-STAPLE approach [8] is semi-local and takesbenefit of a small window or patch around the pixel. In this paper, we rather propose3o estimate the reference shape within a continuous optimization setting by consider-ing such a shape estimation under the umbrella of shape optimization tools [14] anddeformable models [21]. Indeed, the computation of a reference shape can be advanta-geously modeled as the optimum of a well chosen energy criterion and estimated by ashape gradient descent that corresponds to the deformation of an active shape. More-over, we propose a new theoretical criterion based on information theory that allows towell understand the behaviour of our reference shape.Let us also note that shape optimization algorithms have already been proposedin order to compute shape averages [7, 43] or more recently median shapes [2] byminimizing different shape metrics like the Hausdorff distance in [7] or the symmetricarea difference between shapes in [43]. Some other approaches also take advantage ofwell-appropriate distances between level-set shapes (see for example [28]). Comparingwith these previous approaches, our goal is quite different since our aim is to computea consensus shape from N input segmentations.The main contribution of this paper is then to propose a new theoretical model tocarry out the estimation of a consensus or reference shape from several segmentationentries using active contours and shape gradients. This theoretical model is based oninformation theory and justified using the analogies between information theory andarea measures. In order to estimate what we call a “mutual shape”, we then proposeto maximize the mutual information between the N input segmentations while min-imizing the joint entropy. Such a statistical criterion can be interpreted as a robustmeasure of the symmetric area difference. The minimization is performed through thecomputation of the evolution equation of an active contour. This evolution equation iscomputed using advanced shape derivation tools. In order to perform such a derivation,the criterion must be expressed in a continuous settings and non parametric probabilitydensity functions are estimated using Kernel methods [15]. In this variational setting,we also propose to add a classical regularization term based on the curvature of thedeformable contour. Such a term is weighted using a regularization parameter thatcontrols the smoothness of the obtained contour. The advantage of this formalism isto make explicitly appear, in the criterion to minimize, the domain and the associatedcontour. The criterion is then easier to understand and interpret and some geometri-4al and photometric priors could be directly added in the criterion to minimize. Thederivation is directly performed according to the domain using shape derivation tools.The proposed algorithm is first experimented on a synthetic example that allows tounderstand the differences between a classic average shape based on a symmetric areaminimization [43], a simple majority voting shape and the proposed mutual shape. It isalso evaluated for segmentation fusion and evaluation on different images : color realnatural images, old manuscripts or medical images, in order to show the genericity ofthis framework. The first application concerns segmentation evaluation and fusion ona real color natural image using segmentations from the Berkeley database [29]. Thesecond application is dedicated to text segmentation in old manuscripts and we pro-pose two main examples of segmentation. One of them takes benefit of the DIBCOdatabase [33]. The last application is devoted to segmentation fusion and evaluationof different delineation methods of the left ventricular cavity in Magnetic ResonanceImaging (MRI). For this application, we propose to compare the mutual shape to thereference algorithm STAPLE [41] classically used for segmentation fusion and evalu-ation in medical images.In section 2, we present the problem statement and in section 3, the proposed cri-terion for the estimation of the mutual shape. The criterion is then estimated in a con-tinuous framework and expressed using domain or contour integrals in section 4. Sucha continuous criterion can then be derived using shape optimization tools in order tocompute the mutual shape (see section 5). Experimental results on synthetic examplesare detailed in section 6 and the different applications in section 7.
2. Problem statement
Let U be a class of domains (open regular bounded sets, i.e. C ) of R d (with d = d = d = Ω i an element of U of boundary ∂Ω i . We consider { Ω , ..., Ω n } a family of n shapes where each shapecorresponds to the segmentation of the same unknown object O in a given image. Theimage domain is denoted by Ω ∈ R d . Our aim is to compute a reference shape µ that can5losely represent the true object O (Fig.1). We propose to define the problem through astatistical representation of shapes embedded in an information theory criterion. Let usfirst recall the main shape representation models and criteria proposed in the literature. Figure 1: Diagram of the problem statement: evaluation of a reference shape µ from a set of n segmented shapes of the same object. The computation of a reference shape is closely linked to the choice of a repre-sentation. An analytical representation may be used as in [22] where the authors pro-pose a statistical study of shapes by representing them as a finite number of points.Some authors prefer to choose an implicit representation of shapes which avoids theparametrization step. For example, in [2, 6] shapes are represented using their charac-teristic function as follows: d i ( x ) = x ∈ Ω i x Ω i (1)where x ∈ Ω is the location of the pixel within the image. We denote by Ω i the com-plementary shape of Ω i in Ω with Ω i ∪ Ω i = Ω . One may also take advantage of thedistance function associated to each shape. In [28] the authors propose to perform aprincipal component analysis on shapes in order to provide a statistical shape prior. Inthe same vein, some statistical shape priors have been proposed by [13, 32] using thisimplicit representation.More recently shapes have been represented using Legendre moments in order todefine shape priors for segmentation using active contours [17]. This representationcan also be easily included in a variational setting [17, 27].6e may also consider that each shape is a realization of a random variable. Sucha representation has been introduced in [41] in order to evaluate a reference shape ina statistical framework, in [40] for the morphological exploration of shape spaces andstatistics, and also in [18, 23] for image segmentation using information theory. Inthis paper, we take advantage of this statistical representation that appears to be welladapted to the definition of a statistical criterion. The shape is represented through arandom variable D i whose observation is the characteristic function d i defined in (1).The reference shape µ is also represented through an unknown random variable T withthe associated characteristic function t ( x ) = x ∈ µ and t ( x ) = x ∈ µ . In the literature, average shapes are defined through the minimization of the sum ofthe distances of the unknown shape µ to each shape Ω i as follows: µ = arg min µ ∗ n ∑ i = d ( Ω i , µ ∗ ) (2)Of course, the definition of the distance d is crucial and may lead to different resultsand average shapes. For example, an average shape can be computed by minimizingthe area of the symmetric differences [43] using d ( Ω i , µ ) : = | Ω i △ µ | where | . | standsfor the cardinal of the considered domain. In a continuous optimization framework, thecriterion to minimize can be expressed as follows: SD ( µ ) = n ∑ i = | Ω i △ µ | = n ∑ i = (cid:18) Z µ ( − d i ( x )) d x + Z µ d i ( x ) d x (cid:19) (3)In [6, 7], the authors prefer to introduce the Hausdorff distance to perform shape warp-ing while in [2], the authors modify the previous criterion in order to compute a medianshape . In addition to the previous works, we can also cite [40] where the authors pro-pose to explore shape spaces using mathematical morphology. The optimal shape iscomputed using a watershed performed on the squared sum of the distance functionsor using a morphological computation of a median set. Another class of algorithmswas proposed for the estimation of an unknown shape from multiple channels (coloror multimodal segmentation). We can cite the work of Chan et al. [5] or the multi-modal segmentation approaches proposed in [18, 23]. These works were not designed7t first for segmentation evaluation or fusion but they are worth mentioning becausethey propose to treat the different channels in a single criterion (may also be usefulfor information fusion). Moreover in [18, 23], some information theory quantities areused. Our work is different especially due to the fact that we consider both the max-imization of mutual information coupled with the minimization of joint entropies andthe joint estimation of evaluation quantities (sensitivity and specificity measures).
3. Proposition of a criterion for the estimation of a mutual shape
Our goal is here to mutualize the information given by each segmentation to definea consensus or reference shape. Such a shape cannot be considered as a simple averageshape.In this context, we propose to take advantage of the analogies between informationmeasures (mutual information, joint entropy) and area measures. As previously men-tioned, D i represents the random variable associated with the characteristic function d i of the shape Ω i and T the random variable associated with the characteristic function t of the reference shape µ . Using these notations, H ( D i , T ) represents the joint entropybetween the variables D i and T , and I ( D i / T ) the mutual information. We then proposeto minimize the following criterion : E ( T ) = n ∑ i = ( H ( D i , T ) − I ( D i , T )) = JH ( T ) + MI ( T ) (4)where the sum of joint entropies is denoted by JH ( T ) = ∑ ni = H ( D i , T ) and the sum ofmutual information by MI ( T ) = − ∑ ni = I ( D i , T ) .Introducing this criterion can be justified by the fact that ϕ ( D i , T ) = ( H ( D i , T ) − I ( D i , T )) is a metric which satisfies the following properties :1. ϕ ( X , Y ) ≥ ϕ ( X , Y ) = ϕ ( Y , X ) ϕ ( X , Y ) = X = Y ϕ ( X , Y ) + ϕ ( Y , Z ) ≥ ϕ ( X , Z ) ϕ ( D i , T ) = H ( T / D i ) + H ( D i / T ) using the followingclassical relations between the joint entropy and the conditional entropy and the mutualinformation and the conditional entropy : H ( D i , T ) = H ( D i ) + H ( T / D i ) I ( D i , T ) = H ( D i ) − H ( D i / T ) Finally, H ( T / D i ) + H ( D i / T ) is shown to be a metric that satisfies the four propertiesabove [12]. We then minimize a sum of distances between D i and T expressed usinginformation theory quantities.Moreover, we can give a further interesting geometrical interpretation of the pro-posed criterion. We propose to take advantage of the analogies between informationmeasures (mutual information, joint entropy) and area measures. In [35, 42], it is shownthat Shannon’s information measures can be interpreted in terms of area measures asfollows: H ( D i , T ) = mes ( ˜ D i ∪ ˜ T ) and I ( D i , T ) = mes ( ˜ D i ∩ ˜ T ) , (5)with ˜ X the abstract set associated with the random variable X and the term mes cor-responds to a signed measure defined on an algebra of sets with values in ] − ∞ , + ∞ [ .The signed measure must satisfy mes ( /0 ) = ( S nk = A k ) = ∑ nk = mes ( A k ) forany sequence { A k } nk = of disjoint sets. Each quantity can then be viewed as an opera-tion on the sets (Fig.2). These properties help us to better understand the role of eachterm in the criterion to optimize.Indeed, when estimating a classic average shape using the criterion (3), one per-forms the minimization of the sum of the union of the shapes Ω i with µ while max-imizing the sum of the intersection between the same shapes. By analogy with thiscriterion, we minimize a measure of the union while maximizing a measure of the in-tersection through the use of information quantities. In other words, the sum of thejoint entropies (union of sets) will be minimized while the sum of the mutual infor-mation quantities (intersection) will be maximized. The proposed criterion can thenbe interpreted as a statistical measure of the area of the symmetric difference which isreally interesting for the estimation of a consensus shape.9 igure 2: Mutual information and joint entropy as area measures
4. Expression of the criterion in a continuous framework
In order to take advantage of the previous statistical criterion (4) within a contin-uous shape optimization framework, we propose to express the joint and conditionalprobability density functions according to the reference shape µ . This step is detailedin this section for both the mutual information and the joint entropy. Here we try to express MI ( T ) = − ∑ ni = I ( D i , T ) in a continuous setting accord-ing to the unknown shape µ . In order to simplify the criterion, we use the classicrelation between mutual information and conditional entropy: I ( D i , T ) = H ( D i ) − H ( D i / T ) . Since H ( D i / T ) ≥ H ( D i ) is independent of T , we will rather mini-mize ∑ ni = H ( D i / T ) . Denoting by t and d i the observations of the random variables T and D i , the conditional entropy of D i knowing T can be written as follows: H ( D i / T ) = − ∑ t ∈{ , } " p ( t ) ∑ d i ∈{ , } p ( d i / t ) log ( p ( d i / t )) , (6)with p ( T = t ) = p ( t ) and p ( D i = d i / T = t ) = p ( d i / t ) .The conditional probability p ( d i = / t = ) corresponds to the sensitivity parameter p i (true positive fraction): p i ( µ ) = p ( d i = / t = ) = | µ | Z µ K ( d i ( x ) − ) d x . (7)where the function K represents a Gaussian Kernel of 0-mean and variance σ . Thisfunction allows a rigorous application of the shape derivation tools due to the fact that10he function under the integral is differentiable. In this paper, we choose a very small σ = . p ( d i = / t = ) corresponds to the specificity parameter q i (true negative fraction) : q i ( µ ) = p ( d i = / t = ) = | µ | Z µ K ( d i ( x )) d x . (8)In the rest of the paper, for the sake of simplicity, p i ( µ ) is replaced by p i and q i ( µ ) by q i . The random variable T takes the value 1 with a probability p ( t = ) = | µ | / | Ω | and 0 with a probability p ( t = ) = | µ | / | Ω | . The MI criterion can then be expressedaccording to µ : MI ( µ ) = − n ∑ i = h | µ || Ω | (( − p i ) log ( − p i ) + p i log p i )+ | µ || Ω | ( q i log q i + ( − q i ) log ( − q i )) i (9)The parameters p i and q i depend explicitly on µ , which must be taken into account inthe optimization process. Indeed if µ is updated in an iterative process, the parameters p i and q i must also be updated which implies a joint estimation of these quantities withthe unknown mutual shape. Let us now express, according to µ and in a continuous setting, the sum of the jointentropies JH ( T ) = ∑ ni = H ( D i , T ) . The following expression of the joint entropy isconsidered: H ( D i , T ) = − ∑ t ∈{ , } ∑ d i ∈{ , } p ( d i , t ) log ( p ( d i , t )) , (10)with p ( D i = d i , T = t ) = p ( d i , t ) .The following estimates for the joint probabilities are then used ( a = a = p ( d i = a , t = ) = | Ω | Z µ ( K ( d i ( x ) − a )) d x , p ( d i = a , t = ) = | Ω | Z µ ( K ( d i ( x ) − a )) d x . (11)11here the function K represents a Gaussian Kernel of 0-mean and variance σ . Thecriterion to minimize is now denoted by JH ( µ ) and can be written as follows: JH ( µ ) = − n ∑ i = h p ( d i = , t = ) log ( p ( d i = , t = )) (12) − p ( d i = , t = ) log ( p ( d i = , t = )) − p ( d i = , t = ) log ( p ( d i = , t = )) − p ( d i = , t = ) log ( p ( d i = , t = )) i where p ( d i = a , t = ) and p ( d i = a , t = ) depends on µ as expressed in equations (11). Using the two previous sections, we can express the global criterion to minimizeaccording to µ as follows: E ( µ ) = JH ( µ ) + MI ( µ ) (13) = n ∑ i = h − p ( d i = , t = ) log ( p ( d i = , t = )) − p ( d i = , t = ) log ( p ( d i = , t = )) − p ( d i = , t = ) log ( p ( d i = , t = )) − p ( d i = , t = ) log ( p ( d i = , t = )) − n ∑ i = h | µ || Ω | (( − p i ) log ( − p i ) + p i log p i )+ | µ || Ω | ( q i log q i + ( − q i ) log ( − q i )) i In this given form, the minimization of such a criterion can be considered using activecontours and shape gradients as detailed in the following section.
5. Optimization using shape gradients
In order to compute a local minimum of the criterion E defined in (13), we proposeto take advantage of the framework developed in [1] which is based on the shape op-timization tools proposed in [14, Chap.8]. The main idea is to deform an initial curve(or surface) towards the boundaries of the region of interest.12ormally, the contour then evolves according to the following Partial DifferentialEquation (PDE): ∂Γ ( z , τ ) ∂τ = v ( x , µ ) N ( x ) (14)where Γ ( z , τ ) is the evolving curve, z a parameter of the curve, τ the evolution parame-ter, v ( x , µ ) the amplitude of the velocity in x = Γ ( z , τ ) directed along the normal of thecurve N ( x ) . The evolution equation and more particularly the velocity v must be com-puted in order to make the contour evolve towards an optimum of the energy criterion.From an initial curve Γ defined by the user, we will have lim τ → ∞ Γ ( τ ) = µ at convergenceof the process.The main issue lies in the computation of the velocity v in order to find the unknownshape µ at convergence. This term is deduced from the derivative of the criterion ac-cording to the shape. The method of derivation is explained in details in [1] and isbased on shape derivation principles developed formally in [14, 38]. For completeness,we recall some useful definitions and theorems. The following theorem is the central theorem for derivation of integral domains ofthe form R µ k ( x , µ ) d x . It gives a general relation between the Eulerian derivative andthe shape derivative for region-based terms. Theorem 1
Let Ω be a C domain in R n and V a C vector field. Let k be a C function.The functional J ( µ ) = R µ k ( x , µ ) d x is differentiable and its Eulerian derivative in thedirection of V is the following: < J ′ ( µ ) , V > = Z µ k s ( x , µ ) d x − Z ∂ µ k ( x , µ )( V · N ) d a (15) where k s is the shape derivative of k defined by k s ( x , µ ) = lim τ → k ( x , µ ( τ )) − k ( x , µ ) τ . Theterm N denotes the unit inward normal to ∂ µ and d a its area element (in R , we haved a = ds where s stands for the arc length). The Eulerian derivative of J in the direction V is defined as < J ′ ( µ ) , V > = lim τ → J ( µ ( τ )) − J ( µ ) τ
13f the limit exists, with µ ( τ ) = T τ ( V )( µ ) the transformation of µ through the vector field V . The proof of the theorem can be found in [14]. The following proposition gives us a way to compute the evolution equation of theactive contour when the Eulerian derivative can be expressed as an integral over theboundary of the domain.
Proposition 1
Let us consider that the shape derivative of the criterion J ( µ ) in thedirection V may be written in the following way: < J ′ ( µ ) , V > = − Z ∂ µ v ( x , µ )( V · N ) d a (16) Interpreting this equation as the L inner product on the space of velocities, the straight-forward choice in order to minimize J ( µ ) consists in choosing V = v N for the defor-mation. We can then deduce that, from an initial contour Γ , the boundary ∂ µ can befound at convergence of the following evolution equation: ∂Γ∂τ = v ( x , µ ) N (17) where v is the velocity of the curve and τ the evolution parameter. The shape derivatives of the criteria SD ( µ ) (3), MI ( µ ) (9) and JH ( µ ) (12), can bewritten in the form (16) which allows us to find some geometrical PDEs of the form(17) for each criterion. The derivation is developed thereafter. This paragraph details the shape derivatives of the criteria SD ( µ ) (3), MI ( µ ) (9) and JH ( µ ) (12). Proofs of the two main new theorems 3 and 4 are given in the appendix ofthe paper. Theorem 2
The shape derivative in the direction V of the functional SD ( µ ) given in(3) is: < SD ′ ( µ ) , V > = − Z Γ v SD ( V · N ) d a ith the velocity : v SD = n ∑ i = ( − d i ( x )) . (18)The computation of the shape derivative of MI ( µ ) is more complex because the func-tions inside the integrals depend on µ . Theorem 3
The shape derivative in the direction V of the functional MI ( µ ) defined in(9) is: < MI ′ ( µ ) , V > = − Z Γ v MI ( V · N ) d a with the velocity v MI = | Ω | n ∑ i = h ( p i − K ( d i − )) log (cid:18) pi − p i (cid:19) (19) − ( q i − K ( d i )) log (cid:18) qi − q i (cid:19) + q i log q i + ( − q i ) log ( − q i )+ p i log p i + ( − p i ) log ( − p i ) i The computation of the shape derivative of JH ( µ ) is also complex and leads to thefollowing theorem: Theorem 4
The shape derivative in the direction V of the functional JH ( µ ) defined in(12) is: < JH ′ ( µ ) , V > = − Z Γ v JH ( V · N ) d a The velocity v JH is given by the following equation:v JH = − | Ω | n ∑ i = h K ( d i − ) log (cid:18) p ( d i = , t = ) p ( di = , t = ) (cid:19) + K ( d i ) log (cid:18) p ( d i = , t = ) p ( di = , t = ) (cid:19) i . (20) where v JH is directed along N . .4. Global evolution equations for the different criteria A standard regularization term is added in the criterion to minimize in order to favorsmooth shapes. This term corresponds to the minimization of the curve length and isdefined by
Reg ( µ ) = R ∂ µ ds . It is balanced with a positive coefficient λ in the criterionand leads to the following velocity in the evolution equation: v Reg = κ (21)where κ is the curvature of the contour Γ ( τ ) .Finally, we propose to define our mutual reference shape through the minimizationof a global criterion called J IT (Information Theoretic criterion): J IT ( µ ) = JH ( µ ) + MI ( µ ) + λ Reg ( µ ) . (22)In order to minimize this criterion, the following evolution equation is used: (cid:18) ∂Γ∂τ (cid:19) IT = ( v JH + v MI + λ v Reg ) N (23)where v MI , v JH and v Reg are defined respectively in equations (19), (20) and (21). Theterm N designates the inward unit normal of the active contour. In the experimentalresults, the mutual reference shape is also compared to the average shape (SD) thatcorresponds to the minimization of the following criterion: J SD ( µ ) = SD ( µ ) + λ Reg ( µ ) . (24)In order to minimize this criterion, the following evolution equation is applied: (cid:18) ∂Γ∂τ (cid:19) SD = ( v SD + λ v Reg ) N (25)where v SD and v Reg are defined respectively in equations (18) and (21). These velocitiesare directed along the unit inward normal N of the active contour.Note also that using this formalism, some other prior information (photometricor geometric) can be inserted by adding some additional velocities in the PDE. Forexample, we may take advantage of the tools developed in [4, 17, 27, 32].16 .5. Implementation of the active contour As far as the numerical implementation is concerned, we use the level set method[30]. The key idea is to introduce an auxiliary function U ( x , τ ) such that Γ ( τ ) is thezero level set of U . The function U is often chosen to be the signed distance functionof Γ ( τ ) . The evolution equation then becomes: ∂ U ∂τ = F | ∇ U | . (26)The velocity is chosen as F = v JH + v MI + λ v Reg for the estimation of the mutual shapeand F = v SD + λ v Reg for the estimation of the SD shape. This method is accurate andallows to automatically handle the topological changes of the initial curve. However,the same evolution equations could be implemented using faster implementation al-gorithms such as B-splines [34]. Convex optimization methods [3] may perhaps beinteresting but the criterion is not convex and some assumptions are needed before adirect application of these methods.
6. Experimental results on a synthetic example
The behavior of our mutual shape estimation is first tested on a synthetic example.The mutual shape, the classic average shape and a simple majority voting approach arecompared. We also study the joint evolution of the sensitivity and specificity parame-ters.
In this section, a test sequence consisting of different segmentations of a lozenge(Fig.3) was built. The first entry is the true segmentation mask, the other entries repre-sent the segmentation of 1 / ∑ ni = d i / n , we remark (Fig.4.b) that some masks share an intersection. Indeedthe values of the average image belong to the interval [ , . ] . The value 0 correspondsto black points in Fig.4.a and the value 0 . I A in an image named I AT displayed in (Fig.4.b). If I A ( x ) ≥ . Figure 3:
The image to segment is given in (a) and the different segmentation entries (masks) for this imageare given in (b). then I AT = I A ( x ) < . I AT =
255 (white points). This pro-cedure gives us a simple majority voting procedure. The result is the black line insidethe lozenge. (a) (b)
Figure 4:
The average image I A (a) and the corresponding binarized average image I AT (b) of the masks ofthe Fig.3(b) (simple majority voting procedure). We then use the evolution equations of both the mutual shape (23) and of the SDshape (25). The initial contour is chosen as a circle including the lozenge (Fig.5.aand Fig.6.a). The mutual shape algorithm is able to recover the whole lozenge and isthen different from a classic average shape (see Fig.5 and Fig.6). The curve evolvesand segments the whole lozenge by an iterative process (images resulted from differentiterations in Fig.5.b and Fig.5.c). The final contour is given in Fig.5.d. The mutualshape is compared to a shape average computed using the minimization of the classicsymmetrical difference (criterion J SD with evolution equation (25)). The evolutionis given in Fig.6. In this case, the final contour is similar to the result obtained bycomputing a binarized mean I AT (Fig.4.b) since it corresponds to a line due to thesmall overlap between masks 2 and 5 (Fig.3.b). The same small value is taken for theregularization parameter λ in order to give a higher importance to the data term.18a) Initial contour (b) It. 80 (c) It. 140 (d) Mutual shape Figure 5:
Evolution using the mutual shape (evolution equation (23) with λ = (a) Initial contour (b) It. 300 (c) It. 400 (d) SD shape Figure 6:
Evolution using the SD shape (evolution equation (25) with λ = An outlier (Fig.7.a) was introduced in the initial sequence of masks in order to testthe robustness of the mutual shape estimation. Indeed, our goal is to test that the mutualshape is also different to a simple union of the different masks. In Fig.7, the differentsteps of the evolution of the contour are displayed. The final contour (Fig.7.d) fits thelozenge and excludes the outlier from the final contour.
When the active contour evolves using the evolution equation (23), the parameters p i and q i are estimated jointly with the mutual shape as proposed in STAPLE [41].The different values of these parameters along the evolution of the curve are givenin Table 1. These results are obtained using masks displayed in the first row of this19a) Input outlier (b) Initial (c) It. 100 (d) It. 380 (e) Mutual shape Figure 7:
Introduction of an outlier (a) in the initial sequence of masks (Fig.3.b) and estimation of themutual shape (evolution equation (23) with λ = Table. According to the final values reported in Table 1, we can conclude that the bestsegmentation corresponds to the shape 1 with p = q = .
25. Note that the initial values of p i and q i are computed directly usingthe initial contour.We can notice that the specificity parameter q i is less relevant. Indeed this param-eter is estimated using the external domain (¯ µ ) and is then estimated using a highernumber of pixels. It should be normalized in order to be comparable to the p i value.One solution consists in the selection of a smaller working area (a mask that includesthe union of masks chosen in order to get two regions with a comparable size).
7. Experimental results on real images
In this section, our aim is to provide a variety of examples where the proposed mu-tual shape can be valuable. Indeed the theoretical framework proposed above is genericand can be applied to different images, modalities, shapes and applications. First of all,in subsection 7.1, we provide a simple example on a real color image from the Berkeleydatabase [29] to show the robustness of our estimation to an outlier, the accuracy of theobtained contour and the relevance of the classification performed using p i and q i . Asalready mentioned, the implementation is performed using the level set method whichautomatically handles topological changes. Therefore, we then apply the estimation20 terations mask 1 mask 2 mask 3 mask 4 mask 5 mask 6It. 0 p = . p = . p = . p = . p = . p = . q = q = q = q = q = q = p = . p = . p = . p = . p = . p = . q = q = q = q = q = q = p = p = . p = . p = . p = . p = q = q = q = q = q = q = . Joint evolution of the contour and of the sensitivity and specificity parameters p i and q i for themasks 1 to 6. The values correspond to the evolution of the contour displayed in Fig.7 (initial contour,iteration 100 and final contour). of the mutual shape for complicated shapes composed of multiple separated compo-nents such as the text in old documents. In the subsection (7.2), we give two examples: the first one is dedicated to the fusion of very simple binarization techniques whilethe second one performs fusion and evaluation of real automatic binarization methodsfrom the DIBCO challenge [33]. In the subsection 7.3, we propose to test the mutualshape for the fusion and the evaluation without gold standard of different segmentationmethods or expert delineations of the left ventricle in cardiac magnetic resonance im-ages (cardiac MRI) and notably expert segmentations. This estimated mutual shape iscompared to the classical STAPLE estimation [41] and evaluation results are analysedon the basis of some previous works on evaluation without gold standard [26].Let us note that the parameter λ is chosen small (1 to 10) for non convex shapesand may be chosen higher for convex shapes (10 to 100). In this last case, it can helpto get a more regularized contour. The estimation of such a mutual shape is first tested for the unsupervised evalu-ation of segmentation methods of real images. The object of interest is the tiger ofthe image displayed in Fig.9. We then extract the object of interest from the different21egmented images proposed in the Berkeley database[29]. The different segmentationentries m to m are given in Fig.8 and we add an outlier m , which corresponds to thesegmentation of the tree behind the tiger, to the five main segmentation entries. m m m m m m Figure 8: The different segmentation masks m to m and an outlier m are taken as segmentationentries for the mutual shape estimation. i m m m m m m p i q i Table 2: Sensitivity and specificity parameters p i and q i for the segmentations m to m displayedin Fig.8. In Fig.9, we show the evolution of the active contour from the initial contours (bub-bles) given in Fig.9.a. One intermediate contour is given Fig.9.b, and the final mutualshape is shown in Fig.9.c. The mutual shape provided in Fig.9.d provides an interestingresult for segmentation fusion that takes benefit of the different segmentation entrieswhile being robust to the outlier shape. The evolution of the active contour displayedin Fig.9 shows that the initial shape evolves correctly towards the boundaries of the ob-ject of interest. The sensitivity and specificity parameters are computed together withthis reference shape and provided in Table 2. These parameters provide an interest-22a) Initial contour (b) Iteration 200(c) Estimated mutual shape (final contour) (d) Final segmentation
Figure 9: Evolution of the active contour (in white, with λ = m to m (Fig.8). The final mutual shape isgiven in (c) and the segmented tiger in (d). ing classification of the different segmentation without any given reference. The mask m seems to be the best segmentation regarding with the estimated consensus and m clearly appears as an outlier. However, such a classification is dependent on the choiceof the different segmentation entries. We can however conclude that the mutual shapeis robust to the introduction of an outlier segmentation in the entry sequence and thatthe outlier is clearly detected at the end of the process through the low values of itscorresponding p i ( p i = s using Intel-based CPU @ 2.70GHz. The size of the image is 321 ∗ A second real application of our algorithm is dedicated to the fusion of differentsegmentations of the text in old parchments.23s a first example, we propose to combine different basic binarization methods us-ing the mutual shape in order to construct a better segmentation. Let us consider forexample the original image given in Fig.10 where the object of interest is the wholetext. The input masks are obtained using classical binarization techniques providedby the library of image processing Pandore [31]. The techniques used are namely "pmassbinarization” (based on a percentage of pixels, mask 1 and 2), “pcorrelation-binarization” (maximization of the correlation between two classes, mask 3), “pvari-ancebinarization” (maximization of the interclass and intraclass distance, mask 4), “pniblackbinarization” (based on an adaptive binarization technique described in [36]mask 5) and “padaptativemeanbinarization” (based on the analysis of the mean valueof the intensities on a sliding window mask 6) . The corresponding masks (shown inFig.11) are used as segmentation inputs of our mutual shape algorithm.
Figure 10: An original image from an old manuscript from Gallica (Gallica is the online numer-ical library of the BNF (National French Library)). (a) mask 1 (b) mask 2 (c) mask 3(d) mask 4 (e) mask 5 (f) mask 6
Figure 11: The different segmentation masks of the text (a,b,c,d,e,f).
The mutual shape is then computed using active contours (Fig.12). The initial24ontour is chosen as a set of little circles currently named as “bubbles” in the frameworkof active contours. The text is well segmented as displayed in Fig.12.c showing thepotential application of this method to build a consensus segmentation from a set ofdifferent simple binarization techniques not necesseraly all well chosen and composedof a set of pixels that is not connected. This example also shows that our algorithm isable to handle a shape composed of different separated components.Let us now take another example of applicability of the mutual shape for segmen-tation fusion and evaluation of different methods of text binarization taken from theDIBCO database 2013 [33]. Let us consider for example the original image givenin Fig.13.a where the object of interest is the whole text and for which we have thereference segmentation given in Fig.13.b. The input masks corresponds to differentalgorithms tested during this challenge and are all available in the database. They aregiven in Fig.14.For all these masks, we can compute the Dice Coefficient with the reference seg-mentation. The different values are given in Table.3.i m m m m m m m DC Table 3: Computation of the Dice Coefficient with the reference segmentation (Fig.13.b) for thesegmentations m to m (Fig.14). Let us now compute the mutual shape and compare the quality of the obtainedresult to the reference segmentation. The obtained mutual shape (final contour and theassociated mask) are given in Fig.15. For this mutual shape, we find DC = .
93 whichoutperforms the DC coefficient of all the different masks in entry except the mask m . The resulting mutual shape is then interesting for an intelligent fusion of differentsegmentation results. Let us now compare the ranking obtained using the mutual shapealgorithm through the joint computation of the p i and q i coefficients. The differentvalues of p i and q i are given in Table.4 and allow us to rank the different segmentationmethods as follows (from the best one to the worst one according to the sum of p i and25 i ) : m , m , m , m , m , m , m . The ranking obtained using the reference mask and theDC coefficients leads to : m , m , m , m , m , m , m . We can observe that the ranking isthe same for the first and the last mask. There are some difference of ranking betweencomparable masks such as m , m and m . The mask m corresponds to an under-segmentation and the mask m to an over-segmentation which explains their places inthe end of the ranking.i m m m m m m m p i q i Table 4: Sensitivity and specificity parameters p i and q i for the segmentations m to m displayedin Fig.14. In this last example, we show that the mutual shape leads to an interesting segmen-tation result by performing an intelligent fusion of different segmentation entries. Theobtained ranking is interesting but can be different to the ranking performed using areference mask and the DC coefficient.
The segmentation of cardiac structures is an active research field in all medicalmodalities [16], where expert performance is still higher than image segmentation al-gorithms performance. As experts segmentations can vary, it was proposed to useSTAPLE algorithm to define a consensus segmentation between different experts [39].Furthermore, to reduce the drawbacks of each specific image segmentation algorithm,it was proposed to take advantage of the results of different segmentation algorithmsand the interest of combining different segmentation results using STAPLE was shown[26]. When compared to individual methods, using these combined segmentations pro-vided better estimates of the clinical parameters of interest; this was demonstrated bya supervised approach using experts delineations and a non supervised evaluation ap-proach described in [25]. In this specific context, the estimation of a mutual shapewas tested for the non supervised evaluation and the fusion of different segmentation26ethods of the left ventricular cavity from cardiac cine magnetic resonance images[20, 19]. For instance, the excellent behavior of mutual shape towards outliers wasdemonstrated. In this section, we propose a first comparison between the mutual shapeapproach and STAPLE. At the difference of mutual shape, the STAPLE algorithm doesnot introduce any regularization term and can thus provide unsmoothed results, whichare not relevant on a physiological basis.The segmentation entries are selected inside a database that contains the results ob-tained by three experts and different algorithms [9, 11, 24, 37, 10]. The correspondingcontours are displayed in Fig.16 and Fig.17. In this specific example, the endocardiumis not well delimited by the automated algorithms, due to the presence of the aorticroot, which leads to very different segmentations. i Exp Exp Exp p i q i Table 5: Sensitivity and specificity parameters p i and q i for the segmentations Exp to Exp (Fig.16). i m m m m m p i
0. 884 0.715 0.956 0.714 0.787 q i Table 6: Sensitivity and specificity parameters p i and q i for the segmentations m to m displayedin Fig.17. Fig.18 shows the consensus segmentations estimated by STAPLE (a), SD (b), andmutual shape (c), using the three experts entries. The regularization parameter λ was setequal to 100. Table 5 shows that Exp provided for this specific case the best contour,but this result (superiority of Exp ) was already reported elsewhere [26]. Filled masksof STAPLE (Fig.18.d) and mutual shape (Fig.18.e) clearly demonstrate that STAPLEdoes not necessarily provide smoothed contours at the difference of mutual shape.Finally, Fig.19 shows the consensus segmentations, estimated by STAPLE (a), SD27b), and mutual shape (c), using the five segmentation entries of the automated algo-rithms ( m to m ). The regularization parameter λ was set equal to 10. The parameters p i and q i are estimated jointly with the mutual shape (see Table 6). At the differenceof STAPLE, the contour provided by mutual shape is smooth. However both meth-ods provide results that are quite different from the expert entries. This difficult caseshows that mutual shape and STAPLE both depend on the accuracy of the segmentationentries.
8. Conclusion
In this work, we search for a mutual shape that minimizes the sum of joint entropieswhile maximizing the sum of mutual information between each entry shape and theunknown reference shape. We give a geometrical interpretation of this criterion usingarea measures. The optimization is performed using active contours by computing ashape gradient and the associated evolution equation. Shape derivatives are computedand detailed for the given criterion. Our theoretical formalism is valid for 2D slices or3D images. The main contribution of this paper lies in the proposition of a theoreticalcriterion for the estimation of a consensus shape both for segmentation fusion andevaluation without reference.Some experimental results are provided on both synthetic images and real imagesfor segmentation fusion and evaluation. Indeed, the proposed mutual shape is able tobuild a consensus shape from a set of different segmentations of the same object andcan then be used as an intelligent fusion of different segmentation entries. Moreover,the algorithm estimates jointly the sensibility and specificity parameters (as first pro-posed by Warfield et al [41]). It then provides an evaluation or ranking of the proposedsegmentation methods on the basis of these two parameters.The experimental results performed on a synthetic image allow to better understandthe difference between the mutual shape, a simple union, an average shape and a simplemajority voting method. Moreover, the estimated mutual shape is robust to very aber-rant outliers thanks to the joint minimization of the joint entropy with the maximizationof the mutual information. 28e then propose some tests on real images for different applications. A first test isperformed on a real color image and we show that the estimated mutual shape is robustand accurate. The second type of application concerns the segmentation of the text inold manuscripts from a set of simple segmentation entries. The results obtained for thisapplication bring new opportunitues in the field of segmentation by demonstrating thatan improved segmentation method may be designed by taking benefit of the fusion ofseveral simple segmentation methods. The intelligent fusion of simple algorithms canprobably lead to a new powerful segmentation algorithm. For this part, the choice ofthe segmentation entries is still an open issue but one may think of a learning phasefor an interactive choice of the different segmentation methods chosen to build theconsensus. We may call this new kind of process : segmentation by intelligent votingusing a consensus shape. The last application, devoted to medical images, also showsthat the mutual shape can be useful for the evaluation of different segmentation methodswithout any reference. However, such a classification may be a little different than theone obtained using the true reference shape. Indeed it corresponds to a classificationregarding with the most preponderant shapes in the set of entries and not to an absoluteclassification with a fixed reference. The classification is clearly dependent on thechoice of the different segmentation methods in entry. This unsupervised evaluationprocess may however be useful to detect abnormal segmentation methods in a set ofdifferent segmentation entries.One perspective of this work may also concern the addition of prior terms insidethe variational criterion. For example, a shape prior can be interesting when the objec-tive is to segment the left ventricular cavity. If the objective is different, some otherprior shapes may be added (such as the homogeneity of the inside region for example,the gradient, or the target color for color segmentation). Our mathematical frameworkseems well adapted for this purpose since other information may be easily and rigor-ously introduced in the criterion to minimize.29 cknowledgements
The medical application took place in a larger project named MediEval supportedby the GdR 2647 Stic Santé (CNRS-INSERM). We thank all the partners of this projectfor providing us the different segmentations for the evaluation part.
Appendix
Proof of Theorem 3
First of all, we compute the shape derivatives of the probabilities p i and q i whichdepend on the domain. By applying the theorem 1, we find : < p ′ i ( µ ) , V > = − | µ | Z ∂ µ K ( d i ( x ) − )( V · N ) d a + R ∂ µ ( V · N ) d a | µ | Z µ K ( d i ( x ) − ) d x which reduces to : < p ′ i ( µ ) , V > = | µ | Z ∂ µ ( p i − K ( d i ( x ) − ))( V · N ) d a (27)In the same way, we compute the shape derivative of q i : < q ′ i ( µ ) , V > = | µ | Z ∂ µ ( − q i + K ( d i ( x )))( V · N ) d a (28)We can also compute the shape derivatives of µ and µ : < | µ | ′ , V > = − Z ∂ µ ( V · N ) d a < | µ | ′ , V > = Z ∂ µ ( V · N ) d a Let us denote ϕ ( p ) = p log ( p ) + ( − p ) log ( − p ) , the conditional entropy then be-comes : H ( D i / T ) = − h | µ || Ω | ϕ ( p i ) + | µ || Ω | ϕ ( q i ) i By using chain derivation rules, we find : < H ( D i / T ) ′ , V > = − | Ω | h | µ | < p ′ i , V > ϕ ′ ( p i )+ | µ | < q ′ i , V > ϕ ′ ( q i )+ < | µ | ′ , V > ϕ ( p i )+ < | µ | ′ , V > ϕ ( q i ) i ϕ ′ ( p ) = log ( p ) − log ( − p ) . Replacing the shape derivatives by the previousformulas, we find Theorem 3. Proof of Theorem 4
First of all, we compute the shape derivatives of the joint probabilities p ( d i , t ) whichdepend on the domain. Using theorem 1, we find for a=1 or a=0 : < p ( d i = a , t = ) ′ , V > = − | Ω | Z ∂ µ K ( d i ( x ) − a )( V · N ) d a and < p ( d i = a , t = ) ′ , V > = | Ω | Z ∂ µ K ( d i ( x ) − a )( V · N ) d a Let denote Ψ ( p ) = p log p , the joint entropy then becomes : H ( D i , T ) = − Ψ (( p ( d i = , t = )) − Ψ ( p ( d i = , t = )) − Ψ ( p ( d i = , t = )) − Ψ ( p ( d i = , t = )) By using chain derivation rules, we find : < H ( D i , T ) ′ , V > = − < p ( d i = , t = ) ′ , V > Ψ ′ ( p ( d i = , t = )) − < p ( d i = , t = ) ′ , V > Ψ ′ ( p ( d i = , t = )) − < p ( d i = , t = ) ′ , V > Ψ ′ ( p ( d i = , t = )) − < p ( d i = , t = ) ′ , V > Ψ ′ ( p ( d i = , t = )) where Ψ ′ ( p ) = log ( p ) +
1. Replacing the shape derivatives by the previous formulas,we find Theorem 4.
References [1] G. Aubert, M. Barlaud, O. Faugeras, and S. Jehan-Besson. Image segmentation usingactive contours : Calculus of variations or shape gradients. SIAM Journal on AppliedMathematics, 63(6):2128–2154, 2003.[2] B. Berkels, G. Linkmann, and M. Rumpf. An SL(2) invariant shape median. Journal ofMathematical Imaging and Vision, 37(2):85–97, 2010.
3] X. Bresson, S. Esedoglu, P. Vandergheynst, J. Thiran, and S. Osher. Fast global mini-mization of the active contour/snake model. Journal of Mathematical Imaging and Vision,28(2):151–167, 2007.[4] T. F. Chan and L. A. Vese. Active contour without edges. IEEE Transactions on ImageProcessing, 10(2):266–277, 2001.[5] T.F. Chan, B.Y. Sandberg, and L.A. Vese. Active contours without edges for vector-valuedimages. Journal of Visual Communication and Image Representation, 11(2):130–141,2000.[6] G. Charpiat, O. Faugeras, and R. Keriven. Shape metrics, warping and statistics. InInternational Conference on Image Processing, volume 2, pages 627–630, 2003.[7] G. Charpiat, O. Faugeras, and R. Keriven. Approximations of shape metrics and appli-cation to shape warping and empirical shape statistics. Foundations of ComputationalMathematics, 5(1):1–58, 2005.[8] O. Commowick, A. Akhondi-Asl, and S.K. Warfield. Estimating a reference standard seg-mentation with spatially varying performance parameters. IEEE Transactions on MedicalImaging, 31(8):1593–1606, 2012.[9] C. Constantinides, R. El Berbari, A. de Cesare, Y. Chenoune, E. Roullot, A. Herment,E. Mousseaux, and F. Frouin. Development and evaluation of an algorithm for the auto-mated segmentation of the left and right ventricles on cine MRI. IRBM, 30(4):188–191,2009.[10] C. Constantinides, E. Roullot, M. Lefort, and F. Frouin. Fully automated segmentation ofthe left ventricle applied to cine MR images: description and results on a database of 45subjects. In IEEE Engineering in Medicine and Biology Society, pages 3207–3210, 2012.[11] J. Cousty, L. Najman, M. Couprie, S. Clément-Guinaudeau, T. Goissen, and J. Garot. Seg-mentation of 4D cardiac MRI: automated method based on spatio-temporal watershed cuts.Image and Vision Computing, 28(8):1229–1243, 2010.[12] T.M. Cover and J.A. Thomas. Elements of Information Theory. Wiley-Interscience, 1991.[13] D. Cremers, T. Kohlberger, and C. Schnörr. Shape statistics in kernel space for variationalimage segmentation. Pattern Recognition, 36(2):1929–1943, 2003.
14] M.C. Delfour and J.P. Zolésio. Shapes and Geometries: Metrics, Analysis, DifferentialCalculus, and Optimization. Advances in design and control. Society for Industrial andApplied Mathematics (SIAM), 2001.[15] R. Duda and P. Hart. Pattern Classification and Scene Analysis. John Wiley & Sons, Inc.,1973.[16] J. Fleureau, M. Garreau, D. Boulmier, C. Leclercq, and A. Hernandez. 3D multi-object seg-mentation of cardiac MSCT imaging by using a multi-agent approach. IRBM, 30(3):104–113, 2009.[17] A. Foulonneau, P. Charbonnier, and F. Heitz. Affine-invariant geometric shape priorsfor region-based active contours. IEEE Transactions on Pattern Analysis and MachineIntelligence, 28(8):1352–1357, 2006.[18] A. Herbulot, S. Jehan-Besson, S. Duffner, M. Barlaud, and G. Aubert. Segmentationof vectorial image features using shape gradients and information measures. Journal ofMathematical Imaging and Vision, 25(3):365–386, 2006.[19] S. Jehan-Besson, C. Tilmant, A. de Cesare, A. Lalande, A. Cochet, J. Cousty, J. Lebenberg,M. Lefort, P. Clarysse, R. Clouard, L. Najman, L. Sarry, F. Frouin, and M. Garreau. Amutual reference shape based on information theory. In IEEE International Conference onImage Processing, pages 887–891, 2014.[20] S. Jehan-Besson, C. Tilmant, A. De de Cesare, F. Frouin, L. Najman, A. Lalande, L. Sarry,C. Casta, P. Clarysse, C. Constantidinès, J. Cousty, M. Lefort, A. Cochet, and M. Gar-reau. Estimation d’une forme mutuelle pour l’évaluation de la segmentation en imageriecardiaque. In GRETSI - Traitement du Signal et des Images, 2011.[21] M. Kass, A. Witkin, and D. Terzopoulos. Snakes : Active contour models. InternationalJournal of Computer Vision, 1:321–332, 1988.[22] D.G. Kendall. Shape manifolds, procrustean metrics, and complex projective spaces.Bulletin of the London Mathematical Society, 16(2):81–121, 1984.[23] J. Kim, J.W. Fisher, A. Yezzi, M. Cetin, and A.S. Willsky. Nonparametric methods forimage segmentation using information theory and curve evolution. In IEEE InternationalConference on Image Processing, pages 797–800, 2002.
24] A. Lalande, N. Salve, A. Comte, M.-C. Jaulent, L. Legrand, P.M. Walker, Y. Cottin, J.E.Wolf, and F. Brunotte. Left ventricular ejection fraction calculation from automaticallyselected and processed diastolic and systolic frames in short axis cine-MRI. Journal ofCardiovascular Magnetic Resonance, 6(4):817–827, 2004.[25] J. Lebenberg, I. Buvat, A. Lalande, P. Clarysse, C. Casta, A. Cochet, C. Constantinidès,J. Cousty, A. de Cesare, S. Jehan-Besson, M. Lefort, L. Najman, E. Roullot, L. Sarry,C. Tilmant, M. Garreau, and F. Frouin. Non supervised ranking of different segmenta-tion approaches: application to the estimation of the left ventricular ejection fraction fromcardiac cine MRI sequences. IEEE Transactions on Medical Imaging, 31(8):1651–1660,2012.[26] J. Lebenberg, A. Lalande, P. Clarysse, I. Buvat, C. Casta, A. Cochet, C. Constantinidès,J. Cousty, A. de Cesare, S. Jehan-Besson, M. Lefort, L. Najman, E. Roullot, L. Sarry,C. Tilmant, F. Frouin, and M. Garreau. Improved estimation of cardiac function parame-ters using a combination of independent automated segmentation results in cardiovascularmagnetic resonance imaging. PLOS ONE, 10(8):e0135715, 2015.[27] F. Lecellier, S. Jehan-Besson, J. Fadili, G. Aubert, M. Revenu, and E. Saloux. Region-based active contours with noise and shape priors. In IEEE International Conference onImage Processing, volume 1, pages 1649–1652, 2006.[28] M. Leventon, E. Grimson, and O. Faugeras. Statistical shape influence in geodesic activecontours. In IEEE International Conference on Image Processing, pages 316–323, 2000.[29] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented naturalimages and its application to evaluating segmentation algorithms and measuring ecologicalstatistics. In Proceedings Eighth International Conference on Computer Vision, volume 2,pages 416–423, 2001.[30] S. Osher and J.A. Sethian. Fronts propagating with curvature-dependent speed: Algorithmsbased on Hamilton-Jacobi formulations. Journal of Computational Physics, 79(1):12–49,1988.[31] Pandore: A library of image processing operators (Version 6.6). [Software]. Greyc Labo-ratory. https://clouard.users.greyc.fr/Pandore, 2013.
32] N. Paragios. A level set approach for shape-driven segmentation and tracking of the leftventricle. IEEE Transactions on Medical Imaging, 22(6):773–776, 2003.[33] I. Pratikakis, B. Gatos, and K. Ntirogiannis. ICDAR 2013 document image binariza-tion contest (DIBCO 2013). In Proceedings of the 2013 12th International Conferenceon Document Analysis and Recognition, pages 1471–1476, Washington, DC, USA, 2013.IEEE Computer Society.[34] F. Precioso, M. Barlaud, T. Blu, and M. Unser. Robust real-time segmentation of imagesand videos using a smooth-spline snake-based algorithm. IEEE Transactions on ImageProcessing, 14(7):910–924, 2005.[35] F. M. Reza. An Introduction to Information Theory, pages 106–108. McGraw-Hill, 1994.[36] J. Sauvola and M. Pietikainen. Adaptive document image binarization. PatternRecognition, 33(2):225–236, 2000.[37] J. Schaerer, C. Casta, J. Pousin, and P. Clarysse. A dynamic elastic model for segmentationand tracking of the heart in MR image sequences. Medical Image Analysis, 14(6):738–749,2010.[38] J. Sokolowski and J.P. Zolésio. Introduction to shape optimization. Shape sensitivityanalysis. Springer Series in Computational Mathematics. Springer-Verlag, Berlin, 1992.[39] A. Suinesiaputra, B.R. Cowan, A.O. Al-Agamy, M.A. Elattar, N. Ayache, A.S. Fahmy,A.M. Khalifa, P. Medrano-Gracia, M.P. Jolly, A.H. Kadish, D.C. Lee, J. Margeta, S.K.Warfield, and A.A. Young. A collaborative resource to build consensus for automated leftventricular segmentation of cardiac MR images. Medical Image Analysis, 18(1):50–62,2014.[40] S. Velasco-Forero and J. Angulo. Statistical shape modeling using morphological represen-tations. In IEEE International Conference on Image Processing, pages 3537–3540, 2010.[41] S.K. Warfield, K. H. Zou, and W. M. Wells III. Simultaneous truth and performancelevel estimation (STAPLE): an algorithm for the validation of image segmentation. IEEETransactions on Medical Imaging, 23(7):903–921, 2004.[42] R. W. Yeung. A new outlook on shannon’s information measures. IEEE Transactions onInformation Theory, 37(3):466–474, 1991.
43] A.J. Yezzi and S. Soatto. Deformotion: Deforming motion, shape average and the jointregistration and approximation of structures in images. International Journal of ComputerVision, 53(2):153–167, 2003.
Figure 12: Evolution of the active contour (in black) using the mutual shape (evolution equation(23) with λ = Figure 13: An original image from an old manuscript from the DIBCO database (a) and itscorresponding reference binarization. (a) mask 1 (b) mask 2(c) mask 3 (c) mask 4(e) mask 5 (f) mask 6(g) mask 7
Figure 14: The different segmentation masks of the text (a,b,c,d,e,f,g) taken from the 2013DIBCO database.
Figure 15: The mutual shape (final contour and resulting mask) computed using the masks m to m of the DIBCO image PR08. Exp Exp Exp Figure 16: Segmentation methods of the left ventricle provided by three experts
Exp to Exp . m m m m m Figure 17: Segmentation methods of the left ventricle provided by 5 automated algorithms m to m . Figure 18: Estimation of different consensus estimates using the contours given by the threeexperts
Exp to Exp (Fig.16) using STAPLE algorithm (a), SD approach (b), and mutual shape(c). Filled masks correspond to STAPLE (d) and mutual shape (e) (a) (b) (c) (d) Figure 19: Estimation of different consensus estimates using the masks m to m (Fig.17) usingSTAPLE algorithm (a), SD approach (b), and mutual shape (c), contour delineated by Exp (d).(d).