Regional Active Contours based on Variational level sets and Machine Learning for Image Segmentation
IIMT Institute for Advanced Studies Lucca
Lucca, Italy
Regional Active Contours based on Variational level setsand Machine Learning for Image Segmentation
PhD programme in Computer Science and EngineeringXXVII cycle
ByMohammed Mohammed Abdelsamea Ahmed2015 he dissertation of Mohammed Mohammed AbdelsameaAhmed is approved.
Program Coordinator: Prof. Rocco De Nicola, IMT Institute forAdvanced Studies, Lucca, ItalySupervisor: Assistant Prof. Giorgio Gnecco, IMT Institute for AdvancedStudies, Lucca, ItalySupervisor: Prof. Mohamed Medhat Gaber, Robert Gordon University,Aberdeen, UKTutor: Assistant Prof. Giorgio Gnecco, IMT Institute for AdvancedStudies, Lucca, ItalyThe dissertation of Mohammed Mohammed Abdelsamea Ahmed hasbeen reviewed by:Prof. Ivan Jordanov, University of Portsmouth, UKProf. Frederic Stahl, University of Reading, UKProf. Mykola Pechenizkiy, Eindhoven University of Technology,Netherlands
IMT Institute for Advanced Studies, Lucca2015 edicated to my wife Omnia and our sons Abdelrahman and Ziad. able of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . vList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . xviAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . xviiiPublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . xixAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 1
ACM s . . . . . . . . . . . . . 13
ACM s . . . . . . . . . . . 142.1.1 Unsupervised
ACM s . . . . . . . . . . . . . . 172.1.2 Supervised
ACM s . . . . . . . . . . . . . . . 262.1.3 Other level set-based
ACM s . . . . . . . . . . 28
ACM s . . . . . . . . . . . . . . . . . . . . . . 29
SOM s) . . . . . . . . . . . . . 293.2
SOM -based Segmentation Models . . . . . . . . . . 323.2.1 An example of a
SOM -based
ACM . . . . . . 333.2.2 Other
SOM -based
ACM s . . . . . . . . . . . 35ii
Globally Signed Pressure Force Model . . . . . . . . . . 40
GSRPF
Model . . . . . . . . . . . . . . . . . . . . 414.2.1 The
GSRPF sign pressure function formulation 434.3 Implementation . . . . . . . . . . . . . . . . . . . . . 454.4 Experimental study . . . . . . . . . . . . . . . . . . . 464.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 49
SOM -based Chan-Vese Model . . . . . . . . 55
CSOM-CV model . . . . . . . . . . . . . . . . . . 565.2.1 Training session . . . . . . . . . . . . . . . . . 565.2.2 Testing session . . . . . . . . . . . . . . . . . 585.3 Implementation . . . . . . . . . . . . . . . . . . . . . 605.4 Experimental study . . . . . . . . . . . . . . . . . . . 625.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 65 AC Model . . . . . . . . . . . . . . . . . 67
SOAC
Model . . . . . . . . . . . . . . . . . . . . 686.2.1 The
SOAC model for scalar-valued images . 686.2.2 The
SOAC model for vector-valued images . 716.3 Implementation . . . . . . . . . . . . . . . . . . . . . 716.4 Experimental study . . . . . . . . . . . . . . . . . . . 736.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 88 SOM -based Chan-Vese Model . . . . . . . . . . . . . . . 90
SOMCV and
SOMCV s models . . . . . . . . . . 907.2.1 The SOMCV and
SOMCV s models for scalar-valued images . . . . . . . . . . . . . . . . . . 917.2.2 The SOMCV and
SOMCV s models for vector-valued images . . . . . . . . . . . . . . . . . . 957.3 Implementation . . . . . . . . . . . . . . . . . . . . . 957.4 Experimental study . . . . . . . . . . . . . . . . . . . 987.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 114 SOM -based Regional AC Model . . . . . . . . . . . . . . 115 iii.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 1158.2 The
SOM - RAC model . . . . . . . . . . . . . . . . . . 1168.3 Implementation . . . . . . . . . . . . . . . . . . . . . 1198.4 Experimental study . . . . . . . . . . . . . . . . . . . 1218.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 127 iv ist of Figures / multi-channel im-ages considered in this work: images with (a) fore-ground / background intensity overlap and inhomo-geneous regions, (b) intensity inhomogeneity, (c)weak and ill-defined edges and shadows, (d) ad-ditive noise, (e) many intensity levels; sequences ofimages with (f) scene changes. . . . . . . . . . . . . . 61.2 Examples of challenging images (b) that can be ef-fectively handeled by our proposed models (a). . . . 82.1 The parametric representation of a contour. . . . . . 142.2 The geometric representation of a contour. . . . . . . 153.1 The SOM architecture. . . . . . . . . . . . . . . . . . 303.2 The architecture of the SISOM -based
ACM proposedin [104]. . . . . . . . . . . . . . . . . . . . . . . . . . . 36v.1 A synthetic image with multiple classes in the fore-ground, and the performance of the proposed,
SBGFRLS ,and C - V , models for some choices of their parame-ters. (a) the original 123 x 80 image with three di ff er-ent intensities 100, 150 and 200, and its histogram; (b)the same image with Gaussian noise added of stan-dard deviation (SD) 30, and its histogram. Overlaidis also the initial contour (in red) used in all the sub-sequent tests. From left to right: the segmentationresults of our model (c) with di ff erent σ values (1.4,1.6, 1.8, and 2); (d) of SBGFRLS with di ff erent σ and α values ((2,10), (2,50), (2.5, 10), and (2.5,50), respec-tively); and (e) of the C - V model with di ff erent µ values (1.4, 1.6, 1.8, and 2). . . . . . . . . . . . . . . . 504.2 The sensitivity of our model to the parameter σ interms of Recall and Precision, in segmenting the im-age in Fig. 4.1 with Gaussian noise with standarddeviation , SD =
30. . . . . . . . . . . . . . . . . . . 514.3 The segmentation results on a 101 x 99 syntheticimage containing di ff erent objects of variable con-vexity and shape, and noisy background. From leftto right: the original image (with the initial contour),the segmentation obtained by the proposed model( σ = . SBGFRLS and C - V models. . . . . . . . . . . . . . . . . . . . . . 514.4 The rapid evolution of the proposed model ( σ = . ff erent contour initializations, and then, fromleft to right, the segmentation results of the proposed GSRPF model ( σ = . SBGFRLS (with σ = α = C - V (with µ = .
2) models,respectively, when using the same initial contour. . 53vi.6 Segmentation results when di ff erent real images en-countered in natural and life sciences are used. Ar-ranged in rows there are: (a) a 109 x 119 brain MRIimage, from [3]; (b) a 436 x 422 Arabidopsis op-tical image with complex background; (c) a 256 x256 cellulose microscopy image, from [4]; and (d)a 256 x 256 chromosome microscopy image, from[4]. Arranged in columns there are the original im-age (with the initial contour), and then, from left toright the results of the proposed GSRPF model, andof the
SBGFRLS and C - V models respectively, whenusing the same initial contour. (Parameters are as inFig. 4.5, except in (a) for GSRPF ( σ = CSOM-CV and the
CSOM models.By its definition, no supervised pixel is used by the
C-V model. . . . . . . . . . . . . . . . . . . . . . . . . 625.2 The robustness of the
CSOM-CV model to two dif-ferent kinds of noise: the first column shows, fromtop to down, two noisy versions of the image shownin Fig. 5.1(a), and two noisy versions of the imageshown in Fig. 5.1(b), respectively, with the additionof Gaussian noise with standard deviation SD =
50 (first and third row) and salt and pepper noise(second and fourth row). The initial contours usedby the
CSOM-CV and
C-V models are also shown(first and third row); finally, the second, third, andfourth columns show, respectively, the correspond-ing binary segmentation obtained by the
CSOM-CV , CSOM , and
C-V models. . . . . . . . . . . . . . . . . 63vii.3 The segmentation results obtained on real and syn-thetic gray-level images. The first row shows theoriginal images with the initial contours, while thesecond, third, and fourth rows show, respectively,the corresponding segmentation results obtained bythe
CSOM-CV , CSOM , and
C-V models. . . . . . . 646.1 A synthetic image containing objects characterizedby many di ff erent intensities and an overlap in theforeground / background intensity distributions, anda comparison among its segmentations obtained bythe SOAC model and the
LRCV , C - V , KDE -based,
GMM -based, and
CSOM models: (a) the original90 ×
122 image with the three di ff erent intensities100, 150 and 200 in its foreground, and 120 in itsbackground, and (b) its histogram; (c) the same im-age with the addition of Gaussian noise with stan-dard deviation ( SD ) equal to 5, and (d) its histogram;(e) the original image in (a) with the addition of arectangular initial contour (in black), and trainingexamples (in red for the foreground, in blue for thebackground); (g) its ground truth, and its segmenta-tion results obtained - starting from the initial con-tour in (e) - by (f) the SOAC model, (h) the
LRCV model, and (i) the C - V model; (j) the noisy versionof the same image, already shown in (c), with theaddition of the initial contour and the training ex-amples; its segmentation by (k) the CSOM model, (l)the
SOAC model, (m) the
KDE -based model, and (n)the
GMM -based model. . . . . . . . . . . . . . . . . 766.2 (a) the same synthetic image considered in Fig. 6.1,with the supervised training examples; (b) its noisyversion, obtained by the addition of salt and peppernoise; (c) the segmentation result by
SOAC model. . 77viii.3 (a) A synthetic image; (b) its ground truth; (c) train-ing examples (in red for the foreground, in blue forthe background); (d) the initial contour used by the
SOAC model (in white). . . . . . . . . . . . . . . . . 786.4 The sensitivity of
SOAC to di ff erent levels of noiseadded to the image shown in Fig. 6.3(a) (Gaussiannoise with SD = , , , , , , , ,
90, and100, respectively), in terms of Recall, Precision, and F -measure. For a comparison, also the case of CSOM is considered. The number of training pixels for boththe foreground and the background is also shown. . 796.5 The segmentation results obtained on di ff erent realand synthetic images. Arranged in columns, fromleft to right: training examples (in red for the fore-ground, in blue for the background), the initial con-tour, and the segmentation results obtained, respec-tively by the SOAC , KDE -based,
GMM -based, C - V ,and CSOM models. . . . . . . . . . . . . . . . . . . 826.6 The contour evolution of the
GMM -based and
KDE -based models on some images in Fig. 6.5. Arrangedin columns, from left to right: the initial contour,4 intermediate contours, and the final contour. Ar-ranged in rows: the contour evolutions of the
GMM -based model for two images, and the ones of the
KDE -based model for two images. . . . . . . . . . . 836.7 The sensitivity to the training pixels on some real im-ages, taken from [13, 5]. From left to right: trainingexamples (in red for the foreground, in blue for thebackground) of the first (training) image, the initialcontour and the segmentation produced by
SOAC for the second (test) image, and the initial contourand the segmentation produced by
SOAC for thethird (test) image. . . . . . . . . . . . . . . . . . . . 83ix.8 Segmentation results obtained by the
SOAC modelon two real images containing intensity inhomo-geneity. Arranged in columns: the training exam-ples used by the
SOAC model (in red for the fore-ground, in blue for the background), respectively,for (a) a 174 ×
238 brain image and (b) a 152 × SOAC model for the two cases; the curve evolution at threesuccessive stages of
SOAC . . . . . . . . . . . . . . . . 846.9 Segmentation results obtained for the real imagesshown in Fig. 6.8(a) and (b) by three supervised ref-erence models, using the same training data as the
SOAC model. Arranged in columns: the segmen-tation results obtained, respectively, by the
KDE -based,
GMM -based, and
CSOM models. . . . . . . . 856.10 The segmentation results obtained on real multi-spectral images. Arranged in columns: three 481 ×
321 real images with training examples (respectively,in red for the foreground, in blue for the background)and the initial contour (respectively, in black, white,and black); the segmentation results obtained onsuch images, respectively, by the vectorial versionsof the
SOAC , KDE -based,
GMM -based, C - V , and CSOM models. . . . . . . . . . . . . . . . . . . . . . . 866.11 A comparison of the pixel-by-pixel visual represen-tations of the term e SOAC in the
SOAC model (for-mula (6.14)) and the terms e KDE and e GMM (formula(6.15)) in the
GMM -based and
KDE -based models,for two selected images considered in the chapter.Arranged in columns, from left to right: the originalimages, and the pixel-by-pixel visual representationof the terms e SOAC , e KDE and e GMM . . . . . . . . . . . 88x.1 The architecture of
SOMCV for
RGB images: (a)the input intensities of a training voxel; (b) a 3 × SOM neural map (with a three-dimensional proto-type associated with each neuron); (c) the trained
SOM ; (d) the contour evolution process; and (e) theforeground (in red) and background (in black) rep-resentative neurons for the
SOMCV (top) and the
SOMCV s (down) models. For a scalar-valued im-age, a similar model is used, but the prototypes havedimension 1, and a 1- D grid is used. . . . . . . . . . 997.2 The rapid contour evolution of the SOMCV and
SOMCV s models when compared to the contourevolution of the C - V model, in the scalar case. Thefirst and second rows show, respectively, the con-tour evolution of SOMCV and
SOMCV s . From leftto right: initial contour (in black), contour after 3,6, 9, 12 iterations, and final contour (15 iterations).The third row shows the contour evolution of the C - V model. From left to right: initial contour (inblack), contour after 50, 100, 150, 200 iterations, andfinal contour (260 iterations). . . . . . . . . . . . . . 1007.3 The e ff ectiveness of the SOMCV model in dealingwith objects characterized by many di ff erent intensi-ties and skewness / multimodality of the foregroundintensity distribution. Arranged in rows there are:(a) a noisy 140 ×
100 image (with Gaussian noiseadded, standard deviation SD =
10) with six di ff er-ent intensities 80, 100, 140, 170, 200, and 230 in itsforeground; (b) a noisy 90 ×
122 image (with Gaus-sian noise added, standard deviation SD =
10) withthree di ff erent intensities 100, 150, and 200 in itsforeground. The columns from left to right are: theimages with the additions of the initial contours, thehistograms of the intensities of the images, and, re-spectively, the segmentation results of the SOMCV , SOMCV s ( σ = . , .
5, respectively, for (a) and (b)),and C - V models. . . . . . . . . . . . . . . . . . . . . . 101xi.4 The robustness of the SOMCV and
SOMCV s modelsto the additive noise: the first row shows, from leftto right, the image of Fig. 7.3(a) with the addition ofdi ff erent Gaussian noise levels (standard deviation SD =
10, 15, 20, and 25, respectively); the secondand third rows show, respectively, the correspond-ing segmentation results of
SOMCV and
SOMCV s . . 1027.5 The robustness of the SOMCV and
SOMCV s modelsto the additive noise: the first row shows, from leftto right, the image of Fig. 7.3(b) with the addition ofdi ff erent Gaussian noise levels (standard deviation SD =
10, 20, 30, 40, and 50, respectively); the secondand third rows show, respectively, the correspond-ing segmentation results of
SOMCV and
SOMCV s . . 1037.6 The segmentation results obtained on real and syn-thetic scalar-valued images. The first, second andthird row show the original images with the ini-tial contours, the histograms of the image intensi-ties and their ground truth, respectively, while thefourth, fifth, and sixth rows show, respectively, thecorresponding segmentation results of the SOMCV , SOMCV s and C - V models. . . . . . . . . . . . . . . . 1047.7 The segmentation results on real images from [5,13], and synthetic vector-valued images. The firstand second rows show the original images with theinitial contours, respectively, while the third, fourth,and fifth rows show, respectively, the correspondingsegmentation results of the vectorial versions of the SOMCV , SOMCV s and C - V models. Note that σ = . SOMCV and
SOMCV s for theimage in the second column. . . . . . . . . . . . . . . 105xii.8 The segmentation results of the Otsu’s and the multi-threshold Otsu’s methods on some of the scalar-valued images considered in this chapter. The firstrow shows the original images. The second rowshows the segmentation results, corresponding tothe images of first row, obtained by the Otsu’s method.The third row shows the object of interests obtainedby the multi-threshold Otsu’s method when the num-ber of thresholds is five. The fourth row shows themerged objects obtained by first applying the multi-Otsu’s method when the number of thresholds is 2,3, 4, and 5, then merging some of the obtained ob-jects. The fifth row shows the segmentation resultsof the Otsu’s method applied on the images of thefourth row. Finally, the sixth row shows the segmen-tation results obtained by SOMCV on the images ofthe first row. . . . . . . . . . . . . . . . . . . . . . . . 1127.9 The robustness of the
SOMCV model to scene changesand moving objects. (a) The first row shows the orig-inal early frames (frames 50-59, from left to right)of a real-aircraft video while later frames (frames350-359, from left to right) are shown in the secondrow. (b) shows the segmentation results obtained by
SOMCV , on the frames shown in part (a). . . . . . . 1138.1 The segmentation results obtained on real and syn-thetic images by the
SOM - RAC and the
LRCV mod-els. The first column shows the original imageswith the initial contours, while the second and thirdcolumns show, respectively, the corresponding seg-mentation results obtained by the two models. . . . 123xiii.2 The robustness of the
SOM - RAC model with respectto the contour initialization, as compared to the seg-mentation results obtained by the
LRCV model, fora synthetic 127 ×
96 image with intensity inhomo-geneity. The first column shows the original imagewith three di ff erent rectangular initial contours (inwhite). The second and third columns show, re-spectively, the segmentation results obtained by the SOM - RAC and the
LRCV models. . . . . . . . . . . . 1248.3 The robustness of the
SOM - RAC model with re-spect to the contour initialization in handling a real103 ×
131 image in the presence of intensity inhomo-geneity and weak edges, as compared to the segmen-tation results obtained by the
LRCV model on thesame image. The first column shows the original im-age with three di ff erent rectangular initial contours(in black). The second and third columns show, re-spectively, the segmentation results obtained by the SOM - RAC and the
LRCV models. . . . . . . . . . . . 1258.4 The robustness of the
SOM - RAC model with respectto additive noise in handling a synthetic 127 × LRCV model. The first column shows, from topto down, the image of Fig. 8.2 with the addition ofdi ff erent Gaussian noise levels (with standard devi-ations SD =
0, 5, 10, 15, 20, and 25, respectively).The second and third columns show, respectively,the segmentation results obtained by the
SOM - RAC and the
LRCV models. . . . . . . . . . . . . . . . . . 126xiv.5 The robustness of the
SOM - RAC model with respectto scene changes. The first and third columns showthe training images while the second and fourthcolumns show, respectively, the segmentations re-sults obtained by the
SOM - RAC model on di ff erenttest images. For each row, the SOM - RAC modelwas trained on the first (respectively third) image,then it was used to segment the second (respectively,fourth) image. For the second and fourth columns,the initial contours are shown in white, whereas thefinal segmentation results are shown in black. . . . 1298.6 The robustness of the
SOM - RAC model with respectto the locality parameter σ in handling a synthetic100 ×
100 image (already shown in the second row ofFig. 8.5) when compared to
LRCV model (the sameinitial contour has been used in the comparison).Parts (a) shows the segmentation results obtainedby the
SOM - RAC model with σ = , , , ,
25 inthe first row and σ = , , , ,
50 in the secondrow. Part (b) shows the corresponding segmentationresults obtained by the
LRCV model. . . . . . . . . 130xv ist of Tables
ACM )s presented in the thesis. . . . . . . . 94.1 The robustness of the
GSRPF model ( σ = .
4) to thenoise level: Precision and Recall metrics for di ff er-ent Gaussian noise levels, measured by the standarddeviation ( SD ). . . . . . . . . . . . . . . . . . . . . . . 474.2 The CPU time and the number of iterations requiredby the proposed GSRPF model, and by the
SBFRLS and C - V models, to segment the foreground in someof the images considered here. . . . . . . . . . . . . 485.1 The contour evolution time and number of iterationsrequired by the CSOM-CV and
C-V models to seg-ment the foreground for some of the images shownin this chapter. The
CPU time of
CSOM is also in-cluded. . . . . . . . . . . . . . . . . . . . . . . . . . . 655.2 The Precision, Recall, and F -measure metrics for the CSOM-CV , CSOM , and
C-V models. . . . . . . . . . 656.1 For each image considered in the chapter: the ref-erence from which it was taken (apart from the firstartificial one), its size, and the associated number offoreground / background pixels used in the trainingphase. . . . . . . . . . . . . . . . . . . . . . . . . . . . 78xvi.2 The Precision, Recall, and F -measure metrics for the SOAC , KDE -based,
GMM -based, and
CSOM mod-els, applied to the images presented in the chapter.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 806.3 The
CPU time and the number of iterations requiredby the
SOAC model and the
KDE -based model tosegment the foreground for some images consideredin this chapter. . . . . . . . . . . . . . . . . . . . . . 877.1 The Precision, Recall, and F -measure metrics for thescalar SOMCV , SOMCV s and C - V models in the seg-mentation of the scalar images shown in Fig. 7.6. . 1067.2 The Precision, Recall, and F -measure metrics for thevectorial SOMCV , SOMCV s and C - V models in thesegmentation of the RGB images shown in Fig. 7.7. 1067.3 The contour evolution time and number of iterationsrequired by the
SOMCV , SOMCV s , and C - V mod-els to segment the foreground for the scalar-valuedimages shown in Fig. 7.6. . . . . . . . . . . . . . . . 1087.4 The contour evolution time and number of iterationsrequired by the SOMCV , SOMCV s , and C - V mod-els to segment the foreground for the vector-valuedimages shown in Fig. 7.7. . . . . . . . . . . . . . . . 1097.5 The Precision, Recall, and F -measure metrics forthe Otsu’s method and the multi-threshold Otsu’smethod (with post-processing) in the segmentationof the images shown in Fig. 7.8 (second and fifthrows, respectively) compared with the SOMCV model(sixth row). . . . . . . . . . . . . . . . . . . . . . . . 111xvii cknowledgments
First of all I wish to express my sincere gratitude to myadvisor, Assist. Prof. Giorgio Gnecco, for his incredible guidance,serenity, and support. In particular I am highly indebted and ap-preciated his advice and guidance in every line that I wrote duringhis supervision. I would like also to express my deepest gratitudeto my co-advisor, Prof. Mohamed Medhat Gaber, for his advises,guidance, great vision, and the great opportunity for hosting me atRobert Gordon University during my visiting research period.Moreover, I would especially like to thank all of my coau-thors, without whom, in all honesty, the research presented in thisthesis would never have been possible. I also must thank the fac-ulty and sta ff of the IMT Institute for Advanced Studies, Luccafor their professional treatment of me as more of a colleague. Inparticular I would like very much to thank the sta ff of the FacilitiesO ffi ce for helping me during my stay in Italy.Last but not least, I would like to thank my wife for herpatience and encouragement during my study.xviiihe research outcomes of this thesis are reported in the fol-lowing publications: Journal Papers :1. M. M. Abdelsamea and G. Gnecco. Robust Local-GlobalSOM-based ACM. Electronics Letters, vol. 51, pp. 142–143,2015.2. M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. An e ffi cientself organizing active contour model for image segmentation.Neurocomputing, vol. 149, Part B, pp. 820–835, 2015.3. M. Minervini, M. M. Abdelsamea, and S. A. Tsaftaris, Image-based plant phenotyping with incremental learning and ac-tive contours, Ecological Informatics, vol. 23, pp. 35–48.2014, Special Issue on Multimedia in Ecology and Environ-ment.4. M. M. Abdelsamea, G. Gnecco, M. M. Gaber, and E. Elyan,On the relationship between variational level set-based andSOM-based active contours, submitted to Computational In-telligence and Neuroscience, 2014.5. M. M. Abdelsamea, G. Gnecco, M. M. Gaber, A SOM-basedChan-Vese Model for unsupervised image segmentation, sub-mitted to Image and Vision Computing, 2014. Conference Papers :1. M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. A Surveyof SOM-based Active Contours for Image Segmentation. InProceedings of the 10th Workshop on Self- Organizing Maps(WSOM 2014), Advances in Intelligent Systems and Comput-ing, volume 295, pages 293–302. Springer, 2014.2. M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. A concur-rent SOMbased Chan-Vese model for image segmentation. InProceedings of the 10th Workshop on Self-Organizing Maps(WSOM 2014), Advances in Intelligent Systems and Comput-ing, volume 295, pages 199–208. Springer, 2014.xix. M. M. Abdelsamea and S. A. Tsaftaris. Active contour modeldriven by globally signed region pressure force. In Proceed-ings of the 18th International Conference on Digital SignalProcessing (DSP), pages 1–6, 2013.xx bstract
Image segmentation is the problem of partitioning an im-age into di ff erent subsets, where each subset may have a di ff erentcharacterization in terms of color, intensity, texture, and / or otherfeatures. Segmentation is a fundamental component of image pro-cessing, and plays a significant role in computer vision, objectrecognition, and object tracking. Active Contour Models ( ACM s)constitute a powerful energy-based minimization framework forimage segmentation, which relies on the concept of contour evolu-tion. Starting from an initial guess, the contour is evolved with theaim of approximating better and better the actual object boundary.Handling complex images in an e ffi cient, e ff ective, and ro-bust way is a real challenge, especially in the presence of inten-sity inhomogeneity, overlap between the foreground / backgroundintensity distributions, objects characterized by many di ff erent in-tensities, and / or additive noise. In this thesis, to deal with thesechallenges, we propose a number of image segmentation modelsrelying on variational level set methods and specific kinds of neu-ral networks, to handle complex images in both supervised andunsupervised ways. Experimental results demonstrate the highaccuracy of the segmentation results, obtained by the proposedmodels on various benchmark synthetic and real images comparedwith state-of-the-art active contour models.xxi hapter 1 Introduction
Image segmentation is the problem of partitioning an image I ( x ), where x is the pixel location within the image, into di ff erentsubsets Ω i , where each subset may have a di ff erent characterizationin terms of color, intensity, texture, and / or other features used assimilarity criteria. Segmentation is a fundamental component ofimage processing, and plays a significant role in computer vision,object recognition, and object tracking.Traditionally, image segmentation methods can be classi-fied into five categories. The first category is made up of threshold-based segmentation methods [83]. These methods are pixel-based,and usually divide the image into two subsets, i.e., the foregroundand the background, using a threshold on the value of some fea-ture (e.g., gray level, color value). These methods assume that theforeground and background in the image have di ff erent ranges forthe values of the features to be thresholded. Over the years, manydi ff erent thresholding techniques have been developed includingMinimum error thresholding, Moment-preserving thresholding,Otsu’s thresholding, just to mention a few. The most popularthresholding method, Otsu’s algorithm [78], improves the imagesegmentation performance over other threshold-based segmenta-tion methods in the following way. The threshold used in Otsu’s al-1orithm is chosen in such a way to optimize a trade-o ff between themaximization of the inter-class variance (i.e., between pairs of pix-els belonging to the foreground and the background, respectively)and the minimization of the intra-class variance (i.e., between pairsof pixels belonging to the same region). Otsu’s thresholding algo-rithm is good for thresholding an image whose intensity histogramis either bimodal or multimodal (i.e., it provides a satisfactory so-lution in the case of the segmentation of large objects with nearlyuniform intensities, significantly di ff erent from the intensity of thebackground). However, it has not the ability to segment imageswith unimodal distribution (e.g., images containing small objectswith di ff erent intensities), and its output is sensitive to noise. Thus,post-processing operations are usually required to obtain a finalsatisfactory segmentation.The second category of methods is called boundary-basedsegmentation [72]. These methods detect boundaries and discon-tinuities in the image based on the assumption that the intensityvalues of the pixels linking the foreground and the backgroundare distinct. The first / second order derivatives of the image in-tensity are usually used to highlight those pixels (e.g., Sobel andPrewitt edge detectors [72] as first-order methods, and the Laplaceedge detector [83] as a second-order method, respectively). Thedi ff erence between first and second order methods is that the lattercan localize the local displacement and orientation of the bound-ary. By far the most accurate technique of detecting boundariesand discontinuities in an image is the Canny edge detector [25].The Canny edge detector is less sensitive to noise than other edgedetectors, as it convolves the input image with a Gaussian filter.The result is a slightly blurred version of the input image. Thismethod is also very easy to be implemented. However, it is verysensitive to noise, and leads to segmentation results characterizedby a discontinuous detection of the object boundaries.The third category of methods [49] is called region-basedsegmentation. Region-based segmentation techniques divide animage into subsets based on the assumption that all neighboringpixels within one subset have a similar value of some feature, e.g.,image intensity. Region growing [12] is the most popular region-2ased segmentation technique. In region growing, one has to iden-tify at first a set of seeds as initial representatives of the subsets.Then, the features of each pixel are compared to the features ofits neighbors. If a suitable predefined criterion are satisfied, thenthe pixel is classified as belonging the same subset associated withits “most similar” seed. Accordingly, region growing relies on theprior information given by the seeds and the predefined classi-fication criterion. A second popular region-based segmentationmethod is region “splitting and merging”. In such method, the in-put image is first divided into several small regions. Then, on theregions, a series of splitting and merging operations are performedand controlled by a suitable predefined criterion. As region-basedsegmentation is an intensity-based method, the segmentation re-sult in general leads to nonsmooth and badly shaped boundary forthe segmented object.The fourth category of methods [81] is learning-based seg-mentation. There are two general strategies for developing learning-based segmentation algorithms: namely, generative learning anddiscriminative learning. Generative learning [21] utilizes data setof examples to build a probabilistic model, by finding the bestestimates of parameters for some prespecified parametric formof a probability distribution. One problem with these methodsis that the best estimates of the parameters may not provide asatisfatory model, because the parametric model itself may notbe correct. Another problem is that the classification / clusteringframework associated with a parametric probabilistic model maynot provide an accurate description of the data due to limitednumber of parameters in the model even in the case in which itstraining is well performed. Techniques following the generativeapproach include K-means [67], the Expectation-Maximization al-gorithm [94], and Gaussian Mixture Models [36]. Discriminativelearning [88, 15, 48, 113] ignores probability and attempts to con-struct a good decision boundary directly. Such an approach isoften extremely successful, especially when no reasonable para-metric probabilistic model of the data exists. It assumes that thedecision boundary comes from another class of nonparametric so-lutions, and chooses the best element of that class according to a3uitable opimality criterion. Techniques following the discrimina-tive approach include Linear Discriminative Analysis [44], NeuralNetworks [69, 41, 82, 43, 39] and Support Vector Machines [22]. Themain problems with these methods are their sensitivity to noise andthe discontinuity of the resulting object boundaries.The last category of methods [23, 50] are energy-based seg-mentation methods. This class of methods is based on an energyfunctional and deals with the segmentation problem as an op-timization problem, which tries to divide the image into regionsbased on the maximization / minimization of the energy functional.The most well-known energy-based segmentation techniques arecalled “active contours”. The main idea of active contours is tochoose an initial contour inside the image domain to be segmented,then make such a contour evolve by using a series of shrinking andexpanding operations. Some advantages of the active contoursover the aforementioned methods are that topological changes ofthe objects to be segmented can be handled implicitly. More im-portantly, complex shapes can be modeled without the need ofprior knowledge about the image. Finally, rich information canbe inserted into the energy functional (e.g., boundary-based andregion-based information). Current active contour models with / without prior knowl-edge incorporated into the energy functional might be e ffi cient / ef-fective enough to handle complex images. However, they are notalways the most e ffi cient and e ff ective solutions to handle imageswith complex intensity distributions. As a consequence, we be-lieve that a real challenge in active contour models is to improvetheir e ffi ciency and e ff ectiveness in segmenting images character-ized by complex intensity distributions. Motivated by this issue,we mainly focus on developing e ff ective, e ffi cient and / or robustsupervised and unsupervised level set image segmentation frame- Loosely speaking, a functional is defined as a function of a function, i.e., afunction that takes a vector as its input argument and returns a scalar. / local way) with a variety of images char-acterized by many intensity levels, intensity inhomogeneity, and / orpresenting other computer-vision challenges (see Fig. 1.1 for somechallenging images used in this thesis). Both single-channel andmulti-channel images are considered in this work.5 bcd ef Figure 1.1: Some of the challenging single- / multi-channel imagesconsidered in this work: images with (a) foreground / backgroundintensity overlap and inhomogeneous regions, (b) intensity inho-mogeneity, (c) weak and ill-defined edges and shadows, (d) addi-tive noise, (e) many intensity levels; sequences of images with (f)scene changes. 6 .3 Contributions In this thesis, we first present a survey about the state ofthe art of active contour models with a focus on their strengths andweaknesses. Then, we propose a number of novel active contourmodels, which are able to handle images presenting challenges incomputer vision in an e ffi cient, e ff ective, and / or robust way (e.g.,see Fig. 1.2 for examples of images that each model can handle). Wealso compare such approaches with state-of-the-art segmentationmodels, focusing on active contour models ( ACMs ) (e.g., a briefoverview of some of the regional state-of-the-art
ACM s and ourproposed ones is reported in Table 1.1), but considering also othersegmentation methods, such as thresholding ones. In the presentsection, we briefly describe the main contributions of the proposed
ACMs . It is worth noting that some of the terms that are usedhere to describe these contributions will be further illustrated inthe following chapter: 7 b Figure 1.2: Examples of challenging images (b) that can be e ff ec-tively handeled by our proposed models (a).8 a b l e . : A s u mm a r y o f s o m e o f t h e r e g i o n a l A c t i v e C o n t o u r M o d e l s ( A C M ) s p r e s e n t e d i n t h e t h e s i s . A C M R e f . R e g i o n a li n f o r m a ti o n M o d e l ov e r v i e w L o c a l G l o b a l GA C [ ] N o N o M a k e s u s e o f b o u n d a r y i n f o r m a t i o n b u t b e h a v e s b a d l y w i t h w e a k e d g e s . C V [ ] N o Y e s C a nh a n d l e o b j e c t s w i t h b l u rr e d b o u n d a r i e s b y m a k i n g s t r o n g s t a t i s t i c a l a ss u m p t i o n s . S B G F R LS [ ] N o Y e s V e r y e ffi c i e n t c o m p u t a t i o n a ll y b u t s t ill m a k e ss t r o n g s t a t i s t i c a l a ss u m p t i o n s . G S R P F [ ] N o Y e s M o r ee ffi c i e n t a n d r o b u s t c o m p a r e d t o S B G F R LS b u t s t ill m a k e ss t r o n g s t a t i s t i c a l a ss u m p t i o n s . L B F [ ] Y e s N o C a nh a n d l e c o m p l e x d i s t r i b u t i o n s w i t h i nh o m o g e n e i t i e s w h il e s e n s i t i v e t o i n i t i a l c o n t o u r . L I F [ ] Y e s N o B e h a v e s li k e w i s e L B F a n d i s c o m p u t a t i o n a ll y m o r ee ffi c i e n t b u t s t ill s e n s i t i v e t o t h e i n i t i a l c o n t o u r . L RC V [ ] Y e s N o C o m p u t a t i o n a ll y v e r y e ffi c i e n t c o m p a r e d t o L B F a n d L I F b u t s e n s i t i v e t o t h e i n i t i a l c o n t o u r . G MM - A C [ ] N o Y e s E x p l o i t s p r i o r k n o w l e d g e b u t m a k e ss t r o n g s t a t i s t i c a l a ss u m p t i o n s . S I S O M [ ] N o N o L o c a li z e s t h e s a li e n t c o n t o u r s u s i n g a S O M b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . T A S O M [ ] N o N o A d j u s t s a u t o m a t i c a ll y t h e n u m b e r o f S O M n e u r o n s b u t n o t o p o l o g i c a l c h a n g e s c a n b e h a n d l e d . B S O M [ ] N o Y e s E x p l o i t s r e g i o n a li n f o r m a t i o n b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . e B S O M [ ] N o Y e s P r o d u c e ss m oo t h c o n t o u r s b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . F T A - S O M [ ] N o Y e s C o n v e r g e s q u i c k l y b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . C F B L - S O M [ ] N o Y e s E x p l o i t s p r i o r k n o w l e d g e b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . C A M - S O M [ ] N o Y e s C a nh a n d l e o b j e c t s w i t h c o n c a v i t i e s b u tt o p o l o g i c a l c h a n g e s c a nn o t b e h a n d l e d . K D E - A C M [ ] Y e s N o M o d e l s a r b i t r a r y s h a p e s r e l y i n g o nh u g e a m o u n t o f s u p e r v i s e d i n f o r m a t i o n . C S O M - C V [ ] N o Y e s V e r y r o b u s tt o t h e n o i s e r e l y i n g o n s u p e r v i s e d i n f o r m a t i o n . S O A C [ ] Y e s N o C a nh a n d l e c o m p l e x i m a g e s i n a l o c a l w a y b a s e d o n s u p e r v i s e d i n f o r m a t i o n . S O M C V [ ] N o Y e s R e d u c e s t h e i n t e r v e n t i o n o f t h e u s e r a n d h a n d l e s i m a g e s i n g l o b a l w a y . S O M - R A C [ ] Y e s Y e s I s r o b u s tt o n o i s e , s c e n e c h a n g e s , a n d i nh o m o g e n e i t i e s b u t i s c o m p u t a t i o n a ll y e x p e n s i v e . lobally Signed Region Pressure Force ( GSRPF )-based activecontour model . The
GSRPF -based
ACM [11] is designed to seg-ment, using global intensity information, images possibly charac-terized by a non symmetric intensity distribution of the RegionOf Interest (
ROI ). The model has the following strengths: 1) it canaccurately modulate the sign of the “pressure” force inside and out-side the contour which is used to guide the contour evolution; 2) itcan handle images with many intensity levels in the foreground; 3)it is robust to additive noise; and 4) o ff ers high e ffi ciency and rapidconvergence. The proposed GSRPF model is robust to contour ini-tialization and has the ability to stop the curve evolution close evento ill-defined (weak) edges. Our model provides a parameter-freeenvironment which allows a minimal user intervention. Experi-mental results on several synthetic and real images demonstratethe high accuracy of the segmentation results obtained by the pro-posed model in comparison to the segmentations obtained by othermethods adopted from the literature.
Concurrent Self Organizing Map-based Chan-Vese (
CSOM-CV )model . CSOM-CV [8] is a novel regional
ACM , which relies on aConcurrent Self Organizing Map
CSOM to approximate globallythe foreground and background image intensity distributions ina supervised way, and to drive the evolution of the active con-tour accordingly. The model integrates such information into theframework of the Chan-Vese ( C - V ) model, which is the reason forwhich we coined the term CSOM-CV for the proposed model. Themain idea of the
CSOM-CV model is to concurrently integrate theglobal information extracted by a
CSOM from a small percentageof the total number of pixels in the image. The information com-ing from such supervised pixels is incorporated into the level-setframework of the
C-V model to build an e ff ective ACM . The pro-posed model integrates the advantages of
CSOM as a powerfulclassification tool and the
C-V model as an e ff ective tool for theoptimization of a global energy functional. Experimental resultsshow the higher e ff ectiveness of CSOM-CV in segmenting bothsynthetic and real images, when compared with the stand-alone
C-V and
CSOM models. 10 elf Organizing Active Contour (
SOAC ) model . The
SOAC model[9] can be described as a variational level set method driven by theprototypes (weights) of neurons belonging to a Self OrganizingMap (
SOM ), obtained after a training session. Such prototypes areable to keep track of the dissimilarity beween the foreground andbackground intensity distributions. A di ff erence with the CSOM-CV model is that the information in the
SOAC model is local.The
SOAC model can handle images characterized by many in-tensity levels, intensity inhomogeneity, and complex distributions,possibly with a complicated foreground and background overlap.Experimental results demonstrate the higher accuracy of the seg-mentation results obtained by the
SOAC model on several syntheticand real images, when compared with the segmentations obtainedby other well-known active contour models.
SOM-based Chan-Vese (
SOMCV ) model . SOMCV model [10] issimilar to the
CSOM-CV model, with the di ff erence that now thetraining of the model is completely unsupervised. Also in thiscase, the prototypes of the trained neurons encode global intensityinformation. The proposed model can handle images with manyintenisty levels and complex intensity distributions, and is robustto additive noise. Experimental results show the higher accuracyof the segmentation results obtained by the proposed model onseveral synthetic and real images, when compared with the C-V active contour model. A significant di ff erence with the CSOM-CV model is that the intervention of the final user is significantlyreduced in the
SOMCV model, since no supervised information isused.
SOM-based Regional Active Contour (
SOM - RAC ) model . Finally,likewise the
SOMCV model, also the
SOM - RAC model [7] relies onthe global information coming from selected prototypes associatedwith a
SOM , which is trained o ff -line in an unsupervised way tomodel the intensity distribution of an image, and used on-line tosegment an identical or similar image. In order to improve the ro-bustness of the model, global and local information are combined inthe on-line phase. Experimental results show the higher accuracyof the segmentations obtained by the SOM - RAC model on several11ynthetic and real images, when compared with a state-of-the-artlocal
ACM , namely, the
Local Region-based Chan-Vese model . This thesis is organized in a number of chapters. Chapter 2illustrates the main concepts of variational level set-based
ACM s,and reviews the development of the state-of-the-art models from amachine learning perspective. Chapter 3 reviews various kinds of
SOM -based
ACM s, with a focus on their strengths and weaknessesin comparison with level set-based
ACM s. Chapter 4 presents theproposed
GSRPF model as a global unsupervised sign pressureforce
ACM . Chapter 5 and 6 describe our proposed
CSOM - CV and SOAC models as global and local supervised
ACM , respectively.Chapter 7 presents the proposed
SOMCV model as a global un-supervised
ACM . Chapter 8 presents the
SOM - RAC model as alocal-global unsupervised
ACM . Finally, Chapter 9 concludes thethesis, and presents some possible future research directions.12 hapter 2
Variational level set-based
ACM s Active contour, sometimes called “Evolving Front”, is acontour C inside the image domain Ω which evolves and is de-formed through a set of shrink / expansion operations. Such a pro-cess known as “Contour Evolution”, has the purpose of fittingthe contour to the boundary of an object to be segmented froman image I ( x ), and is governed by the minimization of an energyfunctional.Active Contour Models ( ACM s) usually deal with the im-age segmentation problem as a functional optimization problem,as they try to divide an image into several regions by optimizing asuitable functional. Starting from an initial contour, the optimiza-tion is performed in an iterative way, evolving the current contourwith the aim of approximating better and better the actual bound-ary (hence the denomination “active contour” models, which isactually used also for models which are not based on the explicitminimization of a functional [102]).To build an active contour, there are mainly two methods.The first one is an explicit or Lagrangian method, which results inparametric active contours, also called Snakes. The second one isan implicit or Eulerian method, which results in geometric activecontours, known as level set method.13n parametrized
ACM s, the contour C , see Fig. 2.1, is rep-resented as C : = { x ∈ Ω : x = ( x ( s ) , x ( s )) , ≤ s ≤ } , (2.1)where x ( s ) and x ( s ) are functions of the scalar parameter s . Arepresentative parametrized ACM is the Snakes model, proposedby Kass et al. [50] (see also [115] for successive developments). s=0 s=1s=0.2
Figure 2.1: The parametric representation of a contour.The main drawbacks of parametrized
ACM s are the fre-quent occurrence of local minima in the image energy functionalto be optimized (which is mainly due to the presence of a gradientenergy term inside such a functional), and the fact that topologicalchanges of the objects (e.g., merging and splitting) cannot be han-dled during the evolution of the contour. Instead, level set methods- which will be described in the next section - can model arbitrar-ily complex shapes. Moreover, another advantage with respectto parametetric methods is that they can handle also topologicalchanges of the contours.In this chapter, we review some representative variationallevel set-based
ACM s, from a machine learning perspective, witha focus on their advantages and disadvantages in modeling theevolving contour via a level set.
ACM s The di ff erence between parametric active contour and ge-ometric (or variational level set-based) active contour models isthat in geometric active contours, the contour is implemented via14 variational level set method. Such a representation was first pro-posed by Osher and Sethian [77]. In such methods, the contour C , see Fig. 2.2, is implicitly represented by a function φ ( x ), called“level set function”, where x is the pixel location inside the imagedomain Ω . The contour C is then defined as the zero level set ofthe function φ ( x ), i.e., C : = { x ∈ Ω : φ ( x ) = } . (2.2)A common and simple expression for φ ( x ), which is usedby most authors, is φ ( x ) = + ρ, for x ∈ inside(C) , , for x ∈ C , − ρ, for x ∈ outside(C) , (2.3)where ρ is a positive parameter (possibly dependent on x and C , insuch case it is denoted by ρ ( x , C )). (x)<0 (x)>0(x)=0 Figure 2.2: The geometric representation of a contour.In the variational level set method , expressing the contourC in terms of the level set function φ , the energy functional to be15inimized can be expressed as follows: E ( φ ) = E in ( φ ( x )) + E out ( φ ( x )) + E C ( φ ( x )) , (2.4)where E in ( φ ) and E out ( φ ) are integral energy terms inside and out-side the contour, and E C ( φ ) is an integral energy term for the con-tour itself. More precisely, the three terms are defined as: E in ( φ ( x )) = Z φ ( x ) > e ( x ) dx = Z Ω H ( φ ( x )) · e ( x ) dx , (2.5) E out ( φ ( x )) = Z φ ( x ) < e ( x ) dx = Z Ω (1 − H ( φ ( x ))) · e ( x ) dx , (2.6) E C ( φ ( x )) = Z Ω ||∇ H ( φ ( x )) || dx = Z Ω δ ( φ ( x )) · ||∇ φ ( x ) || dx , (2.7)where e ( x ) is a suitable function, and H and δ are, respectively, theHeaviside function and the Dirac delta distribution, i.e., H ( z ) = ( , if z ≥ , , if z < , (2.8)and δ ( z ) = ddz H ( z ) . (2.9)Accordingly, the evolution of the level set function φ pro-vides the evolution of the contour C . In the variational level setframework, the (local) minimization of the energy functional E ( φ )can be obtained by evolving the level set function φ according tothe following Euler-Lagrange partial di ff erential equation : ∂φ∂ t = − ∂ E ( φ ) ∂φ (2.10) In the following, when writing partial di ff erential equations, in general we donot write explicitly the arguments of the involved functions, which are describedeither in the text, or in the references from which such equations are reported. φ is now considered a function of both the pixel location x and time t , and the term ∂ E ( φ ) ∂φ denotes the functional derivativeof E with respect to φ (i.e., loosely speaking, the generalizationof the gradient to an infinite-dimensional setting). So, Eq. (2.10)represents the application to the present optimization problem ofan extension to infinite dimension of the classical gradient methodfor unconstrained optimization. ACM s According to specific kind of partial di ff erential equation( PDE ) (see Eq. 2.10) that models the contour evolution, variationallevel set methods can be divided into two categories:
Global Ac-tive Contour Models ( GACMs ) [24, 33, 32, 118, 73], and
Local ActiveContour Models ( LACMs ) [40, 100, 31, 42].In order to guide e ffi ciently the evolution of the currentcontour, ACM s allow to integrate various kinds of information in-side the energy functional, such as: local information (e.g., featuresbased on spatial dependencies among pixels), global information(e.g., features that are not influenced by such spatial dependen-cies), shape information, prior information, and possibly also a-posteriori information learned from examples. As a consequence,depending on the kind of information used, one can further di-vide
ACM s into several categories: e.g., edge-based
ACM s [128,54, 26, 52, 68, 111], global region-based
ACM s [28, 91, 62, 112, 16],edge / region-based ACM s [29, 93, 119, 38, 107], and local region-based
ACM s [110, 96, 106, 18, 126, 114].In particular, edge-based ACMs make use of an edge-detector(in general, the gradient of the image intensity) to stop the evolu-tion of the active contour on the true boundaries of the objects ofinterest. One of the most popular edge-based active contours is theGeodesic Active Contour (
GAC ) model [26], which is described inthe following.
Geodesic Active Contour (
GAC ) model [26]. The level set formu-17ation of the
GAC model can be described as follows: ∂φ∂ t = g ||∇ φ || div ∇ φ ||∇ φ || ! + α ! + ∇ g · ∇ φ, (2.11)where φ is the level set function, ∇ is the gradient operator, α > g is the Edge Stopping Function ( ESF )defined as follows: g = + ||∇ G σ ∗ I || , (2.12)where G σ is a Gaussian kernel function with width σ , ∗ is theconvolution operator, and I is the original image intensity.Edge-based models, such as the above-mentioned model,make use of an edge-detector, usually the gradient of the imageintensity, to stop the evolution of the initial guess of the contouron the actual boundary. As a result, such kind of models canhandle only images with well-defined edge information. Indeed,when images have ill-defined edges, the evolution of the contourtypically does not converge to the true object boundary.An alternative solution consists in using statistical informa-tion about a region (e.g., intensity, texture, color, etc.) to construct astopping functional that is able to stop the contour evolution on theboundary between two di ff erent regions, as it happens in region-based models [28, 91]. An example of a region-based model isillustrated as follows. Chan-Vese model (
C-V ) [28]. The Chan-Vese ( C - V ) model is a well-known representative state-of-the-art global region-based ACM (atthe time of writing, it has received more than 4000 citations, accord-ing to Scopus). After its initialization, the contour in the C - V modelis evolved iteratively in an unsupervised fashion with the aim ofminimizing a suitable energy functional, constructed in such a waythat its minimum is achieved in correspondence with a close ap-proximation of the actual boundary between two di ff erent regions.The energy functional E CV of the C - V model for a scalar-valued See the survey paper [58] for the recent state of the art region-based
ACM s. E CV ( C ) : = µ · Length( C ) + v · Area(in( C )) + λ + Z in( C ) ( I ( x ) − c + ( C )) dx + λ − Z out( C ) ( I ( x ) − c − ( C )) dx , (2.13)where C is a contour, I ( x ) ∈ R denotes the intensity of the imageindexed by the pixel location x in the image domain Ω , µ ≥ C ) (foreground) and out( C ) (background) represent theregions inside and outside the contour, respectively, and v ≥ c + ( C ) and c − ( C ), i.e., c + ( C ) : = mean( I ( x ) | x ∈ in( C )) , (2.14)and c − ( C ) = mean( I ( x ) | x ∈ out( C )) , (2.15)are the mean intensities of the foreground and the background,respectively, and λ + , λ − ≥ R in( C ) ( I ( x ) − c + ( C )) dx and R out( C ) ( I ( x ) − c − ( C )) dx , respectively, inside and outside the contour.The functional is constructed in such a way that, when the regionsin( C ) and out( C ) are smooth and “match” the true foreground andthe true background, respectively, E CV ( C ) reaches its minimum.Following [125], in the variational level set formulation of(2.13), the contour C is expressed as the zero level set of an auxiliaryfunction φ : Ω → R : C : = { x ∈ Ω : φ ( x ) = } . (2.16)Note that di ff erent functions φ ( x ) can be chosen to express the same19ontour C . For instance, denoting by d ( x , C ) the minimum of theEuclidean distances of the pixel x to the points on the curve C , φ ( x )can be chosen as a signed distance function, defined as follows: φ ( x ) : = d ( x , C ) , x ∈ in( C ) , , x ∈ C , − d ( x , C ) , x ∈ out( C ) , (2.17)This variational level set formulation has the advantage of beingable to deal directly with the case of a foreground and a backgroundthat are not necessarily connected internally.After replacing C with φ and highlighting the dependenceof c + ( C ) and c − ( C ) on φ , in the variational level set formulation ofthe C - V model the (local) minimization of the cost (2.13) is per-formed by applying the gradient-descent technique in an infinite-dimensional setting (see Eq. 2.10 and also the reference [28]), lead-ing to the following PDE , which describes the evolution of thecontour: ∂φ∂ t = δ (cid:16) φ (cid:17) h µ ∇ · (cid:16) ∇ φ/ ||∇ φ || (cid:17) − v − λ + (cid:16) I − c + ( φ ) (cid:17) + λ − (cid:16) I − c − ( φ ) (cid:17) i , (2.18)where δ ( · ) is the Dirac generalized function. The first term in µ of (2.18) keeps the level set function smooth, the second one in ν controls the propagation speed of the evolving contour, while thethird and fourth terms in λ + and λ − can be interpreted, respectively,as internal and external forces that drive the contour toward theactual object boundary. Then, Eq. (2.18) is solved iteratively in[28] by replacing the Dirac delta by a smooth approximation, andusing a finite di ff erence scheme. Sometimes, also a re-initializationstep is performed, in which the current level set function φ isreplaced by its binarization (ie., a level set function of the form(2.17), representing the same current contour).The C - V model can also be derived, in a Maximum Likeli-hood setting, by making the assumption that the foreground andthe background follow Gaussian intensity distributions with the20ame variance [30]. Then, the model approximates globally theforeground and background intensity distributions by the twoscalars c + ( φ ) and c − ( φ ), respectively, which are their mean inten-sities. Similarly, Leventon et al. proposed in [60] to use Gaussianintensity distributions with di ff erent variances inside a parametricdensity estimation method. Also, Tsai et al. in [97] proposed to useinstead uniform intensity distributions to model the two intensitydistributions. However, such models are known to perform poorlyin the case of objects with inhomogeneous intensities [30].Compared to edge-based models, region-based models usu-ally perform better in images with blurred edges, and are less sen-sitive to the contour initialization.Hybrid models that combine the advantages of both edgeand regional information are able to control better the directionof evolution of the contour than the previous mentioned models.The Geodesic-Aided Chan-Vese ( GACV ) model [29] is a popularhybrid model, which includes both region and edge informationin its formulation. An example of a hybrid model is the following.
Selective Binary and Gaussian Filtering Regularized (
SBGFRLS )Model . The
SBGFRLS model [124] combines the advantages ofboth the C - V and GAC models. It utilizes the statistical informa-tion inside and outside the contour to construct a region-basedsigned pressure force (
SPF ) function, which is used in place of theedge stopping function (
ESF ) (i.e., the information related to imageintensity gradients) used in the
GAC model (recall Eq. 2.12). Itslevel set formulation can be described as ∂φ∂ t = sp f ( I ( x )) · α ||∇ φ || , (2.19)where α is a balloon force parameter (controlling the rate of ex-pansion of the level set function) and the function spf is defined as sp f ( I ( x )) = I ( x ) − c + ( C ) + c − ( C )2 max x ∈ Ω (cid:16) || I ( x ) − c + ( C ) + c − ( C )2 || (cid:17) , (2.20)where c + ( C ) and c − ( C ) are defined likewise in the C - V model above.21bserve that compared to the C - V model, in Eq.( 2.18) the Diracfunction term δ ( φ ) has been replaced by ||∇ φ || which according tothe authors, has an e ff ective range on the whole image, rather thanthe small range of the former. Also, the bracket in Eq.( 2.18) isreplaced by the sp f function defined in Eq. 2.20. To regularize thecurve the authors in [124] (following the practice of others, e.g.,[128, 124, 87]), rather than relying on the computationally costly µ ∇ · (cid:16) ∇ φ/ ||∇ φ || (cid:17) term, convolve the level set curve with a Gaussiankernel g σ , i.e., φ ← g σ ∗ φ, (2.21)where the width σ of the Gaussian K σ has a role similar to the oneof µ in Eq.( 2.18) of the C - V model. If the value of σ is small, thenthe level set function is sensitive to the noise and it does not allowthe level set function to flow into the narrow regions of the object.Overall this model is faster, computationally more e ffi cient,and performs better than the conventional C - V model as pointedout [124]. However, it still has similar drawbacks as the C - V model,such as its ine ffi ciency in handling images with several intensitylevels, its sensitivity to the contour initialization, and its inabilityto handle images with intensity inhomogeneity (i.e., the e ff ect ofslow variations in object illumination possibly occurring duringthe image acquisition process).In order to deal with images with intensity inhomogeneity,several authors have introduced in the SPF function terms thatrelate to local and global intensity information [109, 110, 96, 106].However, these models are still sensitive to contour initializationand additive noise. Furthermore, when the contour is close tothe object boundary, the influence of the global intensity force maydistract the contour from the real object boundary, leading to objectleaking [64], i.e., the presence of a final blurred contour.In general, global models cannot segment successfully ob-jects that are constituted by more than one intensity class. Onthe other hand, sometimes this is possible by using local models,which rely on local information as their main component in theassociated variational level set framework. However, such modelsare still sensitive to the contour initialization and may lead to ob-22ect leaking. Some examples of such local region-based
ACM s areillustrated in the following.
Local Binary Fitting (
LBF ) model [61]. The evolution of the contourin the
LBF model is described by the following
PDE : ∂φ∂ t = − δ ǫ (cid:16) φ (cid:17) ( λ e − λ e ) + v δ ǫ ( φ ) div ∇ φ ||∇ φ || ! + µ ∇ φ − div ∇ φ ||∇ φ || !! (2.22)where v and µ are non-negative constants, ǫ >
0, and the functions e and e are defined as follows: e ( x ) = Z Ω g σ x − y (cid:1) || I y (cid:1) − f ( x ) || dy , (2.23) e ( x ) = Z Ω g σ x − y (cid:1) || I y (cid:1) − f ( x ) || dy , (2.24)where f and f are, respectively, internal and external gray-levelfitting functions at point x and g σ ( x ) is the kernel function of width σ . Also, δ ǫ ( φ ), is a regularized Dirac function, defined as follows: δ ǫ ( x ) = π ǫǫ + x , (2.25)Finally, div is the divergence operator, whereas the functions f and f are defined as follows: f ( x ) = g σ ( x ) h H (cid:16) φ ( x ) (cid:17) I ( x ) i g σ ( x ) H (cid:16) φ ( x ) (cid:17) , (2.26) f ( x ) = g σ ( x ) h(cid:16) − H (cid:16) φ ( x ) (cid:17)(cid:17) I ( x ) i g σ ( x ) (cid:16) − H (cid:16) φ ( x ) (cid:17)(cid:17) . (2.27)In general, the LBF model can produce good segmentationsof objects with intensity inhomogeneities. Furthermore, it has a23etter performance than the well-known Piecewise Smooth ( PS )model ([105] , [98]) for what concerns segmentation accuracy andcomputational e ffi ciency. However, the LBF model only takes intoaccount the local gray-level information. Thus in this model, it iseasy to be trapped into a local minimum of the energy functional,and the model is also sensitive to the initial location of the activecontour. Finally, over-segmentation problems may occur.
Local Image Fitting (
LIF ) energy model [122]. K. Zhang et al.proposed in [122] the
LIF energy model to extract local image in-formation in their proposed energy functional. The evolution ofthe contour in their model can be described by the following
PDE : ∂φ∂ t = (cid:16) I − I LFI (cid:17) ( m − m ) δ ǫ (cid:16) φ (cid:17) (2.28)where the local fitted image LFI is defined as follows: I LFI = m H ǫ (cid:16) φ (cid:17) + m (cid:16) − H ǫ (cid:16) φ (cid:17)(cid:17) (2.29)where m and m are the average local intensities inside and outsidethe contour, respectively.The main idea of this model is to use the local image in-formation to construct a functional, which takes into account thedi ff erence between the fitted image and the original one to segmentan image with intensity inhomogeneities.The complexity analysis and experimental results showedthat the LIF model is more e ffi cient than the LBF model, whileyielding similar results.However, the obtained models are still sensitive to contourinitialization and high levels of additive noise. A model that hasbeen shown high accuracy when handling images with intensityinhomogeneity compared to the two above-mentioned models isthe following one.
Local Region-based Chan-Vese (
LRCV ) [64]. The LRCV modelis a natural extension of the already-mentioned Chan-Vese ( CV )model. Such an extension is obtained by integrating local intensityinformation into the objective functional. This is the main feature24f the LRCV model, which provides to it the capability of handlingimages with intensity inhomogeneity, which is missing instead inthe C - V model.The objective functional E LRCV of the
LRCV model has theexpression E LRCV ( C ) : = λ + Z in( C ) ( I ( x ) − c + ( x , C )) dx + λ − Z out( C ) ( I ( x ) − c − ( x , C )) dx , (2.30)where c + ( x , C ) and c − ( x , C ) are functions which represent the localweighted mean intensities of the image around the pixel x , assum-ing that it belongs, respectively, to the foreground / background: c + ( x , C ) : = R in( C ) g σ ( x − y ) I y (cid:1) dy R in( C ) g σ ( x − y ) dy , (2.31) c − ( x , C ) : = R out( C ) g σ ( x − y ) I y (cid:1) dy R out( C ) g σ ( x − y ) dy , (2.32)where g σ is a Gaussian kernel function with R R g σ ( x ) dx = σ > C - V model re-sults in the following PDE , which describes the evolution of thecontour: ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:20) − λ + (cid:16) I − c + ( x , φ ) (cid:17) + λ − (cid:16) I − c − ( x , φ ) (cid:17) (cid:21) , (2.33)Eq. (2.33) can be solved iteratively by replacing the Diracdelta by a smooth approximation, and using a finite di ff erencescheme. Also a regularization step can be performed (as in [64]), inwhich the current level set function φ is replaced by its convolutionby a Gaussian kernel with suitable width σ ′ > LRCV model is that it relies only on thelocal information coming from the current location of the contour,25o it is sensitive to the contour initialization.
ACM s From a machine learning perspective,
ACM s for imagesegmentation use both supervised and unsupervised information.Both kinds of
ACM s rely on parametric and / or nonparametric den-sity estimation methods to approximate the intensity distributionsof the subsets to be segmented (e.g., foreground / background). Themain idea of such models is to make statistical assumptions on theimage intensity distribution and to solve the segmentation problemby a Maximum Likelihood ( ML ) or Maximum A-Posteriori prob-ability ( MAP ) approach. For instance, for scalar-valued images,in both parametric / nonparametric region-based ACM s, the objec-tive energy functional has usually an integral form (see, e.g., [59]),whose integrands are expressed in terms of functions e i ( x ) havingthe form e i ( x ) : = − log( p i ( I ( x ))) , ∀ i ∈ I . (2.34)Here, p i ( I ( x )) : = p ( I ( x ) | x ∈ Ω i ) is the conditional probabilitydensity of the image intensity I ( x ), conditioned on x ∈ Ω i , so thelog-likelihood term log ( p i ( I ( x ))) quantifies how much an imagepixel is likely to be an element of the subset Ω i . In the case ofsupervised ACM s, the models p i ( I ( x )) are estimated from a trainingset, one for each subset Ω i . Similarly, for a vector-valued image I ( x ) with D components, the terms e i ( x ) have the form e i ( x ) : = − log( p i ( I ( x ))) , ∀ i ∈ I , (2.35)where p i ( I ( x )) : = p ( I ( x ) | x ∈ Ω i ).Now, we briefly discuss some supervised ACM s, whichtake advantage of the availability of labeled training data. As anexample, Lee et al. proposed in [59] a supervised
ACM , which isformulated in a parametric form. In the following, we refer to sucha model as a Gaussian Mixture Model (
GMM )-based
ACM , since itexploits supervised training examples to estimate the parametersof multivariate Gaussian mixture densities. In such a model, thelevel set evolution
PDE is given, e.g., in the case of multi-spectral26mages I ( x ), by ∂φ∂ t = δ (cid:16) φ (cid:17) h βκ ( φ ) + log p in ( I ) − log p out ( I ) i , (2.36)where β ≥ κ ( φ ) is the averagecurvature of the level set function φ . The two terms p in ( I ( x )) and p out ( I ( x )) in (2.36) are then expressed as p in ( I ( x )) , p out ( I ( x )) : = K X k = α k N ( µ k , Σ k , I ( x )) , (2.37)where K is the number of computational units, N ( µ k , Σ k , · ), k = , . . . , K are Gaussian functions with centers µ k and covariance ma-trices Σ k , and the α k ’s are the coe ffi cients of the linear combination.All the parameters ( α k , µ k , Σ k ) are then estimated from the trainingexamples. Besides GMM -based
ACM s, also Nonparametric KernelDensity Estimation (
KDE )-based models with Gaussian computa-tional units have been proposed in [34, 35] with the same aim. Inthe case of scalar images, they have the form p in ( I ( x )) : = || L + || || L + || X i = K I ( x ) − I ( x + i ) σ ! , (2.38) p out ( I ( x )) : = || L − || || L − || X i = K I ( x ) − I ( x − i ) σ KDE ! , (2.39)where the pixels x + i and x − i belong, respectively, to given sets L + and L − of training pixels inside the true foreground / background, σ KDE > KDE -basedmodel, and K ( u ) : = √ π exp − u ! . (2.40)Of course, such models can be extended to the case of vector-valuedimages (in particular, replacing σ KDE by a covariance matrix).27 .1.3 Other level set-based
ACM s Supervised Boundary-based
GAC ( sBGAC ) [79]. The sBGAC modelis a supervised level-set based ACM , which was proposed by Para-gios et al. in [79] with the aim of providing a boundary-basedframework that is derived by the
GAC for texture image segmen-tation. Its main contribution is the connection between the mini-mization of a
GAC objective function with a contour propagationmethod for supervised texture segmentation. However, sBGAC isstill limited to boundary-based information, which results in a highsensitivity to the noise and to the initial contour.
Geodesic Active Region Model (
GARM ) [80]. GARM was pro-posed in [80] with the aim of reducing the sensitivity of sBGAC to the noise and to the contour initialization, by integrating theregion-based information along with the boundary information.
GARM is a supervised texture segmentation
ACM implementedby variational level set.The inclusion of supervised examples in
ACM s can improvesignificantly their performance by constructing a Knowledge Base( KB ), to be used as a guide in the evolution of the contour. However,state-of-the-art supervised ACM s often make strong statistical as-sumptions on the image intensity distribution of each subset to bemodeled. So, the evolution of the contour is driven by probabilitymodels constructed based on given reference distributions. There-fore, the applicability of such models is limited by how accuratethe probability models are. 28 hapter 3
SOM-based
ACM s Self Organizing Maps (
SOM s) have attracted the attentionof many computer vision scientists, particularly when dealing withimage segmentation as a contour extraction problem. The idea ofutilizing the prototypes (weights) of a
SOM to model an evolv-ing contour has produced a new class of Active Contour Models(
ACM s), known as
SOM -based
ACM s. Such models have beenproposed in general with the aim of exploiting the specific abil-ity of
SOM s to learn the edge-map information via their topologypreservation property, and overcoming some drawbacks of other
ACM s, such as trapping into local minima of the image energyfunctional to be minimized in such models. In this chapter (in asimilar way to the previous chapter), the main principles of
SOM sand their application in modeling active contours are highlighted.Then, we review existing
SOM -based
ACM s with a focus on theiradvantages and disadvantages in modeling the evolving contourvia di ff erent kinds of SOM s. SOM s) The
SOM [55, 56], which was proposed by Kohonen, is anunsupervised neural network whose neurons update concurrentlytheir weights in a self-organizing manner, in such a way that, dur-ing the learning process, its neurons evolve adaptively into specific29etectors of di ff erent input patterns. A basic SOM , see Fig. 3.1, iscomposed of an input layer, an output layer, and an intermediateconnection layer. The input layer contains a unit for each com-ponent of the input vector. The output layer consists of neuronsthat are typically located either on a 1- D or a 2- D grid, and arefully connected with the units in the input layer. The intermediateconnection layer is composed of weights (also called prototypes)connecting the units in the input layer and the neurons in the out-put layer (in practice, one has one weight vector associated witheach output neuron, where the dimension of the weight vector isequal to the dimension of the input). The learning algorithm of the SOM can be summarized by the following steps: input layerconnection layeroutput layer
Figure 3.1: The SOM architecture.1. initialize randomly the weights of the neurons in the out-put layer, and select a suitable learning rate and a suitableneighborhood size around a “winner” neuron;2. for each training input vector, determine the winner neuron30sing a suitable rule;3. update the weights on the selected neighborhood of the win-ner neuron;4. repeat Steps 2-3 above by selecting another training inputvector, until learning is accomplished (i.e., a suitable stoppingcriterion is satisfied).Given a collection of training samples and a number ofclasses,
SOM s can be also used as a concurrent system for patternclassification, hence for image segmentation. In this context, aspecific model is represented by the Concurrent Self OrganizingMaps (
CSOM s) [74]. The classification process of a
CSOM startsby training a series of
SOM s (one for each class) in a parallel way,using for each
SOM a subset of samples coming from its associatedclass. During the training process, the neurons of each
SOM aretopologically arranged in the corresponding map on the basis oftheir prototypes (weights) and of the ones of the neurons within acertain geometric distance from them, and are moved toward thecurrent input using the classical self-organization learning rule ofa
SOM , which is expressed by w n ( t +
1) : = w n ( t ) + η ( t ) h bn ( t )[ x ( t ) − w n ( t )] , (3.1)where t = , , , , . . . is a time index, w n ( t ) is the prototype ofthe neuron n at time t , x ( t ) is the input vector at time t , η ( t ) is alearning rate, and h bn ( t ) is the neighborhood kernel at time t ofthe neuron n around a specific neuron b , called best-matching unit( BMU ). More precisely, in each
SOM and at the time t , an inputvector x ( t ) ∈ R D is presented to feed the network, then the neuronsin the map compete one with the other to be the winner neuron b , which is the chosen as the one whose weight w b ( t ) is the closestto the input vector x ( t ) in terms of a similarity measure, which isusually the Euclidean distance k · k . In this case, k x ( t ) − w b ( t ) k : = min n k x ( t ) − w n ( t ) k , where n varies in the set of neurons of the map.Once the learning of all the SOM s has been accomplished,the class label of a previously-unseen input test pattern is deter-31ined by the criterion of the minimum quantization error. Moreprecisely, the
BMU neuron associated with the input test patternis determined for each
SOM , and the winning
SOM is the one forwhich the prototype of the associated
BMU neuron has the small-est distance from the input test pattern, which is consequently as-signed to the class associated with that
SOM . SOM s have been usedextensively for image segmentation, but often not in combinationwith
ACM s [99, 89, 76, 75, 95, 17, 127, 92, 85, 53]. In the followingsubsection, we review, in brief, some of the existing
SOM -basedsegmentation models.
SOM -based Segmentation Models
In [65], a
SOM -based clustering technique has been usedas a thresholding technique for image segmentation. First, theintensity histogram of the image was used to feed a
SOM in orderto partition the histogram into several regions. The algorithm wasapplied to text recognition, after choosing the number of regionsin a suitable way.Huang et al. in [45] proposed to use a
SOM in two stages,for color image segmentation. The first stage aims to identify alarge initial set of color classes, while the second ones aims toidentify a final batch of segmented clusters.In [47], Y. Jiang et al. used a
SOM to segment a multi-spectral image (composed of five-dimension vectors), by clusteringthe pixels based on their color and spatial features. Then, thoseclustered blocks were merged into a specific number of regions, andsome morphological operations were applied. In general
SOM s,have been extensively used in the field of segmentation and all thedeveloped
SOM -based segmentation models (as stated in [37, 117,14, 90, 51, 92]) yielded improved segmentations in comparison totraditional non
SOM - based techniques.Moreover, also other kinds of neural networks have beenused with the aim of approximating the edge map: e.g., multi-layer perceptrons [71], whose approximation capability has beenextensively investigated (see, e.g., [19, 57]).32e conclude mentioning that, when a
SOM is used as asupervised / unsupervised image segmentation technique, the seg-mented objects so-obtained have usually disconnected boundaries,which are often sensitive to the noise. However, in order to im-prove the robustness of edge-based ACM s to the blur and to ill-defined edge information,
SOM s have been also used in combi-nation with
ACM s, with the explicit aim of modelling the activecontour and controlling its evolution, adopting a learning schemesimilar to Kohonen’s learning algorithm [55], resulting in
SOM -based
ACM s [104, 86] (which belong, in this case, to the class ofedge-based
ACM s). The evolution of the active contour in a
SOM -based
ACM is guided by the feature space constructed by the
SOM when learning the weights associated with the neurons of the map.A review of
SOM -based
ACM s is provided in the following sub-sections.
SOM -based
ACM
The basic idea of existing
SOM -based
ACM s is to modeland implement the active contour using a
SOM neural map, rely-ing in the training phase on the edge map of the image (i.e., the setof points obtained by an edge-detection algorithm) to update theweights of the neurons of the
SOM network, and consequently tocontrol the evolution of the active contour. The points of the edgemap act as inputs to the network, which is trained in an unsuper-vised fashion (in the sense that no supervised samples belonging tothe foreground / background, respectively, are provided). As a re-sult, during training the weights associated with the neurons in theoutput map move toward points belonging to the nearest salientcontour.In the following, we illustrate the general ideas of usinga SOM in modeling the active contour, by describing a classicalexample of a
SOM -based
ACM , which was proposed in [104] byVenkatesh and Rishikesh.
Spatial Isomorphism Self Organizing Map (
SISOM )-based
ACM [104]. The
SISOM -based
ACM is the first
SOM -based
ACM
SOM to modelthe evolving contour. The
SOM is composed of a fixed numberof neurons (and consequently a fixed number of “knots” or con-trol points for the evolving curve) and has a fixed structure. Themodel requires a rough approximation of the true boundary as aninitial contour. Its
SOM network is constructed and trained in anunsupervised fashion, based on the initial contour and the edgemap information. The contour evolution is controlled by the edgeinformation extracted from the image by an edge detector. AsFig. 3.2 illustrates, the main steps of the
SISOM -based
ACM can besummarized as follows:1. construct the edge map of the image to be segmented;2. initialize the contour to enclose the object of interest in theimage;3. obtain the x - and x - coordinates of the edge points to bepresented as inputs to the network;4. construct a SOM with a number of neurons equal to the num-ber of the edge points of the initial contour and two weightsassociated with each neuron; the points on the initial contourare used to initialize the weights of the
SOM ;5. repeat the following steps for a fixed number of iterations:(a) select randomly an edge point and feed its coordinatesto the network;(b) determine the best-matching neuron;(c) update the weights of the neurons in the network by theclassical unsupervised learning scheme of the
SOM [55],which is composed of a competitive phase and a coop-erative one;(d) compute a neighborhood parameter for the contour ac-cording to the updated weights and a threshold.34ig. 3.2 illustrates the evolution procedure of the
SISOM -based
ACM . On the left-side of the figure, the neurons of the mapare represented by gray circles, while the black circle representsthe winner neuron associated with the current input to the map(in this case, the red circle on the right-hand side of the figure,which is connected by the blue segments to all the neurons of themap). On the right-hand side, instead, the positions of the whitecircles represent the initial prototypes of the neurons, whereas thepositions of the black circles represent their final values, at theend of learning. The evolution of the contour is controlled bythe learning algorithm above, which guides the evolution of theprotoypes of the neurons of the
SOM (hence, of the active contour)using the points of the edge map as inputs to the
SOM learningalgorithm. As a result, the final contour is represented by a seriesof prototypes of neurons located near the actual boundary of theobject to be segmented.We conclude by mentoning that, in order to produce goodsegmentations, the
SISOM -based
ACM requires the initial contour(which is used to initialize the prototypes of the neurons) to bevery close to the true boundary of the object to be extracted, andthe points of the initial contour have to be assigned to the neuronsof the
SOM in a suitable order: if such assumptions are satisfied,the contour extraction process performed by the model is robust tothe noise. Moreover, di ff erently from other ACM s, the model doesnot require a particular energy functional to be optimized.
SOM -based
ACM s In this subsection, we describe other
SOM -based
ACM s,and highlight their advantages and disadvantages.
Time Adaptive Self Organizing Map (
TASOM )-based
ACM [86].The
TASOM -based
ACM was proposed by Shah-Hosseini andSafabakhsh as a development of the
SISOM -based
ACM , withthe aim of inserting neurons incrementally into the
SOM map ordeleting them incrementally, thus determining automatically therequired number of control points of the extracted contour. More-35igure 3.2: The architecture of the
SISOM -based
ACM proposedin [104].over, each neuron is provided with its specific dynamic learningrate and neighbourhood function. As a consequence, the
TASOM -based
ACM can overcome one of the main limitations of the
SISOM -based
ACM , i.e., its sensitivity to the contour initialization, in thesense that the initial guess of the contour in the
TASOM -based
ACM can be far from the actual object boundary. Likewise the
SISOM -based
ACM , topological changes of the objects (e.g., splitting andmerging) cannot be handled, since both models rely completelyon the edge information (instead than on regional information) todrive the contour evolution.
Batch Self Organizing Map (
BSOM )-based
ACM [102, 103]. Thismodel is a modification of the
TASOM -based
ACM , and was pro-posed by Venkatesh et al. with the aim of dealing better withthe leaking problem (i.e., the presence of a final blurred contour),which often occurs when handling images with ill-defined edges.Such a problem is due to the explicit usage by the
TASOM -based
ACM of only edge information to model and control the evolu-tion of the contour. In the
BSOM -based
ACM , instead, the image36ntensity variation inside a local region is used along with theedge information to control the movement of the contour. In thisway, the robustness of the model is increased in handling imageswith blurred edges. At the same time, the
BSOM -based
ACM isless sensitive to the initial guess of the contour, when comparedto parametrized
ACM s like Snakes, and to the
SOM -based
ACM sdescribed above. However, likewise all such models, the
BSOM -based
ACM has not the ability to handle topological changes of theobjects to be segmented. An extension of the
BSOM -based
ACM was proposed in [101, 20] and applied therein to the segmentationof pupil images. Such a modified version of the basic
BSOM -based
ACM increases the smoothness of the extracted contour, andprevents the extracted contour from being extended over the trueboundaries of the object.
Fast Time Adaptive Self Organizing Map (
FTA - SOM )-based
ACM [46]. This is another modification of the
TASOM -based
ACM , andwas proposed by Izadi and Safabakhsh with the aim of decreasingits computational complexity. The
FTA - SOM -based
ACM is basedon the observation that choosing the learning rate parameters of theprototypes of the neurons of the
SOM in such a way that they areequal to a large fixed value when they are far from the boundary,and to a small value when they are near the boundary, can lead to asignificant increase of the convergence speed of the active contour.Accordingly, in each iteration, the
FTA - SOM -based
ACM finds theminimum distance of each neuron from the boundary, then its setsthe associated learning rate as a fraction of that distance.
Coarse to Fine Boundary Location Self Organizing Map (
CFBL - SOM )-based
ACM [121]. The above
SOM -based
ACM s work inan unsupervised fashion, as the user is required only to providean initial contour to be evolved automatically. In [121], Zeng etal. proposed the
CFBL - SOM -based
ACM as the first supervised
SOM -based
ACM , i.e., a model in which the user is allowed toprovide supervised points (supervised “seeds”) from the desiredboundaries. Starting from this coarse information, the neuronsof the
SOM are then employed to evolve the active contour tothe desired boundaries in a “coarse-to-fine” approach. The
CFBL -37 OM -based ACM follows such a strategy when controlling theevolution of the contour. So, an advantage of the
CFBL - SOM -based
ACM over the
SOM -based
ACM s described above is that itallows to integrate prior knowledge on the desired boundaries ofthe objects to be segmented, which comes from the interaction of theuser with the
SOM -based
ACM s segmentation framework. Whencompared with such
SOM -based
ACM , this property provides the
CFBL - SOM -based
ACM with the ability of handling objects withmore complex shapes, inhomogeneous intensity distributions, andweak boundaries.
Conscience, Archiving and Mean-movement mechanisms SelfOrganizing Map (
CAM - SOM )-based
ACM [84]. The
CAM - SOM -based
ACM was proposed by Sadeghi et al. as an extension of the
BSOM -base
ACM , by introducing three mechanisms called Con-science, Archiving and Mean-Movement. The main achievementof the
CAM - SOM -based
ACM is to allow more complex boundaries(such as concave boundaries) to be captured, and to provide a re-duction of the computational cost. By the Conscience mechanism,the neurons are not allowed to “win” too much frequently, whichmakes the capture of complex boundaries possible. The Archiv-ing mechanism allows a significant reduction in the computationalcost. By such mechanism, neurons whose prototypes are closeto the boundary of the object to be segmented and whose valueshave not changed significantly in the last iterations are archivedand eliminated from subsequent computations. Finally, in orderto ensure a continuous movement of the active contour towardsconcave regions, the Mean-Movement mechanism is used in eachepoch to force the winner neuron to move towards the mean ofa set of feature points, instead of a single feature point. Together,the Conscience and Mean-Movement mechanisms prevent the con-tour from stopping the contour evolution at the entrance of objectconcavities.
Extracting Multiple Objects . The main limitation of various
SOM -based
ACM s is their inability to detect multiple contours and to rec-ognize multiple objects. As mentioned above, a similar problemarises in parametric
ACM s such as Snakes. To deal with the multi-38le contour extraction problem, Venkatesh et al. proposed in [103]to use a splitting criterion. However, if the initial contour is out-side the objects, contours inside an object still cannot be extracted.Sadeghi et al. proposed in [84] a splitting criterion (to be checkedat each epoch) such that the main contour can be divided into sev-eral sub-contours whenever the criterion is satisfied. The processis repeated until each of the sub-contours encloses one single ob-ject. However, the merging process is still not handled implicitlyby the model, which reduces its scope, especially when handlingimages containing multiple objects in the presence of noise or ill-defined edges. Moreover, Ma et al. proposed in [66] to use a
SOM to classify the edge elements in the image. This model relies firston detecting the boundaries of the objects. Then, for each edgepixel, a feature vector is extracted and normalized. Finally, a
SOM is used as a clustering tool to detect the object boundaries whenthe feature vectors are supplied as inputs to the map. As a result,multiple contours can be recognized. However, the model sharesthe same limitations of other models that use a
SOM as a clusteringtool for image segmentation [99, 116], resulting in disconnectedboundaries and sensitivity to the noise.39 hapter 4
Globally Signed PressureForce Model
One of the most popular and widely used global activecontour models is the region-based
ACM , which often relies on theassumption of homogeneous intensity in the regions of interest.As a result, most often than not, when images violate this assump-tion the performance of this method is limited. Thus, handlingimages that contain foreground objects characterized by multipleintensity classes present a challenge. In this chapter, we present anovel active contour model based on a new Signed Pressure Force(
SPF ) function which we term
Globally Signed Region Pressure Force ( GSRPF ). It is designed to take into account, in a global way, ofthe skewness of the intensity distribution of the region of interest(
ROI ). It can accurately modulate the signs of the pressure forceinside and outside the contour, and handle images with multiple in-tensity classes in the foreground. Moreover, it is robust to additivenoise, and o ff ers high e ffi ciency and rapid convergence. The pro-posed GSRPF model is robust to contour initialization and has theability to stop the curve evolution close even to ill-defined (weak)edges.
GSRPF provides a nearly parameter-free segmentation en-vironment, requiring minimal user intervention. Experimental re-40ults on several synthetic and real images have demonstrated thehigher accuracy of the segmentation results obtained by
GSRPF ,in comparison to the segmentations obtained by other methodsadopted from the literature.The majority of global intensity-based active contour mod-els assume that the regions of interest are composed by subregionsthat are nearly homogeneous in intensity. Consequently, whenthese assumptions are violated, the performance of these modelsis far from the desired one. In this chapter, we propose the
GSRPF as a new intensity-driven region-based
ACM that can e ffi cientlysegment the foreground (i.e., the object(s)) when it is character-ized by a non symmetric distribution. This non symmetry couldarise either from intensity variations or from the fact that the objectcould be composed into two or more intensity classes. To providea computationally e ffi cient solution and reduce the possibility oftrapping into local minima, the GSRPF model is based on an SPF -like formulation.
GSRPF
Model
It is obvious that relying only on the global mean (insideand outside the contour) as in the C - V model is not su ffi cient tomodel intensity distributions when the images to be segmentedhave foregrounds characterized by more complex intensity dis-tributions. To overcome this problem, we introduce the globalmedian in addition to a global mean inside the energy term to beminimized. Given a contour C , x the pixel location in the image I ( x ), the energy term is defined as41 GSRPF C , c + , m + , c − (cid:1) : = Z in ( C ) λ + e + ( x ) dx + Z out ( C ) λ − e − ( x ) dx , (4.1) e + ( x ) : = (cid:12)(cid:12)(cid:12) I ( x ) − c + (cid:12)(cid:12)(cid:12) + (cid:12)(cid:12)(cid:12) I ( x ) − m + (cid:12)(cid:12)(cid:12) , (4.2) e − ( x ) : = (cid:12)(cid:12)(cid:12) I ( x ) − c − (cid:12)(cid:12)(cid:12) , (4.3)where the positive constants λ + and λ − define the weight of eachterm (inside and outside the contour), c + and m + are scalars approx-imating the mean and median intensity respectively, for the image I inside the contour, and c − is a scalar approximating the meanoutside the contour. Following the standard variational level setformulations [28], we replace the contour curve C with the levelset function φ [125], thus obtaining E GSRPF (cid:16) φ, c + , m + , c − (cid:17) = Z φ> λ + e + ( x ) dx + Z φ< λ − e − ( x ) dx . (4.4)In a similar way to other intensity-driven active contourmodels, the statistical descriptors c + , m + , and c − are defined c + ( φ ) = mean( I ( x ) | φ ( x ) ≥ , m + ( φ ) = median( I ( x ) | φ ( x ) ≥ , c − ( φ ) = mean( I ( x ) | φ ( x ) < . (4.5)Using the level set function φ to represent the contour C inthe domain Ω , the energy E GSRPF can be written as a functional asfollows: 42
GSRPF (cid:16) φ, c + , m + , c − (cid:17) = Z Ω λ + e + ( x ) H ( φ ( x )) dx + Z Ω λ − e − ( x )(1 − H ( φ ( x ))) dx , (4.6)where H is the Heaviside function.By keeping c + , m + , and c − fixed, we minimize the energyfunctional E GSRPF (cid:16) φ, c + , m + , c − (cid:17) with respect to φ to obtain the gra-dient descent flow as ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + ( x ) + λ − e − ( x ) (cid:3) , (4.7)where δ is the generalized Dirac delta function.By considering the higher order statistics m + , the proposedmodel can overcome the limitation of the C - V model about thesymmetry of the intensity distribution, which is not accurate inmost of the real-life images. In the binary gray level images, ourmodel as an energy minimization model behaves exactly the sameas the C - V model, where m + = c + . However, in order to improvethe robustness to the contour initialization when handling gray-level images, in the next subsection we include in the model an SPF function.
GSRPF sign pressure function formulation
Although we could rely on Eq. 4.7 to update our level set,obtaining an “SPF” like formulation would reduce the possibilityof trapping into local minima by well modulating the interior andexterior forces.In this section, we propose one such formulation, whichwe term Globally Signed Region Pressure Force (
GSRPF ) model.It is proposed in such a way that it can modulate the signs of thepressure force inside and outside the object of interest using thestatistical quantities defined in Eq. 4.5 to be combined with theminimization of an energy functional similar to the one in Eq. 4.6.43irst, we assume λ + = λ − =
1, then we define the
SPF function as follows: sp f ( I ( x )) = sp f · sp f ( I ( x )) , (4.8)where sp f : = sign (2 c + + m + − c − ) , sp f ( I ( x )) : = sign ( I ( x ) − c + + m + − c − c + + m + − c − ) , (4.9)where c + , m + , and c − are defined in Eq. 4.5.Rather than a constant force, we use a force that is a quadraticfunction of I ( x ) to control the propagation of the evolving curve ,i.e., we define α ( I ( x )) : = I ( x ) − c + + m + − c − c + + m + − c − ! . (4.10)The motivation behind the proposed propagation function α ( I ( x )) is to dynamically increase the interior and exterior forcesacting on the curve when it is far from the boundaries (thus reduc-ing such possibility of entrapment in local minimal) and decreasethe forces when the curve is close to the boundaries (thus allowingthe curve to stop very close to the actual boundaries).The (per-pixel) multiplication of the proposed α ( I ( x )) and sp f ( I ( x )) results in a new region-based signed pressure force func-tion, which we term Globally Signed Region Pressure Force ( GSRPF )function: gsrp f ( I ( x )) : = α ( I ( x )) · sp f ( I ( x )) . (4.11)The proposed GSRPF function has the capacity to modulatethe sign of the pressure forces and implicitly control the propaga-tion of the evolving curve so that the contour shrinks when it isoutside the object of interest and expands when it is inside theobject. Following the sp f formulation described in Chapter 2, thefinal level set formulation of our model is described by the follow-44ng PDE: ∂φ∂ t = gsrp f ( I ( x )) · (cid:12)(cid:12)(cid:12) ∇ φ (cid:12)(cid:12)(cid:12) . (4.12)In order to achieve computational e ffi ciency, we use a Gaussiankernel to regularize the level set function φ to keep the interfaceregular. The parameter σ of the Gaussian kernel is the only tunableparameter of the model.As we will demonstrate in the Section 4.4 the proposedmodel: • is capable of identifying objects of complex intensity distribu-tion (by taking into account the skewness of the distributionin the model); • is robust to additive noise (e.g., a higher order statistics isconsidered in our model to accommodate non symmetricand noisy distributions); • is not sensitive to the contour initialization (since only globalinformation is considered for the curve evolution); • is computationally e ffi cient (since it does not require a re-initialization of the level set function, and regularizes thecontour e ffi ciently); and • requires a few iterations to converge. To illustrate the ease of implementation of our model, themain steps of the algorithm can be summarized as follows.1. Initialize the level set function φ to be binary i.e., set φ ( x , t =
0) : = − ρ x ∈ Ω \ Ω ′ , x ∈ Ω ′ ,ρ x ∈ Ω \ Ω , (4.13)where ρ > Ω is a subset in the image domain Ω and Ω ′ is the boundary of Ω .45. Calculate the GSRPF function according to Eq. 4.11.3. Evolve the level set according to Eq. 4.12.4. Regularize the level set using a Gaussian kernel function.5. If the curve evolution has converged, stop and return theresult. Otherwise return to Step 2. In this section we demonstrate the superiority of the pro-posed method, compared to implementations of some of the meth-ods reviewed in Chapter 2, when applied to challenging syntheticand real images. We implemented the proposed algorithm in Mat-lab R2009b on a PC (2.5-GHz Intel(R) Core(TM) 2 Duo, 2.00 GBRAM). For a fair comparison, we used reference Matlab imple-mentations of the C - V and SBGFRLS .To demonstrate the e ff ectiveness of our approach in han-dling images where the background has multiple intensity classes,we created a synthetic image for this purpose (shown in Fig. 4.1),without additive noise and with noise. We compare the perfor-mance of the proposed model with the C - V and SBGFRLS models,and vary the parameter σ . As Fig. 4.1(a) illustrates, by increasingthe value of σ , the proposed GSRPF is not sensitive to the noise andfinds all the regions of the object for a large rang of σ . On the otherhand, the SBGFRLS model (see Fig. 4.1(d)) is not able to evolve thecontour properly through the noisy regions, even when alteringthe values of α and σ . Similarly, as Fig. 4.1(e) shows, the C − V model is unable to segment the image, even when considering µ values. To demonstrate the accuracy of the proposed method quan-titatively, we adopt the precision and recall metrics, and comparethe segmentation results with the ground truth. Fig. 4.2 showsthe e ff ect of σ on the accuracy of the segmentation result usingthe synthetic image with noise shown in Fig. 4.1(a) as the groundtruth. Based on the results of this experiment, the value σ = . ff erentlevels of noise are added to the synthetic image of Fig. 4.1. Thehigh precision of the obtained segmentations at most noise levelsconfirms the ability of the proposed GSRPF model to find all theregions of the object, irrespective of noise strength.Table 4.1: The robustness of the
GSRPF model ( σ = .
4) to the noiselevel: Precision and Recall metrics for di ff erent Gaussian noiselevels, measured by the standard deviation ( SD ). SD
10 20 30 40 50Precision(%) 100 100 100 99 89Recall (%) 10 99 89 80 71Fig. 4.3(b) illustrates the ability of the
GSRPF model tofind accurately the boundaries of objects with various convexi-ties, shapes, and noisy background. Even though
SBGFRLS canalso identify the objects, it is unable to segment the hole inside oneof the objects, as shown in Fig. 4.3(c). The C - V model is unableto segment the same image (as shown in Fig. 4.3(d)) because it istrapped into a local minimum.To demonstrate the speed and adaptability of the proposedmethod, in Fig. 4.4 we show the curve evolution for a few iterations.It is readily evident that our model converges fast to an accuratedelineation of the foreground object.Fig. 4.5 shows the robustness of the proposed GSRPF modeland also the sensitivity of the
SBGFRLS and C - V models to di ff erentcontour initializations. The interior and exterior forces are able toguide e ffi ciently the evolution of the contour, nonwithstanding thelocation of the initial contour. Indeed, the initial position of thecontour does not a ff ect the final segmentation, as Fig. 4.5(b), (f), (j),and (n) show, and the presence of the shadow of the plane doesnot lead to over-segmentation. On the other hand, the SBGFRLS model is unable to accurately segment the object when the contour47s initialized outside the object, as shown in Fig. 4.5(g), (k), and (o).On the other hand, the C - V model is more robust to the initializationcompared to SBGFRLS , with the exception of Fig. 4.5(p).Fig. 4.6 demonstrates the ability of our method in handlingimages arising in natural and life sciences. In Fig. 4.6(a), all themodels accurately delineate the boundaries of a brain malignancy.Fig. 4.6(b) shows the ability of our model to extract accurately anArabidopsis rosette from a complicated background (e.g. soil, pot,tray); however, the other two models are not able to extract all theplant parts, as seen in Fig. 4.6(c) and 4.6(d). Similarly, Fig. 4.6(c)and (d) show the ability of
GSRPF to segment multiple objects inthe scene, such as cells and chromosomes. On the other hand, thesegmentation results of the
SBGFRLS and C - V models are not satis-factory. This is mainly to be attributed to the fact that both modelsimpose certain conditions on the foreground intensity distribution,and as such they cannot minimize the overlap between the objectand background intensity distributions.To demonstrate the computational e ffi ciency of the pro-posed method when compared to other global methods, Table 4.2shows the CPU time in seconds and the number of iterations to con-vergence for all the images considered here. Overall, the proposedmethod is able to segment the images in roughly half the numberof iterations when compared to SBGFRLS , another sp f -like model.Table 4.2: The CPU time and the number of iterations required bythe proposed
GSRPF model, and by the
SBFRLS and C - V models,to segment the foreground in some of the images considered here. Figure GSRPF SBGFRLS C-VCPU Time(s) Iterations CPU Time(s) Iterations CPU Time(s) IterationsFig. 3(a) 0.06 11 0.12 17 - -Fig. 5(a) 0.56 21 0.82 45 4.89 339Fig. 6(a) .03 10 .05 13 1.5 75Fig. 6(b) 4.02 46 7.58 84 82.89 806Fig. 6(c) 1.93 41 2.79 67 16.36 406Fig. 6(d) 0.92 29 - - - - .5 Summary In this chapter, we have proposed a novel energy-basedactive contour model based on a new Globally Signed RegionPressure Force (
GSRPF ) function. The
GSRPF model considersthe global information extracted from an image and accommo-dates also foreground intensity distributions that are not neces-sarily symmetric. It automatically and e ffi ciently modulates thesigns of the pressure forces inside and outside the contour. Com-pared with other methods, the proposed model is less sensitiveto noise, contour initialization, and can handle images with com-plex intensity distributions in the foreground and / or background.Our model is a Gaussian regularizing level set model that reliesonly on a single parameter. It is designed to have a quadratic be-havior, and converges in a few iterations without penalizing thesegmentation accuracy. Results on synthetic and real images froma variety of scenarios demonstrate the superiority of our model insegmentation accuracy when compared with well regarded globallevel set methods. As a global signed pressure force model, GSRPF relies on strong statistical assumptions. As a consequence, a globallevel set-based model, termed Concurrent Self Organizing Map-based Chan-Vese (
CSOM-CV ) model is proposed and presented inthe following chapter, with the aim of attenuating such kinds ofassumptions. 49 bcde
Figure 4.1: A synthetic image with multiple classes in the fore-ground, and the performance of the proposed,
SBGFRLS , and C - V ,models for some choices of their parameters. (a) the original 123x 80 image with three di ff erent intensities 100, 150 and 200, andits histogram; (b) the same image with Gaussian noise added ofstandard deviation (SD) 30, and its histogram. Overlaid is also theinitial contour (in red) used in all the subsequent tests. From leftto right: the segmentation results of our model (c) with di ff erent σ values (1.4, 1.6, 1.8, and 2); (d) of SBGFRLS with di ff erent σ and α values ((2,10), (2,50), (2.5, 10), and (2.5,50), respectively); and (e) ofthe C - V model with di ff erent µ values (1.4, 1.6, 1.8, and 2).50 .8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 Σ % PrecisionRecall
Figure 4.2: The sensitivity of our model to the parameter σ in termsof Recall and Precision, in segmenting the image in Fig. 4.1 withGaussian noise with standard deviation , SD = a b c d Figure 4.3: The segmentation results on a 101 x 99 synthetic imagecontaining di ff erent objects of variable convexity and shape, andnoisy background. From left to right: the original image (with theinitial contour), the segmentation obtained by the proposed model( σ = . SBGFRLS and C - V models.51igure 4.4: The rapid evolution of the proposed model ( σ = .
5) ona 481 x 321 real image (downloaded from [1]). From left to right:initial contour, contour after 6 and 9 iterations, and final contour(15 iterations). 52 b c dhgfei j k lponm
Figure 4.5: Robustness to the contour initialization when segment-ing a 135 x 125 plane image obtained from [2]. Arranged incolumns there are the original image with di ff erent contour initial-izations, and then, from left to right, the segmentation results ofthe proposed GSRPF model ( σ = . SBGFRLS (with σ = α = C - V (with µ = .
2) models, respectively,when using the same initial contour.53 cba
Figure 4.6: Segmentation results when di ff erent real images en-countered in natural and life sciences are used. Arranged in rowsthere are: (a) a 109 x 119 brain MRI image, from [3]; (b) a 436 x 422Arabidopsis optical image with complex background; (c) a 256 x256 cellulose microscopy image, from [4]; and (d) a 256 x 256 chro-mosome microscopy image, from [4]. Arranged in columns thereare the original image (with the initial contour), and then, fromleft to right the results of the proposed GSRPF model, and of the
SBGFRLS and C - V models respectively, when using the same ini-tial contour. (Parameters are as in Fig. 4.5, except in (a) for GSRPF ( σ = hapter 5 Concurrent
SOM -basedChan-Vese Model
Concurrent Self Organizing Maps (
CSOM s) deal with thepattern classification problem in a parallel processing way, aim-ing to minimize a suitable objective function. Similarly, ActiveContour Models (
ACM s) (e.g., the Chan-Vese (
C-V ) model) dealwith the image segmentation problem as an optimization problemby minimizing a suitable energy functional. The e ff ectiveness of ACM s is a real challenge in many computer vision applications. Inthis chapter, we propose a novel regional
ACM , which relies on a
CSOM to approximate the foreground and background image in-tensity distributions in a supervised way, and to drive the evolutionof the active contour accordingly. We term our model ConcurrentSelf Organizing Map-based Chan-Vese (
CSOM-CV ) model [8]. Themain idea of the
CSOM-CV model is to concurrently integrate theglobal information extracted by a
CSOM from a few supervisedpixels into the level-set framework of the
C-V model to build ane ff ective ACM . The proposed model integrates the advantages of
CSOM as a powerful classification tool and
C-V as an e ff ective toolfor the optimization of a global energy functional. Experimental re-sults show the e ff ectiveness of CSOM-CV in segmenting synthetic55nd real images, when compared with the stand-alone
C-V and
CSOM models.Most of the existing global regional
ACM s rely explicitlyon a particular probability model (e.g., Gaussian, Laplacian, etc.),which results in restricting their scope in handling images in aglobal fashion, and a ff ects negatively their performance when pro-cessing noisy images. On the other hand, SOM -based models havethe advantage of being able to predict the underlying image inten-sity distribution relying on their “topology preserving property”,which is typical of
SOM s. However, the application of such modelsin segmentation usually results in disconnected boundaries. More-over, they are often quite sensitive to the noise. Motivated by theissues above, we propose the
CSOM - CV model to combine SOM sand global
ACM s in order to deal with the image segmentationproblem reducing the disadvantages of both approaches, whilepreserving the aforementioned advantages.
CSOM-CV model
In this section, we describe our Concurrent Self Organiz-ing Map based Chan-Vese Model (
CSOM-CV ). Such model is com-posed of an o ff -line session and an on-line one, which are described,respectively, in Subsections 5.2.1 and 5.2.2. The
CSOM-CV model we propose makes use of two
SOM s,one associated with the foreground, the other to the background.We make a distinction between the two
SOM s by using, respec-tively, the superscripts + and − for the associated weights. Weassume that two sets of training samples belonging to the trueforeground Ω + and the true background Ω − of a training image I ( tr ) are available. They are defined as: L + : = { x + , . . . , x + | L + | ∈ Ω + } and L − : = { x − , . . . , x −| L − | ∈ Ω − } , where | L + | and | L − | are their cardinal-ities. In the following, we describe first the learning procedure of56he SOM trained with the set of foreground training pixels L + . Inthe training session, after choosing a suitable topology of the SOM associated with the foreground, the intensity I ( tr ) ( x + t ) of a randomly-extracted pixel x + t ∈ L + of the foreground of the training image isapplied as input to the neural map at time t = , , . . . , t ( tr )max − t ( tr )max is the number of iterations in the training of the neuralmap. Then, the neurons are self-organized in order to preserve- at the end of training - the topological structure of the imageintensity distribution of the foreground. Each neuron n of the SOM is connected to the input by a weight vector w + n of the samedimension as the input (which - in the case of gray-level imagesconsidered in this work - has dimension 1). After their randominitialization, the weights w + n of the neurons are updated by theself-organization learning rule (3.1), which we re-write in the formspecific for the case considered here: w + n ( t +
1) : = w + n ( t ) + η ( t ) h bn ( t )[ I ( tr ) ( x + t ) − w + n ( t )] , (5.1)In this case, the BMU neuron b is the one whose weight vector isthe closest to the input I ( tr ) ( x t ) at time t . Both the learning rate η ( t ) and the neighborhood kernel h bn ( t ) are designed to be time-decreasing in order to stabilize the weights w + n ( t ) for t su ffi cientlylarge. In this way - due to the well-known properties [55] of theself-organization learning rule (5.1) - when the training session iscompleted, one can accurately model and often approximate theinput intensity distribution of the foreground, by associating theintensity of each input to the weight of the corresponding BMU neuron. In particular, in the following we make the choice η ( t ) : = η exp − t τ η ! , (5.2)where η > τ η > h bn ( t ) is selected as a Gaussian function centered on the57 MU neuron, i.e., it has the form h bn ( t ) : = exp − k r b − r n k r ( t ) , (5.3)where r b , r n ∈ R are the location vectors in the output neural mapof neurons b and n , respectively, and r ( t ) > h bn ( t ) guaran-tees that, for fixed t , when k r b − r n k increases, h bn ( t ) decreases tozero gradually to smooth out the e ff ect of the BMU neuron on theweights of the neurons far from the
BMU neuron itself, and when t increases, the influence of the BMU neuron becomes more andmore localized). In particular, in the following we choose r ( t ) : = r exp (cid:18) − t τ r (cid:19) , (5.4)where r > τ r > SOM di ff ers only inthe random choice of the training pixel (which is now denoted by x − t , and belongs to the set L − ), and in the weights of the network,which are denoted by w − n . Once the training of the two
SOM s has been accomplished,the two trained networks are applied on-line in the testing session,during the evolution of the contour C , to approximate and describeglobally the foreground and background intensity distributions ofa similar test image I ( x ). Indeed, during the contour evolution, thetwo mean intensities mean( I ( x ) | x ∈ in( C )) and mean( I ( x ) | x ∈ out( C ))in the current approximations of the foreground and backgroundare presented as inputs to the two trained networks. We now definethe quantities w + b ( C ) : = argmin n | w n − mean( I ( x ) | x ∈ in( C )) | , (5.5) w − b ( C ) : = argmin n | w n − mean( I ( x ) | x ∈ out( C )) | , (5.6)58here w + b ( C ) is the prototype of the BMU neuron to the meanintensity inside the current contour, while w − b ( C ) is the prototypeof the BMU neuron to the mean intensity outside it. Then, wedefine the functional of the
CSOM − CV model as E CSOM − CV ( C ) : = λ + Z in( C ) e + ( x , C ) dx + λ − Z out( C ) e − ( x , C ) dx , (5.7) e + ( x , C ) : = (cid:16) I ( x ) − w + b ( C ) (cid:17) , (5.8) e − ( x , C ) : = (cid:16) I ( x ) − w − b ( C ) (cid:17) . (5.9)where the parameters λ + , λ − ≥ R in( C ) e + ( x , C ) dx and R out( C ) e − ( x , C ) dx ,inside and outside the contour.Now, as in [28], we replace the contour curve C with thelevel set function φ , obtaining E CSOM − CV (cid:16) φ (cid:17) = λ + Z φ> e + ( x , φ ) dx + λ − Z φ< e − ( x , φ ) dx , (5.10)where we have also made explicit the dependence of e + and e − on φ . In terms of the Heaviside step function H ( · ), the CSOM-CV energy functional can be also written as follows: E CSOM − CV (cid:16) φ (cid:17) = λ + Z Ω e + ( x , φ ) H ( φ ( x )) dx + λ − Z Ω e − ( x , φ )(1 − H ( φ ( x ))) dx . (5.11)Finally, proceeding likewise in [28], by an application of the gradient-descent technique in an infinite-dimensional setting, the evolution59f the contour is described by the PDE ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + + λ − e − (cid:3) , (5.12)which shows how the learned neurons of the two SOM s are usedto determine the internal and external forces acting on the contour.Moreover, in a similar way to [122], we perform - at each iteration ofa finite-di ff erence approximation of (5.12) - the regularization of thecurrent level set function by replacing it with its convolution witha Gaussian filter of suitable width. Finally, the contour evolutionis performed for t ( evol )max iterations (unless convergence is obtainedbefore). Another di ff erence with the C - V model is the absence ofthe regularization terms in µ and ν . This can be justified as follows.As pointed out in [122, 123], the convolution of the current levelset function with a Gaussian filter can be used as an e ffi cient androbust approach to regularize it. In such an approach, the width ofthe Gaussian filter is used to control the regularization strength, asthe parameters µ and ν do in the C - V model. So, in a similar wayto our previous models, we have not included in our formulationthe regularization parameters µ and ν . The procedural steps of the training and testing sessionsfor the
CSOM - CV model are summarized in Algorithm 5.3.60 lgorithm 1 CSOM − CV segmentation framework procedure • Input: – Training and test scalar-valued images, and supervised pixels of the training image belonging,respectively, to the sets L + and L − . – Topology of the two neural maps (with 2-dimensional prototypes), and their respective numbers FN and BN of neurons in the output layer. – Number of iterations t ( tr )max for training the two neural maps. – Maximum number of iterations t ( evol )max for the contour evolution. – η >
0: starting learning rate. – r >
0: starting radius of the maps. – τ η ,τ r >
0: time constants in the learning rate and contour smoothing parameter. – λ + , λ − ≥
0: weights of the energy terms, respectively, inside and outside the contour. – σ : Gaussian smoothing parameter. – ρ >
0: constant in the binary approximation of the level set function. • Output: – Segmentation result.
TRAINING SESSION: Initialize randomly the prototypes of the neurons of the two maps. repeat Choose randomly a pixel x + t ∈ L + and determine the BMU neuron of the
SOM associated with the foregroundto the input intensity I ( tr ) ( x + t ). Update the prototypes w + n of the SOM associated with the foreground using (5.1), (5.2), (5.3), and (5.4). until learning of the prototypes is accomplished (i.e., the number of iterations t ( tr )max is reached). Proceed similarly for the training of the
SOM associated with the background, with x + t ∈ L + and w + n replaced,respectively, by x − t ∈ L − and w − n . TESTING SESSION: Choose a subset Ω (e.g., a rectangle) in the image domain Ω with boundary Ω ′ , and initialize the level set functionas: φ ( x ) : = ρ, x ∈ Ω \ Ω ′ , , x ∈ Ω ′ , − ρ, x ∈ Ω \ ( Ω ∪ Ω ′ ) . (5.13) repeat Calculate the functions w + b and w − b from (5.5) and (5.6). Evolve the level set function φ according to a finite di ff erence approximation of (5.12). At each iteration of the finite-di ff erence scheme, re-initialize the current level set function to be binary byperforming the update φ ← ρ (cid:16) H ( φ ) − H ( − φ ) (cid:17) , (5.14)then regularize by convolution the obtained level set function: φ ← g σ ′ ⊗ φ, (5.15)where g σ ′ is a Gaussian kernel with R R g σ ′ ( x ) dx = σ ′ . until the curve evolution converges (i.e., the curve does not change anymore) or the maximum number ofiterations t ( evol )max is reached. end procedure .4 Experimental study In this section, we demonstrate the e ff ectiveness of the CSOM-CV model, when compared to the stand-alone
CSOM and
C-V models, in handling synthetic and real images. For a fair com-parison, the
CSOM-CV , C-V and the
CSOM models used in thisexperiment are all implemented in Matlab R2012a on a PC with thefollowing configuration: 1.8 GHz Intel(R) Core(TM) i3-3217U, and4.00 GB RAM. In each experiment, the
CSOM-CV parameters arefixed as follows: η = . σ = .
5, and the weight parameters (i.e., λ + , λ − ) are fixed to 1. Also, r = max( M , N ) /
2, where M and N arethe numbers of rows and columns of the neural map, t ( tr )max = t ( evol )max = τ η = t ( tr ) max , τ r = t ( tr )max / ln( r ), ρ =
1. The
SOM s arecomposed of 3 × M = N = C - V model, λ + , λ − are also fixed to 1, µ is chosen such thatthe final contour is smooth enough, and ν = da b c e Figure 5.1: The training images used in this chapter together withthe supervised foreground pixels (red) and the supervised back-ground pixels (blue) used in training sessions of the
CSOM-CV andthe
CSOM models. By its definition, no supervised pixel is usedby the
C-V model.To demonstrate the robustness of
CSOM-CV to the noise,in the experiment described in Fig. 5.2 we have used the noise-freeimages of Fig. 5.1(a) and (b) in the training sessions of
CSOM-CV and
CSOM , then the trained
SOM s have been applied on-line bythe two models to their noisy versions as test images. As shown in62ig. 5.2, for this case
CSOM-CV is more robust and less sensitiveto the noise than
C-V (which does not make use of supervisedtraining examples) and
CSOM , since the regions of the foregroundare detected more accurately by
CSOM-CV .Figure 5.2: The robustness of the
CSOM-CV model to two di ff erentkinds of noise: the first column shows, from top to down, twonoisy versions of the image shown in Fig. 5.1(a), and two noisyversions of the image shown in Fig. 5.1(b), respectively, with theaddition of Gaussian noise with standard deviation SD =
50 (firstand third row) and salt and pepper noise (second and fourth row).The initial contours used by the
CSOM-CV and
C-V models are alsoshown (first and third row); finally, the second, third, and fourthcolumns show, respectively, the corresponding binary segmenta-tion obtained by the
CSOM-CV , CSOM , and
C-V models.Fig. 5.3 illustrates the e ff ectiveness of CSOM-CV in han-dling other images. The segmentation results of the
CSOM-CV model shown in the first row demonstrate its ability to segmentobjects with blurred edges and background, while on the same im-63ges the
CSOM and
C-V models incur, respectively, in over- andunder- segmentation problems. Similarly, as shown, respectively,in the second and third rows,
CSOM-CV outperforms
CSOM and
C-V also in handling images characterized by nonhomogeneousbackground intensity distribution, and in the presence of a shadow.Figure 5.3: The segmentation results obtained on real and syntheticgray-level images. The first row shows the original images withthe initial contours, while the second, third, and fourth rows show,respectively, the corresponding segmentation results obtained bythe
CSOM-CV , CSOM , and
C-V models.To demonstrate the computational e ffi ciency of the CSOM-CV model when compared to the
CSOM and
C-V models, Table 5.1shows, for each of the three methods, the
CPU time (in seconds)required to segment the images shown in Fig. 5.2 and 5.3. For the
CSOM-CV and
C-V models, the number of iterations performedbefore convergence of the active contour is also reported in thetable. As illustrated by Table 5.1, we can observe that the
CSOM-CV model has demonstrated to be much faster than the
CSOM and
C-V models in all the listed cases, thus confirming the e ffi ciency64f the CSOM-CV model. Moreover, as illustrated in Table 5.2,we have also used the Precision, Recall, and F -measure metrics(where the “positive” pixels are the foreground pixels) to evaluatequantitatively the segmentation results of all the models, confimingthe e ff ectiveness of the CSOM-CV model when compared to the
CSOM and
C-V models.Table 5.1: The contour evolution time and number of iterationsrequired by the
CSOM-CV and
C-V models to segment the fore-ground for some of the images shown in this chapter. The
CPU time of
CSOM is also included.
Image in Image size
CSOM-CV model
CSOM model
C-V model
CPU
Time (s)
CPU
Time (s)
CPU
Time (s) ×
101 0.73 20 16.7 3.2 158Fig. 5.2 row 2 114 ×
101 0.62 18 14.7 3.64 219Fig. 5.2 row 3 64 ×
61 0.078 10 4.9 0.04 4Fig. 5.2 row 4 64 ×
61 0.07 10 5 0.98 30Fig. 5.3 row 1 118 ×
93 0.04 10 6.12 2.12 137Fig. 5.3 row 2 300 ×
225 0.62 37 42.03 6.68 205Fig. 5.3 row 3 135 ×
125 0.15 17 10.1 4.18 266
Table 5.2: The Precision, Recall, and F -measure metrics for the CSOM-CV , CSOM , and
C-V models.
Image in
CSOM-CV model
CSOM model
C-V model P (%) R (%) F -m.(%) P (%) R (%) F -m.(%) P (%) R (%) F -m.(%)Fig. 5.2 row 1 99.7 99.8 99.8 93.5 94.7 94 97 88.3 92.5Fig. 5.2 row 2 99.8 99.9 99.8 94.7 97.5 96.1 94.2 87.2 90.5Fig. 5.2 row 3 48.2 93.9 63.7 16.4 72.6 26.8 12.7 96.4 22.5Fig. 5.2 row 4 56.8 97.4 71.1 48.7 96.4 64.7 12.2 100 21.7Fig. 5.3 row 1 100 94.3 97.1 99.6 92.1 95.7 92.8 82.9 87.6Fig. 5.3 row 2 63.5 89.5 74.3 39.2 95.4 55.6 73 60 65.9Fig. 5.3 row 3 95.7 99.8 97.7 46.5 100 63.5 94.9 61.4 74.6 In this chapter, we have proposed a novel
SOM -based
ACM model, the Concurrent Self Organizing Map-based Chan-Vese (
CSOM-CV ) model, which relies mainly on a set of proto-types coming from two trained
SOM s to guide the evolution ofthe active contour. The
CSOM-CV model is a supervised and65lobal region-based
ACM . It has been demonstrated to be e ffi cientand robust to two di ff erent kinds of noise. As compared to the C-V model, our proposed solution consists instead in modelingglobally in a supervised way the intensity distributions of the fore-ground / background (relying on a few supervised pixels) withoutusing parametric models, but relying on a set of prototypes result-ing from the training of a CSOM . So, the main reasons for which,as shown experimentally in Section 5.4, the proposed model a ff ectspositively the C-V model in terms of speed-up in the testing phaseand robustness to the noise are that - di ff erently from the proposedmodel - the C-V model refers to Gaussian intensity distributionsof the foreground / background, and does not include supervisedexamples. Moreover, as compared to CSOM and in general to
SOM -like models used in image segmentation, our solution con-sists in modeling the active contour using a variational level setmethod and relying at the same time on a few prototypes comingfrom the learned
CSOM . In this way, the
CSOM-CV model is able toproduce a final segmentation result characterized by a smooth con-tour while most
SOM -like models usually produce segmentationscharacterized by disconnected boundaries. The enhanced versionof
CSOM-CV , termed
Self Organizing Active Contour ( SOAC ) model,has been proposed with the aim of enlarging its scope and appli-cability. 66 hapter 6
Self Organizing AC Model
Active contour models can significantly improve their per-formance by constructing a knowledge base, and use it to super-vise the movement of the contour during its evolution. However,the state-of-the-art supervised
ACM s make usually statistical as-sumptions on the image intensity distribution of each subset to bemodeled. This results in that the evolution of the contour is drivenby the probability model constructed based on the given referencedistributions. As a consequence, the scope of those models is lim-ited by how accurate the probability model is. The situation iseven worsened by the limited availability of training samples. Inthis chapter, we present a supervised
ACM , termed
Self OrganizingActive Contour ( SOAC ) model [9], which is an extension and im-provement of our previous
CSOM - CV model presented in Chapter5. Compared to the CSOM-CV model, the
SOAC model makesthe following important improvement: its regional descriptors w + b ( x , C ) and w − b ( x , C ) depend on the pixel location x , while thethe CSOM - CV model uses regional descriptors of the form w + b ( C )and w − b ( C ), which are constant functions. So, the CSOM - CV modelis a global ACM (i.e., the spatial dependences of the pixels are nottaken into account in such a model, since it just considers only the67verage intensities inside and outside the contour), whereas the
SOAC model makes also use of local information, which providesit the ability of handling more complex images. For this reason,the
CSOM - CV model is not able to deal successfully with some ofthe images presented in this chapter, although it still showed betterperformance than CSOM and C - V for some images (as detailed inthe previous chapter). However, di ff erently from the SOAC model,it is not able to deal properly with images presenting challengessuch as intensity inhomogeneity and foreground / background in-tensity overlap. SOAC
Model
Our proposed solution to address e ffi ciently the limitationsof current ACM s, mentioned in Chapter 2, and unsupervised
SOM -based
ACM s, discussed in Chapter 6, is to deal implicitly - through
SOMS s - with the decision boundary between the subsets, insteadof relying on a particular probability model for a pixel to be anelement of each subset. In this way, images that contain intensityinhomogeneity, an overlap between the foreground / backgroundintensity distributions, and / or objects characterized by many di ff er-ent intensities, can be handled in an e ffi cient way, as demonstratedby the obtained experimental comparisons of our model with other ACM s, which are reported in Section 6.4 after the presentation ofour model.In this section, we describe in details our proposed
Self Or-ganizing Active Contour ( SOAC ) model. We first consider the case ofscalar-valued images in Subsection 6.2.1. Then, in Subsection 6.2.2,we detail the changes needed to deal with the case of vector-valuedimages. Finally, in Subsection 6.3, algorithmic details are providedfor the two cases.
SOAC model for scalar-valued images
The
SOAC segmentation framework is composed of twosessions: a supervised training session, and a testing session. The68raining process is perfomed in a similar way as the training sessionof the
CSOM - CV model, as described in Subsection 5.2.1 of chapter5). Once the training sessions of both SOM s have been com-pleted, the trained networks are applied online, during the evo-lution of the contour C , to a test image of intensity I ( x ), with theaim of approximating and describing locally, respectively, the fore-ground and background intensity distributions. Indeed, duringthe contour evolution , the two local weighted average intensities R in( C ) g σ ( x − y ) I y (cid:1) dy R in( C ) g σ ( x − y ) dy (6.1)and R out( C ) g σ ( x − y ) I y (cid:1) dy R out( C ) g σ ( x − y ) dy (6.2)are presented as inputs to the two trained networks, respectively,for each pixel x ∈ in( C ) and x ∈ out( C ). Here, g σ is a Gaussian kernelfunction with R R g σ ( x ) dx = σ , which determines thee ff ective size of the neighborhood of x on which the integrals in(6.1) and (6.2) are performed. The prototypes of the BMU neuronsassociated to the inputs (6.1) and (6.2) are, respectively, w + b ( x , C ) : = argmin n ∈{ ,..., FN } (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) w + n − R in( C ) g σ ( x − y ) I y (cid:1) dy R in( C ) g σ ( x − y ) dy (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (6.3) w − b ( x , C ) : = argmin n ∈{ ,..., BN } (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) w − n − R out( C ) g σ ( x − y ) I y (cid:1) dy R out( C ) g σ ( x − y ) dy (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) . (6.4)Such prototypes are extracted as local regional intensity descrip- In practice, this process of the testing session can be done o ff -line becausethere is no need here for the user to initialize a contour to determine the domainsof the interior and exterior forces as they are automatically determined by thepreviously trained SOM s. SOAC model, which has the following expression: E SOAC ( C ) : = λ + Z in( C ) e + ( x , C ) dx + λ − Z out( C ) e − ( x , C ) dx , (6.5) e + ( x , C ) : = (cid:16) I ( x ) − w + b ( x , C ) (cid:17) , (6.6) e − ( x , C ) : = (cid:16) I ( x ) − w − b ( x , C ) (cid:17) , (6.7)where the parameters λ + , λ − ≥ R in( C ) e + ( x , C ) dx and R out( C ) e − ( x , C ) dx , respectively,inside and outside the current contour.The terms e + ( x , C ) and e − ( x , C ) in (6.6) and (6.7) are ableto model more complex intensity distributions than the terms( I ( x ) − c + ( C )) and ( I ( x ) − c − ( C )) used in the energy formulationof the C - V model: in particular, they are able to model skewed andmultimodal intensity distributions.Now, we replace the contour curve C with the level setfunction φ , obtaining E SOAC (cid:16) φ (cid:17) = λ + Z φ> e + ( x , C ) dx + λ − Z φ< e − ( x , C ) dx . (6.8)where we have also made explicit the dependence of e + and e − on φ .In terms of the Heaviside step function H ( · ), the energy functionalof the SOAC model can be also written as follows: E SOAC (cid:16) φ (cid:17) = λ + Z Ω e + ( x , C ) H ( φ ( x )) dx + λ − Z Ω e − ( x , C )(1 − H ( φ ( x ))) dx . (6.9)70inally, the evolution of the contour can be described by the PDE ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + + λ − e − (cid:3) , (6.10)which shows how the learned neurons for each subset are used todetermine the internal and external forces acting on the contour. SOAC model for vector-valued images
The
SOAC model can be extended to the case of vector-valued images. Such an extension is particularly useful for thesegmentation of multi-spectral images (see Section 6.4 for somerelated experiments). In the vectorial case, the image I ( x ) is madeup of D channels I i ( x ) ( i = , . . . , D ), and also the SOM weightsare vectors of dimension D . The only significant change withrespect to the scalar case described in Subsection 6.2.1 is that, in thedetermination of the BMU neurons, the absolute values in formulas(6.3) and (6.4) are replaced by Euclidean norms in R D . The procedural steps of the training and testing sessionsfor the
SOAC model are summarized in Algorithm 2 for the scalarcase (only a slight modification is needed in the vectorial case, asdiscussed in Subsection 6.2.2). 71 lgorithm 2
SOAC segmentation framework for scalar-valued im-ages procedure • Input: – Training and test scalar-valued images, and supervised pixels of the training image belonging,respectively, to the sets L + and L − . – Topology of the two neural maps (with 1-dimensional prototypes), and their respective numbers FN and BN of neurons in the output layer. – Number of iterations t ( tr )max for training the two neural maps. – Maximum number of iterations t ( evol )max for the contour evolution. – η >
0: starting learning rate. – r >
0: starting radius of the maps. – τ η ,τ r >
0: time constants in the learning rate and contour smoothing parameter. – λ + , λ − ≥
0: weights of the energy terms, respectively, inside and outside the contour. – σ,σ ′ >
0: Gaussian intensity and contour smoothing parameters. – ρ >
0: constant in the binary approximation of the level set function. • Output: – Segmentation result.
TRAINING SESSION: Initialize randomly the prototypes of the neurons of the two maps. repeat Choose randomly a pixel x + t ∈ L + and determine the BMU neuron of the
SOM associated with the foregroundto the input intensity I ( tr ) ( x + t ). Update the prototypes w + n of the SOM associated with the foreground using (5.1), (5.2), (5.3), and (5.4). until learning of the prototypes is accomplished (i.e., the number of iterations t ( tr )max is reached). Proceed similarly for the training of the
SOM associated with the background, with x + t ∈ L + and w + n replaced,respectively, by x − t ∈ L − and w − n . TESTING SESSION: Choose a subset Ω (e.g., a rectangle) in the image domain Ω with boundary Ω ′ , and initialize the level set functionas: φ ( x ) : = ρ, x ∈ Ω \ Ω ′ , , x ∈ Ω ′ , − ρ, x ∈ Ω \ ( Ω ∪ Ω ′ ) . (6.11) repeat Calculate the functions w + b and w − b from (6.3) and (6.4). Evolve the level set function φ according to a finite di ff erence approximation of (6.10). At each iteration of the finite-di ff erence scheme, re-initialize the current level set function to be binary byperforming the update φ ← ρ (cid:16) H ( φ ) − H ( − φ ) (cid:17) , (6.12)then regularize by convolution the obtained level set function: φ ← g σ ′ ⊗ φ, (6.13)where g σ ′ is a Gaussian kernel with R R g σ ′ ( x ) dx = σ ′ . until the curve evolution converges (i.e., the curve does not change anymore) or the maximum number ofiterations t ( evol )max is reached. end procedure .4 Experimental study In this section, we demonstrate the e ff ectiveness and ro-bustness of our proposed method, compared to implementationsof some of the state-of-the-art ACM s described in chapter 2, in han-dling synthetic and real images which present well-known chal-lenges in computer vision. More precisely, we study the behaviorof the
SOAC model - when compared to other state-of-the-art im-age segmentation models - in handling scalar-valued and vector-valued images containing objects with non-homogeneous intensity(e.g., in the presence of various di ff erent gray values and simple / -complex object shapes, intensity inhomogeneity in both the fore-ground and background, di ff erent kinds of noise, ill-defined edges,foreground / background intensity overlap, etc.). For a fair compar-ison, all models have been implemented in Matlab R2012a on aPC with the following configuration: 2.5 GHz Intel(R) Core(TM) 2Duo, and 2.00 GB RAM.Moreover, all the ACM reference modelsused in this experiment are Gaussian regularizing level set models.In each experiment, the r , σ and σ ′ parameters are ex-pressed in pixels. In the experiments performed, the parameterswere tuned by trial and error on a validation image (the one inFig. 6.5(row 1)), and their values reported in the chapter were theones that yielded the best results on it. Then, the parameters werefixed to such values for all the other images considered in the chap-ter, with the exception of the images in rows 3 and 4 of Fig. 6.5 andthe images in Fig. 6.8, which required an additional manual tuningof the locality parameter σ , because of the presence of intensityinhomogeneity in such images (see also [61] and [64] as other ex-amples of local ACM s for a which a similar locality parameter wastuned manually for each experiment). When parameters slightlydi ff erent from the ones reported in the chapter were used, only asmall di ff erence in performance was observed. In more details,in the experiments, the SOAC parameters were fixed as follows: η = . σ = . σ = σ ′ = .
5, andthe weight parameters (i.e., λ + , λ − for the scalar-valued case, and λ + i , λ − i in the vector-valued case) were fixed to 1. Also, r = . ( tr )max = t ( evol )max = τ η : = t ( tr )max , τ r : = t ( tr )max / ln( r ), ρ =
1. Thetwo
SOM s had the same 1-D structure, and were composed of 3output neurons. In the C - V model, λ + , λ − for the scalar-valued caseand λ + i , λ − i in the vector-valued case were also fixed to 1, µ waschosen such that the final contour was smooth enough and ν = K of the GMM - based model (see Chapter 2) were chosen tobe 2 (also larger numbers of computational units were considered,but the best results were obtained for K = σ ′ parameterwas fixed for both the GMM -based and
KDE -based models to beequal to the one of the
SOAC model. The other parameters of themodels used in the comparison with the
SOAC model were chosenfollowing the recommendations of the papers in which the modelswere, respectively, proposed: for instance, the parameter σ KDE inthe
KDE -based model was fixed as the average nearest neighbordistance, as recommended in [35]. For the case of gray-level im-ages, the range of the values assumed by the intensity is 0-255,as all the considered gray-level images are 8-bit images. Unlessstated otherwise, the training image used in the training sessioncoincided with the test image. Otherwise, it was an image similarto the test image (obtained, e.g., by adding Gaussian noise). In allthe testing sessions, the initial contour was chosen as rectangular.The images considered in the chapter were chosen for theircomplexity, and because several known
ACM s often showed un-desirable results when tested on them. In Table 6.1, we report thereference from which each image was taken (apart from the firstartificial image), its size, and the associated number of pixels in theforeground and background that have been used in the trainingphase. In more details, the image in Fig. 6.5(row 1) was taken from[109] as an example of a real infrared image with very weak bound-aries, whereas the one in Fig. 6.5(row 2) comes from [120], and isan example of an image with some shadows. The other imagesin Fig. 6.5(rows 3, 4, and 5) were downloaded from the ComputerVision Database [5], and represent real images corrupted by inten-sity inhomogeneity, whereas the one in Fig. 6.8(row 1) was takenfrom [120] as an example of a real
MRI image with intensity inho-mogeneity. The image in Fig. 6.8(row 2) comes from [108], and is74 real CT brain image with intensity inhomogeneity. The one inFig. 6.10(row 1) was taken from [123] as a real image with a verynoisy background. The other images in Fig. 6.10(rows 2 and 3)were downloaded from the Berkeley image segmentation data set[6], and represent real images containing a significant overlap inthe foreground / background intensity distributions.In order to demonstrate the e ff ectiveness of our model inhandling images containing an overlap in the foreground / back-ground intensity distributions and / or containing objects in theforeground characterized by several intensities, we created firsta synthetic image with such characteristics, which is shown inFig. 6.1(a), whereas Fig. 6.1(b) shows its histogram. In the remain-ing of Fig. 6.1, we illustrate the segmentation performance obtainedby the SOAC model when compared to unsupervised
ACM s suchas
LRCV and C - V in handling such an image. Also, the perfor-mance of SOAC in handling the noisy version of the same imagewas tested and compared with some supervised
ACM s trained onthe same data. As a motivation for the use of supervised data,Fig. 6.1(f) illustrates the good capability of
SOAC in handling theimage shown in Fig. 6.1(a), whereas both the
LRCV (Fig. 6.1(h))and the C - V (Fig. 6.1(i)) models, which are unsupervised, failed tosegment the same image. Moreover, due to the foreground / back-ground intensity overlap, both the KDE -based (Fig. 6.1(m)) and the
GMM -based (Fig. 6.1(n)) models failed to separate the three objectsof the foreground when segmenting the same synthetic image withthe addition of Gaussian noise (Fig. 6.1(j)). On the other hand, asshown in Fig. 6.1(l), in such a case
SOAC outperformed both thesupervised
KDE -based and
GMM -based models. In order to have afair comparison with another
SOM -based model, Fig. 6.1(k) showsfor the same image the output mask of the
CSOM classifier, whichillustrates its high sensitivity to noise. This confirms that
SOAC was not biased by the performance of the
CSOM classifier. Addi-tionally,
SOAC was tested on the same synthetic image shown inFig. 6.1(a) with the addition of salt and pepper noise (see Fig. 6.2),and was able to correctly find the desired object also in such asituation.To demonstrate the accuracy of
SOAC quantitatively and75 l m j a b c d e g h if n Figure 6.1: A synthetic image containing objects characterized bymany di ff erent intensities and an overlap in the foreground / back-ground intensity distributions, and a comparison among its seg-mentations obtained by the SOAC model and the
LRCV , C - V , KDE -based,
GMM -based, and
CSOM models: (a) the original 90 × ff erent intensities 100, 150 and 200 in itsforeground, and 120 in its background, and (b) its histogram; (c)the same image with the addition of Gaussian noise with standarddeviation ( SD ) equal to 5, and (d) its histogram; (e) the originalimage in (a) with the addition of a rectangular initial contour (inblack), and training examples (in red for the foreground, in blue forthe background); (g) its ground truth, and its segmentation resultsobtained - starting from the initial contour in (e) - by (f) the SOAC model, (h) the
LRCV model, and (i) the C - V model; (j) the noisyversion of the same image, already shown in (c), with the additionof the initial contour and the training examples; its segmentationby (k) the CSOM model, (l) the
SOAC model, (m) the
KDE -basedmodel, and (n) the
GMM -based model.76 ca Figure 6.2: (a) the same synthetic image considered in Fig. 6.1, withthe supervised training examples; (b) its noisy version, obtained bythe addition of salt and pepper noise; (c) the segmentation resultby
SOAC model.show the robustness of our model to increasing levels of noise, weadopt the Precision ( P ), Recall ( R ), and F -measure ( F -m) metrics.As it can be observed obviously, SOAC and
CSOM share asimilar mechanism in discovering, in the training session, the un-derlying intensity distribution of a given image. However,
SOAC di ff ers from CSOM for the additional presence of its variationallevel set framework. For these reasons, in the following we com-pare the segmentation behaviors of
SOAC and
CSOM . As expected,such results show a smaller noise sensitivity of
SOAC , when ad-ditive noise appears in the test image. More precisely, Fig. 6.4shows the values assumed by the Precision, Recall, and F -measuremetrics for the SOAC and
CSOM models applied to the syntheticimage shown in Fig. 6.3(a), corrupted by several levels of additivenoise. Fig. 6.3(b) shows the corresponding ground truth, whereasFig. 6.3(c) and Fig. 6.3(d) show, respectively, the training examples,and the initial contour used by the
SOAC model. The performanceof
SOAC is compared with the
CSOM classifier, relying on the sametraining data (Fig. 6.3(c)). As Fig. 6.4 illustrates, in the presence ofadditive noise
SOAC achieved higher performance than
CSOM ,confirming that the
SOAC model is less sensitive to additive noisethan
CSOM . 77 c db
Figure 6.3: (a) A synthetic image; (b) its ground truth; (c) trainingexamples (in red for the foreground, in blue for the background);(d) the initial contour used by the
SOAC model (in white).Table 6.1: For each image considered in the chapter: the referencefrom which it was taken (apart from the first artificial one), its size,and the associated number of foreground / background pixels usedin the training phase. Fig. Ref. Image size ×
122 134 1656.5 (row 1) [109] 118 ×
93 160 1726.5 (row 2) [120] 319 ×
127 171 2366.5 (row 3) [5] 300 ×
225 300 5576.5 (row 4) [5] 300 ×
225 957 28816.5 (row 5) [5] 300 ×
203 1349 12846.8 (row 1) [120] 152 ×
128 995 1516.8 (row 2) [108] 174 ×
238 615 7186.10 (row 1) [123] 481 ×
321 713 14646.10 (row 2) [6] 481 ×
321 731 17786.10 (row 3) [6] 481 ×
321 897 2452
20 40 60 80 100 SD020406080100 % CSOM - F - measureProposed Model H SOAC L - F - measureCSOM - RecallProposed Model H SOAC L - RecallCSOM - PrecisionProposed Model H SOAC L - Figure 6.4: The sensitivity of
SOAC to di ff erent levels of noiseadded to the image shown in Fig. 6.3(a) (Gaussian noise with SD = , , , , , , , ,
90, and 100, respectively), in termsof Recall, Precision, and F -measure. For a comparison, also the caseof CSOM is considered. The number of training pixels for both theforeground and the background is also shown.79 a b l e . : T h e P r e c i s i o n , R e c a ll , a n d F - m e a s u r e m e t r i c s f o r t h e S O A C , K D E - b a s e d , G MM - b a s e d , a n d C S O M m o d e l s , a pp li e d t o t h e i m a g e s p r e s e n t e d i n t h e c h a p t e r . F i g . S O A CK D E - b a s e d m o d e l G MM - b a s e d m o d e l C S O M m o d e l ( r o w ) P ( % ) R ( % ) F - m . ( % ) P ( % ) R ( % ) F - m . ( % ) P ( % ) R ( % ) F - m . ( % ) P ( % ) R ( % ) F - m . ( % ) . ( ) . . . . . . . . . . ( ) . . . . . . . . . . . . ( ) . . . . . . . . . ( ) . . . . . . . . . . . ( ) . . . . . . . . . . . ( ) . . . . . . . . . . . . . ( ) . . . . . . . . . . . ( ) . . . . . . . . . . . . ( ) . . . . . . . . . . . . ( ) . . . . . . . . . . . ( ) . . . . . . . . . . ff ectiveness of SOAC in handlingdi ff erent synthetic and real images. Fig. 6.5 shows images withblurred edges, noisy background, intensity inhomogeneity, andshadows. For the case of Fig. 6.5(row 1), among the consideredmodels, only SOAC was able to find the whole object, while boththe
KDE -based model and the
GMM -based model failed in thesegmentation, due to the occurrence of a leaking problem. InFig. 6.5(row 2), the
GMM -based model had a similar performanceto
SOAC , while the
KDE -based model failed to find accurately theobject. Di ff erently from the SOAC model, both the
KDE -basedmodel and the
GMM -based model failed completely to find theobject in Fig. 6.5(row 3), due to the increased amount of intensityinhomogeneity. However, they showed a similar performance to
SOAC when handling the image shown in Fig. 6.5(row 4), whilethey failed again on the image in Fig. 6.5(row 5). For these images,the (unsupervised) C - V model generated good segmentations onlyin Fig. 6.5(row 2) and (row 4), while the CSOM model producedacceptable results only in Fig. 6.5(row 1) and (row 2). In any case,their performance was in general smaller than the one of the
SOAC model (with the exception of Fig. 6.5(d), in which the C - V modelhad slightly better performance). Finally, we observe that, despitethe rectangular contour initialization, some of the segmentationsreported in Fig. 6.5 (and also in the next Fig. 6.10 for the case of RGB images) and obtained by the considered level set-based
ACM s- especially the
KDE -based and
GMM -based models - containedholes. Their appearance depends on various reasons, such as thefollowing: 1) since such models are used inside a variational levelset-based framework, topological changes such as object mergingand splitting - hence, also changes in the number of connectedcomponents - were allowed during the contour evolution; 2) thesmall number of training examples - compared to the complexityof such models - did not provide the
KDE -based and
GMM -basedmodels the ability to drive the contour evolution in a proper way.As a consequence, during the contour evolution some of the fore-ground pixels were considered as background ones; 3) the sensi-tivity of such models to the contour initialization determined theextraction of undesirable objects during the contour evolution.81igure 6.5: The segmentation results obtained on di ff erent real andsynthetic images. Arranged in columns, from left to right: trainingexamples (in red for the foreground, in blue for the background),the initial contour, and the segmentation results obtained, respec-tively by the SOAC , KDE -based,
GMM -based, C - V , and CSOM models.Fig. 6.6 shows the contour evolution over time for the
GMM -based and
KDE -based models, for some images consideredin Fig. 6.5. For three cases in Fig. 6.6 (row 1 for the
GMM -basedmodel, and rows 3 and 4 for the
KDE -based model) a leakingproblem (i.e., in these cases, the appearance of holes in the finalsegmentation) occurred, while the final segmentation result wasacceptable for the
GMM -based model in row 2 (which refers to thesame image in row 4, which was badly segmented, instead, by the
KDE -based model).To illustrate the robustness of the
SOAC model with respectto the selection of the training examples, we trained the
SOAC model with some foreground and background pixels belongingto the first image shown in Fig. 6.7. Then, in the on-line session,we applied the trained model to the remaining images shown inFig. 6.7. As shown in the figure, the segmentations produced by the
SOAC model in such a situation demonstrate its ability in finding82igure 6.6: The contour evolution of the
GMM -based and
KDE -based models on some images in Fig. 6.5. Arranged in columns,from left to right: the initial contour, 4 intermediate contours, andthe final contour. Arranged in rows: the contour evolutions of the
GMM -based model for two images, and the ones of the
KDE -basedmodel for two images.objects in test images di ff erent from the one used in the trainingphase.Figure 6.7: The sensitivity to the training pixels on some real im-ages, taken from [13, 5]. From left to right: training examples (in redfor the foreground, in blue for the background) of the first (train-ing) image, the initial contour and the segmentation produced by SOAC for the second (test) image, and the initial contour and thesegmentation produced by
SOAC for the third (test) image.To confirm the e ff ectiveness of SOAC in handling images83ith intensity inhomogeneity, we tested
SOAC on some real biomed-ical images. Fig. 6.8 illustrates the contour evolution for an in-creasing number of iterations, and confirms the ability of
SOAC in handling images with intensity inhomogeneity, when a limitednumber of supervised examples is provided. On the other hand,the other supervised reference models considered in the compar-ison and trained on the same data failed to correctly segment thesame images, as shown in Fig. 6.9. ab Figure 6.8: Segmentation results obtained by the
SOAC model ontwo real images containing intensity inhomogeneity. Arranged incolumns: the training examples used by the
SOAC model (in redfor the foreground, in blue for the background), respectively, for(a) a 174 ×
238 brain image and (b) a 152 ×
128 heart image; theinitial contours used by the
SOAC model for the two cases; thecurve evolution at three successive stages of
SOAC .To illustrate the e ff ectiveness of the extension of the SOAC model to the case of vector-valued images, we tested its perfor-mance in handling multi-spectral images. For the case consid-ered in Fig. 6.10(first row) (an image with noisy background), theextended
SOAC model and all the vectorial extensions of the su-pervised reference models were successfull in segmenting the im-84igure 6.9: Segmentation results obtained for the real imagesshown in Fig. 6.8(a) and (b) by three supervised reference mod-els, using the same training data as the
SOAC model. Arrangedin columns: the segmentation results obtained, respectively, by the
KDE -based,
GMM -based, and
CSOM models.age, apart from the vectorial extension of the C - V model, whichwas highly sensitive to the presence of noise. Fig. 6.10(secondrow) illustrates a comparison of the segmentation results obtainedby the same models, when the multi-spectral image containedintensity inhomogeneity. In this case, only the vectorial exten-sion of the SOAC model was able to accurately segment the im-age, whereas the
KDE -based model showed unsatisfactory re-sults, due to the small number of learning data, and the C - V model was not able to find the whole object, due to the over-lap between the foreground / background intensity distributions.Fig. 6.10(third row) illustrates a comparison of segmentation re-sults, when the multi-spectral image contained complicated over-laps in the foreground / background intensity distributions. Inter-estingly, the CSOM model failed completely to find the object ofinterest in Fig. 6.10(third row), due to the significant overlap in85hat image of the intensity distributions of the foreground and thebackground. However, the
SOAC model (which, instead, embedsthe concurrent
SOM s in a variational level set framework) was ableto find the same object.Figure 6.10: The segmentation results obtained on real multi-spectral images. Arranged in columns: three 481 ×
321 real imageswith training examples (respectively, in red for the foreground,in blue for the background) and the initial contour (respectively,in black, white, and black); the segmentation results obtained onsuch images, respectively, by the vectorial versions of the
SOAC , KDE -based,
GMM -based, C - V , and CSOM models.Table 6.2 shows the high accuracy of
SOAC quantitativelyin terms of Precision, Recall, and F -metrics, when compared withthe proposed reference models, for the test images considered inthis chapter. To keep the size of the table small, only the resultsof the supervised models are reported in the table. Additionally,to demonstrate the computational e ffi ciency of SOAC when com-pared to the
KDE -based model (which is a non-parametric model),Table 6.3 shows the
CPU time in seconds and the final number ofiterations for the two models, for the images considered in this ex-perimental study. The three
RGB images in Fig. 6.10 do not appearin the table since the convergence of the
KDE -based model was tooslow on them, because of their large size (due to the presence ofthree channels).Fig. 6.11 provides a comparison of the pixel-by-pixel visual86able 6.3: The
CPU time and the number of iterations requiredby the
SOAC model and the
KDE -based model to segment theforeground for some images considered in this chapter.
Fig.
SOAC model
KDE -based model
CPU
Time(s)
CPU
Time(s) representations of the term e SOAC ( x , C ) : = − λ + (cid:16) I ( x ) − w + b ( x , C ) (cid:17) + λ − (cid:16) I ( x ) − w − b ( x , C ) (cid:17) (6.14)inside the level set formula of the SOAC model, and the ones of theterms e KDE ( x ) , e GMM ( x ) : = log p in ( I ( x )) − log p out ( I ( x )) (6.15)inside the level set formula of the KDE -based and
GMM -basedmodels, for two selected images considered in the chapter. Forsimplicity of comparison, we have chosen two cases in which asmall value of the parameter σ was used, in order to have thetwo terms w + b ( x , C ) and w − b ( x , C ) not significantly influenced by thechoice of the contour C , see formulas (6.3) and (6.4). As demon-strated in the figure, the term e SOAC in the
SOAC model was ableto identify accurately the actual pixels of the foreground and back-ground, without being a ff ected by the shadows in Fig. 6.11(firstrow) or by the intensity inhomogeneity and the overlap betweenthe foreground / background intensity distributions in Fig. 6.11(sec-ond row). On the other hand, likely due to the small numberof training examples (compared to the complexity of the models),both terms e KDE and e GMM in the
KDE -based and
GMM -based mod-els failed to identify the pixels correctly.87igure 6.11: A comparison of the pixel-by-pixel visual representa-tions of the term e SOAC in the
SOAC model (formula (6.14)) and theterms e KDE and e GMM (formula (6.15)) in the
GMM -based and
KDE -based models, for two selected images considered in the chapter.Arranged in columns, from left to right: the original images, andthe pixel-by-pixel visual representation of the terms e SOAC , e KDE and e GMM .The above results clearly show that our proposed
Self Or-ganizing Active Contour ( SOAC ) model is an accurate, e ffi cient androbust technique in image segmentation. In this chapter, we have proposed a new supervised
ACM ,which we have termed
Self Organizing Active Contour ( SOAC ) model.It is based on two sets of self organizing neurons to learn the dissim-ilarity between the intensity distributions of the foreground / back-ground. In this way, the information about such distributions isintegrated implicitly into the energy functional of the model bythe learned prototypes of the two SOM s, helping in the guide ofthe contour evolution.
SOAC is a Gaussian regularizing level setmethod, and it is robust to additive noise. The experimental re-sults obtained on several synthetic and real images for both scalar-88alued and vector-valued cases images demonstrate the high e ff ec-tiveness and robustness of our model, when compared with state-of-the-art ACM s, in segmenting images with overlap between theforeground / background intensity distributions, intensity inhomo-geneity, and / or containing objects characterized by many di ff erentintensities. In order to take advantage of SOM as an unsupervisedvisualization tool, in this thesis we describe also an unsupervised
SOM -based
ACM , termed
SOM-based Chan-Vese ( SOMCV ) , whichis presented in the following chapter.89 hapter 7
SOM -based Chan-VeseModel
In this chapter, we present a novel active contour model,which we termed
SOM-based Chan-Vese ( SOMCV ) [10]. It works byexplicitly integrating the information coming from the prototypesof the neurons in a trained
SOM to help choosing whether to shrinkor expand the current contour during the optimization process,which is performed in an iterative way. Similarly to the
CSOM - CV (chapter 5) and SOAC (chapter 6) models,
SOMCV relies on a se-ries of trained self-organizing neurons as a discriminative machinelearning framework to approximate the image distribution. It alsointegrates their prototypes implicitly into the energy framework.However,
SOMCV is presented as a global and unsupervised
SOM -based
ACM , which does not rely on training samples, di ff erentlyfrom CSOM - CV and SOAC . SOMCV and
SOMCV s models In this section, we describe our
SOM -based Chan-Vese(
SOMCV ) active contour model and its modification
SOMCV s . Wefirst consider the case of scalar-valued images in Subsection 7.2.1.90hen, in Subsection 7.2.2, we briefly discuss the changes neededto deal with the case of vector-valued images. Finally, in Subsec-tion 7.3, algorithmic details are provided. SOMCV and
SOMCV s models for scalar-valuedimages Both the
SOMCV and
SOMCV s segmentation frameworksfor scalar-valued images are composed of two sessions: an un-supervised training session and a testing session, which are per-formed, respectively, o ff -line and on-line.In the training session, after choosing a suitable topologyof the SOM , the intensity I ( tr ) ( x t ) of a randomly-extracted pixel x t of a training image is applied as input to the SOM at time t = , , . . . , t ( tr )max −
1, where t ( tr )max is the number of iterations inthe training of the SOM . Then, the neurons are trained in a self-organized way in order to be able to preserve the topological struc-ture of the image intensity distribution at the end of training. Eachneuron n is connected to the input by a weight vector w n of the samedimension as the input (which - in this scalar case - is of dimension1). After their random initialization, the weights w n are updatedby the self-organization learning rule (as stated in formula (5.1)).Since, after training, the inputs to the network are topolog-ically arranged in the output map on the basis of the prototypes ofthe neurons that have the smallest distances from the inputs, wesay that the learned prototypes have a global Self-Organizing Topol-ogy Preservation ( SOTP ) property, which allows one to representthe intensity distributions inside and outside the contour globallyduring the contour evolution.Once the training of the
SOM has been accomplished, thetrained network is applied on-line in the testing session, during theevolution of the contour C , to approximate and describe globallythe foreground and background intensity distributions of a similartest image I ( x ). Indeed, during the contour evolution, the twoscalar intensities mean( I ( x ) | x ∈ in( C )) and mean( I ( x ) | x ∈ out( C )) arepresented as inputs to the trained network. We now define, for91ach neuron n , the quantities A + n ( C ) : = | w n − mean( I ( x ) | x ∈ in( C )) | , (7.1) A − n ( C ) : = | w n − mean( I ( x ) | x ∈ out( C )) | , (7.2)which are, respectively, the distances of the associated prototype w n from the mean intensities of the current approximations of theforeground and the background. Then, we define the two sets { w + j ( C ) } : = { w n : A + n ( C ) ≤ A − n ( C ) } , (7.3) { w − j ( C ) } : = { w n : A + n ( C ) > A − n ( C ) } , (7.4)of cardinalities N + ( C ) : = |{ w + j ( C ) }| and N − ( C ) : = |{ w − j ( C ) }| , whichare the sets of neurons whose prototypes are associated, respec-tively, with the current approximations of the foreground and thebackground. Such prototypes are chosen as representatives of theforeground and background intensity distributions according totheir closeness to the two mean intensities. So, they are extractedas global regional intensity descriptors and included in the energyfunctional to be minimized in our proposed SOMCV model, whichhas the following expression: E SOMCV ( C ) : = λ + Z in( C ) e + ( x , C ) dx + λ − Z out( C ) e − ( x , C ) dx , (7.5) e + ( x , C ) : = X j = ,..., N + ( C ) (cid:16) I ( x ) − w + j ( C ) (cid:17) , (7.6) e − ( x , C ) : = X j = ,..., N − ( C ) (cid:16) I ( x ) − w − j ( C ) (cid:17) , (7.7)where the parameters λ + , λ − ≥ R in( C ) e + ( x , C ) dx and R out( C ) e − ( x , C ) dx ,inside and outside the contour.Now, we replace the contour curve C with the level set92unction φ , obtaining E SOMCV (cid:16) φ (cid:17) = λ + Z φ> e + ( x , φ ) dx + λ − Z φ< e − ( x , φ ) dx , (7.8)where we have also made explicit the dependence of e + and e − on φ . In terms of the Heaviside step function H ( · ), the SOMCV energyfunctional can be also written as follows: E SOMCV (cid:16) φ (cid:17) = λ + Z Ω e + ( x , φ ) H ( φ ( x )) dx + λ − Z Ω e − ( x , φ )(1 − H ( φ ( x ))) dx . (7.9)Finally, proceeding likewise in Chapter 5 and 6, the evo-lution of the contour in the SOMCV model is described by thePDE ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + + λ − e − (cid:3) , (7.10)which shows how the trained neurons are used to determine theinternal and external forces acting on the contour.In the following, we describe also a simplification of the SOMCV model (which we term
SOMCV s model), which is basedon an energy functional whose evaluation is easier from a compu-tational point of view than the one of (7.5). This is obtained byreplacing the sets { w + j ( C ) } and { w − j ( C ) } above by single prototypes w + b and w − b , defined as follows: w + b ( C ) : = argmin n | w n − mean( I ( x ) | x ∈ in( C )) | , (7.11) w − b ( C ) : = argmin n | w n − mean( I ( x ) | x ∈ out( C )) | , (7.12)where w + b ( C ) is the prototype of the BMU neuron to the meanintensity inside the current contour, while w − b ( C ) is the prototypeof the BMU neuron to the mean intensity outside it. Then, we93efine the functional of the
SOMCV s model as E SOMCV s ( C ) : = λ + Z in( C ) e + s ( x , C ) dx + λ − Z out( C ) e − s ( x , C ) dx , (7.13) e + s ( x , C ) : = (cid:16) I ( x ) − w + b ( C ) (cid:17) , (7.14) e − s ( x , C ) : = (cid:16) I ( x ) − w − b ( C ) (cid:17) . (7.15)Then, proceeding as above, after replacing C with the level setfunction φ , the evolution of the contour is described by the PDE ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + s + λ − e − s (cid:3) . (7.16)Although the expressions of e + s ( x , C ) and e − s ( x , C ) are similar to thoseof the terms ( I ( x ) − c + ( C )) and ( I ( x ) − c − ( C )) used in the C - V model,the prototypes w + b ( C ) and w − b ( C ) may represent globally the two re-gional intensity distributions better than the mean intensities inthe two regions. This can be shown in the following way: supposethat the current contour C coincides with the actual object bound-ary, but that the image contains additive noise: then, the valuesof the mean regional intensities c + ( C ) : = mean( I ( x ) | x ∈ in( C )) and c − ( C ) : = mean( I ( x ) | x ∈ out( C )) depend on C in a continuous way,likely making the contour evolve toward a worse approximationof the object boundary. Instead, the values of w + b ( C ) and w − b ( C )may not change at all for small changes of C , providing more ro-bustness of the model with respect to additive noise. In orderto obtain such a behavior, one should keep the size of the net-work small. Otherwise, when using a network with a large num-ber of neurons (then of propotypes), one may more likely obtain w + b ( C ) (cid:27) mean( I ( x ) | x ∈ in( C )) and w − b ( C ) (cid:27) mean( I ( x ) | x ∈ out( C )),losing the just-mentioned robustness.Moreover, when the foreground / background intensity dis-tributions are characterized by many di ff erent intensities, mini-mizing the functional of the C - V model - in which the dependence94n the foreground / background intensity distributions is expressedonly in terms of the mean regional intensities c + ( C ) and c − ( C ) -may result in under(over)-segmentation problems. Of course, suchproblems are still not solved by replacing c + ( C ) and c − ( C ) with theprototypes w + b ( C ) and w − b ( C ), since also w + b ( C ) and w − b ( C ) are onlyscalar quantities. So, in the case of skewness / multimodality of thetwo distributions, one expects better segmentation results whenusing the functional (7.5) of the SOMCV model, which representsthe foreground / background intensity distributions by larger sets ofweights for each of the two regions, as compared to the functional(7.13) of the SOMCV s model.In Section 7.4, the robustness of the proposed model toadditive noise and to intensity distributions characterized by manyintensity values is investigated experimentally. SOMCV and
SOMCV s models for vector-valuedimages The
SOMCV and
SOMCV s models can be extended to thecase of vector-valued images. Such an extension is particularlyuseful for the segmentation of multi-spectral images (see Section7.4 for some related experiments). In the vectorial case, the image I ( x ) is made up of D channels I i ( x )( i = , ..., D ), and also the SOM weights are vectors of dimension D . The only significant changewith respect to the scalar case described in Subsection 7.2.1 is that,in the determination of the BMU neuron, the absolute values informulas (7.1) and (7.2) are replaced by Euclidean norms in R D . Having discussed the formulations of the
SOMCV and
SOMCV s models, in the following, the procedural steps of theirtraining and testing sessions are summarized in Algorithm 4 (toavoid redundancy, only the case of scalar-valued images is detailedhere). 95 lgorithm 3 SOMCV and
SOMCV s segmentation frameworks forscalar-valued images procedure • Input: – Training and test scalar-valued images. – Topology of the network (with 1-dimensional prototypes). – Number of iterations t ( tr )max for training the neural map. – Maximum number of iterations t ( evol )max for the contour evolution. – η >
0: starting learning rate. – r >
0: starting radius of the map. – τ η ,τ r >
0: time constants in the learning rate and contour smoothing parameter. – λ + , λ − ≥
0: weights of the energy terms, respectively, inside and outside the contour. – σ >
0: Gaussian contour smoothing parameter. – ρ >
0: constant in the binary approximation of the level set function. • Output: – Segmentation result.
TRAINING SESSION: Initialize randomly the prototypes of the neurons. repeat Choose randomly a pixel x t in the image domain Ω and determine the BMU neuron to the input intensity I ( tr ) ( x t ). Update the prototypes w n using (5.1), (5.2), (5.3), and (5.4). until learning of the prototypes is accomplished (i.e., the number of iterations t ( tr )max is reached). TESTING SESSION: Choose a subset Ω (e.g., a rectangle) in the image domain Ω with boundary Ω ′ , and initialize the level set functionas: φ ( x ) : = ρ, x ∈ Ω \ Ω ′ , , x ∈ Ω ′ , − ρ, x ∈ Ω \ ( Ω ∪ Ω ′ ) . (7.17) Choose the functional to be minimized (the E SOMCV functional (7.5) or the E SOMCVs functional (7.13)). repeat if E SOMCV functional (7.5) has been chosen then
Determine, for each neuron, the quantities A + n and A − n from (7.1) and (7.2), then the sets { w + j } and { w − j } from (7.3) and (7.4). Evolve the level set function φ according to a finite di ff erence approximation of (8.14). else Calculate w + b and w − b from (7.11) and (7.12). Evolve the level set function φ according to a finite di ff erence approximation of (7.16). end if At each iteration of the finite-di ff erence scheme, re-initialize the current level set function to be binary byperforming the update φ ← ρ (cid:16) H ( φ ) − H ( − φ ) (cid:17) , (7.18)then regularize by convolution the obtained level set function: φ ← g σ ⊗ φ, (7.19)where g σ is a Gaussian kernel with R R g σ ( x ) dx = σ . until the curve evolution converges (i.e., the curve does not change anymore) or the maximum number ofiterations t ( evol )max is reached. end procedure × × SOMCV model - or of the best-matchingneurons to the mean intensities in the two regions, in the case ofthe functional (7.13) of the
SOMCV s model - are used as globalregional descriptors for the foreground and background intensitydistributions. Then, in the testing phase, they are used as core com-ponents of the level set energy functional to guide the evolution ofthe contour. Moreover, in order to keep the contour and the levelset function smooth at each iteration without losing informationon the displacement of the current contour, the current level setfunction φ is first re-initialized to be binary, then convolved with aGaussian kernel function. The smoothness degree of the updatedlevel set function is controlled by the width parameter σ of theGaussian as described in Subsection 7.2.1.Fig. 7.1 illustrates the o ff -line and on-line components of the SOMCV and
SOMCV s models in a vector-valued (more specifically, RGB ) image segmentation framework (the scalar case is similar, butuses scalar prototypes and preferably a 1- D grid). Fig. 7.1(a) showsthe input layer of the SOM , whose dimension is equal to the one ofthe voxel intensities of the image to be segmented. For example, inthe case of
RGB images, the input layer of the map has dimension3, since it receives the R , G , and B channels of the vector-valuedimage. The red cube in Fig. 7.1(a) represents a voxel intensity pre-sented as input to the SOM , in this case made up of 3 × SOMCV s (see Fig. 7.1(e), down),only one prototype is used as a global intensity descriptor for eachregion. In this section, we demonstrate the e ff ectiveness and ro-bustness of the SOMCV and
SOMCV s models, compared to the C - V model described in chapter 2, in handling real and syntheticimages. For a fair comparison, the SOMCV , SOMCV s and the C - V model used in this experiment are all implemented in Mat-lab R2012a on a PC with the following configuration: 2.5 GHzIntel(R) Core(TM) 2 Duo, and 2.00 GB RAM.In each experiment,the r and σ parameters are expressed in pixels. Moreover, the SOMCV and
SOMCV s parameters are fixed as follows: η = . σ = .
5, and the weight parameters (i.e., λ + , λ − for the scalar-valued case, and λ + i , λ − i in the vector-valued case) are fixed to In the experiments presented in Fig. 7.3 and 7.7, also the choice σ = . σ = . raining Session Online Sessiona b c d e Figure 7.1: The architecture of
SOMCV for
RGB images: (a) theinput intensities of a training voxel; (b) a 3 × SOM neural map(with a three-dimensional prototype associated with each neuron);(c) the trained
SOM ; (d) the contour evolution process; and (e)the foreground (in red) and background (in black) representativeneurons for the
SOMCV (top) and the
SOMCV s (down) models. Fora scalar-valued image, a similar model is used, but the prototypeshave dimension 1, and a 1- D grid is used.1. Also, r : = max( M , N ) /
2, where M and N are the numbersof rows and columns of the installed neural map, t ( tr )max = t ( evol )max = τ η : = t ( tr )max , τ r : = t ( tr )max / ln( r ), ρ =
1. For the exper-iments performed on the scalar-valued images considered in thechapter, the
SOM network has been chosen as a 1- D neural mapcomposed of 5 neurons (i.e., M = N = × M = N =
3) and a 2 × M = N =
2) for the otherexperiments (see Tables 7.3 and 7.4). In the C - V model, λ + , λ − forthe scalar-valued case and λ + i , λ − i in the vector-valued case are alsofixed to 1, µ is chosen such that the final contour is smooth enoughand ν = SOMCV s model is considered with the same parameters of the SOMCV model. Unless stated otherwise, the training image usedin the unsupervised training session coincides with the test image.Otherwise, it is an image similar to the test image (obtained, e.g.,99y adding Gaussian noise). In all the testing sessions, the initialcontour has been chosen as rectangular. For the case of gray-levelimages, the range of the values assumed by the intensity is 0-255as all the considered gray-level images are 8-bit images.Fig. 7.2 illustrates the fast convergence of
SOMCV (and itsvariation
SOMCV s ) for scalar-valued images and the associatedcontour evolution process when compared to the C - V model. AsFig. 7.2 shows, the final contours obtained by the SOMCV and
SOMCV s models converge with similar numbers of performed it-erations and similar performances because of the large intensityhomogeneity of the image considered in the experiment.Figure 7.2: The rapid contour evolution of the SOMCV and
SOMCV s models when compared to the contour evolution of the C - V model, in the scalar case. The first and second rows show, re-spectively, the contour evolution of SOMCV and
SOMCV s . Fromleft to right: initial contour (in black), contour after 3, 6, 9, 12 it-erations, and final contour (15 iterations). The third row showsthe contour evolution of the C - V model. From left to right: initialcontour (in black), contour after 50, 100, 150, 200 iterations, andfinal contour (260 iterations).Fig. 7.3 illustrates the e ff ectiveness and robustness of SOMCV in handling images containing objects characterized by many dif-ferent intensities and skewness / multimodality of the foregroundintensity distribution, in the presence of noise. Compared to the100 - V model, the SOMCV model shows better results, due to itsautomated ability to preserve the topological structure of the fore-ground intensity distribution (this is not needed, instead, for thebackground distribution, which is simpler). Moreover, the seg-mentation performance of the
SOMCV s model is quite similar tothe one of the SOMCV model but more sensitive to the noise. ab Figure 7.3: The e ff ectiveness of the SOMCV model in dealing withobjects characterized by many di ff erent intensities and skewness / -multimodality of the foreground intensity distribution. Arrangedin rows there are: (a) a noisy 140 ×
100 image (with Gaussian noiseadded, standard deviation SD =
10) with six di ff erent intensities80, 100, 140, 170, 200, and 230 in its foreground; (b) a noisy 90 × SD = ff erent intensities 100, 150, and 200 in its foreground.The columns from left to right are: the images with the additionsof the initial contours, the histograms of the intensities of the im-ages, and, respectively, the segmentation results of the SOMCV , SOMCV s ( σ = . , .
5, respectively, for (a) and (b)), and C - V models.Then, in order to demonstrate the robustness of SOMCV and
SOMCV s to the additive noise, in the experiment described inFig. 7.4 we have used the top left image of Fig. 7.3 in the trainingsession of SOMCV and
SOMCV s , then the trained SOM (whosevalues of the weights are common to the two models) has beenapplied on-line to various test images obtained adding to such animage di ff erent levels of Gaussian noise. As shown in Fig. 7.4, forthis case SOMCV is more robust and less sensitive to the addi-tive noise than
SOMCV s , since the regions of the foreground are101etected more accurately by SOMCV .Similarly, the image of Fig. 7.3(b) has been used in the train-ing session of
SOMCV and
SOMCV s , then the trained SOM hasbeen applied on-line to various test images obtained by addingto such an image di ff erent levels of Gaussian noise, as shown inFig. 7.5. The results of these two experiments show the abilityof SOMCV to find all the di ff erent regions of the object (which ischaracterized by many di ff erent intensities), and also its robust-ness to the additive noise and to the skewness / multimodality ofthe foreground intensity distribution. They also demonstrate that,in the case of images containing objects characterized by many dif-ferent intensities or by skewed / multimodal intensity distributions, SOMCV usually produces better results than
SOMCV s .Figure 7.4: The robustness of the SOMCV and
SOMCV s modelsto the additive noise: the first row shows, from left to right, theimage of Fig. 7.3(a) with the addition of di ff erent Gaussian noiselevels (standard deviation SD =
10, 15, 20, and 25, respectively);the second and third rows show, respectively, the correspondingsegmentation results of
SOMCV and
SOMCV s .Fig. 7.6 illustrates the e ff ectiveness of SOMCV in handlingreal and synthetic scalar-valued images. The segmentation resultsof the
SOMCV model on the real images shown in the first and sec-ond columns show the ability of
SOMCV to segment objects withblurred edges and background, while the C - V model provides a102igure 7.5: The robustness of the SOMCV and
SOMCV s modelsto the additive noise: the first row shows, from left to right, theimage of Fig. 7.3(b) with the addition of di ff erent Gaussian noiselevels (standard deviation SD =
10, 20, 30, 40, and 50, respectively);the second and third rows show, respectively, the correspondingsegmentation results of
SOMCV and
SOMCV s .worse segmentation for the image in the first column, and incursin an under-segmentation problem for the image in the secondcolumn. Similarly, SOMCV outperforms C - V also in handling syn-thetic images as shown in the third and fourth columns. Moreover, SOMCV and
SOMCV s behave exactly the same as C - V in han-dling binary gray images as in the case of the image shown in theright-most column. This is because in this case the mean intensitiesinside and outside the contour are accurate enough to approximatethe foreground / background intensity distributions. For the imagespresented in Fig. 7.6, SOMCV outperforms also
SOMCV s .To illustrate the e ff ectiveness of SOMCV and its variation
SOMCV s in handling real and synthetic vector-valued images, wehave tested the extension of SOMCV and
SOMCV s to the vectorialframework on RGB real and synthetic images, which is shown inFig. 7.7 in comparison with the vectorial C - V model from [27]. Thesegmentation results of SOMCV are similar to the ones of C - V inhandling the image shown in the fourth column, while SOMCV outperforms C - V in all the other shown images. For these images, SOMCV outperforms also
SOMCV s , which, however, provides bet-ter results than C - V , apart from the cases of the images considered103n the first two columns, for which the results are similar.In the following, we provide also a quantitative study toconfirm the e ff ectiveness of SOMCV and
SOMCV s , when com-pared to C - V . To demonstrate quantitatively the accuracy of theFigure 7.6: The segmentation results obtained on real and syntheticscalar-valued images. The first, second and third row show theoriginal images with the initial contours, the histograms of theimage intensities and their ground truth, respectively, while thefourth, fifth, and sixth rows show, respectively, the correspondingsegmentation results of the SOMCV , SOMCV s and C - V models.104igure 7.7: The segmentation results on real images from [5, 13],and synthetic vector-valued images. The first and second rowsshow the original images with the initial contours, respectively,while the third, fourth, and fifth rows show, respectively, the cor-responding segmentation results of the vectorial versions of the SOMCV , SOMCV s and C - V models. Note that σ = . SOMCV and
SOMCV s for the image in the second col-umn. SOMCV and
SOMCV s models in segmenting the images shown inFig. 7.6 and 7.7, we have also compared the obtained segmentationresults with their corresponding ground-truth data by adoptingthe Precision ( P ), Recall ( R ), and F -measure metrics.Tables 7.1 and 7.2 illustrate the high segmentation accuracyof the SOMCV model and its variation
SOMCV s when comparedto the C - V model, in terms of the three metrics defined above. Asthe two tables illustrate, the SOMCV model has shown a betterperformance than the C - V model in both the scalar and vectorialcases and for all the tested images used, respectively, in Fig. 7.6105nd 7.7. Moreover, the SOMCV s model has usually shown a similarperformance as the SOMCV model.Table 7.1: The Precision, Recall, and F -measure metrics for thescalar SOMCV , SOMCV s and C - V models in the segmentation ofthe scalar images shown in Fig. 7.6. Image in
SOMCV SOMCV s C - VP (%) R (%) F -m (%) P (%) R (%) F -m (%) P (%) R (%) F -m (%)column 1 98.8 99.9 99.3 75.5 100 86 91.8 83.3 87.4column 2 60.6 98.5 75 60.6 98.5 75 42.7 98.5 59.6column 3 100 100 100 100 100 100 99.2 88 93.3column 4 96.3 99.3 97.8 98.8 98.4 98.6 96.5 96.4 96.4column 5 100 100 100 100 100 100 99 100 99.5 Table 7.2: The Precision, Recall, and F -measure metrics for thevectorial SOMCV , SOMCV s and C - V models in the segmentationof the RGB images shown in Fig. 7.7.
Image in
SOMCV SOMCV s C - VP (%) R (%) F -m. (%) P (%) R (%) F -m. (%) P (%) R (%) F -m. (%)column 1 89.6 96.8 93 91.3 91.8 91.5 94.7 83.1 88.5column 2 71.7 97.6 82.7 72.3 97.3 82.9 84.5 81.9 83.2column 3 94.4 90.1 92.2 95 89 91.9 89.5 88.9 89.2column 4 96.1 85.5 90.5 93.5 91.7 92.6 96.1 86.9 91.3column 5 99.6 100 99.8 100 100 100 96.8 89.6 93.1 To demonstrate the computational e ffi ciency of the SOMCV and
SOMCV s models when compared to the C - V model, Table 7.3shows, for each of the three methods, the CPU time (in seconds) thatwas required for the contour evolution (i.e., the time required inthe testing session) and the number of iterations performed beforeconvergence for the real and synthetic images used in Fig. 7.6.Moreover, the computational e ff ectiveness of the vectorial versionsof SOMCV and
SOMCV s with respect to the vectorial C - V model isillustrated in Table 7.4 for the RGB images in Fig. 7.7 by showing, forall methods, the
CPU times and the number of iterations requiredin the testing session (note that, in the common training sessionof
SOMCV and
SOMCV s , the CPU time is fixed by the numberof iterations t ( tr )max ). The sizes of the training and test scalar-valuedand vector-valued images are also listed in the two tables. From106hese tables, we can observe that the SOMCV and
SOMCV s modelswere much faster than the C - V model in all the listed cases, as thecontour evolution for SOMCV and
SOMCV s required less iterationsto converge than for the C - V model, and also the computationaltime per iteration for the SOMCV and
SOMCV s models was smallerthan the one for the C - V model. This is due to the fact that SOMCV and
SOMCV s models are Gaussian Regularizing Level Set Models,whereas the original C - V model has not this feature.Concluding, the results shown in Tables 7.1-7.4 highlightseveral advantages of the SOMCV and
SOMCV s models with re-spect to the C - V model. 107 a b l e . : T h e c o n t o u r e v o l u t i o n t i m e a n d n u m b e r o f i t e r a t i o n s r e q u i r e d b y t h e S O M C V , S O M C V s , a n d C - V m o d e l s t o s e g m e n tt h e f o r e g r o u n d f o r t h e s c a l a r - v a l u e d i m a g e ss h o w n i n F i g . . . I m a g e i n I m a g e s i z e S O M t o p o l o gy S O M C V S O M C V s C - V C P U T i m e ( s ) I t e r a t i o n s C P U T i m e ( s ) I t e r a t i o n s C P U T i m e ( s ) I t e r a t i o n s C o l u m n × . . . C o l u m n × . . . C o l u m n × . . . C o l u m n × . . . C o l u m n × . . . a b l e . : T h e c o n t o u r e v o l u t i o n t i m e a n d n u m b e r o f i t e r a t i o n s r e q u i r e d b y t h e S O M C V , S O M C V s , a n d C - V m o d e l s t o s e g m e n tt h e f o r e g r o u n d f o r t h e v e c t o r - v a l u e d i m a g e ss h o w n i n F i g . . . I m a g e i n I m a g e s i z e S O M t o p o l o gy S O M C V S O M C V s C - V C P U T i m e ( s ) I t e r a t i o n s C P U T i m e ( s ) I t e r a t i o n s C P U T i m e ( s ) I t e r a t i o n s C o l u m n × × . . C o l u m n × × . . C o l u m n × × . . . C o l u m n × × . . . C o l u m n × × . . . SOMCV model with some rep-resentative global pixel-based segmentation techniques, we haveapplied the Otsu’s method [78] and the multi-threshold Otsu’smethod [63] to some of the scalar-valued images considered in thischapter. Such methods belong to the class of thresholding imagesegmentation methods, as they segment a scalar-valued image bycomparing the pixel intensity with one or multiple thresholds, re-spectively. The main reason for selecting the Otsu’s method is thatits threshold is chosen in such a way to optimize a trade-o ff be-tween the maximization of the inter-class variance (i.e., betweenpairs of pixels beloging to the foreground and the background,respectively) and the minimization of the intra-class variance (i.e.,between pairs of pixels belonging to the same region). The multi-threshold the Otsu’s method is similar but uses more thresholds,segmenting the image in more than 2 regions. Fig. 7.8 shows thesegmentation results obtained by the Otsu’s method (second row)and the multi-threshold Otsu’s method (third row) on some of thescalar-valued images considered in this chapter. For a fair com-parison, in the case of multi-threshold Otsu’s method we havealso merged some of the objects found for di ff erent numbers ofthresholds (as shown in the fourth row), then we have appliedthe classical Otsu’s method to the resulting image (fifth row). Asillustrated by Fig. 7.8, the Otsu’s and multi-threshold Otsu’s meth-ods demonstrated to be more sensitive to noise than our proposed SOMCV model. As an additional drawback, post-processing op-erations were also required for the multi-threshold Otsu’s method.The quantitative results corresponding to Fig. 7.8 are reported inTable 7.5. 110 a b l e . : T h e P r e c i s i o n , R e c a ll , a n d F - m e a s u r e m e t r i c s f o r t h e O t s u ’ s m e t h o d a n d t h e m u l t i - t h r e s h o l d O t s u ’ s m e t h o d ( w i t h p o s t - p r o c e ss i n g ) i n t h e s e g m e n t a t i o n o f t h e i m a g e ss h o w n i n F i g . . ( s e c o n d a n d fi f t h r o w s , r e s p e c t i v e l y ) c o m p a r e d w i t h t h e S O M C V m o d e l ( s i x t h r o w ) . I m a g e i n O t s u ’ s m e t h o d m u l t i - t h r e s h o l d O t s u ’ s m e t h o d S O M C V P ( % ) R ( % ) F - m ( % ) P ( % ) R ( % ) F - m ( % ) P ( % ) R ( % ) F - m ( % ) c o l u m n . . . . . . . c o l u m n . . . . . c o l u m n . . . . . . . SOMCV on the images of the first row.Finally, we have trained the neural map on a single frame112f a real aircraft video [70] (the top left image in Fig. 7.9(a)) and ap-plied the trained network on-line to segment individually - using
SOMCV - some of its
RGB -frames, which are shown in Fig. 7.9(a)(the initial contours for the video frames are similar to the initialcontour - shown in red - which has been used for the first image).Fig. 7.9(b) shows the segmentation results of
SOMCV in handlingthe selected frames in Fig. 7.9(a) and demonstrates its robustnessto scene changes and object motions. Concluding, this experimenthightlights the robustness of
SOMCV model to the contour ini-tialization, scene changes and illumination variations when beingused in an on-line framework. ba Figure 7.9: The robustness of the
SOMCV model to scene changesand moving objects. (a) The first row shows the original earlyframes (frames 50-59, from left to right) of a real-aircraft videowhile later frames (frames 350-359, from left to right) are shownin the second row. (b) shows the segmentation results obtained by
SOMCV , on the frames shown in part (a).113 .5 Summary
In this chapter, we have proposed a novel global
ACM ,termed
SOM-based Chan-Vese ( SOMCV ). The
SOMCV model is aglobal and an unsupervised
ACM that integrates e ff ectively theadvantages of ACM s and self-organizing networks.
SOMCV hasa
Self-Organizing Topology Preservation ( SOTP ) property, which al-lows to preserve the topological structures of the foreground / back-ground intensity distributions during the active contour evolution.Indeed, SOMCV relies on a set of self-organized neurons by auto-matically extracting the prototypes of selected neurons as globalregional descriptors and iteratively, in an unsupervised way, inte-grates them during the evolution of the contour.In order to highlight the robustness of
SOMCV , severalsynthetic and real images with di ff erent kinds of intensity distribu-tions have been handled e ff ectively in the experimental studies pre-sented in Section 8.4. Also the variation of SOMCV - the
SOMCV s model - has provided good results in most cases. The capability of SOMCV and
SOMCV s to handle images globally without relyingon a particular statistical assumption is the main contribution ofthis chapter. Moreover, the e ff ectiveness and robustness of SOMCV and
SOMCV s may find applications in various other problems incomputer vision. In a similar way to the SOMCV , in the followingchapter we also describe another
SOM -based
ACM , that we haveproposed which takes advantage of both local and global informa-tion in order to improve the robustness of the segmentation to thecontour initialization, and to the presence of noise and intensityinhomogenity. 114 hapter 8
SOM -based Regional AC Model
Local Active Contour Models (local
ACM s) constitute ane ffi cient image segmentation framework, which is driven by localinformation about the intensity of an image. Most of the existinglocal ACM s can handle images with intensity inhomogeneity byintegrating explicity local intensity information into the objectivefunctional to be optimized. However, in this case, the success oflocal
ACM s depends on how accurate the initial contour is. Then,a challenge in the use of such models consists in handling imageswith intensity inhomogeneity in such a way to obtain robustnesswith respect to the contour initialization. In this chapter we pro-pose a model, termed
SOM-based Regional Active Contour ( SOM - RAC ) model, with such a property. The
SOM - RAC model relies onthe global information coming from selected prototypes associatedwith a Self Organizing Map (
SOM ), which is trained o ff -line tomodel the intensity distribution of an image, and used on-line tosegment an identical or similar image. In order to improve the ro-bustness of the model, global and local information are combinedin the on-line process, as the selection of the weights of the trained SOM is driven by local information on the intensity of the image.115xperimental results show the high accuracy of the segmentationsobtained by the
SOM - RAC model on several synthetic and real im-ages, when compared with a state-of-the-art local
ACM , the
LocalRegion-based Chan-Vese model .The main motivation for this model is to combine globaland local intensity information in an
ACM through a
SOM -basedapproach. In the first phase of
SOM - RAC , a set of neurons istrained to model globally the intensity distribution of the imageby a self-organization learning procedure. Such weights are usedto integrate the intensity distribution implicitly - as global Regionof Interest (
ROI ) descriptors - into the objective functional of theproposed
SOM - RAC model, to guide the evolution of the activecontour. In a second phase, local image intensity information isused during the actual segmentation process in combination withthe global information coming from the weights of the trained
SOM , and is combined with the global information above insidethe objective functional. In this way, the proposed
ACM model isable to make use of both global and local information.
SOM - RAC model
In this section, we describe our
SOM -based Regional ActiveContour (
SOM - RAC ) model. Although the model is presented herefor the case of scalar-valued images (e.g., gray-level images), it canbe extended straightforwardly to the case of vector-valued images(e.g.,
RGB images).The
SOM - RAC segmentation framework is composed ofan unsupervised training session and a testing session. The twosessions are performed, respectively, o ff -line and on-line.The training session of this model is the same as the trainingsession of the SOMCV model, which was presented in chapter 7(see Subsection 7.2.1).Once its training has been accomplished, the
SOM networkis applied in the testing session, during the evolution of the activecontour C , to approximate and describe globally the foregroundand background intensity distributions of a similar test image I ( x )116o be segmented. The use of such a global information helps in pro-viding to the model robustness to the contour initialization and tothe additive noise. Moreover, during the active contour evolution,a combination of local and global information is exploited in orderto provide to the model robustness to the intensity inhomogeneityand to possible changes in the intensity distribution itself, whenmoving from the training image to the test one. More precisely, asa first step, one determines, for each pixel x ∈ Ω (the domain ofthe image), the best-matching-unit ( BMU ) neuron w b ( x ) to the localweighted mean intensity of the image c ( x ) : = R Ω g σ ∗ ( x − y ) I y (cid:1) dy R Ω g σ ∗ ( x − y ) dy , (8.1)i.e., w b ( x ) : = argmin w n | w n − c ( x ) | , (8.2)where g σ ∗ is a Gaussian kernel function with R R g σ ∗ ( x ) dx = σ ∗ >
0. Such a choice of w b ( x ) does not depend on thecurrent contour. Then, the two local weighted mean intensitiesin the foreground and the background, respectively as defined inEquations 8.3 and 8.4, are compared to w b ( x ), to define suitableregional intensity descriptors. c + ( x , C ) : = R in( C ) g σ ( x − y ) I y (cid:1) dy R in( C ) g σ ( x − y ) dy , (8.3) c − ( x , C ) : = R out( C ) g σ ( x − y ) I y (cid:1) dy R out( C ) g σ ( x − y ) dy . (8.4)117here 0 < σ ∗ < σ . More precisely, one sets A + b ( x , C ) : = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) w b ( x ) − R in( C ) g σ ( x − y ) I y (cid:1) dy R in( C ) g σ ( x − y ) dy (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (8.5) A − b ( x , C ) : = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) w b ( x ) − R out( C ) g σ ( x − y ) I y (cid:1) dy R out( C ) g σ ( x − y ) dy (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) , (8.6)which are, respectively, the distances of the weight w b ( x ) from thetwo local weighted mean intensities in the two regions around thepixel x . Then, one defines the two weights w + b ( x , C ) : = w b ( x ) , if A + b ( x , C ) < A − b ( x , C ) , , otherwise , (8.7) w − b ( x , C ) : = w b ( x ) , if A + b ( x , C ) > A − b ( x , C ) , , otherwise . (8.8)Such weights are extracted as regional intensity descriptors and in-cluded in the objective functional to be minimized in our proposed SOM - RAC model, which has the following expression : E SOM − RAC ( C ) : = λ + Z in( C ) e + ( x , C ) dx + λ − Z out( C ) e − ( x , C ) dx , (8.9) e + ( x , C ) : = (cid:16) I ( x ) − w + b ( x , C ) (cid:17) , (8.10) e − ( x , C ) : = (cid:16) I ( x ) − w − b ( x , C ) (cid:17) . (8.11)where the parameters λ + , λ − ≥ R in( C ) e + ( x , C ) dx and R out( C ) e − ( x , C ) dx , insideand outside the contour. When one of the index set is empty, one can use several strategies: e.g.,replacing the associated summation with its value at its last previous evaluation(however, this was not needed for the case studies shown in Section 8.4). C with the level setfunction φ , obtaining E SOM − RAC (cid:16) φ (cid:17) = λ + Z φ> e + ( x , φ ) dx + λ − Z φ< e − ( x , φ ) dx , (8.12)where we have also made explicit the dependence of the functions e + and e − on φ . In terms of the Heaviside step function H ( · ), the SOM - RAC objective functional can be also written as follows: E SOM − RAC (cid:16) φ (cid:17) = λ + Z Ω e + ( x , φ ) H ( φ ( x )) dx + λ − Z Ω e − ( x , φ )(1 − H ( φ ( x ))) dx . (8.13)Finally, the evolution of the contour in the SOM - RAC is describedby the
PDE ∂φ∂ t = δ (cid:16) φ (cid:17) (cid:2) − λ + e + + λ − e − (cid:3) , (8.14)which shows how the learned neurons are used to determine theinternal and external forces acting on the contour during its evolu-tion. Apart from this di ff erence, Eq. (8.14) can be solved iterativelyusing the same smoothing and discretization techniques used inthe C - V model. Moreover, at each iteration of a finite-di ff erenceapproximation of (8.14), we also perform a regularization of thecurrent level set function by replacing it with its convolution witha Gaussian filter of suitable width σ ′ >
0. Such a convolution canbe preceded by a binarization of the function φ , without loss ofinformation about the current contour. The procedural steps of the two sessions of the
SOM - RAC model are summarized in the following Algorithm 4. See formula (8.16) in Algorithm 4. lgorithm 4
SOM - RAC segmentation framework procedure • Input: – Training and test scalar-valued images. – Topology of the network. – Number of iterations t ( tr )max for training the neural map. – Maximum number of iterations t ( evol )max for the contour evolution. – η >
0: starting learning rate. – r >
0: starting radius of the map. – τ η ,τ r >
0: time constants in the learning rate and contour smoothing parameter. – λ + , λ − ≥
0: weights of the energy terms, respectively, inside and outside the contour. – σ ∗ ,σ,σ ′ >
0: Gaussian intensity and contour smoothing parameters. – ρ >
0: constant in the binary approximation of the level set function. • Output: – Segmentation result.
TRAINING SESSION: Initialize randomly the weights of the neurons. repeat Choose randomly a pixel x t in the image domain Ω and determine the BMU neuron to the input intensity I ( tr ) ( x t ). Update the weights w n using (5.1), (5.2), (5.3), and (5.4). until learning of the weights is accomplished (i.e., the number of iterations t ( tr )max is reached). TESTING SESSION: Choose a subset Ω (e.g., a rectangle) in the image domain Ω with boundary Ω ′ , and initialize the level set functionas: φ ( x ) : = ρ, x ∈ Ω \ Ω ′ , , x ∈ Ω ′ , − ρ, x ∈ Ω \ ( Ω ∪ Ω ′ ) . (8.15) repeat Determine, for each pixel x ∈ Ω , the weight w b ( x ) from (8.2), then, for the current contour C and each pixel x ∈ Ω , the quantities A + b ( x , C ) and A − b ( x , C ) from (8.5) and (8.6), finally the weights w + b ( x , C ) and w − b ( x , C ) from (8.7)and (8.8). Evolve the level set function φ according to a finite di ff erence approximation of (8.14). At each iteration of the finite-di ff erence scheme, re-initialize the current level set function to be binary byperforming the update φ ← ρ (cid:16) H ( φ ) − H ( − φ ) (cid:17) , (8.16)then regularize by convolution the obtained level set function: φ ← g σ ′ ⊗ φ, (8.17)where g σ ′ is a Gaussian kernel with R R g σ ′ ( x ) dx = σ ′ . until the curve evolution converges (i.e., the curve does not change anymore) or the maximum number ofiterations t ( evol )max is reached. end procedure .4 Experimental study In this section, we demonstrate experimentally the e ffi -ciency and robustness of the SOM - RAC model in handling real andsynthetic images, as compared with the
LRCV model describedin Chapter 2. For a fair comparison, in these experiments boththe
SOM - RAC and the
LRCV models are implemented in MatlabR2012a on a PC with the following configuration: 1.8 GHz Intel(R)Core(TM) i3-3217U, and 4.00 GB RAM.In each experiment, the r , σ and σ ′ parameters are ex-pressed in pixels. Moreover, the SOM - RAC parameters are fixedas follows: η = . σ ∗ = . σ =
30 (apart from the experimentsdescribed in Figure 8.6, in which several values for σ have beenconsidered), σ ′ = .
5, and λ + = λ − =
1. Also, r : = max( M , N ) / M = N = SOM in the
SOM - RAC model, t ( tr )max = t ( evol )max = τ η : = t ( tr )max , τ r : = t ( tr )max / ln( r ), ρ =
1. In the
LRCV model, λ + , λ − are also fixedto 1, and the same values of σ and σ ′ as above are considered. Un-less stated otherwise, the training image used in the unsupervisedtraining session coincides with the test image. Otherwise, it is animage similar to the test image (obtained, e.g., by adding Gaussiannoise). In all the testing sessions, the initial contour has been cho-sen as rectangular (which is a standard choice for ACM s). All theconsidered gray-level images are 8-bit images, so the range of thevalues assumed by the intensity is 0-255.To illustrate the e ff ectiveness of SOM - RAC in handling realand synthetic images with respect to the
LRCV model, in Fig. 8.1we compare the segmentation results obtained by the
SOM - RAC and the
LRCV models, the contour initialization and amount of lo-cal information (controlled by the parameter σ ) being the same forthe two models. As such figure illustrates, the SOM - RAC modelis more e ff ective in handling real and synthetic images. More pre-cisely, the segmentation results obtained by the SOM - RAC modelon the images considered in this experiment show its ability ofsegmenting images with intensity inhomogeneity and objects withblurred edges (first and second rows), images with intensity in-homogeneity only (third and fourth rows), and a synthetic image121ontaining a shadow (fifth row). The final contours obtained bythe
SOM - RAC model demonstrate its high e ff ectiveness, whereasunsatisfying results are obtained in these cases by LRCV model,due to its sensitivity to the contour initialization.The experiments illustrated in the remaining of this sectioncan be divided into four parts, as they test the robustness of the
SOM - RAC model, respectively, to the contour initialization, ad-ditive noise, scene changes, and choice of the locality parameter σ . In Fig. 8.2 and 8.3, we test the robustness of the SOM - RAC model in handling images with di ff erent initial rectangular con-tours. Fig. 8.2 illustrates the robustness of SOM - RAC to contourinitialization in handling a synthetic image with intensity inhomo-geneity when compared to the
LRCV model. As such figure shows,in this experiment the
SOM - RAC model is less sensitive to the lo-cation of the initial contour than the
LRCV model, and the finalcontours obtained by the
SOM - RAC model converge to the trueobject boundary with similar performances for all the consideredcontour initializations. Additionally, the robustness of
SOM - RAC to contour initialization in handling a real image with intensityinhomogeneity and weak boundaries is illustrated in Fig. 8.3 incomparison with the
LRCV model. In this case, the final contoursobtained by
SOM - RAC for all the initial contours show its abilityto find the object with a high accuracy, while for the case of the
LRCV model a leaking problem occurs for all the initial contours.Fig. 8.4 illustrates the e ff ectiveness and robustness of the SOM - RAC model in handling the synthetic image already shownin Fig. 8.2 with the addition of di ff erent Gaussian noise levels. Inthis experiment, the SOM - RAC model has been trained o ff -line onthe image shown in the first row, then it has been used to segmentits noisy versions in the on-line phase. As shown by the figure, thesegmentations obtained in this experiment by the SOM - RAC modeldemonstrate its small sensitivity to the additive noise. On the otherhand, in this experiment the
LRCV model is more sensitive to theadditive noise, which a ff ects gradually its segmentation results.In order to demonstrate the robustness of the SOM - RAC model with respect to scene changes, we have trained o ff -line122igure 8.1: The segmentation results obtained on real and syn-thetic images by the SOM - RAC and the
LRCV models. The firstcolumn shows the original images with the initial contours, whilethe second and third columns show, respectively, the correspond-ing segmentation results obtained by the two models.123igure 8.2: The robustness of the
SOM - RAC model with respect tothe contour initialization, as compared to the segmentation resultsobtained by the
LRCV model, for a synthetic 127 ×
96 image withintensity inhomogeneity. The first column shows the original im-age with three di ff erent rectangular initial contours (in white). Thesecond and third columns show, respectively, the segmentationresults obtained by the SOM - RAC and the
LRCV models.the
SOM - RAC model on the images shown in the first columnof Fig. 8.5, then we have applied it on-line to segment di ff erentimages, as shown in the second column of Fig. 8.5. Then, we havealso trained it on the images shown in the third column to segmentthe images in the fourth column. In this experiment, the robust-ness of SOM - RAC has been confirmed in several situations: whenintensity inhomogeneity occurs in real and synthetic images withweak boundaries (first row of Fig. 8.5), in the presence / absence124igure 8.3: The robustness of the SOM - RAC model with respectto the contour initialization in handling a real 103 ×
131 imagein the presence of intensity inhomogeneity and weak edges, ascompared to the segmentation results obtained by the
LRCV modelon the same image. The first column shows the original image withthree di ff erent rectangular initial contours (in black). The secondand third columns show, respectively, the segmentation resultsobtained by the SOM - RAC and the
LRCV models.of the intensity inhomogeneity (second row), in synthetic images(third row), and in handling images containing an overlap of theforeground / background intensity distributions.Finally, in order to study the robustness of the SOM - RAC model with respect to the locality parameter σ , we have trained the SOM - RAC model on the same synthetic image already consideredin Fig. 8.5, then we have used it on-line to segment the same imagefor several values of σ (the initial contour being the same as the125igure 8.4: The robustness of the SOM - RAC model with respect toadditive noise in handling a synthetic 127 ×
96 image in the presenceof intensity inhomogeneity, as compared to the segmentation re-sults obtained by the
LRCV model. The first column shows, fromtop to down, the image of Fig. 8.2 with the addition of di ff erentGaussian noise levels (with standard deviations SD =
0, 5, 10, 15,20, and 25, respectively). The second and third columns show,respectively, the segmentation results obtained by the
SOM - RAC and the
LRCV models. 126ne shown in Fig. 8.5 for the same image). Then we have alsoapplied the
LRCV model to segment the same image, for the samechoices of σ . Fig. 8.6 (a) demonstrates the robustness of SOM - RAC to changes in the parameter σ , since in all the considered cases theobjects are segmented accurately when σ is more than or equal to20. On the other hand, as shown in Fig. 8.6 (b), in this experimentthe LRCV model is able to find the object correctly only when σ isequal to 15. In this chapter, with the aim of developing an
ACM that is atthe same time e ff ective and robust in handling complex images con-taining intensity inhomogeneity, we have proposed a novel ACM ,termed
SOM-based Regional Active Contour ( SOM - RAC ) model. Themain motivation for the
SOM - RAC model is to deal with somedrawbacks of global
ACM s and local
ACM s through the combi-nation of global and local information by a
SOM -based approach.Indeed, global information plays an important role to improve therobustness of
ACM s against the contour intialization and the addi-tive noise but - if used alone - it is usually not su ffi cient to handleimages containing intensity inhomogeneity. On the other hand,local information allows one to deal e ff ectively with the intensityinhomogeneity but - if used alone - it produces usually ACM svery sensitive to the contour initialization. The
SOM - RAC modelcombines both kinds of information relying on global regional de-scriptors (i.e., suitably selected weights of a trained
SOM ) on thebasis of local regional descriptors (i.e., the local weighted meanintensities). In this way, the
SOM - RAC model is able to integratethe advantages of local
ACM s and Self Organizing Maps.In order to highlight the robustness of the proposed
SOM - RAC model, we have tested and compared it with a state-of-the-artlocal
ACM . In the experimental studies presented in Section 8.4,several synthetic and real images with various intensity distribu-tions and locations of the initial contours have been handled ef-fectively by the proposed
SOM - RAC model, which has also out-127erformed the other
ACM model, showing more robustness withrespect to several factors. In the following chapter, we concludethe thesis by highlighting the motivation and contributions of thepresented models and, identifying some possible future researchdirections. 128igure 8.5: The robustness of the
SOM - RAC model with respectto scene changes. The first and third columns show the trainingimages while the second and fourth columns show, respectively,the segmentations results obtained by the
SOM - RAC model on dif-ferent test images. For each row, the
SOM - RAC model was trainedon the first (respectively third) image, then it was used to segmentthe second (respectively, fourth) image. For the second and fourthcolumns, the initial contours are shown in white, whereas the finalsegmentation results are shown in black.129 b Figure 8.6: The robustness of the
SOM - RAC model with respect tothe locality parameter σ in handling a synthetic 100 ×
100 image(already shown in the second row of Fig. 8.5) when comparedto
LRCV model (the same initial contour has been used in thecomparison). Parts (a) shows the segmentation results obtainedby the
SOM - RAC model with σ = , , , ,
25 in the first rowand σ = , , , ,
50 in the second row. Part (b) shows thecorresponding segmentation results obtained by the
LRCV model.130 hapter 9
Conclusions and FutureWork
In this thesis, we first provided a literature review to showvarious kinds of active contour models to deal with some challeng-ing problems in computer vision. Moreover, a number of novelmodels have been presented to deal (in e ff ective, e ffi cient, and / orrobust way) with the image segmentation problem, and have beencompared with state-of-the-art active contour models.In chapter 4, a novel energy based-active contour modelbased on a new Globally Signed Region Pressure Force ( GSRPF )function has been proposed. The
GSRPF considers the global infor-mation extracted from a ROI and accommodates also foregroundintensity distributions that are not necessarily symmetric. It auto-matically and e ffi ciently modulates the signs of the pressure forcesinside and outside the contour. Compared with other ACM s, theresulting method is less sensitive to noise, contour initialization,and can handle images with complex intensity distributions in theforeground and / or background. GSRPF is a Gaussian regulariz-ing level set model that relies only on a single parameter. It isdesigned to have a quadratic behaviour and converge in a few it-erations without penalizing the segmentation accuracy. Results onsynthetic and real images from a variety of scenarios have demon-strated the superior segmentation accuracy of
GSRPF , when com-131ared with well regarded global level set methods, such as the
SBGFRLS and
C-V models.Chapter 5 describes a novel
SOM -based
ACM model, theConcurrent Self Organizing Map-based Chan-Vese model (
CSOM-CV ), which relies mainly on a set of prototypes coming from twotrained
SOM s to guide the evolution of the active contour.
CSOM-CV is a supervised and global region-based
ACM . It has beendemonstrated to be e ffi cient and robust to the noise. As com-pared to the C-V model, our proposed solution consists insteadin modeling globally in a supervised way the intensity distribu-tions of the foreground / background (relying on a few supervisedpixels) without using parametric models, but relying on a set ofprototypes resulting from the training of a CSOM . Moreover, ascompared to
CSOM and in general to previous
SOM -like modelsused in image segmentation, our solution consists in modeling theactive contour using a variational level set method and relyingat the same time on a few prototypes coming from the learned
CSOM . In this way, the
CSOM-CV model is able to produce a finalsegmentation result characterized by a smooth contour while most
SOM -like models usually produce segmentations characterized bydisconnected boundaries. Moreover, the
CSOM-CV has shown tobe more robust to two di ff erent kinds of noise.We have also proposed a new supervised ACM , whichwe have termed
Self Organizing Active Contour ( SOAC ) model inchapter 6. It is based on two sets of self organizing neurons tolearn the dissimilarity between the foreground / background inten-sity distributions. In this way, the information about such distri-butions is integrated implicitly into the energy functional of themodel by the learned prototypes of the two SOM s, helping in theguide of the contour evolution.
SOAC is a Gaussian regularizinglevel set method, and it is robust to additive noise. The experi-mental results obtained on several synthetic and real images forboth scalar-valued and vector-valued cases images demonstratethe high e ff ectiveness and robustness of our model, when com-pared with state-of-the-art ACM s (e.g.,
KDE -based,
GMM -based, C - V , LRCV models), in segmenting images with overlap betweenthe foreground / background intensity distributions, intensity inho-132ogeneity, and / or containing objects characterized by many dif-ferent intensities.A novel global ACM , termed
SOM-based Chan-Vese ( SOMCV )was proposed in chapter 7. The
SOMCV model is a global andan unsupervised
ACM that integrates globally and e ff ectively theadvantages of ACM s and self-organizing networks.
SOMCV hasa
Self-Organizing Topology Preservation ( SOTP ) property, which al-lows to preserve the topological structures of the foreground / back-ground intensity distributions during the active contour evolution.Indeed, SOMCV relies on a set of self-organized neurons by auto-matically extracting the prototypes of selected neurons as globalregional descriptors and iteratively, in an unsupervised way, inte-grates them during the evolution of the contour. The robustness,e ff ectiveness and e ffi ciency of the SOMCV model on several syn-thetic and real images with di ff erent kinds of intensity distributionshas been demonstrated in that chapter and compared to a global ACM (e.g., the C - V ) and other thresholding-based models (e.g.,Otsu’s and Multi-level Otsu’s methods).Eventually, with the aim of developing an ACM that is atthe same time e ff ective and robust in handling complex imagescontaining intensity inhomogeneity, we have proposed, in chapter8, another novel ACM , termed
SOM-based Regional Active Contour ( SOM - RAC ) model. The main motivation for the
SOM - RAC modelis to deal with the sensitivity of Local
ACM s to the contour initial-ization (when intensity inhomogeneity and additive noise occur inthe images) through the combination of global and local informa-tion by a
SOM -based approach. Indeed, global information playsan important role to improve the robustness of
ACM s against thecontour intialization and the additive noise but - if used alone - it isusually not su ffi cient to handle images containing intensity inho-mogeneity. On the other hand, local information allows one to deale ff ectively with the intensity inhomogeneity but - if used alone - itproduces usually ACM s very sensitive to the contour initialization.The
SOM - RAC model combines both kinds of information relyingon global regional descriptors (i.e., suitably selected weights of atrained
SOM ) on the basis of local regional descriptors (i.e., the lo-cal weighted mean intensities). In this way, the
SOM - RAC model is133ble to integrate the advantages of local
ACM s and Self OrganizingMaps. In order to highlight the robustness of the proposed
SOM - RAC model, we have tested and compared it with a state-of-the-artlocal
ACM (e.g.,
LRCV ) on several synthetic and real images withvarious intensity distributions and locations of the initial contours,showing more robustness with respect to several factors.As discussed above, several
ACM s have been proposed inthe thesis, relying mainly on prior intensity information. As apossible future research direction, our models could benefit fromother kind of prior information such as shape information. Withthe addition of such information, our models could behave nicelyin handling complex images with some other kinds of challengingproblems such as occlusion. Other future research directions in-clude developing the machine learning components of our modelsfrom a streaming learning perspective, in order to better under-stand the content of videos, and handling them in real time. Thiscould be possible by integrating streaming learning algorithms intothe segmentation framework of our models.134 ibliography [1] .[2] .[3] .[4] http://decsai.ugr.es/cvg/dbimagenes/gbio256.php .[5] .[6] .[7] M. M. Abdelsamea and G. Gnecco. robust local-global SOM-based ACM.
Electronics Letters , 51:142–143, 2015.[8] M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. A concur-rent SOM-based Chan-Vese model for image segmentation.In
Proceedings of the 10th Workshop on Self-Organizing Maps(WSOM 2014), Advances in Intelligent Systems and Computing ,volume 295, pages 199–208. Springer, 2014.[9] M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. An e ffi cientself organizing active contour model for image segmentation. Neurocomputing , 2014. DOI:10.1016 / j.neucom.2014.07.052.[10] M. M. Abdelsamea, G. Gnecco, and M. M. Gaber. A som-based chan-vese model for unsupervised image segmenta-tion. Technical Report , 2014.13511] M. M. Abdelsamea and S. A. Tsaftaris. Active contour modeldriven by globally signed region pressure force. In
Proceed-ings of the 18th International Conference on Digital Signal Pro-cessing (DSP) , pages 1–6, 2013.[12] R. Adams and L. Bischof. Seeded region growing.
IEEE Trans-actions on Pattern Analysis and Machine Intelligence , 16(6):641–647, 1994.[13] S. Alpert, M. Galun, R. Basri, and A. Brandt. Image seg-mentation by probabilistic bottom-up aggregation and cueintegration. In
Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR) , pages 1–8, 2007.[14] R.F. A. Aluizio and C. C. Diogo. Local adaptive receptive fieldself-organizing map for image color segmentation.
Image andVision Computing , 27(9):1229 – 1239, 2009.[15] Y. N. Andrew and I. J. Michael. On discriminative vs. gen-erative classifiers: a comparison of logistic regression andnaive Bayes. In
Proceedings of Advances in Neural InformationProcessing Systems 14 (NIPS 2001) , 2001.[16] Y. J. Anthony, A. Tsai, and A. Willsky. A fully global ap-proach to image segmentation via coupled curve evolutionequations.
Journal of Visual Communication and Image Repre-sentation , 13:195 – 216, 2002.[17] M. A. Bakhshali, M. Shamsi, and A. Golzarfar. Applicationof SOM network to study facial skin segmentation after fa-cial surgery. In
Proceedings of the 2012 IEEE Symposium onIndustrial Electronics and Applications (ISIEA) , pages 310–314,2012.[18] S. Balla-Arabe, X. Gao, B. Wang, F. Yang, and V. Brost. Multi-kernel implicit curve evolution for selected texture regionsegmentation in vhr satellite images.
IEEE Transactions onGeoscience and Remote Sensing , 52(8):5183–5192, 2014.13619] A. R. Barron. Universal approximation bounds for superpo-sitions of a sigmoidal function.
IEEE Transactions on Informa-tion Theory , 39(3):930–945, 1993.[20] C. A. C. M. Bastos, I. R. Tsang, G. S. Vasconcelos, and G. D. C.Cavalcanti. Pupil segmentation using pulling & pushing andBSOM neural network. In
Proceedings of the 2012 IEEE Inter-national Conference on Systems, Man, and Cybernetics (SMC) ,pages 2359–2364, 2012.[21] L. E. Baum and T. Petrie. Statistical inference for probabilisticfunctions of finite state Markov chains.
The Annals of Mathe-matical Statistics , 37(6):1554–1563, 1966.[22] B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training al-gorithm for optimal margin classifiers. In
Proceedings of thefifth Annual Workshop on Computational Learning Theory , pages144–152, New York, NY, USA, 1992. ACM.[23] Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energyminimization via graph cuts.
IEEE Transactions on PatternAnalysis and Machine Intelligence , 23(11):1222–1239, 2001.[24] X. Bresson, S. Esedoglu, P. Vandergheynst, J. Thiran, andS. Osher. Fast global minimization of the active contour / s-nake model. Journal of Mathematics and Imaging Vision ,28(2):151–167, 2007.[25] J. Canny. A computational approach to edge detection.
IEEETransactions on Pattern Analysis and Machine Intelligence , pages679–698, 1986.[26] V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active con-tours.
International Journal of Computer Vision , 22(1):61–79,1997.[27] T. F. Chan, Sandberg, and L. A. Vese. Active contours withoutedges for vector-valued images.
Journal of Visual Communi-cation and Image Representation , 11(2):130–141, 2000.13728] T. F. Chan and L. A. Vese. Active contours without edges.
IEEE Transactions on Image Processing , 10(2):266–277, 2001.[29] L. Chen, Y. Zhou, Y. Wang, and J. Yang. GACV: geodesic-aided CV method.
Pattern Recognition , 39(7):1391–1395, 2006.[30] S. Chen and R. J. Radke. Level set segmentation with bothshape and intensity priors. In
Proceedings of the 12th IEEEInternational Conference on Computer Vision , pages 763–770,2009.[31] Y. M. Cheung, X. Liu, and X. You. A local region basedapproach to lip tracking.
Pattern Recognition , 45(9):3336–3347,2012.[32] L. D. Cohen and I. Cohen. Finite element methods for ac-tive contour models and balloons for 2d and 3d images.
IEEE Transactions on Pattern Analysis and Machine Intelligence ,15:1131–1147, 1991.[33] L. D. Cohen and R. Kimmel. Global minimum for active con-tour models: a minimal path approach.
International Journalof Computer Vision , 24(1):57–78, 1997.[34] D. Cremers, S. J. Osher, and S. Soatto. Kernel density estima-tion and intrinsic alignment for knowledge-driven segmen-tation: teaching level sets to walk. In
International Journal ofComputer Vision , pages 36–44, 2004.[35] D. Cremers and M. Rousson. E ffi cient kernel density estima-tion of shape and intensity priors for level set segmentation.In Deformable Models. Topics in Biomedical Engineering. Inter-national Book Series 2007, Springer , pages 447–460, 2007.[36] A. Declercq and J. H. Piater. Online learning of Gaussianmixture models - a two-level approach. In
Proceedings of thethird International Conference on Computer Vision Theory andApplications (VISAPP) , pages 605–611. INSTICC - Institutefor Systems and Technologies of Information, Control andCommunication, 2008. 13837] G. Dong and M. Xie. Color clustering and learning for imagesegmentation based on neural networks.
IEEE Transactionson Neural Networks , 16(4):925–936, 2005.[38] V. Estellers, D. Zosso, X. Bresson, and J.-P. Thiran. Har-monic active contours.
IEEE Transactions on Image Processing ,23(1):69–82, 2014.[39] A. Fonseka and D. Alahakoon. Exploratory data analysiswith multi-layer growing self-organizing maps. In
Interna-tional Conference on Information and Automation for Sustainabil-ity (ICIAFs) , pages 132–137, 2010.[40] Y. Gan and Q. Zhao. An e ff ective defect inspection methodfor LCD using active contour model. IEEE Transactions onInstrumentation and Measurement , 62(9):2438–2445, 2013.[41] A. Georgieva and I. Jordanov. Intelligent visual recognitionand classification of cork tiles with neural networks.
IEEETransactions on Neural Networks , 20(4):675–685, 2009.[42] Y. Gu, V. Kumar, L. O. Hall, D. B. Goldgof, C. Li, R. Korn,C. Bendtsen, E. R. Velazquez, A. Dekker, H. J. W. L. Aerts,P. Lambin, X. Li, J. Tian, R. A. Gatenby, and R. J. Gillies.Automated delineation of lung tumors from CT images us-ing a single click ensemble segmentation approach.
PatternRecognition , 46(3):692–702, 2013.[43] K. Gunawardana, J. Rajapakse, and D. Alahakoon. Brain-inspired self-organizing model for incremental learning. In
International Joint Conference on Neural Networks (IJCNN) ,pages 1–8, 2013.[44] W. H. Highleyman.
Linear decision functions, with applicationto pattern recognition . Polytechnic Institute of Brooklyn, 1961.[45] H. Huang, Y. Chen, and W. Hsu. Color image segmentationusing a self-organizing map algorithm.
Journal of ElectronicImaging , 11(2):136–148, 2002.13946] M. Izadi and R. Safabakhsh. An improved time-adaptiveself-organizing map for high-speed shape modeling.
PatternRecognition , 42(7):1361–1370, 2009.[47] Y. Jiang, K. Chen, and Z. Zhou. SOM based image segmen-tation. In G. Wang, Q. Liu, Y. Yao, and A. Skowron, editors,
Rough Sets, Fuzzy Sets, Data Mining, and Granular Comput-ing , volume 2639 of
Lecture Notes in Computer Science , pages640–643. Springer Berlin Heidelberg, 2003.[48] I. Jordanov and A. Georgieva. Neural network learning withglobal heuristic search.
IEEE Transactions on Neural Networks ,18(3):937–942, 2007.[49] I. Karoui, R. Fablet, J. Boucher, and J. Augustin. Variationalregion-based segmentation using multiple texture statistics.
IEEE Transactions on Image Processing , 19(12):3146–3156, 2010.[50] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: active con-tour models.
International Journal of Computer Vision , 1(4):321–331, 1988.[51] M. Y. Kiang, M. Y. Hu, and D. M. Fisher. An extendedself-organizing map network for market segmentation -a telecommunication example.
Decision Support Systems ,42(1):36–47, 2006.[52] S. Kichenassamy, A. Kumar, and P. Olver. Conformal curva-ture flows: from phase transitions to active vision.
Archivefor Rational Mechanics and Analysis , 134(3):275–301, 1996.[53] T. H. Kim, I. Kyu Eom, and Y. S. Kim. Texture segmenta-tion using statistical characteristics of SOM and multi-scaleBayesian estimation. In
Proceedings of the 2006 IEEE Inter-national Conference on Acoustics, Speech and Signal Processing(ICASSP) , volume 2, pages II–II, 2006.[54] W. Kim and C. Kim. Active contours driven by the salientedge energy model.
IEEE Transactions on Image Processing ,22(4):1667–1673, 2013. 14055] T. Kohonen. The self-organizing map.
Proceedings of the IEEE ,78(9):1464–1480, 1990.[56] T. Kohonen. Essentials of the self-organizing map.
NeuralNetworks , 37:52–65, 2013.[57] V. Kurkov and M. Sanguineti. Comparison of worst-caseerrors in linear and neural network approximation.
IEEETransactions on Information Theory , 48(1):264–275, 2002.[58] F. Lecellier, S. Jehan-Besson, and J. Fadili. Statistical region-based active contours for segmentation: an overview.
IRBM ,35(1):3 – 10, 2014. Biomedical image segmentation usingvariational and statistical approaches.[59] C. P. Lee, W. Snyder, and C. Wang. Supervised multispectralimage segmentation using active contours. In
Proceedings ofthe 2005 IEEE International Conference on Robotics and Automa-tion, 2005. ICRA 2005. , pages 4242–4247. IEEE, 2005.[60] M. E. Leventon, W. E. L. Grimson, and O. Faugeras. Statisticalshape influence in geodesic active contours. In
Proceedings ofthe 5th IEEE EMBS International Summer School on BiomedicalImaging , pages 316 – 323, 2002.[61] C. Li, C. Kao, J. C. Gore, and Z. Ding. Implicit active contoursdriven by local binary fitting energy. In
Proceedings of the IEEEConference on Computer Vision and Pattern Recognition (CVPR) ,pages 1–7, 2007.[62] Z. Li, Z. Liu, and W. Shi. A fast level set algorithm for buildingroof recognition from high spatial resolution panchromaticimages.
IEEE Transactions on Geoscience and Remote SensingLetters , 11(4):743–747, 2014.[63] P. Liao, T. Chen, and P. Chung. A fast algorithm for multilevelthresholding.
Journal of Information Science and Engineering ,17(5):713–727, 2001. 14164] S. Liu and Y. Peng. A local region-based Chan-Vese modelfor image segmentation.
Pattern Recognition , 45(7):2769–2779,2012.[65] J. Lzaro, J. Arias, J. L. Martn, A. Zuloaga, and C. Cuadrado.SOM segmentation of gray scale images for optical recogni-tion.
Pattern Recognition Letters , 27(16):1991 – 1997, 2006.[66] Y. Ma, X. Gu, and Y. Wang. Contour detection based onself-organizing feature clustering.
Proceedings of the 2013 In-ternational Conference on Computing, Networking and Commu-nications (ICNC) , 2:221–226, 2007.[67] J. B. MacQueen. Some methods for classification and analysisof multivariate observations. In
Proceedings of the fifth Berke-ley Symposium on Mathematical Statistics and Probability , vol-ume 1, pages 281–297. University of California Press, 1967.[68] P. Marquez-Neila, L. Baumela, and L. Alvarez. A morpho-logical approach to curvature-based evolution of curves andsurfaces.
IEEE Transactions on Pattern Analysis and MachineIntelligence , 36(1):2–17, 2014.[69] W. S. McCulloch and W. Pitts. Neurocomputing: foundationsof research. chapter A Logical Calculus of the Ideas Imma-nent in Nervous Activity, pages 15–27. MIT Press, 1988.[70] A. S. Mian. Realtime visual tracking of aircrafts. In
Pro-ceedings of the 2008 Digital Image Computing: Techniques andApplications , pages 351–356, 2008.[71] I. Middleton and R. I. Damper. Segmentation of magneticresonance images using a combination of neural networksand active contour models.
Medical Enginering and Physics ,26(1):71–86, 2004.[72] Z. Musoromy, S. Ramalingam, and N. Bekooy. Edge detec-tion comparison for license plate detection. In
Proceedings ofthe 11th International Conference on Control Automation RoboticsVision (ICARCV) , pages 1133–1138. IEEE, 2010.14273] A. Myronenko and X. B. Song. Global active contour-basedimage segmentation via probability alignment. In
Proceed-ings of International Conference on Computer Vision and PatternRecognition (CVPR) , pages 2798–2804. IEEE, 2009.[74] V. Neagoe and A. Ropot. Concurrent self-organizing mapsfor pattern classification. In
Proceedings of the 1st IEEE Inter-national Conference on Cognitive Informatics (ICCI 2002) , pages304–312, 2002.[75] A. Ortiz, J.M. Gorriz, J. Ramirez, and D. Salas-Gonzalez.Improving { MR } brain image segmentation using self-organising maps and entropy-gradient clustering. Informa-tion Sciences , 262(0):117 – 136, 2014.[76] A. Ortiz, J.M. Grriz, J. Ramrez, D. Salas-Gonzlez, and J.M.Llamas-Elvira. Two fully-unsupervised methods for { MR } brain image segmentation using SOM-based strategies. Ap-plied Soft Computing , 13(5):2668 – 2682, 2013.[77] S. Osher and J. A. Sethian. Fronts propagating with curvaturedependent speed: algorithms based on Hamilton-Jacobi for-mulations. In
Journal Of Computational Physics , pages 12–49,1988.[78] N. Otsu. A threshold selection method from gray-level his-tograms.
IEEE Transactions on Systems, Man and Cybernetics ,9(1):62–66, 1979.[79] N. Paragios and R. Deriche. Geodesic active contours for su-pervised texture segmentation. In
Proceedings of InternationalConference on Computer Vision and Pattern Recognition (CVPR) ,pages 2422–2427. IEEE Computer Society, 1999.[80] N. Paragios and R. Deriche. Geodesic active regions andlevel set methods for supervised texture segmentation.
In-ternational Journal of Computer Vision , pages 223–247, 2002.[81] M. P. Pathegama and O. Gol. Edge-end pixel extraction foredge-based image segmentation.
International Journal of Com- uter, Information, Systems and Control Engineering , 1(2):434 –437, 2007.[82] N. Petrov, A. Georgieva, and I. Jordanov. Self-organizingmaps for texture classification.
Neural Computing and Appli-cations , 22(7-8):1499–1508, 2013.[83] S. Raut, M. Raghuvanshi, R. Dharaskar, and A. Raut. Imagesegmentation - a state-of-art survey for prediction. In
Pro-ceedings of the International Conference on Advanced ComputerControl, ICACC ’09 , pages 420–424. IEEE, 2009.[84] F. Sadeghi, H. Izadinia, and R. Safabakhsh. A new activecontour model based on the conscience, archiving and mean-movement mechanisms and the SOM.
Pattern RecognitionLetters , 32(12):1622–1634, 2011.[85] M. Sarma and K. K. Sarma. Segmentation of assamesephonemes using SOM. In
Proceedings of the 3rd National Con-ference on Emerging Trends and Applications in Computer Science(NCETACS) , pages 121–125, 2012.[86] H. Shah-Hosseini and R. Safabakhsh. A TASOM-based algo-rithm for active contour modeling.
Pattern Recognition Letters ,24:1361 – 1373, 2003.[87] Y. Shi and W. C. Karl. Real-time tracking using level sets. In
Proceedings of IEEE International Conference on Computer Visionand Pattern Recognition (CVPR) , volume 2, pages 34–41, vol.2. IEEE, 2005.[88] P. Singla and P. Domingos. Discriminative training of markovlogic networks. In
Proceedings of the 20th National Conferenceon Artificial Intelligence , pages 868–873. AAAI Press, 2005.[89] S. Skakun. A neural network approach to flood mapping us-ing satellite imagery.
Computing and Informatics , 29(6):1013–1024, 2010. 14490] A. Soria-Frisch. Unsupervised construction of fuzzy mea-sures through self-organizing feature maps and its appli-cation in color image segmentation.
International Journal ofApproximate Reasoning , 41(1):23 – 42, 2006.[91] M. F. Talu. ORACM: Online region-based active contourmodel.
Expert Systems with Applications , 40(16):6233 – 6240,2013.[92] D. Tian and L. Fan. A brain MR images segmentation methodbased on SOM neural network. In
Proceedings of the 1st Inter-national Conference on Bioinformatics and Biomedical Engineer-ing (ICBBE). , pages 686–689, 2007.[93] Y. Tian, F. Duan, M. Zhou, and Z. Wu. Active contour modelcombining region and edge information.
Machine Vision andApplications , pages 1–15, August 2011.[94] D. M. Titterington, A. F. M. Smith, and U. E. Makov.
Statisticalanalysis of finite mixture distributions . Wiley, New York, 1985.[95] N. Torbati, A. Ayatollahi, and A. Kermani. An e ffi cient neu-ral network based method for medical image segmentation. Computers in Biology and Medicine , 44(0):76 – 87, 2014.[96] T. Tran, V. Pham, Y. Chiu, and K. Shyu. Active contourwith selective local or global segmentation for intensity in-homogeneous image. In
Proceedings of the 3rd IEEE Interna-tional Conference on Computer Science and Information Technol-ogy (ICCSIT) , volume 1, pages 306–310, 2010.[97] A. Tsai, A. Yezzi, W. Wells, C. Tempany, D. Tucker, A. Fan,W. E. Grimson, and A. Willsky. A shape-based approach tothe segmentation of medical imagery using level sets.
IEEETransactions on Medical Imaging , 22(2):137–154, 2003.[98] A. Tsai, A. Yezzi, and A. S. Willsky. Curve evolution im-plementation of the Mumford-Shah functional for imagesegmentation, denoising, interpolation, and magnification.
IEEE Transactions on Image Processing , 10(8):1169–1186, 2001.14599] S. R. Vantaram and E. Saber. Survey of contemporary trendsin color image segmentation.
Journal of Electronic Imaging ,21(4), 2012.[100] A. Vard, K. Jamshidi, and N. Movahhedinia. Small objectdetection in cluttered image using a correlation based ac-tive contour model.
Pattern Recognition Letters , 33(5):543–553,2012.[101] G. S. Vasconcelos, C. A. C. M. Bastos, I. R. Tsang, and G. D. C.Cavalcanti. BSOM network for pupil segmentation. In
Pro-ceedings of the 2011 IEEE International Joint Conference on NeuralNetworks (IJCNN) , pages 2704–2709, 2011.[102] Y. V. Venkatesh, S. Kumar Raja, and N. Ramya. A novelSOM-based approach for active contour modeling. In
Pro-ceedings of the 2004 Conference on Intelligent Sensors, SensorNetworks and Information Processing (ISSNIP) , pages 229–234,December 2004.[103] Y. V. Venkatesh, K. S. Raja, and N. Ramya. Multiple contourextraction from graylevel images using an artificial neuralnetwork.
IEEE Transactions on Image Processing , 15(4):892–899, 2006.[104] Y. V. Venkatesh and N. Rishikesh. Self-organizing neuralnetworks based on spatial isomorphism for active contourmodeling.
Pattern Recognition , 33(7):1239–1250, 2000.[105] L. A. Vese, T. F. Chan, Tony, and F. Chan. A multiphase levelset framework for image segmentation using the Mumfordand Shah model.
International Journal of Computer Vision ,pages 271–293, 2002.[106] U. Vovk, F. Pernus, and B. Likar. A review of methods for cor-rection of intensity inhomogeneity in MRI.
IEEE Transactionson Medical Imaging , 26(3):405–421, 2007.[107] B. Wang, X. Gao, D. Tao, and X. Li. A nonlinear adaptive levelset for image segmentation.
IEEE Transactions on Cybernetics ,44(3):418–428, 2014. 146108] L. Wang, C. Li, Q. Sun, D. Xia, and C. Kao. Active con-tours driven by local and global intensity fitting energy withapplication to brain MR image segmentation.
ComputerizedMedical Imaging and Graphics , 33(7):520 – 531, 2009.[109] L. Wang, C. Li, Q. Sun, D. Xia, and C. Y. Kao. Active con-tours driven by local and global intensity fitting energy withapplication to brain MR image segmentation.
Computerizedmedical imaging and graphics , 33(7):520–531, 2009.[110] P. Wang, K. Sun, and Z. Chen. Local and global intensityinformation integrated geodesic model for image segmen-tation. In
Proceedings of the 2012 International Conference onComputer Science and Electronics Engineering (ICCSEE) , vol-ume 2, pages 129–132, 2012.[111] W. Wang, L. Zhu, J. Qin, Y. Chui, B. N. Li, and P. Heng.Multiscale geodesic active contours for ultrasound imagesegmentation using speckle reducing anisotropic di ff usion. Optics and Lasers in Engineering , 54(0):105 – 116, 2014.[112] W. Wen, C. He, and M. Li. Transition region-based activecontour model for image segmentation.
Journal of ElectronicImaging , 22(1):013021–013021, 2013.[113] M. Wickramasinghe, J. Rajapakse, and D. Alahakoon. Un-derstanding individual play sequences using growing selforganizing maps.
Neural Information Processing , pages 48–55,2012.[114] X. Xie, C. Wang, A. Zhang, and X. Meng. A robust levelset method based on local statistical information for noisyimage segmentation.
Optik - International Journal for Light andElectron Optics , (0), 2014.[115] C. Xu and J. L. Prince. Snakes, shapes, and gradient vec-tor flow.
IEEE Transactions on Image Processing , 7(3):359–369,1998. 147116] Z. Yang, Z. Bai, J. Wu, and Y. Chen. Target region locationbased on texture analysis and active contour model.
Trans-actions of Tianjin University , 15(3):157–161, 2009.[117] N. C. Yeo, K. H. Lee, Y. V. Venkatesh, and S. H. Ong.Colour image segmentation using the self-organizing mapand adaptive resonance theory.
Image Vision Computational ,23(12):1060–1079, 2005.[118] A. Yezzi, A. Tsai, and A. Willsky. A fully global approachto image segmentation via coupled curve evolution equa-tions.
Journal of Visual Communication and Image Representa-tion , 13:195–216, 2002.[119] J. Yin and J. Yang. A modified level set approach for segmen-tation of multiband polarimetric sar images.
IEEE Transac-tions on Geoscience and Remote Sensing , PP(99):1–11, 2014.[120] J. Yuan, P. Li, Y. Wen, and Y. Xu. Level set segmentationof intensity inhomogeneous images based on local linearapproximation of di ff erence image. IET Image Processing ,6(5):473–482, July 2012.[121] D. Zeng, Z. Zhou, and S. Xie. Coarse-to-fine boundary lo-cation with a SOM-like method.
IEEE Transactions on NeuralNetworks , 21(3):481–493, 2010.[122] K. Zhang, H. Song, and L. Zhang. Active contours drivenby local image fitting energy.
Pattern Recognition , 43(4):1199–1206, 2010.[123] K. Zhang, L. Zhang, H. Song, and D. Zhang. Reinitialization-free level set evolution via reaction di ff usion. IEEE Transac-tions on Image Processing , 22(1):258–271, 2013.[124] K. Zhang, L. Zhang, H. Song, and W. Zhou. Active contourswith selective local or global segmentation: a new formu-lation and level set method.
Image and Vision Computing ,28(4):668–676, 2010. 148125] H. Zhao, T. Chan, B. Merriman, and S. Osher. A variationallevel set approach to multiphase motion.
Journal of Computa-tional Physics , 127(1):179–195, 1996.[126] Q. Zheng, E. Dong, Z. Cao, W. Sun, and Z. Li. Active contourmodel driven by linear speed function for local segmenta-tion with robust initialization and applications in MR brainimages.
Signal Processing , 97(0):117 – 133, 2014.[127] Z. Zhiming and J. Yingying. Application of SOM neuralnetwork in customer segmentation model in coal enterprises.In
Proceedings of the International Forum on Computer Science-Technology and Applications (IFCSTA) , volume 3, pages 451–454, 2009.[128] G. Zhu. Boundary-based image segmentation using binarylevel set method.