A reusable pipeline for large-scale fiber segmentation on unidirectional fiber beds using fully convolutional neural networks
Alexandre Fioravante de Siqueira, Daniela Mayumi Ushizima, Stéfan van der Walt
AA reusable pipeline for large-scale fibersegmentation on unidirectional fiber bedsusing fully convolutional neural networks
Alexandre Fioravante de Siqueira ∗ , Daniela M. Ushizima † ,and St´efan J. van der Walt ‡ Berkeley Institute for Data Science, University of California, Berkeley, USA Lawrence Berkeley National Laboratory, Berkeley, USA
January, 2021
Abstract
Fiber-reinforced ceramic-matrix composites are advanced materi-als resistant to high temperatures, with application to aerospace engi-neering. Their analysis depends on the detection of embedded fibers,with semi-supervised techniques usually employed to separate fiberswithin the fiber beds. Here we present an open computational pipelineto detect fibers in ex-situ X-ray computed tomography fiber beds. Toseparate the fibers in these samples, we tested four different archi-tectures of fully convolutional neural networks. When comparing ourneural network approach to a semi-supervised one, we obtained Diceand Matthews coefficients greater than 92 . ± . . ± . ∗ [email protected] † [email protected] ‡ [email protected] a r X i v : . [ ee ss . I V ] J a n icense, and can be freely adapted and re-used in other domains. Alldata and instructions on how to download and use it are also available. Keywords:
Computer Vision, Deep Learning, Image Segmenta-tion, 3D Analysis, Metrology.
Fiber-reinforced ceramic-matrix composites are advanced materials used inaerospace gas-turbine engines [51, 35] and nuclear fusion [22], due to theirresistance to temperatures 100–200 ◦ C higher than allows for the same ap-plications.Larson et al. investigated new manufacturing processes for curing pre-ceramic polymer into unidirectional fiber beds, studying the microstruc-ture evolution during matrix impregnation and aiming to reinforce ceramic-matrix composites [24, 23]. They used X-ray computed tomography (CT) tocharacterize the three-dimensional microstructure of their composites non-destructively, studying their evolution in-situ while processing the materialsat high temperatures [24] and describing overall fiber bed properties andmicrostructures of unidirectional composites [23]. The X-ray CT images ac-quired from these fiber beds are available at Materials Data Facility [5].Larson et al.’s fiber beds have widths of approximately 1 . mm , containing5000–6200 fibers per stack. Each fiber has an average radius of 6 . ± . µm ,with diameters ranging from 13 to 20 pixels in the micrographs [23]. Theypresent semi-supervised techniques to separate the fibers within the fiberbeds; their segmentation is available for five samples [25]. However, we con-sidered their results could be improved using different techniques. This mo-tivated us to test alternative solutions.In this study we separate fibers in ex-situ X-ray CT fiber beds of ninesamples from Larson et al. The samples we used in this study correspondto two general states: wet — obtained after pressure removal — and cured.These samples were acquired using microtomographic instruments from theAdvanced Light Source at Lawrence Berkeley National Laboratory operatedin a low-flux, two-bunch mode [23]. We used their reconstructions obtainedwithout phase retrieval; Larson et al. provide segmentations for five of thesesamples [25], which we compare to our results.To separate the fibers in these samples, we tested four different fullyconvolutional neural networks (CNN, section 4.1), algorithms from computer2ision and deep learning. When comparing our neural network approach toLarson et al. results, we obtained Dice [13] and Matthews [30] coefficientsgreater than 92 . ± . . ± . Larson et al. provide segmentations for their fibers (Fig 1) in five of the wetand cured samples, obtained using the following pipeline [23]:1. Fiber detection using the circular Hough transform [48, 3];2. Correction of improperly identified pixels using filters based on con-nected region size and pixel value, and by comparisons using ten slicesabove and below the slice of interest;3. Separation of fibers using the watershed algorithm [31].However, their proposed method briefly describes these steps. There areno details on parameters used, or the source code for their segmentation. Wetried different approaches to reproduce their results, focusing on separatingthe fibers in the fiber bed samples. Our first approach was to create a classic,unsupervised image processing pipeline. We used histogram equalization [45],Chambolle’s total variation denoising [38, 7], multi-Otsu threshold [34, 28],and the WUSEM algorithm [12] to separate each single fiber. The resultis a labeled image containing the separated fibers (Fig 2). The pipelinepresented limitations when processing fibers on the edges of fiber beds, notbeing equivalent to the solution presented by Larson et al. We restricted thesegmentation region to have a satisfactory result (Fig 2(d)), but the numberof detected fibers is reduced.To obtain more accurate results, we evaluated four fully convolutionalneural network architectures: Tiramisu [19] and U-net [37], as well as their3Figure 1: Slice number 1000 from the sample “232p3 wet”, provided in [25].The whole sample contains 2160 slices. This slice represents the structure ofthe samples we processed: they contain the fiber bed (large circular structure)and the fibers within it (small round elements).three-dimensional counterparts, 3D Tiramisu and 3D U-net [52]. We also in-vestigated whether three-dimensional networks generate better segmentationresults, leveraging the structure of the material.
We implemented four architectures of fully convolutional neural networks(CNN) — Tiramisu, U-net, 3D Tiramisu, and 3D U-net — to reproducethe results provided by Larson et al. Labeled data, in our case, consists4f fibers within fiber beds. To train the neural networks to recognize thesefibers, we used slices from two different samples: 232p3 wet and 232p3 cured,registered according to the wet sample. Larson et al. provided the fibersegmentation for these samples [25], which we used as labels in the training.The training and validation datasets contained 250 and 50 images from eachsample, respectively, in a total of 600 images. Each image from the originalsamples have width and height size of 2560 × • “232p1”: wet • “232p3”: wet, cured, cured registered • “235p1”: wet • “235p4”: wet, cured, cured registered • “244p1”: wet, cured, cured registered • “245p1”: wetHere, the first three numeric characters correspond to a material sample,and the last character correspond to different extrinsic factors, e.g. deforma-tion. Despite being samples from similar materials, the reconstructed files5resented several differences, for example regarding amount of ringing ar-tifacts, intensity variation, noise, therefore they are considered as differentsamples in this paper.We calculated the average processing time for each sample (Fig 5). Theprediction time results are similar to the training ones; 2D U-net and 2DTiramisu are the fastest architectures to process a sample, while 3D Tiramisuis the slowest. After processing all samples, we compared our predictions with the resultsthat Larson et al. made available on their dataset [25]. They provided fivedatasets from the twelve we processed: “232p1 wet” , “232p3 cured” , “232p3wet” , “244p1 cured” , “244p1 wet” .First, we compared our predictions to their results using receiver oper-ating characteristic (ROC) curves and the area under curve (AUC, Fig 6).AUC is larger than 98% for all comparisons; therefore, our predictions are ac-curate when compared with the semi-supervised method suggested by Larsonet al. The 2D versions of U-net and Tiramisu have similar results, performingbetter than 3D U-net and 3D Tiramisu.We also examined the binary versions of our predictions and comparedthem with Larson et al. results. For each slice from the dataset, similarly tothe volume, we used a hard threshold of 0 .
5; values above that are consideredas fibers, while values below that are treated as background. We used Dice[13] and Matthews [30] correlation coefficients for our comparison (1). Thecomparison using U-net yields the highest Dice and Matthews coefficients forthree of five datasets. Tiramisu had highest Dice/Matthews coefficients forthe “244p1, cured” dataset, and both networks have approximate results for“232p1, wet”. 3D Tiramisu had the lowest Dice and Matthews coefficientsin our comparison.
The analysis of ceramic matrix composites (CMC) depends on the detec-tion of its fibers. Semi-supervised algorithms such as the one presented byLarson et al [23] can perform satisfactorily for that end. However, their6 iramisu U-net 3D Tiramisu 3D U-netSample Dice Matthews Dice Matthews Dice Matthews Dice Matthews232p1, wet . ± .
29% 96 . ± .
93% 97 . ± .
20% 96 . ± .
13% 94 . ± .
73% 92 . ± .
65% 95 . ± .
74% 93 . ± . . ± .
04% 97 . ± .
06% 98 . ± .
04% 97 . ± .
06% 95 . ± .
36% 93 . ± .
88% 95 . ± .
00% 94 . ± . . ± .
15% 96 . ± .
70% 97 . ± .
12% 96 . ± .
99% 94 . ± .
90% 92 . ± .
87% 95 . ± .
97% 93 . ± . . ± .
03% 97 . ± .
05% 98 . ± .
04% 97 . ± .
05% 94 . ± .
74% 92 . ± .
54% 96 . ± .
25% 94 . ± . . ± .
53% 97 . ± .
15% 98 . ± .
39% 97 . ± .
23% 94 . ± .
81% 92 . ± .
71% 96 . ± .
00% 95 . ± . Table 1: Dice and Matthews coefficients for each sample, obtained from thecomparison of our neural network results and data from Larson et al [25]. U-net yields the highest Dice and Matthews coefficients for three of five samples.Tiramisu had highest Dice/Matthews coefficients for one of the datasets. 3DTiramisu had the lowest Dice and Matthews coefficients.specific algorithm lack information on the parameters necessary for replica-tion. Reimplementing such methods without that information would lead toinaccurate results, since the reported approach includes manual steps thatrequire human curation.Convolutional neural networks are being used successfully in the segmen-tation of different two- and three-dimensional scientific data (e.g., [4, 43, 16,29, 39, 27]), including microtomographies. For example, fully convolutionalneural networks were used to generate 3D tau inclusion density maps [2],to segment the tidemark on osteochondral samples [42], and 3D models ofstructures of temporal-bone anatomy [33].Researchers are studying fiber-analysis detection for a while, using differ-ent tools. There are several approaches using tracking, statistical approaches,or classical image processing (e.g., [10, 6, 40, 44, 50, 14, 15, 9]). To the bestof our knowledge, there are two different deep learning approaches for thisproblem: • Yu et al. [47] use an unsupervised learning approach based on FasterR-CNN [36] and a Kalman filter based tracking. They compare theirresults with Zhou et al. [50], reaching a Dice coefficient of up to 99 %. • Miramontes et al. [32] reach an average accuracy of 93.75% using a 2DLeNet-5 CNN [26] to detect fibers in a specific sample.Our study builds upon previous work by using similar material samples,but it expands tests to many more samples as well as it includes the im-plemention and training of four architectures: 2D U-net, 2D Tiramisu, 3DU-net, and 3D Tiramisu, used to process twelve large datasets ( ≈
140 GB),and comparing our results with the gold standard data provided by Larson7t al. [25] for five of them. We used ROC curves and their area under curve(AUC) to ensure the quality of our predictions, obtaining AUC larger than98% (Fig 6). Also, Dice and Matthews coefficients were used to compare ourresults with Larson et al’s solutions (Table 1), reaching coefficients of up to98 . ± . .
5, that suited the sigmoid on the lastlayer of the CNN we implemented. We could also use conditional random8eld networks for that end.
We implemented four architectures — two dimensional U-net [37] and Tiramisu[19], and their three-dimensional versions — to attempt reproducing the re-sults provided by Larson et al. We used supervised algorithms: they rely onlabeled data to learn what are the regions of interest — in our case, fiberswithin microtomographies of fiber beds.All CNN algorithms were implemented using TensorFlow [1] and Keras[8] on a computer with two Intel Xeon Gold processors 6134 and two NvidiaGeForce RTX 2080 graphical processing units. Each GPU has 10 GB ofRAM.To train the neural networks on how to recognize the fibers, we usedslices from two different samples: “232p3 wet” and “232p3 cured”, registeredaccording to the wet sample. Larson et al. provided the fiber segmentationfor these samples, which we used as labels in the training. The trainingand validation procedures processed 350 and 149 images from each sample,respectively; a total of 998 images. Each image from the original sampleshave width and height size of 2560 × × ×
288 pixels, in a total of 50,000 images for the training set,and 10,000 for the validation set.We needed to pre-process the training images differently to train thethree-dimensional networks. We loaded the entire samples, each with size2160 × × × ×
64 voxels, each 32 pixels. Hence, the training andvalidation sets for the three-dimensional networks have 96,000 and 19,200cubes, respectively.We implemented data augmentation in our pipeline, aiming for a networkcapable of processing samples with different characteristics. We augmented9he images on the training sets using rotations, horizontal and vertical flips,width and height shifts, zoom and shear transforms. For that, we used Kerasembedded tools within the
ImageDataGenerator module to augment imagesfor the two-dimensional networks. Since Keras’s
ImageDataGenerator is notable to process three-dimensional input so far, we adapted the
ImageDataGenerator module. The adapted version we used in this study is named
ChunkDataGenerator ,and is available in the Supplementary Material.To reduce the possibility of overfitting, we implemented dropout regu-larization [41] in our pipeline. We followed the suggestions in the originalpapers for U-net architectures: 2D U-net received a dropout rate of 50% inthe last analysis layer and in the bottleneck, while 3D U-net [52] did not re-ceive any dropout. The Tiramisu structures received a dropout rate of 20%,as suggested by J´egou et al [19].For a better comparison, we maintained the same training hyperparam-eters when possible. Due to the large amount of training data and the sim-ilarities between training samples (2D tiles or 3D cubes), our preliminarytests indicated that we would have a higher accuracy for all networks in thefirst training epochs. Therefore, we decided to train all architectures duringfive epochs. The 2D architectures were trained with batches of four images,while the batches for 3D architectures had two cubes each. For all archi-tectures, we used a learning rate of 1 E −
4, and binary cross entropy [49]as the loss function. We followed the original papers regarding to optimiza-tion algorithms: we used the Adam optimizer [20] in the U-net architectures,while the Tiramisu ones were trained using the RMSProp optimizer [11].We implemented batch normalization [18] in all architectures, including the2D U-net. Ronneberger et al. do not suggest it in their preliminary study,although it is known that architectures using batch normalization tend toconverge faster.
We used Dice [13] and Matthews [30] correlation coefficients (Equations 1, 2])to evaluate our results, assuming that the fiber detections from [25] containa reasonable gold standard.
Dice = 2 × T P × T P + F P + F N (1)10 atthews = T P × T N − F P × F N (cid:113) ( T P + F N )( T P + F P )( T N + F N )( T N + F P ) (2)Dice and Matthews coefficients receive true positive (TP), false positive(FP), true negative (TN), and false negative (FN) pixels, which are deter-mined as: • TP: pixels correctly labeled as being part of a fiber. • FP: pixels incorrectly labeled as being part of a fiber. • TN: pixels correctly labeled as background. • FN: pixels incorrectly labeled as background.TP, FP, TN, and FN are obtained when the prediction data is comparedwith a certain gold standard, which in this study is Larson’s semi-supervisedsegmentation data [25].
Imaging CMC specimens at high-resolution as Larson et al samples [25] leadsto large datasets — each stack we used in this paper has around 14 GB afterthe reconstruction, for example .Frequently, the specialist needs software to visualize the result of theirdata collection, but most of them fail to produce meaningful graphs withoutconsidering advanced image analysis and/or computational platforms withgenerous amounts of memory. One may use Jupyter Notebooks [21], whichenable domain scientists to quickly probe specimens imaged with X-ray mi-croCT during their beamtime. For this reason, the figures in this paper areall generated on standard laptops with no more than 16 GB of RAM, whichis the typical computation system at hand.We used matplotlib [17] and ITK [46] (Fig 9) to generate our figures.Despite our use of methods that consider either global or local information,we designed protocols that allow any user to visualize essential content fromtheir experiments recorded as 3D image stacks. The exceptions are the registered versions of cured samples 232p3, 235p4 and 244p1,with 11 GB each, and the sample 232p3 wet with around 6 GB. DATA AVAILABILITY
The supplementary data generated in this study is available at https://datadryad.org/stash/dataset/doi:10.6078/D1069R , under a CC0 (pub-lic domain) license.
The software we produced throughout this study is available at https://github.com/alexdesiqueira/fcn_microct/ , under a BSD license.
AFS would like to thank Sebastian Berg, Ross Barnowski, Silvia Miramontes,Ralf Gommers, and Matt Rocklin for the discussions on fully convolutionalnetworks, their structure and different frameworks. This research was fundedin part by the Gordon and Betty Moore Foundation through Grant GBMF3834and by the Alfred P. Sloan Foundation through Grant 2013-10-27 to the Uni-versity of California, Berkeley.
References [1] Mart´ın Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis,Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving,Michael Isard, and et al. Tensorflow: a system for large-scale machinelearning. In
Proceedings of the 12th USENIX conference on OperatingSystems Design and Implementation , OSDI’16, page 265–283. USENIXAssociation, Nov 2016.[2] Maryana Alegro, Yuheng Chen, Dulce Ovando, Helmut Heinser, RanaEser, Daniela Ushizima, Duygu Tosun, and Lea T. Grinberg. Deeplearning for alzheimer’s disease: Mapping large-scale histological tauprotein for neuroimaging biomarker validation. bioRxiv , page 698902,May 2020.[3] T. J. Atherton and D. J. Kerbyson.
Size Invariant Circle Detection .1999. 124] Samik Banerjee, Lucas Magee, Dingkang Wang, Xu Li, Bing-XingHuo, Jaikishan Jayakumar, Katherine Matho, Meng-Kuan Lin, KeerthiRam, Mohanasankar Sivaprakasam, and et al. Semantic segmenta-tion of microscopic neuroanatomical data by combining topological pri-ors with encoder–decoder deep networks.
Nature Machine Intelligence ,2(1010):585–594, Oct 2020.[5] B. Blaiszik, K. Chard, J. Pruyne, R. Ananthakrishnan, S. Tuecke, andI. Foster. The materials data facility: Data services to advance materialsscience research.
JOM , 68(8):2045–2052, August 2016.[6] Stephen Bricker, J. P. Simmons, Craig Przybyla, and Russell Hardie.Anomaly detection of microstructural defects in continuous fiber rein-forced composites. page 94010A, Mar 2015.[7] A. Chambolle. An algorithm for total variation minimization and ap-plications.
Journal of Mathematical Imaging and Vision , 20(1):89–97,2004.[8] Fran¸cois Chollet et al. Keras. https://keras.io , 2015.[9] Peter J. Creveling, William W. Whitacre, and Michael W. Czabaj. Afiber-segmentation algorithm for composites imaged using x-ray micro-tomography: Development and validation.
Composites Part A: AppliedScience and Manufacturing , 126:105606, Nov 2019.[10] Michael W. Czabaj, Mark L. Riccio, and William W. Whitacre. Numeri-cal reconstruction of graphite/epoxy composite microstructure based onsub-micron resolution x-ray computed tomography.
Composites Scienceand Technology , 105:174–182, Dec 2014.[11] Yann N. Dauphin, Harm de Vries, and Yoshua Bengio. Equilibratedadaptive learning rates for non-convex optimization. arXiv:1502.04390[cs] , Feb 2015. arXiv: 1502.04390.[12] Alexandre Fioravante de Siqueira, Wagner Massayuki Nakasuga, San-dro Guedes, and Lothar Ratschbacher. Segmentation of nearly isotropicoverlapped tracks in photomicrographs using successive erosions as wa-tershed markers.
Microscopy Research and Technique , 0(0):0, 2019.1313] Lee R. Dice. Measures of the amount of ecologic association betweenspecies.
Ecology , 26(3):297–302, Jul 1945.[14] Monica J. Emerson, Kristine M. Jespersen, Anders B. Dahl, Knut Con-radsen, and Lars P. Mikkelsen. Individual fibre segmentation from 3dx-ray computed tomography for characterising the fibre orientation inunidirectional composite materials.
Composites Part A: Applied Scienceand Manufacturing , 97:83–92, Jun 2017.[15] Monica Jane Emerson, Vedrana Andersen Dahl, Knut Conradsen,Lars Pilgaard Mikkelsen, and Anders Bjorholm Dahl. Statistical valida-tion of individual fibre segmentation from tomograms and microscopy.
Composites Science and Technology , 160:208–215, May 2018.[16] James P. Horwath, Dmitri N. Zakharov, R´emi M´egret, and Eric A.Stach. Understanding important features of deep learning models forsegmentation of high-resolution transmission electron microscopy im-ages. npj Computational Materials , 6(11):1–9, Jul 2020.[17] J. D. Hunter. Matplotlib: A 2d graphics environment.
Computing inScience & Engineering , 9(3):90–95, 2007.[18] Sergey Ioffe and Christian Szegedy. Batch normalization: Accel-erating deep network training by reducing internal covariate shift. arXiv:1502.03167 [cs] , Mar 2015. arXiv: 1502.03167.[19] Simon J´egou, Michal Drozdzal, David Vazquez, Adriana Romero, andYoshua Bengio. The one hundred layers tiramisu: Fully convolutionaldensenets for semantic segmentation. arXiv:1611.09326 [cs] , Oct 2017.arXiv: 1611.09326.[20] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochasticoptimization. arXiv:1412.6980 [cs] , Jan 2017. arXiv: 1412.6980.[21] Thomas Kluyver, Benjamin Ragan-Kelley, Fernando P´erez, BrianGranger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, JessicaHamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Dami´an Avila, SafiaAbdalla, and Carol Willing. Jupyter notebooks – a publishing format forreproducible computational workflows. In F. Loizides and B. Schmidt,editors,
Positioning and Power in Academic Publishing: Players, Agentsand Agendas , pages 87 – 90. IOS Press, 2016.1422] T. Koyanagi, Y. Katoh, T. Nozawa, L. L. Snead, S. Kondo, C. H.Henager, M. Ferraris, T. Hinoki, and Q. Huang. Recent progress in thedevelopment of sic composites for nuclear fusion applications.
Journalof Nuclear Materials , 511:544–555, Dec 2018.[23] Natalie M. Larson, Charlene Cuellar, and Frank W. Zok. X-ray com-puted tomography of microstructure evolution during matrix impregna-tion and curing in unidirectional fiber beds.
Composites Part A: AppliedScience and Manufacturing , 117:243–259, February 2019.[24] Natalie M. Larson and Frank W. Zok. In-situ 3d visualization of com-posite microstructure during polymer-to-ceramic conversion.
Acta Ma-terialia , 144:579–589, Feb 2018.[25] Natalie M. Larson and Frank W. Zok. Ex-situ xct datasetfor ”x-ray computed tomography of microstructure evolution dur-ing matrix impregnation and curing in unidirectional fiber beds”.http://dx.doi.org/doi:10.18126/M2QM0Z, 2019.[26] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basedlearning applied to document recognition.
Proceedings of the IEEE ,86(11):2278–2324, Nov 1998.[27] Wei Li, Kevin G. Field, and Dane Morgan. Automated defect analysisin electron microscopic images. npj Computational Materials , 4(11):1–9,Jul 2018.[28] P.-S. Liao, T.-S. Chen, and P.-C. Chung. A fast algorithm for mul-tilevel thresholding.
Journal of Information Science and Engineering ,17(5):713–727, 2001.[29] Boyuan Ma, Xiaoyan Wei, Chuni Liu, Xiaojuan Ban, Haiyou Huang,Hao Wang, Weihua Xue, Stephen Wu, Mingfei Gao, Qing Shen, andet al. Data augmentation in microscopic images for material data min-ing. npj Computational Materials , 6(11):1–9, Aug 2020.[30] Brian W. Matthews. Comparison of the predicted and observed sec-ondary structure of t4 phage lysozyme.
Biochimica et Biophysica Acta(BBA) - Protein Structure , 405(2):442–451, 1975.1531] Fernand Meyer. Topographic distance and watershed lines.
Signal Pro-cessing , 38(1):113–125, July 1994.[32] Silvia Miramontes, Daniela M. Ushizima, and Dilworth Y. Parkinson.Evaluating fiber detection models using neural networks. In George Be-bis, Richard Boyle, Bahram Parvin, Darko Koracin, Daniela Ushizima,Sek Chai, Shinjiro Sueda, Xin Lin, Aidong Lu, Daniel Thalmann, andet al., editors,
Advances in Visual Computing , Lecture Notes in Com-puter Science, page 541–552. Springer International Publishing, 2019.[33] Soodeh Nikan, Sumit K. Agrawal, and Hanif M. Ladak. Fully automatedsegmentation of the temporal bone from micro-ct using deep learning.In
Medical Imaging 2020: Biomedical Applications in Molecular, Struc-tural, and Functional Imaging , volume 11317, page 113171U. Interna-tional Society for Optics and Photonics, Feb 2020.[34] N. Otsu. A threshold selection method from gray-level histograms.
IEEETransactions on Systems, Man and Cybernetics , 9(1):62–66, 1979.[35] Nitin P. Padture. Advanced structural ceramics in aerospace propulsion.
Nature Materials , 15(8):804–809, Aug 2016.[36] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Fasterr-cnn: Towards real-time object detection with region proposal net-works.
IEEE transactions on pattern analysis and machine intelligence ,39(6):1137–1149, 2017.[37] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolu-tional Networks for Biomedical Image Segmentation. In Nassir Navab,Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors,
Medical Image Computing and Computer-Assisted Intervention – MIC-CAI 2015 , Lecture Notes in Computer Science, pages 234–241. SpringerInternational Publishing, 2015.[38] Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total vari-ation based noise removal algorithms.
Physica D: Nonlinear Phenomena ,60(1):259–268, 1992.[39] Yu Saito, Kento Shin, Kei Terayama, Shaan Desai, Masaru Onga, YujiNakagawa, Yuki M. Itahashi, Yoshihiro Iwasa, Makoto Yamada, and16oji Tsuda. Deep-learning-based quality filtering of mechanically exfo-liated 2d crystals. npj Computational Materials , 5(11):1–6, Dec 2019.[40] R. M. Sencu, Z. Yang, Y. C. Wang, P. J. Withers, C. Rau, A. Parson,and C. Soutis. Generation of micro-scale finite element models fromsynchrotron x-ray ct images for multidirectional carbon fibre reinforcedcomposites.
Composites Part A: Applied Science and Manufacturing ,91:85–95, Dec 2016.[41] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, andRuslan Salakhutdinov. Dropout: a simple way to prevent neural net-works from overfitting.
The Journal of Machine Learning Research ,15(1):1929–1958, Jan 2014.[42] Aleksei Tiulpin, Mikko Finnil¨a, Petri Lehenkari, Heikki J. Nieminen,and Simo Saarakkala. Deep-learning for tidemark segmentation in hu-man osteochondral tissues imaged with micro-computed tomography. InJacques Blanc-Talon, Patrice Delmas, Wilfried Philips, Dan Popescu,and Paul Scheunders, editors,
Advanced Concepts for Intelligent VisionSystems , Lecture Notes in Computer Science, page 131–138. SpringerInternational Publishing, 2020.[43] Yuta Tokuoka, Takahiro G. Yamada, Daisuke Mashiko, Zenki Ikeda,Noriko F. Hiroi, Tetsuya J. Kobayashi, Kazuo Yamagata, and AkiraFunahashi. 3d convolutional neural networks-based segmentation to ac-quire quantitative criteria of the nucleus during mouse embryogenesis. npj Systems Biology and Applications , 6(11):1–12, Oct 2020.[44] Daniela M. Ushizima, Hrishikesh A. Bale, E. Wes Bethel, Peter Ercius,Brett A. Helms, Harinarayan Krishnan, Lea T. Grinberg, Maciej Ha-ranczyk, Alastair A. Macdowell, Katarzyna Odziomek, and et al. Ideal:Images across domains, experiments, algorithms and learning.
JOM ,68(11):2963–2972, Nov 2016.[45] R.E. Woods and R.C. Gonzalez. Real-time digital image enhancement.
Proceedings of the IEEE , 69(5):643–654, May 1981.[46] Terry S Yoo, Michael J Ackerman, William E Lorensen, Will Schroeder,Vikram Chalana, Stephen Aylward, Dimitris Metaxas, and RossWhitaker. Engineering and algorithm design for an image processing17pi: A technical report on itk - the insight toolkit.
Studies in healthtechnology and informatics , pages 586–592, 2002.[47] Hongkai Yu, Dazhou Guo, Zhipeng Yan, Wei Liu, Jeff Simmons, Craig P.Przybyla, and Song Wang. Unsupervised learning for large-scale fiber de-tection and tracking in microscopic material images. arXiv:1805.10256[cs] , May 2018. arXiv: 1805.10256.[48] H. K. Yuen, J. Princen, J. Dlingworth, and J. Kittler. A ComparativeStudy of Hough Transform Methods for Circle Finding. In
Procedingsof the Alvey Vision Conference 1989 , pages 29.1–29.6, Reading, 1989.Alvey Vision Club.[49] Zhilu Zhang and Mert R. Sabuncu. Generalized cross entropy loss fortraining deep neural networks with noisy labels. In
Proceedings of the32nd International Conference on Neural Information Processing Sys-tems , NIPS’18, page 8792–8802. Curran Associates Inc., Dec 2018.[50] Youjie Zhou, Hongkai Yu, Jeff Simmons, Craig P. Przybyla, and SongWang. Large-scale fiber tracking through sparsely sampled image se-quences of composite materials.
IEEE Transactions on Image Process-ing , 25(10):4931–4942, Oct 2016.[51] Frank W Zok. Ceramic-matrix composites enable revolutionary gains inturbine engine efficiency.
American Ceramic Society Bulletin , 95(5):7,2016.[52] ¨Ozg¨un C¸ i¸cek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox,and Olaf Ronneberger. 3d u-net: Learning dense volumetric segmenta-tion from sparse annotation. arXiv:1606.06650 [cs] , Jun 2016. arXiv:1606.06650. 18Figure 2: Rendering fibers detected in the limited region of interest by theclassic pipeline. We exemplify the classic image processing pipeline usingFig 1 as the input image. This solution presented limitations when process-ing fibers on the edfes of fiber beds. (a)
Histogram equalization and TVChambolle’s filtering (parameter: weight=0.3 ). (b) Multi Otsu’s resultingregions (parameter: classes=4 ). Fibers are located within the fourth re-gion (in yellow). (c)
Binary image obtained considering region four in (b)as the region of interest, and the remaining regions as the background. (d) the processed region from (c), as shown in Fig 1. (e)
Regions resultingfrom the application of WUSEM on the region shown in (d) (parameters: initial radius=0 , delta radius=2 , watershed line=True ). Colormaps:(a, c, d) gray , (b) viridis , (e) nipy spectral .19 (a)(b) Tiramisu U-net 3D Tiramisu 3D U-net
Accuracy
Tiramisu U-net 3D Tiramisu 3D U-net
Loss
Figure 3: Accuracy (a) and loss (b) through time for each training epoch.All networks were trained during five epochs, reaching accuracy higher than0.9 and loss lower than 0.1 on the first training epoch, except for the two-dimensional U-net. However, 2D U-net is the fastest to finish training, andreaches the lowest loss between the candidates. We attribute the subtleloss increase or accuracy decrease on the start of each epoch to the dataaugmentation process. 20
Loss
Tiramisu U-net 3D Tiramisu 3D U-net
Figure 4: Accuracy vs. loss on the first epoch. Accuracy surpasses 0.9 andloss is lower than 0.1 for all networks during the first epoch, except for 2D U-net (loss of 0.23). The large size of the training set and the similarities in thedata are responsible for such numbers. Validation accuracy and validationloss on the first epoch are represented by diamonds.21
True positive rate
Tiramisu (AUC : 99.8367%)U-net (AUC : 99.8396%) 3D Tiramisu (AUC : 99.375%)3D U-net (AUC : 99.6882%) (a)
True positive rate
Tiramisu (AUC : 99.9422%)U-net (AUC : 99.9482%) 3D Tiramisu (AUC : 99.5499%)3D U-net (AUC : 99.7842%) (b)
True positive rate
Tiramisu (AUC : 99.8787%)U-net (AUC : 99.8911%) 3D Tiramisu (AUC : 99.4246%)3D U-net (AUC : 99.7389%) (c)
True positive rate
Tiramisu (AUC : 99.959%)U-net (AUC : 99.9597%) 3D Tiramisu (AUC : 99.439%)3D U-net (AUC : 99.8415%) (d)
True positive rate
Tiramisu (AUC : 99.8999%)U-net (AUC : 99.9056%) 3D Tiramisu (AUC : 99.4688%)3D U-net (AUC : 99.8632%) (e)
Figure 5: Mean and standard deviation for prediction times for each sam-ple. As with processing, during training 2D U-net and 2D Tiramisu werethe fastest architectures to process a sample in one hour, on average. 3DTiramisu, being the slowest, takes in average more than a day to process onesample. 22
Tiramisu U-net 3D Tiramisu 3D U-net05101520
Time (hours)
Figure 6: Receiver operating characteristic (ROC) and area under curve(AUC) from the comparison between the prediction for each network andthe segmentation made available for five samples by Larson et al [25]. ROCcurves were calculated to all slices in a dataset; their mean areas and standarddeviation intervals are presented. AUC is larger than 98% in all comparisons,showing that our predictions are accurate when compared with Larson et al.semi-supervised method. The 2D versions of U-net and Tiramisu performbetter when compared to their 3D alternatives.23Figure 7: A defective slice on the sample “232p3 wet” and the segmentationresulting from each architecture. While the 2D architectures results are im-paired by the defects present in the input image, the 3D ones leverage fromthe sample structure to present a better segmentation result. (a)
Originaldefective image, (b)
U-net prediction, (c)
3D U-net prediction, (d)
Tiramisuprediction, (e)
3D Tiramisu prediction.24 (a) (b) (c)
Figure 8: Visual comparison between 2D U-net and Larson et al. resultsfor sample “232p3 wet”. Each part of this image is obtained combiningboth ours and Larson et al.’s results; we compared each slice, and presentedthe ones that return the lowest Matthews comparison coefficient. Labelspresent the Matthews coefficient for each slice. (b, c) slices presenting fibersfound only by U-net (in red), while some well-defined structures close to theborders are found only by Larson et al. (in yellow). Slice size: 256 ××