A Systematic Approach for MRI Brain Tumor Localization, and Segmentation using Deep Learning and Active Contouring
Shanaka Ramesh Gunasekara, H.N.T.K.Kaldera, Maheshi B. Dissanayake
HHindawi Template version: Apr19 A Systematic Approach for MRI Brain Tumor Localization, and Segmentation using Deep Learning and Active Contouring
Shanaka Ramesh Gunasekara *, H.N.T.K.Kaldera , Maheshi B. Dissanayake * Dept. Of Electrical and ElectronicEngineering,Faculty of Engineering, University ofPeradeniya, 20400, Sri Lanka. Correspondence should be addressed to Shanaka R.Gunasekara; [email protected] ,orMaheshi B. Dissanayake; [email protected]
Abstract
One of the main requirements of tumor extraction is the annotation and segmentation of tumor boundaries correctly. For this purpose, we present a threefold deep learning architecture. First classifiers are implemented with a deep convolutional neural network(CNN) andsecond a region-based convolutional neural network (R-CNN) is performed on the classified images to localize the tumor regions of interest. As the third and final stage, the concentratedtumor boundary is contoured for the segmentation process by using the Chan-Vesesegmentation algorithm. As the typical edge detection algorithms based on gradients of pixel intensity tend to fail in the medical image segmentation process, an active contour algorithm defined with the level set function is proposed. Specifically, Chan- Vese algorithm was applied to detect the tumor boundaries for the segmentation process. To evaluate the performance of the overall system, Dice Score,Rand Index (RI), Variation of Information (VOI), Global Consistency Error (GCE), Boundary Displacement Error (BDE), Mean absolute error (MAE), and Peak Signal to Noise Ratio (PSNR) werecalculated by comparing the segmented boundary area which is the final output of the proposed, against the demarcations of the subject specialists which is the gold standard. Overall performance of the proposed architecture for both glioma and meningioma segmentation is with average dice score of 0.92, (also, with RI of 0.9936, VOI of 0.0301, GCE of 0.004, BDE of 2.099, PSNR of 77.076 and MAE of 52.946), pointing to high reliability of the proposed architecture.
Introduction
Medical image classification and segmentation is a field, where Deep Learning can make a huge impact with promising results. It facilitates automation of non-invasive imaging based diagnosis. Interestingly, computer aided brain tumor diagnosis has effectively utilized the advances in medical image processing in the past and has opened up many promising research activities in the domain of deep learning, with the expectation of developing entirely computerized automatic accurate diagnostic systems for physicians. A braintumor is a mass or growth of abnormal cells in the brain which might be cancerous (malignant) or non-cancerous (benign). The early, comprehensive diagnosis and proper treatments are essential for a patient’s survival in brain tumor management. During the past decades, more than 120 types of brain tumors were discovered by medical scientists. These brain tumors can be broadly categorized into two main groups, namely; primary brain tumors, which originate in the brain itself and secondary deposits in the brain, where primary indawi Template version: Apr19 indawi Template version: Apr19
We present a simplified CNN architecture based on small number oflayers and faster R-CNN, for the classification of axial MRI into glioma, and meningioma brain tumors, and produce a bounding box of the tumor with 94% of accuracy confidence level [21][22]. One of the key challenges in medical image analysis is the scarcity of labelled data. Hence, in this research we specifically focus on applying R-CNN based tumorlocalisation for ascenario with lower number of annotated data. Specifically, we have used a dataset with less than 500axial MRI images for our research.Furthermore, we kept our network simple, to reduce the total number of trainable parameters. 2.
The exaction of the exact tumor boundary was performed by means of unsupervised active contour detection method, developed by means of Chan and Vese[26] algorithm.One of the inherent drawbacks of using active contour algorithms for segmentation is the requirement of initial search area. Without correct initial search area, the algorithm is at the risk of segmenting unwanted regions in the brain MRI, such as orbital cavity and lateral ventricles. In this research, we used tumor bounding box coordinates obtained at faster R-CNN based localisation step as the initial search area for Chan-Vesealgorithm to obtain a refined segmentation of the tumor.This in turn reduces the search area of the Chan-Vese algorithm and helps the algorithm to converge to an accurate boundary in a shorter time frame.The resultant segmented boundary carries a high accuracy according to objective quality evaluation measures, compared to the state of the art. In this article we were able to present an end to end complete systematic approach for both meningioma and glioma tumor detection and segmentation using MRI. The complete system indawi Template version: Apr19
Architecture of the ProposedAlgorithm
In this paper, we propose a threefold completearchitecture to classify and segment brain tumors using T1 weighted MRI sequence. The proposed system architecture consists of three cascaded algorithms; namely in the orderof application, Convolutional neural network for classification, Faster R-CNN network for tumor localization, Chan-Vese algorithm for precisetumor segmentation.The flow diagram of the complete architecture is illustrated in Figure 1.
Figure 1: Proposed algorithm indawi Template version: Apr19
5 Initially, a down sampled MR image is fed into a typical CNN to be classified as Meningioma or Glioma. Then at the second stage, the classified image goes through a faster RCNN networkfor tumor localization. The faster RCNN model adopted uses pre trained weights generated using COCO net data for its feature extracting CNN, which isfollowed by a Region Proposal network (RPN) and a classifier. The output of this second step is a boundary box around the tumor in 128*128 down sampled image. As the third step, this boundary box coordinates are mapped to 512*512 image with the original resolution, and Chan-Vese algorithm is applied only for the boundary box area. This approach assures that the Chan-Vesealgorithm converges to an accurate boundary. Hence, we were able to obtain a much precious boundary for tumor segmentation within a low computational time. The outputs obtained at the step 2 and 3 are presented in Figure 2.
Background Works
Basic Operation of CNN and R-CNN
CNN is a class of layered deep neural network architecture built using convolution, activation, pooling, and fully connected layers to analyse visual imagery. The convolutional layer uses set of learnable filters with different sizes to extract various feature maps to learn the correlation between neighbouring pixels, while drastically reducing the number of weighted parameters. The pooling layer introduce non-linear down sampling to the system architecture while the activation increase the nonlinear properties of the decision function of the overall network independent of the convolution layer. Followed by several combinations of convolution, pooling and activation, CNN has the fully connected layers, where high-level decision making takes place. At the final stage of the design dense layer or loss layer maps the trained outcome with the predefined output class. In a fully connected CNN architecture, these operations are executed forward and backward, through forward learning and back
Figure 2. (a)Brain tumor localization bounding box obtained at the step 2 (after Faster RCNN) (b) Tumor segmentation output obtained at the step 3 (after Chan-Vese). (a) (b) indawi Template version: Apr19 (cid:1)(cid:2)(cid:3)(cid:4) (cid:5) (cid:6), (cid:3)(cid:8) (cid:5) (cid:6)(cid:9) = (cid:12)(cid:13) (cid:14)(cid:15)(cid:16) ∑ (cid:1) (cid:18)(cid:19)(cid:20)(cid:5) (cid:2)(cid:4) (cid:5) , (cid:4) (cid:5)∗ (cid:9) + (cid:23) (cid:12)(cid:13) (cid:24)(cid:25)(cid:26) ∑ (cid:4) (cid:5)∗ (cid:1) (cid:27)(cid:28)(cid:29)(cid:5) (cid:2)(cid:8) (cid:5) , (cid:8) (cid:5)∗ (cid:9) , (1) where, (cid:30) is the index of an anchor in a mini-batch, (cid:4) (cid:5) is the predicted probability of an anchor (cid:30) being an object, (cid:8) (cid:5) is the vector representing the 4 parameterized coordinates of the predicted bounding box, (cid:1) (cid:18)(cid:19)(cid:20) is the classification loss (log loss over two classes), (cid:1) (cid:27)(cid:28)(cid:29) is the regression loss, (cid:31) (cid:18)(cid:19)(cid:20) is the mini-batch size and (cid:31) (cid:27)(cid:28)(cid:29) is the number of the anchor locations.
Figure 3: Region Proposal Network indawi Template version: Apr19
7 It should be noted that when defining the loss function of the RPN for training purposes, a binary class label was assigned to each anchor. If the desired object isinside the anchor,algorithm assigns 1 for (cid:4) (cid:5)∗ to indicatea positive anchor, whereas 0 is assigned to indicate a negative anchor.The (cid:8) (cid:5)∗ is that of the ground-truth box associated with a positive anchor. The regression loss, (cid:1) (cid:27)(cid:28)(cid:29) , of the loss function, adopted in faster R-CNN isdefined as, (cid:1) (cid:27)(cid:28)(cid:29) (cid:2)(cid:8) (cid:5) , (cid:8) (cid:5)∗ (cid:9) = (cid:2)(cid:8) (cid:5) − (cid:8) (cid:5)∗ (cid:9) , (2) where represents robust loss function. It should be noted that the regression loss is activated only for positive anchors ( (cid:4) (cid:5)∗ = 1),and is disabled otherwise ( (cid:4) (cid:5)∗ = 0), due to (cid:4) (cid:5)∗ (cid:1) (cid:27)(cid:28)(cid:29) term in (1). A trained RPN generates different sizes of feature maps as its output. As it is not easy to work with different sizes of feature maps, ROI pooling splits the input feature map into equal size regions and applies max pooling to every region .It is worth noting that, output of the ROI pooling is always independent from the input size. Chan –Vese Segmentation
In image processing, many edge detection techniques based on gradient of intensity, such as Sobel, Prewitt and Roberts, are used for object boundary detection and segmentation[23]. In MRI, it is challenging to get accurate boundary detection using these operators as MRI itself contains Rician noise, which causes irregularities in the edge estimation. As both Rician noise and the edges of the image contain high-frequency components, it is challenging to get accurate results with these edge detection operators which depend on the gradient of the intensities. In[28], although Gaussian smoothing filters were used to reduce the Rician noise, it resulted in blurred and distorted MRI, which may lead toerroneous diagnosis. Therefore it is worthwhile to exploit the edge detection algorithms which are not defined based on the gradient of the intensity for MRIs.
Figure 4: 9 Anchors used by RPN at (320,320) indawi Template version: Apr19 ∅(cid:2)(cid:30), , where (cid:30), are coordinates of the image and (cid:8) is the artificial time. Segmentation is defined as two regions where { ∅ < 0 } belongs to region 1 and { ∅ ≥ 0 } belongs to region 2. The output of the level set function, edge contour, is defined in literatureusing { (cid:30), . This approach is used in Chan–Vese algorithm to form the level set function[30]. The Chan–Vese model, which is based on the Mumford–Shah functional for segmentation[14]is used to detect objects whose boundaries are not necessarily detectedby the gradient[33][30]. Mumford-Shah model is based on energy function and it finds a pair of (cid:2)( , )(cid:9) for a given inputimage ( *. In here, ) denotes a smooth and closed segmentation curve of the object and ( represents nearly piecewise smooth approximation of the given image. Chan-Vesealgorithm presents an alternative solution to the Mumford-Shah model. It refers the fitting energy function and achieves the minimization through solving (3). , -. (cid:2)/1, /2, )(cid:9) = μ . (cid:1)234(cid:8)ℎ(cid:2))(cid:9) + λ (cid:12) . 6 |( * (cid:2)8, 9(cid:9) − / (cid:12) | :(cid:5);(cid:20)(cid:5)<(cid:28)(cid:2)-(cid:9) =8 =9 + λ : . 6 |( * (cid:2)8, 9(cid:9) − / : | :>?@(cid:20)(cid:5)<(cid:28)(cid:2)-(cid:9) =8 (3) where μ , λ (cid:12) and λ : are positive constants. Typically λ (cid:12) = λ : = 1 , c (cid:12) and c : are intensity average of given image μ * inside ) and outside ) respectively. This minimization problem can be solved by replacing curve ) with level set function B(cid:2)8, 9(cid:9) .If
B(cid:2)8, 9(cid:9) > 0 the point (cid:2)8, 9(cid:9) is inside the ) , and if B(cid:2)8, 9(cid:9) < 0 the point (cid:2)8, 9(cid:9) is outside ) . In adition, (cid:2)8, 9(cid:9) is on the curve if B(cid:2)8, 9(cid:9) = 0 . For further details on Chan-Vese model reader is directed to[30][33][34][35]. The advantage of this method is that, even if the image is very noisy and initial conditions are not well defined, still the locations of the boundaries are accurately estimatedby the model.
Materials and Methods
Preparation of the dataset indawi Template version: Apr19
Pre-processing and the parameter setting
At thepre-processing stage, the input images were resized into 128x128 as it was not feasible to train the neural network with the original size of 512*512.Yet, some finer features of the input present at original resolution, could be lost during the down sampling process, reducing the sensitivity of the output by a small fraction. Nevertheless, we have down sampled the input image to 128x128 to reduce the complexity of the network with the objective ofreducing the training of thenetwork.Initially, the dataset is randomly split into 3 sets as; training, testing and validation with the ratio of 0.70: 0.15: 0.15, and5fold cross-validation is applied to the training set with the scikit-learn library of python. Batch normalization was applied to input image to re-arrange the input intensities to the scale 0-1.
Applying Chan-Vese Segmentation
After extracting the bounding box through the proposed FasterR-CNN based model, a segmentation algorithm is applied to obtain a precise tumor boundary definition. As the first step of the segmentation process, the bounding box obtained at faster R-CNN from the down sampled 128*128 image was mapped to the original image size 512*512. Then general image pre-processing techniques such as contrast adjustment and histogram equalization were applied to adjust the contrast levels, brightness level, and the sharpness of the images, to reduce the noise levels while enhancing the details of the image. After applying the pre-processing techniques, Morphological Active Contours (Morphological Chan-Vese) technique was applied to identify the precise tumor region. In this research, Chan–Vese algorithm uses a square-level set function, instead of general circle-level set functions, because the input boundary boxes are defined as squares at the faster R-CNN.Furthermore the Parameters (cid:23) ,and (cid:23) in (3) were set to 1 following [30]. During the simulations100 iterations were followed to obtain the best convergence of the contour lines, and at each iteration, smoothing operators is applied 8 times. The values for these number of iteration and smoothing operator were selected by adopting trial and error method. After the segmentation, the output of the Chan-Vese algorithm is compared against segmentation output of Prewitt edge detection technique and ground truth demarcations provided by neurologists, to justify the performance of our proposed system. indawi Template version: Apr19 Statistical Performance Analysis
For the performance evaluation of the overall cascaded MRI tumor segmentation system proposed, the ground truth demarcations provided by experts in the field (neurologists) were compared against the mask obtained from the prediction process of the designed system. Dice Score, Rand Index (RI), Variation of Information (VOI), Global Consistency Error (GCE), Boundary Displacement Error (BDE), Peak Signal to Noise Ratio (PSNR) and Mean absolute error (MAE) were calculated as the objective performance measure parameters [38]. Dice score (F1 score) is a statistical parameter which used to evaluate the similarity of two samples. Dice score lies between 0 and 1, with 1 signifying the highest similarity between predicted and truth. F1 score is calculated using,
Dice score (cid:2)F1 score(cid:9) = : KL: KLMNLMN(cid:13) (4) where TP is true positive, FP is False positive, ,FN is False negative and TN is True negatives. Rand index (RI) counts the fraction of pairs of pixels whose labelling is consistent between the computed segmentation and the ground truth image. RI lies between 0 and 1, and if the two images are identical, the RI should be equals to 1. The RI value is calculated using
O =
KLMK(cid:13)KLMNLMN(cid:13)MK(cid:13) (5) VOI computes the measure of information content ineach of the segmentations and how much of information one segmentation gives about the other, i.e it measures the information distance between the two segmentation.VOI is defined using the entropy and mutual information as,
PQORS (cid:29) , S @ T = U RS (cid:29)
T + U(cid:2)S @ (cid:9) − 2 VO RS (cid:29) , S @ T (6) where S (cid:29) and S @ are the fuzzy segmentations of the image, U (cid:2)S(cid:9) is the marginal entropy,
U RS (cid:29) , S @ T is the joint entropy between two images, and VORS (cid:29) , S @ T is the Mutual information between two images[39]. The W), measures the extent to which,a particular segmentation can be viewed as a refinement of the other. Segmentationsthat are related are considered to be consistent, since they could represent the same image segmented at a different scale. The mathematical expression for GCE can be written as,
W), (cid:2)S (cid:12) , S : (cid:9) = (cid:12); min(cid:2)∑ ,(cid:2)S (cid:12) , S : , (cid:4) (cid:5) (cid:9) (cid:5) , ∑ ,(cid:2)S : , S (cid:12) , (cid:4) (cid:5) (cid:9) (cid:5) (cid:9) (14) where S (cid:12) and S : are two segmentations, and (cid:4) (cid:5) is a pixel position. The boundary displacement error measures the average displacement error between the boundary pixels in the predicted segmentation and the corresponding closes boundary pixels in the ground truth segmentation as given bellow, indawi Template version: Apr19 Z [\ (cid:2)(, ](cid:9) = (cid:3) ?^_[^(cid:12) ; 0 < ( − ] (15) Mean absolute error (MAE) is the average of the difference between predicted and the actual values in all test cases, i.e it is the average prediction error. Itis a quantity used to measure how close forecasts or predictions are to the eventual outcomes and the mathematical representation is given as,
V‘, = (cid:12)(cid:13) ∑ ∑ | , (cid:5)a − Q (cid:5)a | a(cid:5) (16) where N is the size of the image, E is the edge image, and O is the original image. Experimental Results
The experimental outcome of the implemented architecture presented in Figure 1, is analyzed at two stages; namely first after classification stage and second after segmentation stage. Both the training and validation accuracies of the classifier are presented using the confusion matrices in Figure 5.The ROC curves of the training and validationstages of the classification model are presented in Figure 6. Also, the performance of the segmentation algorithm is illustrated visually in Figure 7. Table2summarizes the statistical performance evaluations of the complete model for selected MRIs,whereas Table 3 presents the overall performance summery of the segmentation performance at the final stage of the proposed architecture.
Results of classification model
We present confusion matrix to illustrate the performance of the classification model against the ground truth. The Confusion matrices for the training dataset and the test samples are shown in the Figure 5.In each confusion matrix, green squares represents TP and TN values, light orange squaresrepresentsFP and FN values, and, blue squares were used to represents Positive Predictive Value(PPV), Negative Predictive Values (NPV), Specificity (Sp), and Sensitivity (Sn)respectively in clockwise, from the top right to bottom left. Overall correct classification rate (accuracy) was given in the purple square. The classification error for the trainingand testing set is equal to 7.69% and 6.42%.Overall summery of the classification process is tabulated in Table 1. As the dataset is more biased towards glioma, we have used Cohen’s kappa statistic[40]as itis a very good measure of the performance of a classifier against the imbalanced class problem. It estimates the designed classifier performance against a random guessing classifier based on the frequency of the class occurrence. In general,Cohen’s kappa value above 0.81 is an indication of a perfect classifier while value less than 0 indicates a non performing classification outcome.The Cohen’s kappa statistics value for the proposed model is of 0.843 in training and 0.872 in testing which indicatesagoodagreementin the classification process. Area Under the Curve -Receiver Operating Characteristic (AUC-ROC) isanother powerful metrics used to evaluate the performance of machine learning algorithms with imbalance dataset. ROCcurveillustratesTPrate versus FPrate at various thresholdvalues and is commonly used in medical statistics.The ROC curve for training model and testing are shown in Figure 6 and AUC values are at 0.93 and 0.94 respectively. indawi Template version: Apr19 Actual Classes
Actual Classes
Meningioma Glioma
Meningioma Glioma
Meningioma
TP 15 100% FP 3 12.5%
PPV
Meningioma
TP 104 100% FP 14 12.5%
PPV
Glioma
FN 0 0% TN 21 87.5%
NPV
Glioma
FN 0 0% TN 100 87.5%
NPV Sn Sp Accuracy Sn Sp Accuracy (b) Confusion matrix for Training (a)
Confusion matrix for Testing
Figure 5: Confusion matrices
Predicted Classes Predicted Classes
Table 1:Performance of the classification network
Brain tumor class Glioma Meningioma
No if images taken for transfer learning 114 104 Training accuracy 88.5% 100% Testing accuracy 87.5% 100% No if images taken for testing 24 15 No of correctly predicted images 21 15 (a)Training (b) Testing Figure 6: Receiver Operation Characteristic (ROC) Curve indawi Template version: Apr19 Results of segmentation model
In our proposed model, the Faster R-CNN extracts the bounding box of the tumor and it is followed by Chan-Vese algorithm to obtain a precise tumorboundary for an accurate segmentation of the tumors. The Faster R-CNN model was able to generatethe boundary boxes with 93.6%confidence intervaland 99.81% accuracy. Figure 7 illustrates random sample outputs of the faster R-CNN.As in Figure 7, (a), thetumoris localized correctly with 99% confidence interval. There are two possible outcomes predicted in Figure 7(b). Once closely examined, it is evident that the false positive area, which comprises of brain matter, is predicted as tumour only with 50% confidence interval. The correcttumorarea in Figure 7. (b) is identified with 99% confidence interval. However, in certain slices of axial MRIs where the image consists of complex anatomical structures such as skull and the eye socket, a false detection could happen with high confidence interval (Example: Tumor 5 of table 2). In such scenarios the system fails to completely recover from the erroneous detection. Yet, close examination of the results presented in table 2, yields that most accurate detections have a confidence level greater than 90%, compared to the false detections which is of less than 80%. Another false detection is presented in Figure 7. (c), where the false detection carries 80% of confidence level,while all the other positive detections carry confidence level of 98% (a) (b) (c)
Figure 7: Random Sampled of Tumor localization of the faster RCNN network
Table 2:Performance comparison between Chan
Sample MRIs Tumor 1 Tumor 2Confidence level of detection
Boundary outlining method CV prewitt CV Dice score 0.9256 0.9025 0.9125Accuracy 0.9427 0.9254 0.9485RI 0.9872 0.9542 0.9953VOI 0.0112 0.1626 0.0252GCE 0.0034 0.1795 0.005 BDE 1.83 2.57 2.0298PSNR 30.5287 13.18 27.2865MAE 58.54 143.23 51.28
To determine the overall performance of the proposed Chanwe compare the output of Chanwell as against the output of a simple gradient based edge detection technique. Prewitt is adopted in the experiment as the gradient based edge detection method. The statisticalquality measures obtained for the segmented output of both the Chanedge detection, are tabulated in Table detections, while column with Tumor 5 presents a false detection. and BDE must be low, whereas the RI should be high for a good segmented image. It is observed from the results that the Chanthe gradient-based edge detection algorithm, Prewitt. This is also evident from the visual inspections shown in Figure have a significantly higher score for all the test images, the RI values of Chanhave a relatively higher value than the Prewitt. Also, the MAE value, which should be low for better prediction, is lowest for Chanthat the tumor boundary detected by Chan 14 ance comparison between Chan - Vese (CV) vs Prewitt for selected tumor
Tumor 2 Tumor 3 Tumor 4 prewitt CV prewitt CV prewitt0.9125 0.8545 0.9237 0.8723 0.9164 0.88450.9485 0.9123 0.9486 0.9406 0.9503 0.93720.9953 0.9648 0.9967 0.9865 0.9955 0.99470.0252 0.2683 0.0281 0.0965 0.0376 0.0752 0.068 0.0031 0.046 0.0044 0.00892.0298 3.325 1.3171 2.56 1.7234 2.78.2865 15.26 27.8105 9.56 26.499 16.25 121.56 65.23 128.26 45.6 99.57
To determine the overall performance of the proposed Chan-Vese algorithm based system, we compare the output of Chan-Vese algorithm against the ground truth, gold standard, as f a simple gradient based edge detection technique. Prewitt is adopted in the experiment as the gradient based edge detection method. The statisticalquality measures obtained for the segmented output of both the Chan-Vesealgorithm, and the Prewitt ection, are tabulated in Table 2. Columns with Tumors 1-4 presents positive detections, while column with Tumor 5 presents a false detection. The value of GCE, VOI and BDE must be low, whereas the RI should be high for a good segmented image. It is d from the results that the Chan-Vese algorithm exhibits a superior performance than based edge detection algorithm, Prewitt. This is also evident from the visual inspections shown in Figure 8.Although the RI for both Chan-Vese and Prewitt alghave a significantly higher score for all the test images, the RI values of Chanhave a relatively higher value than the Prewitt. Also, the MAE value, which should be low for better prediction, is lowest for Chan-Vese according to Table 2. Hence we can conclude that the tumor boundary detected by Chan-Vese is superior than that of Perwitt
Hindawi Template version: Apr19 for selected tumorMRIs
Tumor 4 Tumor 5
Correct detection with 80%, Errornous diagnosis 98%. prewitt CV prewitt 0.8845 0.6256 0.5941 0.9372 0.7831 0.7232 0.9947 0.8263 0.7639 0.0752 0.873 1.94 0.0089 0.7361 0.927 2.78 5.8263 7.7836 16.25 13.578 9.354 99.57 86.6521 145.764
Vese algorithm based system, Vese algorithm against the ground truth, gold standard, as f a simple gradient based edge detection technique. Prewitt is adopted in the experiment as the gradient based edge detection method. The statisticalquality Vesealgorithm, and the Prewitt 4 presents positive The value of GCE, VOI and BDE must be low, whereas the RI should be high for a good segmented image. It is Vese algorithm exhibits a superior performance than based edge detection algorithm, Prewitt. This is also evident from the visual Vese and Prewitt algorithms have a significantly higher score for all the test images, the RI values of Chan-Vese algorithm have a relatively higher value than the Prewitt. Also, the MAE value, which should be low Hence we can conclude Vese is superior than that of Perwitt indawi Template version: Apr19 Table 3: Overall performance of the system
Boundary estimation method Chan-Vese Prewitt Dice score 0.92 0.90 Accuracy 0.9457 0.9125 RI 0.9936 0.9642 VOI 0.0301 0.1542 GCE 0.004 0.083 BDE 2.099 3.581 PSNR 77.076 13.43 MAE 52.946 126.47
However, in some critical casesas illustrated in Figure 7 (c), the system may not perform as desired. Furthermore, the performance measurements for such a scenario is presented in the Last column of Table 2, Tumor 5. It is evident that some of the performance measures for false detection, specifically GCE, VOI and BDE, shows a deviation from that of positive detection. The main reason behind false detection is the miss-prediction and classification of brain matters as tumors. These inaccuracies in the model can be easily rectified by further tuning the proposed model using a larger dataset. The objective of the system presented in this manuscript is to obtain a finer segmentation of brain tumors. To achieve this objective, we firstly employed Faster R-CNN to obtain the initial boundary region and secondly used Chan-Veseactive contour to refine initial boundary assessment to obtain an exact boundary extraction. Table 3 summarise the overall performance of the proposed architecture. According to the objective measurements, the overall system achieved a Dice Score of 0.92, accuracy of 0.946, RI of 0.99 and PSNR of 77.1, against the gold standard,pointing to excellent precision of the proposed segmentation method.
Figure 87: Segmented images vs Ground truth (a)
Ground truth (b) Prewitt (c) Chan- indawi Template version: Apr19 Discussion
This study presents an automated MRI tumor classification and segmentation algorithm based on deep learning techniques and active contours. Results obtained from the experiments demonstrate remarkable performance at braintumor segmentation with dice score of 0.92, accuracy of 0.9457, RIof 0.9936, VOI of 0.0301, GCE of 0.004, BDE of 2.099, PSNR of 77.076 and MAE of 52.946. The model presented in this manuscriptprovides 0.915 dice score for glioma segmentation with thefigshare data set [13][11], with the faster R-CNN and Chan-Vesealgorithms. In comparison, authors in [41]has developed a CNN based glioma segmentation algorithm and achieved a 0.87 Dice score on BRATS 2013 and 2015 datasets. Experimental results were presented in [20] for accurate glioma segmentation algorithm which obtained 0.897 Dice score. An automatic semantic segmentation model was developed on the BRATS 2013 dataset by the authors in [42] and the Dice score was around 0.80. In our study, 0.926 dice core was obtained only for the meningioma segmentation which is comparably high amongsimilar research works. Authors at [43] obtained dice coefficient around 0.81 for 56 meningioma cases by using deep learning on routine multi-parametric model. One recent study [44] achieved dice coefficients ranging between 0.82–0.91 by employing an algorithm based on 3D deep convolutional neural network. Furthermore authors in [45] proposed a CNN based algorithm on same figsharedataset [13] and achieved a dice score of 0.71 for both meningioma and glioma together with axial MR images. Authors in [37] were able to increase the segmentation accuracy using Cascaded Deep Neural Networks and obtained around 0.8 in dice score in both Meningioma and Glioma segmentation. In our study, we achieved an average dice score of 0.92 for both meningioma and glioma segmentation using the same dataset. Hence, comparison with the comparable state-of-the-art methods shows that the proposed methodology exhibits a remarkable improvement in brain tumor segmentation. Nevertheless, all these researchers proved that deep neural networks are capable of performing significantly accurate brain tumor segmentation in MR images. We show that the segmentation output can be further improved by using Active contour algorithms along with deep learning architectures.
Conclusions
In the research presented, we have proposed a R-CNN and Chan-Vese algorithms based model for meningioma and glioma brain tumor classification and segmentation. The proposed model is validated using figshare dataset with 5-fold cross-validation and objective quality metrics dice score, RI, VOI, GCE, BDE, PSNR and MAE are calculated to analyse the performance of the proposed architecture. We have used R-CNN to obtain the initial tumor boundary box which is followed by active contouring to obtain exact tumoroutline. We adopt level set functions based, Chan-Vese algorithm which is independent of Rician noise, for both meningioma and glioma brain tumor segmentation and we compare the performance of the proposed segmentation method against that of the typical-gradient based edge detection algorithm Prewitt. We were able to achieve much more accurate segmentation result through the Chan-Vese algorithm with an average dice score of 0.92 for both tumor types.In addition experts in the field have cross checked our segmentation output and have indawi Template version: Apr19
Data Availability
All MRI data are available in the brain tumor imaging archive (https://figshare.com/articles/brain_tumor_dataset/1512427)
Conflicts of Interest
The authors declare that there is no conflict of interest.
Acknowledgments
The authors wish to acknowledge the insightful and extremely helpful reviews and comments provide by Dr. S.C Weerasinghe, Neurologist, atTeaching hospital Anuradhapura during the design and validation stages of the research . References [1] A. Kabir Anaraki, M. Ayati, and F. Kazemi, “Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms,”
Biocybern. Biomed. Eng. , vol. 39, no. 1, pp. 63–74, 2019. [2] J. Ker, L. Wang, J. Rao, and T. Lim, “Deep Learning Applications in Medical Image Analysis,”
IEEE Access , vol. 6, pp. 9375–9389, 2018. [3] S. Khan and S.-P. Yong, “A deep learning architecture for classifying medical images of anatomy object,” in , 2017, pp. 1661–1668. [4] M. W. Nadeem et al. , “Brain tumor analysis empowered with deep learning: A review, taxonomy, and future challenges,”
Brain Sci. , vol. 10, no. 2, pp. 1–33, 2020. [5] “Health in the European Union – facts and figures - Statistics Explained.” [Online]. Available: https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Health_in_the_European_Union_–_facts_and_figures. [Accessed: 01-Dec-2019]. [6] Bureau of Labor Statistics, “Radiologic and MRI Technologists : Occupational Outlook Handbook: : U.S. Bureau of Labor Statistics,”
United States Department of Labor ş in, C. Direko ǧ lu, and M. Ş ah, “Review of MRI-based Brain Tumor Image Segmentation Using Deep Learning Methods,” Procedia Comput. Sci. , vol. 102, no. indawi Template version: Apr19
J. Med. Imaging , vol. 6, no. 03, p. 1, 2019. [9] C. G. Madamombe, “Deep Learning Techniques to Classify and Analyze Medical Imaging Data,”
Int. J. Comput. Sci. Eng. , vol. 7, no. 04, pp. 109–113, 2018. [10] M. Soltaninejad et al. , “Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels,”
Comput. Methods Programs Biomed. , vol. 157, pp. 69–84, 2018. [11] H. Dong, G. Yang, F. Liu, Y. Mo, and Y. Guo, “Automatic brain tumor detection and segmentation using U-net based fully convolutional networks,” arXiv , vol. 1, pp. 506–517, 2017. [12] M. Soltaninejad et al. , “Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI,”
Int. J. Comput. Assist. Radiol. Surg. , vol. 12, no. 2, pp. 183–203, 2017. [13] Cheng Jun, “brain tumor dataset”, figshare. Dataset, 2017, Available: https://doi.org/10.6084/m9. figshare.1512427.v5 [14] J. Cheng et al. , “Enhanced Performance of Brain Tumor Classification via Tumor Region Augmentation and Partition,”
PLoS One , vol. 10, no. 10, p. e0140381, Oct. 2015. [15] Z. N. K. Swati et al. , “Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning,”
IEEE Access , vol. 7, pp. 17809–17822, 2019. [16] M. M. Badža and M. C. Barjaktarovi ć , “Classification of brain tumors from mri images using a convolutional neural network,” Appl. Sci. , vol. 10, no. 6, 2020. [17] N. Noreen, S. Palaniappan, A. Qayyum, I. Ahmad, M. Imran, and M. Shoaib, “A Deep Learning Model Based on Concatenation Approach for the Diagnosis of Brain Tumor,”
IEEE Access , vol. 8, pp. 55135–55144, 2020. [18] S. Tripathi, A. Verma, and N. Sharma, “Automatic segmentation of brain tumour in MR images using an enhanced deep learning approach,”
Comput. Methods Biomech. Biomed. Eng. Imaging Vis. , vol. 00, no. 00, pp. 1–10, 2020. [19] A. M. Alqudah, H. Alquraan, I. A. Qasmieh, A. Alqudah, and W. Al-Sharu, “Brain tumor classification using deep learning technique - A comparison between cropped, uncropped, and segmented lesion images with different sizes,”
Int. J. Adv. Trends Comput. Sci. Eng. , vol. 8, no. 6, pp. 3684–3691, 2019. [20] X. Chen et al. , “Automatic Prediction of MGMT Status in Glioblastoma via Deep Learning-Based MR Image Analysis,” vol. 2020, 2020. [21] H. N. T. K. Kaldera, S. R. Gunasekara, and M. B. DIssanayake, “Brain tumor Classification and Segmentation using Faster R-CNN,” , 2019. [22] H. N. T. K. Kaldera, S. R. Gunasekara and M. B. Dissanayake, "MRI based Glioma segmentation using Deep Learning algorithms," International Research Conference on Smart Computing and Systems Engineering (SCSE), Colombo, Sri Lanka, pp. 51-56, 2019. [23] Rafael C. Gonzalez , Richard E. Woods, Digital Image Processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2001, pp 714-735 [24] T. R. Shultz and S. E. Fahlman,
Webb G.I. Sammut C. 2017. Clustering. In: Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA [25] S. Yang and G. Berdine, “The receiver operating characteristic (ROC) curve,”
Southwest Respir. Crit. Care Chronicles , vol. 5, no. 19, p. 34, 2017. [26] T. F. Chan, B. Y. Sandberg, and L. A. Vese, “Active Contours without Edges for Vector-Valued Images 1,” vol. 141, pp. 130–141, 2000. indawi Template version: Apr19 arXiv1506.01497 [cs] , Jun. 2015. [28] V. G. Narendra and K. S. Hareesh, “Study and comparison of various image edge detection techniques used in quality inspection and evaluation of agricultural and food products by computer vision,”
Int. J. Agric. Biol. Eng. , vol. 4, no. 2, 2011. [29] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour models,”
Int. J. Comput. Vis. , vol. 1, no. 4, pp. 321–331, 1988. [30] X. F. Wang, D. S. Huang, and H. Xu, “An efficient local Chan-Vese model for image segmentation,”
Pattern Recognit. , vol. 43, no. 3, pp. 603–618, 2010. [31] S. Z. Lin Sun , Xinchao Meng , Jiucheng Xu, “An Image Segmentation Method Based on Improved Regularized Level Set Model,”
Appl. Sci. , vol. 8, no. 12, 2018. [32] X. Jiang, R. Zhang, and S. Nie, “Image Segmentation Based on Level Set Method,”
Phys. Procedia , vol. 33, pp. 840–845, 2012. [33] P. Getreuer, “Chan – Vese Segmentation Simplified Mumford – Shah Model Level Set Functions,” vol. 2, pp. 214–224, 2012. [34] T. F. Chan and L. A. Vese, “Active Contours Without Edges,” vol. 10, no. 2, pp. 266–277, 2001. [35] Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42(5), 577–685. doi:10.1002/cpa.3160420503 [36] Lenka Lhotska, Lucie Sukupova, Igor Lackovi ć , Geoffrey S. Ibbott,"World congress on medical physics and biomedical engineering 2018": June 3–8, 2018, 1, Springer, Prague, Czech Republic (2018) [37] Z. Sobhaninia, S. Rezaei, N. Karimi, A. Emami, and S. Samavi, “Brain tumor segmentation by cascaded deep neural networks using multiple image scales,” , pp. 11–14, 2020. [38] R. Kumar and A. M. Arthanariee, “Performance Evaluation and Comparative Analysis of Proposed Image Segmentation Algorithm,” vol. 7, no. January, pp. 39–47, 2014. [39] A. A. Taha and A. Hanbury, “Metrics for evaluating 3D medical image segmentation: Analysis, selection, and tool,” BMC Med. Imaging , vol. 15, no. 1, 2015. [40] J. R. Landis and G. G. Koch, “The Measurement of Observer Agreement for Categorical Data Published by : International Biometric Society Stable URL
Neurocomputing , vol. 282, pp. 248–261, 2018. [42] S. Cui, L. Mao, J. Jiang, C. Liu, and S. Xiong, “Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network,”
J. Healthc. Eng. , vol. 2018, 2018. [43] K. R. Laukamp et al. , “Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI,”
Eur. Radiol. , vol. 29, no. 1, pp. 124–132, 2019. [44] K. R. Laukamp et al. , “Automated Meningioma Segmentation in Multiparametric MRI: Comparable Effectiveness of a Deep Learning Model and Manual Segmentation,”
Clin. Neuroradiol. , 2020. [45] Z. Sobhaninia et al. , “Brain Tumor Segmentation Using Deep Learning by Type Specific Sorting of Images,” 2018. [46] “models/research/object_detection/samples/configs at master · tensorflow/models.” [Online]. Available: https://github.com/tensorflow/models/tree/master/research/object_detection/samples/c indawi Template version: Apr19
Appendix
Proposed algorithm
The first stage of the proposed system is the classification, where input MRI images are analysed to detect the presence and absence of the tumours. To achieve this objective, a simple CNN architecture with total of 5 layers was designed. The CNN network designed consist of two 2D convolutional (2DConv) layer with 20-3 x
3 kernals and 10-3 x
3 kernels respectively.Both of the 2DConv layers adopts non-linearity through, Relu activation functionand each layer is followed by maxpooling layers with 2 xx