Region and Location Based Indexing and Retrieval of MR-T2 Brain Tumor Images
aa r X i v : . [ c s . C V ] D ec Region and Location Based Indexing and Retrieval of MR-T2Brain Tumor Images
Krishna A N a , Dr. B G Prasad ba Associate Professor, Department of Computer Science and Engineering, S J B Institute ofTechnology, Bangalore-560 060, India. Contact:[email protected] b Professor and Head, Department of Computer Science and Engineering, B N M Institute ofTechnology, Bangalore-560 070, India. Contact: [email protected]
In this paper, region based and location based retrieval systems have been implemented for retrieval of MR-T2axial 2-D brain images. This is done by extracting and characterizing the tumor portion of 2-D brain slices by useof a suitable threshold computed over the entire image. Indexing and retrieval is then performed by computingtexture features based on gray-tone spatial-dependence matrix of segmented regions. A Hash structure is used toindex all images. A combined index is adopted to point to all similar images in terms of the texture features. Atquery time, only those images that are in the same hash bucket as those of the queried image are compared forsimilarity, thus reducing the search space and time.
Keywords :
Content Based Retrieval, Indexing, Segmentation, Texture.
1. INTRODUCTION
In medical field, a large number of diverse radi-ological and pathological images in digital formatare generated everyday in hospitals and medicalcenters with sophisticated image acquisition de-vices and digital scanners. Medical images aregenerally complex in nature and are used for di-agnosis, therapy, research and education. Sup-port of prior image references is critical to radiolo-gists or physicians current examination of images.To support their prior image reference needs, thegenerated images need to be processed and orga-nized so that efficient retrieval of similar imagesfor a current examination image is achieved.Content-Based Image Retrieval (CBIR) hasbeen initially proposed to overcome the problemcaused by the subjectivity of a users perceptionin Text-Based Image Retrieval (TBIR). CBIR ismore challenging in medical domain due to thecomplex nature of images. In medical domain,visual features between normal and pathologicalimages may have only subtle differences; thesemay not be captured by traditional feature ex-traction such as color, texture or shape based on entire images. The main reason is that, impor-tant features in biomedical images are often localfeatures of pathological regions or lesions, ratherthan global features of entire image. Generatinglocal features is much more complex than globalfeatures; however, it can describe fine details ofthe images and allow efficient retrieval of relevantimages based on local object properties. To ex-tract regional or local features, segmentation isvery important in medical imaging and generallytreated as a pre-processing step.Manual segmentation is a very time-consumingtask and not feasible in real-time needs. More-over, results from manual operations are not re-peatable and suffer from intra-observer and inter-observer variability. In the past few decades,researchers have proposed many effective algo-rithms to perform automated segmentation. Thesuccessful implementation of modern mathemat-ical and physical techniques, such as Bayesiansanalysis, template matching and deformablemodels, greatly enhances the accuracy of segmen-tation results. Compared with common imagesegmentation algorithms, the ones used for med-16 egion and Location Based Indexing and Retrieval of MR-T2 Brain Tumor Images
2. REGION-OF-INTEREST SEGMEN-TATION
Image segmentation by use of a suitable thresh-old is one of the main techniques. It is the oldestand still most commonly used technique becauseof its simplicity and efficiency. Many methods[21][22][23][24][25][26] have been proposed in theliterature to find one or more thresholds from thehistogram of an image and perform the segmen-tation based on the threshold. The proposed seg-mentation algorithm uses the global threshold se-lection method which uses gray-level distribution.A global threshold is the that partitions the en-tire image with a single threshold value. After thethresholding, region labeling algorithm is appliedto obtain clusters of different sizes. The clusterof largest size is considered as tumor (region ofinterest). An image can be represented by a 2-Dgray-level intensity function f ( x, y ). The value of f ( x, y ) is the gray-level ranging from 0 to L − L is the number of distinct gray-levels. Themajor steps of the proposed segmentation algo-rithm is shown in Table 1.8 Krishna A N and Dr. B G Prasad
Table 1Segmentation Algorithm by use of a Global Threshold1. Compute T by iterative threshold selection algorithm (shown in Table 2)2. Find t ∗ , the maximum between-class variance3. The optimal threshold T ∗ is defined by the sum T and t ∗
4. The segmented image, g ( x, y ) is given by g ( x, y ) = (cid:26) f ( x, y ) > T ∗ f ( x, y ) ≤ T ∗ where T ∗ is a constant applicable over an entire image5. Labelling of segmented image is done for eliminating small clustersTable 2Algorithm: To Find T Iteratively1. Initialize T = Average gray level of the image.2. Compute µ for pixels less than or equal to T .3. Compute µ for pixels greater than T .4. Compute a new threshold T = ( µ + µ )5. Repeat step 2 through 4 until the differencein T in successive iterations is smallerthan a predefined T (= 0). t ∗ by OtsuMethod Suppose that there are N pixels and L graylevels (0, 1, ..., L −
1) in an image. Let n l denotethe number of pixels at level l , then N = P l − l =0 n l .The histogram of an image can be normalized asa probability distribution by P l = n l N , l − X l =0 P l = 1Assume that a threshold t divides the gray levelsinto two clusters S = 0. 1, ..., t and S = t + 1, t + 2,..., L − σ B ( t ) be the between-classvariance of the gray levels [21]. Then the optimalthreshold t ∗ is obtained by t ∗ = arg max ≤ t
8; ++i) { j = x + delta[i][0];k = y + delta[i][1];if(inImage(j,k)&&in.getSample(j,k,0) > }} private final boolean inImage(int x, int y) { return x ≥ <
256 && y ≥ < } region have been set to 0 in the input image, mak-ing them indistinguishable from the backgroundand the corresponding pixels in the output imagehave been assigned a region number. The regionnumber is then incremented, ready for the nextconnected region. The recursive labeling proce-dure is given in Table 3. Table 4 shows represen-tative snapshots depicting tumor identification ofMR-T2 brain images.
3. FEATURE REPRESENTATION
Texture information is specified by a set ofgray-tone spatial-dependence matrices that arecomputed for various angular relationships anddistances between neighboring resolution cellpairs on the image. All the textural featuresare derived from these angular nearest-neighborgray-tone spatial-dependence matrices. Suppose an image to be analyzed is rectangular and has N x resolution cells in the horizontal direction and N y , resolution cells in the vertical direction. Sup-pose that the gray tone appearing in each resolu-tion cell is quantized to N g levels. Let L x = 1,2, ... , N x be the horizontal spatial domain, L y =1, 2, ..., N y be the vertical spatial domain, andG = 1, 2, ..., N g be the set of N g quantized graytones. The set L y x L x is the set of resolutioncells of the image ordered by their row-columndesignations. The image I can be represented asa function which assigns some gray tone in G toeach resolution cell in L y X L x ; I : L y X L x → G .Four closely related measures from which thetexture features we have used are derived us-ing angular nearest-neighbor gray-tone spatial-dependence matrices: P ( i, j, d, o ), P ( i, j, d, o ), P ( i, j, d, o ) and P ( i, j, d, o ). We assume thatthe texture-context information in an image I iscontained in the ”overall” or ”average” spatial re-lationship which the gray tones in image I haveto one another. More specifically, this texture-context information has been adequately specifiedby a matrix of relative frequencies P ij with whichtwo neighboring resolution cells, one with graytone i and the other with gray tone j separatedby distance d occur on the image. Such matricesof gray-tone spatial-dependence frequencies are afunction of the angular relationship between theneighboring resolution cells as well as a functionof the distance between them. Formally, for an-gles quantized to 45 o intervals, the unnormalizedfrequencies are defined by Eqs. (1)-(4). The de-tails of computing these texture measures is pre-sented in [27]. P ( i, j, d, o ) = { (( k, l ) , ( m, n )) ∈ ( L y X L x ) X ( L y X L x ) | k − m = 0 , | l − n | = d, I ( k, l ) = i, I ( m, n ) = j } (1) P ( i, j, d, o ) = { (( k, l ) , ( m, n )) ∈ ( L y X L x ) X ( L y X L x ) | ( k − m = d,l − n = − d ) or ( k − m = − d, l − n = d ) ,I ( k, l ) = i, I ( m, n ) = j } (2)0 Krishna A N and Dr. B G Prasad
Table 4Some Results of Tumor SegmentationQueryImagesSegmentedImages P ( i, j, d, o ) = { (( k, l ) , ( m, n )) ∈ ( L y X L x ) X ( L y X L x ) || k − m | = d, | l − n | = 0 , I ( k, l ) = i, I ( m, n ) = j } (3) P ( i, j, d, o ) = { (( k, l ) , ( m, n )) ∈ ( L y X L x ) X ( L y X L x ) | ( k − m = d,l − n = d ) or ( k − m = − d, l − n = − d ) ,I ( k, l ) = i, I ( m, n ) = j } (4)where Note : These matrices are symmetric; P ( i, j ; d, a ) = P ( j, i ; d, a ).We compute four closely related measures P ( i, j, d, θ ) quantized to 45 intervals with d = 1from which all our three texture features are de-rived. Out of the equations which define a to-tal set of 14 measures of textural features [27],we have used the three most distinguishing pa-rameters to describe the texture of an image asdepicted by Eqs. (5)-(7). Energy = X i X j P ( i, j ) (5) Entropy = − X i X j P ( i, j ) log ( P ( i, j )) (6) Contrast = N g − X n =0 n N g X i =1 N g X j =1 | i − j | = n P ( i, j ) (7)
4. REGION BASED INDEXING ANDRETRIEVAL (RBIR)
A data structure based on hashing techniqueis used to store all images along with the texturefeature data. A combined index is adopted topoint to all similar images in terms of the texturefeatures. When a query is made based on an ex-ample image, the example image is processed forindex value. Only those images that are in thesame hash bucket as those of the queried imageare compared for similarity. For each image inthe database, segmentation procedure discussedin section 2 is applied to identify region-of-interestand describe segmented region by texture fea-tures: entropy , energy and contrast . The texturefeatures extracted are quantized to integer valuesbetween 0 to 9. The combined index of these fea- egion and Location Based Indexing and Retrieval of MR-T2 Brain Tumor Images ∗ [ entropy ]+10 ∗ [ energy ]+[ contrast ],where [ ] represents quantization. Each combinedindex stores feature data along with the image ob-ject. For a query image, after finding the region-of-interest, the above mentioned texture featureshave to be computed, quantized and the com-bined index derived. Only those images that arestored at the combined index matching those ofthe query index, are extracted as resultant targetimages for a given query image. These resultantimages are sorted using Euclidean distance mea-sure in the decreasing order of similarity againstthe query image and displayed four images at atime using JAVA-AWT based GUI. A few repre-sentative snapshots of region-based indexing andretrieval are shown in Figure 1. Hash table offersvery fast insertion and searching. Irrespective ofthe size of the data, insertion and searching cantake close to constant time O (1). Not only arethey fast, hash tables are simple and easy to im-plement. Searching using hash tables are signif-icantly faster than using tree, which operate in O ( logN ) time.
5. LOCATION BASED INDEXING ANDRETRIEVAL (LBIR)
Location Based Indexing and Retrieval is per-formed by finding spatial location of a segmentedregion. The importance of location of objectsis to identify the area of involvement of tumorlike sensory or motor. The brain has uniqueareas for speech, hearing, visual, temperatureregulation etc.. If the tumor which may notbe centrally located, occurs at any particulararea/location, then the corresponding organ getsaffected. Hence location is an important featureto be indexed. To compute the location of a re-gion, we divide the image space into 3x3 grid cellsand number them 0-8 as shown in Figure 2. Theregion is likely to overlap number of cells in theimage space. The index assigned is the cell num-ber that is maximally covered by the region. Aprogram segment to find location of a region isgiven in Table 5. We have considered an imagesize of 256x256 pixels in our work. The positionof a region forms the location index. For eachimage in the database, segmentation procedure Table 5Program Segment to Find the Location of a Re-gionpublic void findPos(x1, y1, x2, y2) { x1, y1 and x2, y2 specifies thetop left corner and bottom rightcorner position of the regionfor which position is to be computedint pos = 0, row=0, col = 0;;row = findLoc(x1, x2);col = findLoc(y1, y2);pos = row*3 + col; } int findLoc(int loc1, int loc2) { int loc = 0;for(int i=1; i ≤ loc2/85; i++) { if(Abs(85*i-loc2) ≥ Abs(85*i-loc1))loc++; } return loc; } is applied to identify region-of-interest and de-scribe segmented region by texture features: en-tropy , energy and contrast . Each location indexstores region texture feature data along with theimage object. For a query image, after findingthe region-of-interest, the above mentioned tex-ture features have to be computed and the lo-cation index is derived. Only those images thatare stored at the location index matching those ofthe query index, are extracted as resultant targetimages for a given query image. These resultantimages are sorted using Euclidean distance mea-sure in the decreasing order of similarity againstthe query image and displayed four images at atime using JAVA-AWT based GUI. A few rep-resentative snapshots of location-based indexingand retrieval are shown in Figure 2.
6. PERFORMANCE ANALYSIS
The two indexing and retrieval techniques im-plemented are: RBIR and LBIR. The retrievalperformance is measured using precision and re-2
Krishna A N and Dr. B G Prasad
Figure 1. A Few Representative Snapshots of Region Based Indexing and RetrievalFigure 2. A Few Representative Snapshots of Location Based Indexing and RetrievalTable 6Retrieval Results of Percentage Precision of CBIR for Top 10 RetrievalsClasses Number of Retrievals1 2 3 4 5 6 7 8 9 10Glioma 100 100 100 75 80 66.66 71.42 75 66.66 60Meningioma 100 100 100 75 60 66.66 57.14 50 44.44 40Carcinoma 100 100 66.66 50 40 50 42.85 37.5 44.44 40Sarcoma 100 50 66.66 50 60 50 42.85 50 44.44 40Average 100 87.5 83.33 62.5 60 58.33 53.56 53.125 50 45 egion and Location Based Indexing and Retrieval of MR-T2 Brain Tumor Images
Precision rates for 4 differ-ent classes tabulated for retrieval from top 1 totop 10 retrieved images. The results for RBIR isshown to be better when compared to CBIR andLBIR, which leads to almost 10 percent increasein precision rates. Table 9 depicts the
Recall rates for the same 4 different classes in the database.Here also, each of the 100 images were used asa query image and the number of matches in thetop 20 retrieved images was counted and is shownto drastically increase recall rates by almost 10percent. The precision recall graph for plottingthe average precision retrieval rates for top 10 re-trievals of the three indexing schemes is shown inFigure 3. Table 9Retrieval Results of Percentage Recall Rate forTop 20 RetrievalsClasses CBIR LBIR RBIRGlioma 60 60 65Meningioma 45 45 50Carcinoma 45 45 50Sarcoma 50 55 60Average 50 51.25 58.75
7. CONCLUSIONS
We have implemented two methods of index-ing and retrieval namely: i) region-based index-ing and retrieval and ii) location-based index-ing and retrieval. Hash structure is used to in-dex images. The retrieved images are sorted us-ing Euclidean distance measure in the decreas-ing order of similarity against the query image.The performance of both the systems have beenmeasured using standard precision versus recallgraphs. Region-based indexing and retrieval givessignificantly better results of 81.7 percent preci-4
Krishna A N and Dr. B G Prasad
Figure 3. Precision Recall Graph for Top 10 Retrievalssion as compared to location-based indexing andretrieval which gives 72.74 percent precision. Thisis because, in most of the cases the tumor is lo-cated in the central position. The results are alsocompared with CBIR which gives 65.33 percentprecision.
REFERENCES
1. Dzung L Pham et al..
A Survey of Current Meth-ods in Medical Image Segmentation.
TechnicalReport, Annual Review of Biomedical Engineer-ing , Jan 1998.2. Daniel J Withey and Zoltan J Koles. A Reviewof Medical Image Segmentation: Methods andAvailable Software.
International Journal of Bio-electromagnetism , 10(3):125-148, 2008.3. Dickson S and Thomas B. Using Neural Networksto Automatically Detect Brain Tumors in MR Im-ages.
International Journal of Neural Systems ,8:91-99, 1997.4. D T Gering, W E L Grimson and R Kikinis. Rec-ognizing Deviations from Normalcy for Brain Tu-mor Segmentation. In
Springer MICCAI 2002 ,LNCS 2488:388-395, 2002. 5. Z Lao et al..
Computer-Assisted Segmentationof White Matter Lesions in 3D MR Images us-ing Support Vector Machine.
Academic Radiol-ogy , 15(3):300-313, 2008.6. Moon N et al..
Model-Based Brain and TumorSegmentation. In
International Conference onPattern Recognition , (1):528-531, 2002.7. L M Fletcher-Heath et al..
Automatic Segmen-tation of Non-Enhancing Brain Tumors in Mag-netic Resonance Images.
Artificial Intelligence inMedicine , 21:43-63, 2001.8. Kaus M et al..
Automated Segmentation of MRImages of Brain Tumors.
Radiology , 218:586-591,2001.9. Liu J et al..
A System for Brain Tumor Vol-ume Estimation via
MR Imaging and Fuzzy Con-nectedness.
Computerized Medical Imaging andGraphics , 29:21-34, 2005.10. Prastawa M et al..
A Brain Tumor SegmentationFramework Based on Outlier Detection.
MedicalImage Analysis , 8:275-283, 2004.11. Nie J et al..
Automated Brain Tumor Segmen-tation using Spatial Accuracy-Weighted HiddenMarkov Random Field.
Computerized MedicalImaging and Graphics , 33:431-441, 2009.12. Ahmed M and Mohamad D. Segmentation of egion and Location Based Indexing and Retrieval of MR-T2 Brain Tumor Images Brain MR Images for Tumor Extraction by Com-bining K-means Clustering and Perona-MalikAnisotropic Diffusion Model.
International Jour-nal of Image Processing , 2:27-34, 2008.13. Ho S, Bullitt E and Gerig G. Level-Set Evolu-tion with Region Competition: Automatic 3-DSegmentation of Brain Tumors. In
InternationalConference on Pattern Recognition , pages 532-535, 2002.14. Cobzas D et al..
3D Variational Brain Tumor Seg-mentation using a High Dimensional Feature Set.In
IEEE International Conference on ComputerVision , pages 1-8, 2007.15. Ayed B et al..
A Statistical Overlap Prior forVariational Image Segmentation.
InternationalJournal of Computer Vision , 85:115-132, 2009.16. Taheri S et al. Level-Set Segmentation of BrainTumors using a Threshold-Based Speed Function.
Image and Vision Computing , 28:26-37, 2010.17. Jinman Kim et al..
A New Way for Multi-dimensional Medical Data Management: Vol-ume of Interest (VOI)-Based Retrieval of Medi-cal Images With Visual and Functional Features.
IEEE Transactions on Information Technology inBiomedicine , 10(3):598-607, July 2006.18. W Liu and Q Y Tong. Medical Image Retrievalusing Salient Point Detector. In , pages 6:6352-6355, September 2005.19. Feng Jing et al..
Relevance Feedback in Region-Based Image Retrieval.
IEEE Transactionson Circuits and Systems for Video Technology ,14(5):672-681, May 2004.20. Khanh Vu et al..
Image Retrieval Based onRegions of Interest.
IEEE Transactions onKnowledge and Data Engineering , 15(4):1045-1049, July/August 2003.21. N Otsu. A Threshold Selection Method fromGray-Level Histograms.
IEEE Transactions onSystems, Man and Cybernetics , 9:62-66, 1979.22. J N Kanpur, P K Sahoo and A K C Wong. A NewMethod for Gray-Level Picture Thresholding us-ing the Entropy of the Histogram.
Computer Vi-sion, Graphics and Image Processing , 29(3):273-285, 1985. 23. P K Shaoo et al..
A Survey of Thresholding Tech-niques.
Computer Vision, Graphics and ImageProcessing , 41(2):233-260, 1988.24. N R Pal and S Pal. A Review on Image Segmenta-tion Techniques.
Pattern Recognition , 26(9):1277-1294, 1993.25. M Cheriet, J N Said and C Y Suen. A Recur-sive Thresholding Technique for Image Segmen-tation.
IEEE Transactions on Image Processing ,7(6):918-921, June 1998.26. Debashis Sen and Sankar K Pal. HistogramThresholding using Fuzzy and Rough Measuresof Association Error.
IEEE Transactions on Im-age Processing , 18(4):879-888, April 2009.27. R Haralick et al..
Textural Features for ImageClassification.
IEEE Transactions on Systems,Man and Cybernetics , 3(6):610-621, 1973.
Krishna A N is currentlyAssociate professor, Depart-ment of Computer Science andEngineering, S J B Instituteof Technology, Bangalore. Heobtained his Bachelors andMasters degree in ComputerScience and Engineering fromUniversity Visvesvaraya Collegeof Engineering. He has publicat-ions in International Conferences and Journals. Hisresearch interests includes Image Processing, PatternRecognition and Content Based Image Retrieval.