Haksoo Kim

Quantitative evaluation of image segmentation incorporating medical consideration functions

PURPOSE A quantitative and objective metric, the medical similarity index (MSI), has been developed for evaluating the accuracy of a medical image segmentation relative to a reference segmentation. The MSI uses the medical consideration function (MCF) as its basis. METHODS Currently, no indices provide quantitative evaluations of segmentation accuracy with medical considerations. Variations in segmentation can occur due to individual skill levels and medical relevance--curable or palliative intent, boundary uncertainty due to volume averaging, contrast levels, spatial resolution, and unresolved motion all affect the accuracy of a patient segmentation. Current accuracy measuring indices are not medically relevant. For example, undercontouring the tumor volume is not differentiated from overcontouring tumor. Dice similarity coefficient (DSC) and Hausdorff distance (HD) are two similarity measures often used. However, these metrics consider only geometric difference without considering medical implications. Two segments (under- vs overcontouring tumor) with similar DSC and HD measures could produce significantly different medical treatment results. The authors are proposing a MSI involving a user-defined MCF derived from an asymmetric Gaussian function. The shape of the MCF can be determined by a user, reflecting the anatomical location and characteristics of a particular tissue, organ, or tumor type. The peak of MCF is set along the reference contour; the inner and outer slopes are selected by the user. The discrepancy between the test and reference contours is calculated at each pixel by using a bidirectional local distance measure. The MCF value corresponding to that distance is summed and averaged to produce the MSI. Synthetic segmentations and clinical data from a 15 multi-institutional trial for a head-and-neck case are scored and compared by using MSI, DSC, and Hausdorff distance. RESULTS The MSI was shown to reflect medical considerations through the choice of MCF penalties for under- and overcontouring. Existing similarity scores were either insensitive to medical realities or simply inaccurate. CONCLUSIONS The medical similarity index, a segmentation evaluation metric based on medical considerations, has been proposed, developed, and tested to incorporate clinically relevant considerations beyond geometric parameters alone.

Quantitative Analysis Tools and Digital Phantoms for Deformable Image Registration Quality Assurance

This article proposes quantitative analysis tools and digital phantoms to quantify intrinsic errors of deformable image registration (DIR) systems and establish quality assurance (QA) procedures for clinical use of DIR systems utilizing local and global error analysis methods with clinically realistic digital image phantoms. Landmark-based image registration verifications are suitable only for images with significant feature points. To address this shortfall, we adapted a deformation vector field (DVF) comparison approach with new analysis techniques to quantify the results. Digital image phantoms are derived from data sets of actual patient images (a reference image set, R, a test image set, T). Image sets from the same patient taken at different times are registered with deformable methods producing a reference DVFref. Applying DVFref to the original reference image deforms T into a new image R′. The data set, R′, T, and DVFref, is from a realistic truth set and therefore can be used to analyze any DIR system and expose intrinsic errors by comparing DVFref and DVFtest. For quantitative error analysis, calculating and delineating differences between DVFs, 2 methods were used, (1) a local error analysis tool that displays deformation error magnitudes with color mapping on each image slice and (2) a global error analysis tool that calculates a deformation error histogram, which describes a cumulative probability function of errors for each anatomical structure. Three digital image phantoms were generated from three patients with a head and neck, a lung and a liver cancer. The DIR QA was evaluated using the case with head and neck.

Toward a novel dosimetry system using acrylic disk radiation sensor for proton pencil beam scanning

PURPOSE Fabricate an acrylic disk radiation sensor (ADRS) and characterize the photoluminescence signal generated from the optical device as basis for the development and evaluation of a new dosimetry system for pencil beam proton therapy. METHODS Based on the characteristics of the proposed optical dosimetry sensor, we established the relation between the photoluminescence output and the applied dose using an ionization chamber. Then, we obtained the relative integral depth dose profiles using the photoluminescence signal generated by pencil beam irradiation at energies of 99.9 and 162.1 MeV, and compared the results with the curve measured using a Bragg peak ionization chamber. RESULTS The relation between the photoluminescence output and applied dose was linear. In addition, the ADRS was dose independent for beam currents up to 6 Gy/min, and the calibration factor for energy was close to 1. Hence, the energy dependence on the optical device can be disregarded. The integral depth dose profiles obtained for the ADRS suitable agreed with the curve measured in the Bragg peak ionization chamber without requiring correction. CONCLUSIONS These results suggest that the ADRS is suitable for dosimetry measurements in pencil beam scanning, and it will be employed as a low-cost and versatile dosimetry sensor in upcoming developments.

Development and Validation of a Small Animal Immobilizer and Positioning System for the Study of Delivery of Intracranial and Extracranial Radiotherapy Using the Gamma Knife System

The purpose of this research is to establish a process of irradiating mice using the Gamma Knife as a versatile system for small animal irradiation and to validate accurate intracranial and extracranial dose delivery using this system. A stereotactic immobilization device was developed for small animals for the Gamma Knife head frame allowing for isocentric dose delivery. Intercranial positional reproducibility of a reference point from a primary reference animal was verified on an additional mouse. Extracranial positional reproducibility of the mouse aorta was verified using 3 mice. Accurate dose delivery was validated using film and thermoluminescent dosimeter measurements with a solid water phantom. Gamma Knife plans were developed to irradiate intracranial and extracranial targets. Mice were irradiated validating successful targeted radiation dose delivery. Intramouse positional variability of the right mandible reference point across 10 micro-computed tomography scans was 0.65 ± 0.48 mm. Intermouse positional reproducibility across 2 mice at the same reference point was 0.76 ± 0.46 mm. The accuracy of dose delivery was 0.67 ± 0.29 mm and 1.01 ± 0.43 mm in the coronal and sagittal planes, respectively. The planned dose delivered to a mouse phantom was 2 Gy at the 50% isodose with a measured thermoluminescent dosimeter dose of 2.9 ± 0.3 Gy. The phosphorylated form of member X of histone family H2A (γH2AX) staining of irradiated mouse brain and mouse aorta demonstrated adjacent tissue sparing. In conclusion, our system for preclinical studies of small animal irradiation using the Gamma Knife is able to accurately deliver intracranial and extracranial targeted focal radiation allowing for preclinical experiments studying focal radiation.

SU‐F‐T‐599: Volume‐Independent Conformity Index for Stereotactic Brain Tumors

PURPOSE To develop a volume-independent metric called the Gaussian Weighted Conformity Index (GWCI) for assessing conformality of stereotactic radiosurgery plans for small brain tumors. METHODS The GWCI calculates bi-directional distance by searching for corresponding points between the prescription isodose line and tumor contour, assigning different scoring weights to tumor coverage with a score of 1.0 being ideal assuming an idealized Gaussian distribution of dose around the tumor. (Figure 1, left) The GWCI penalizes tumor under-dosing three times more heavily than the prescription isodose falling outside the tumor. (Figure 1, middle) A user interface was created to calculate GWCI from images and RT structures (Figure 1, right). Patients receiving radiosurgery were randomly selected and images and RT structures were exported to MiM (MiMVista, Cleveland, OH) to calculate traditional conformality indices (CI). CIs were calculated for 39 tumors from patients receiving Gamma Knife radiosurgery (GKSRS) and from 10 tumors from patients receiving linac-based stereotactic radiosurgery (L-SRS). GWCIs were calculated for 14 tumors from patients receiving GKSRS and for 10 tumors from patients receiving L-SRS. RESULTS Conformality indices calculated from 39 GKSRS plans and 10 L-SRS plans are plotted in Figure 2 demonstrating that as tumour volume gets smaller, conformality index increases. GWCIs for 14 tumors were plotted against CIs and linear regression was performed (Figure 3) yielding GWCI = -.077*CI + 1.044 (R2 = .52). Utilizing this regression, the corresponding GWCI to a traditionally-acceptable CI of 1.5 was calculated as 0.927. CONCLUSION Limitations of current conformity metrics become apparent when applied to radiosurgery treatment plans. A GWCI tool was successfully developed which can be used to accurately score the quality of an individual treatment plan while eliminating small volume effects. A GWCI of 0.93 may be used as a volume-independent cutoff for plan conformality.

SU‐E‐J‐105: Applying Fuzzy Similarity Index and Ground Truth Fuzzy Contour to Evaluate the Segmentation Accuracy

Purpose: To evaluate the segmentation accuracy by using our novel Fuzzy Similarity Index (FSI) and Ground Truth Fuzzy Contour (GTFC) with the consideration of inter‐and intra‐observer variation Methods: We developed GTFC to build consensus truth segmentation(contour) and FSI to score segmentation for an objective and quantitative evaluation of in‐vivo medical image segmentation. GTFC is built by applying Fuzzy theory to consider with inter‐and intra‐observer variation. GTFC has the Fuzzy Membership Function(FMFn) which can assign a weight to each expert depending on their experience, unlike STAPLE. By using GTFC, we formulate a quantitative scoring index to evaluate the segmentation accuracy.When a test segmentation is evaluated, we calculate the membership value of FMFn at every point in the test segmentation(contour). Then, we can make a distribution of membership value as Membership Score Histogram(MSH). We enhanced FSI to make more responsive. While generating a single value index(FSI) from MSH, we adopt the strategy of penalizing lower membership values. The resultant FSI equation is formulated by combining MSH with a penalty constant. We tested the FSI by applying to a brain case. Ten experts segmented a region in the brain and six non‐experts independently delineated the same place. GTFC was created from segmentations of expert to evaluate the accuracy of segmentations of non‐experts. Then, we calculated FSI after making a MSH per test segmentation. Results: The order in higher similarity to GTFC is non‐expert 6, 1, 5, 2, 3, and 4. Non‐expert 2, 3, and 4 are significantly deviated from GTFC. Their FSIs are 0.845, 0.476, 0.125, 0.085, 0.078, and 0.005, respectively. Conclusions:FSI can sensitively reflect the accuracy of test segmentation. It can be used to develop unbiased educational tools or credential process for clinicians. It can be also used to evaluate the performance of automated segmentation tools.

SU‐E‐J‐113: A New Quantification Measure of the Difference Between Two Organ Contours

PURPOSE To develop a measuring method between two contours, which can be used for validating a PTV during IGRT, organ motion and/or deformation studies. METHODS Quantifying the geometric difference between two organ/target surfaces is essential for Radiation therapy planning and delivery. Point-to-surface distance measures have been utilized to evaluate and visualize the local surface differences. However, previously well-known distance measures have critical shortfalls. Normal distance (ND) measure suffers when the reference surface is strongly curved. Minimum distance (MD) measure (a.k.a. Hausdorff distance) suffers when the test surface is strongly curved. Our new distance measure named Error-Proof Distance (EPD) can deal with both difficult cases.EPD measure calculates the maximum value between the Forward Minimum Distance (FMinD) and the Backward Maximum Distance (BMaxD) at each point. The FMinD denotes the minimum distance to the test surface from a point on the reference surface. The BMaxD means the maximum value among the minimum distances from all points of the test surface to the point on the reference surface. We tested EPD using three 2-D contour examples including a 20mm shifted contour, and two 3-D clinical cases. RESULTS In case of 2-D contour examples, ND and MD measure failed in strongly curved areas, but EPD measure outperformed the others. The maximum distance measured between a reference and a 20mm shifted test contour should be equal to 20mm, but ND erroneously measured 24mm. Furthermore, ND reported erroneous distances where the reference surface is strongly curved in 3-D clinical cases. CONCLUSIONS We succeeded to prove that a new EPD is arobust and accurate distance measure to compare two 2D or 3D surfaces. EPD measure can be used to evaluate and visualize the surface difference of organ contours. It is also helpful for proving PTV margin during IGRT, and organ motion and/or deformation studies. This project is partially supported by the Agency for Healthcare Research and Quality (AHRQ) grant 1R18HS017424-01A2.