Darrin Byrd

Quantitative radiomics: impact of stochastic effects on textural feature analysis implies the need for standards

Abstract. Image heterogeneity metrics such as textural features are an active area of research for evaluating clinical outcomes with positron emission tomography (PET) imaging and other modalities. However, the effects of stochastic image acquisition noise on these metrics are poorly understood. We performed a simulation study by generating 50 statistically independent PET images of the NEMA IQ phantom with realistic noise and resolution properties. Heterogeneity metrics based on gray-level intensity histograms, co-occurrence matrices, neighborhood difference matrices, and zone size matrices were evaluated within regions of interest surrounding the lesions. The impact of stochastic variability was evaluated with percent difference from the mean of the 50 realizations, coefficient of variation and estimated sample size for clinical trials. Additionally, sensitivity studies were performed to simulate the effects of patient size and image reconstruction method on the quantitative performance of these metrics. Complex trends in variability were revealed as a function of textural feature, lesion size, patient size, and reconstruction parameters. In conclusion, the sensitivity of PET textural features to normal stochastic image variation and imaging parameters can be large and is feature-dependent. Standards are needed to ensure that prospective studies that incorporate textural features are properly designed to measure true effects that may impact clinical outcomes.

Evaluation of Cross-Calibrated ⁶⁸Ge/⁶⁸Ga Phantoms for Assessing PET/CT Measurement Bias in Oncology Imaging for Single- and Multicenter Trials

Quantitative PET imaging is an important tool for clinical trials evaluating the response of cancers to investigational therapies. The standardized uptake value, used as a quantitative imaging biomarker, is dependent on multiple parameters that may contribute bias and variability. The use of long-lived, sealed PET calibration phantoms offers the advantages of known radioactivity activity concentration and simpler use than aqueous phantoms. We evaluated scanner and dose calibrator sources from two batches of commercially available kits, together at a single site and distributed across a local multicenter PET imaging network. We found that radioactivity concentration was uniform within the phantoms. Within the regions of interest drawn in the phantom images, coefficients of variation of voxel values were less than 2%. Across phantoms, coefficients of variation for mean signal were close to 1%. Biases of the standardized uptake value estimated with the kits varied by site and were seen to change in time by approximately ±5%. We conclude that these biases cannot be assumed constant over time. The kits provide a robust method to monitor PET scanner and dose calibrator biases, and resulting biases in standardized uptake values.

TU-AB-BRA-04: Quantitative Radiomics: Sensitivity of PET Textural Features to Image Acquisition and Reconstruction Parameters Implies the Need for Standards

Purpose: Despite increased use of heterogeneity metrics for PET imaging, standards for metrics such as textural features have yet to be developed. We evaluated the quantitative variability caused by image acquisition and reconstruction parameters on PET textural features. Methods: PET images of the NEMA IQ phantom were simulated with realistic image acquisition noise. 35 features based on intensity histograms (IH), co-occurrence matrices (COM), neighborhood-difference matrices (NDM), and zone-size matrices (ZSM) were evaluated within lesions (13, 17, 22, 28, 33 mm diameter). Variability in metrics across 50 independent images was evaluated as percent difference from mean for three phantom girths (850, 1030, 1200 mm) and two OSEM reconstructions (2 iterations, 28 subsets, 5 mm FWHM filtration vs 6 iterations, 28 subsets, 8.6 mm FWHM filtration). Also, patient sample size to detect a clinical effect of 30% with Bonferroni-corrected α=0.001 and 95% power was estimated. Results: As a class, NDM features demonstrated greatest sensitivity in means (5–50% difference for medium girth and reconstruction comparisons and 10–100% for large girth comparisons). Some IH features (standard deviation, energy, entropy) had variability below 10% for all sensitivity studies, while others (kurtosis, skewness) had variability above 30%. COM and ZSM features had complex sensitivities; correlation, energy, entropy (COM) and zone percentage, short-zone emphasis, zone-size non-uniformity (ZSM) had variability less than 5% while other metrics had differences up to 30%. Trends were similar for sample size estimation; for example, coarseness, contrast, and strength required 12, 38, and 52 patients to detect a 30% effect for the small girth case but 38, 88, and 128 patients in the large girth case. Conclusion: The sensitivity of PET textural features to image acquisition and reconstruction parameters is large and feature-dependent. Standards are needed to ensure that prospective trials which incorporate textural features are properly designed to detect clinical endpoints. Supported by NIH grants R01 CA169072, U01 CA148131, NCI Contract (SAIC-Frederick) 24XS036-004, and a research contract from GE Healthcare.

Measuring temporal stability of positron emission tomography standardized uptake value bias using long-lived sources in a multicenter network

Abstract. Positron emission tomography (PET) is a quantitative imaging modality, but the computation of standardized uptake values (SUVs) requires several instruments to be correctly calibrated. Variability in the calibration process may lead to unreliable quantitation. Sealed source kits containing traceable amounts of Ge68/Ga68 were used to measure signal stability for 19 PET scanners at nine hospitals in the National Cancer Institute’s Quantitative Imaging Network. Repeated measurements of the sources were performed on PET scanners and in dose calibrators. The measured scanner and dose calibrator signal biases were used to compute the bias in SUVs at multiple time points for each site over a 14-month period. Estimation of absolute SUV accuracy was confounded by bias from the solid phantoms’ physical properties. On average, the intrascanner coefficient of variation for SUV measurements was 3.5%. Over the entire length of the study, single-scanner SUV values varied over a range of 11%. Dose calibrator bias was not correlated with scanner bias. Calibration factors from the image metadata were nearly as variable as scanner signal, and were correlated with signal for many scanners. SUVs often showed low intrascanner variability between successive measurements but were also prone to shifts in apparent bias, possibly in part due to scanner recalibrations that are part of regular scanner quality control. Biases of key factors in the computation of SUVs were not correlated and their temporal variations did not cancel out of the computation. Long-lived sources and image metadata may provide a check on the recalibration process.

Improved attenuation correction for respiratory gated PET/CT with extended-duration cine CT: a simulation study

Due to the wide variability of intra-patient respiratory motion patterns, traditional short-duration cine CT used in respiratory gated PET/CT may be insufficient to match the PET scan data, resulting in suboptimal attenuation correction that eventually compromises the PET quantitative accuracy. Thus, extending the duration of cine CT can be beneficial to address this data mismatch issue. In this work, we propose to use a long-duration cine CT for respiratory gated PET/CT, whose cine acquisition time is ten times longer than a traditional short-duration cine CT. We compare the proposed long-duration cine CT with the traditional short-duration cine CT through numerous phantom simulations with 11 respiratory traces measured during patient PET/CT scans. Experimental results show that, the long-duration cine CT reduces the motion mismatch between PET and CT by 41% and improves the overall reconstruction accuracy by 42% on average, as compared to the traditional short-duration cine CT. The long-duration cine CT also reduces artifacts in PET images caused by misalignment and mismatch between adjacent slices in phase-gated CT images. The improvement in motion matching between PET and CT by extending the cine duration depends on the patient, with potentially greater benefits for patients with irregular breathing patterns or larger diaphragm movements.

An algorithm for automated ROI definition in water or epoxy-filled NEMA NU-2 image quality phantoms

Drawing regions of interest (ROIs) in positron emission tomography/computed tomography (PET/CT) scans of the National Electrical Manufacturers Association (NEMA) NU‐2 Image Quality (IQ) phantom is a time‐consuming process that allows for interuser variability in the measurements. In order to reduce operator effort and allow batch processing of IQ phantom images, we propose a fast, robust, automated algorithm for performing IQ phantom sphere localization and analysis. The algorithm is easily altered to accommodate different configurations of the IQ phantom. The proposed algorithm uses information from both the PET and CT image volumes in order to overcome the challenges of detecting the smallest spheres in the PET volume. This algorithm has been released as an open‐source plug‐in to the Osirix medical image viewing software package. We test the algorithm under various noise conditions, positions within the scanner, air bubbles in the phantom spheres, and scanner misalignment conditions. The proposed algorithm shows runtimes between 3 and 4 min and has proven to be robust under all tested conditions, with expected sphere localization deviations of less than 0.2 mm and variations of PET ROI mean and maximum values on the order of 0.5% and 2%, respectively, over multiple PET acquisitions. We conclude that the proposed algorithm is stable when challenged with a variety of physical and imaging anomalies, and that the algorithm can be a valuable tool for those who use the NEMA NU‐2 IQ phantom for PET/CT scanner acceptance testing and QA/QC. PACS number: 87.57.C‐Drawing regions of interest (ROIs) in positron emission tomography/computed tomography (PET/CT) scans of the National Electrical Manufacturers Association (NEMA) NU-2 Image Quality (IQ) phantom is a time-consuming process that allows for interuser variability in the measurements. In order to reduce operator effort and allow batch processing of IQ phantom images, we propose a fast, robust, automated algorithm for performing IQ phantom sphere localization and analysis. The algorithm is easily altered to accommodate different configurations of the IQ phantom. The proposed algorithm uses information from both the PET and CT image volumes in order to overcome the challenges of detecting the smallest spheres in the PET volume. This algorithm has been released as an open-source plug-in to the Osirix medical image viewing software package. We test the algorithm under various noise conditions, positions within the scanner, air bubbles in the phantom spheres, and scanner misalignment conditions. The proposed algorithm shows runtimes between 3 and 4 min and has proven to be robust under all tested conditions, with expected sphere localization deviations of less than 0.2 mm and variations of PET ROI mean and maximum values on the order of 0.5% and 2%, respectively, over multiple PET acquisitions. We conclude that the proposed algorithm is stable when challenged with a variety of physical and imaging anomalies, and that the algorithm can be a valuable tool for those who use the NEMA NU-2 IQ phantom for PET/CT scanner acceptance testing and QA/QC. PACS number: 87.57.C.

Test-retest reproducibility of FDG-PET/CT uptake in cancer patients within a qualified and calibrated local network

Calibration and reproducibility of quantitative 18F-FDG PET measures are essential for adopting integral 18F-FDG PET/CT biomarkers and response measures in multicenter clinical trials. We implemented a multicenter qualification process using National Institute of Standards and Technology–traceable reference sources for scanners and dose calibrators, and similar patient and imaging protocols. We then assessed SUV in patient test–retest studies. Methods: Five 18F-FDG PET/CT scanners from 4 institutions (2 in a National Cancer Institute–designated Comprehensive Cancer Center, 3 in a community-based network) were qualified for study use. Patients were scanned twice within 15 d, on the same scanner (n = 10); different but same model scanners within an institution (n = 2); or different model scanners at different institutions (n = 11). SUVmax was recorded for lesions, and SUVmean for normal liver uptake. Linear mixed models with random intercept were fitted to evaluate test–retest differences in multiple lesions per patient and to estimate the concordance correlation coefficient. Bland–Altman plots and repeatability coefficients were also produced. Results: In total, 162 lesions (82 bone, 80 soft tissue) were assessed in patients with breast cancer (n = 17) or other cancers (n = 6). Repeat scans within the same institution, using the same scanner or 2 scanners of the same model, had an average difference in SUVmax of 8% (95% confidence interval, 6%–10%). For test–retest on different scanners at different sites, the average difference in lesion SUVmax was 18% (95% confidence interval, 13%–24%). Normal liver uptake (SUVmean) showed an average difference of 5% (95% confidence interval, 3%–10%) for the same scanner model or institution and 6% (95% confidence interval, 3%–11%) for different scanners from different institutions. Protocol adherence was good; the median difference in injection-to-acquisition time was 2 min (range, 0–11 min). Test–retest SUVmax variability was not explained by available information on protocol deviations or patient or lesion characteristics. Conclusion: 18F-FDG PET/CT scanner qualification and calibration can yield highly reproducible test–retest tumor SUV measurements. Our data support use of different qualified scanners of the same model for serial studies. Test–retest differences from different scanner models were greater; more resolution-dependent harmonization of scanner protocols and reconstruction algorithms may be capable of reducing these differences to values closer to same-scanner results.

Multicenter survey of PET/CT protocol parameters that affect standardized uptake values

Abstract. Clinical trials that evaluate cancer treatments may benefit from positron emission tomography (PET) imaging, which for many cancers can discriminate between effective and ineffective treatments. However, the image metrics used to quantify disease and evaluate treatment may be biased by many factors related to clinical protocols and PET system settings, many of which are site- and/or manufacturer-specific. An observational study was conducted using two surveys that were designed to record key sources of bias and variability in PET imaging. These were distributed to hospitals across the United States. The first round of surveys was designed and distributed by the American College of Radiology’s Centers of Quantitative Imaging Excellence program in 2011. The second survey expanded on the first and was completed by the National Cancer Institute’s Quantitative Imaging Network. Sixty-three sites responded to the first survey and 36 to the second. Key imaging parameters varied across participating sites. The range of reported methods for image acquisition and reconstruction suggests that signal biases are not matched between sites. Patient preparation was also inconsistent, potentially contributing additional variability. For multicenter clinical trials, efforts to control biases through standardization of imaging procedures should precede patient measurements.

Abstract P4-02-05: Test-retest fidelity of FDG SUVmax in bone and non-boney metastatic breast cancer lesions in local area network PET/CT scanners

Background: Metabolic activity in lesions, measured by FDG-PET, is often used for assessing tumor aggressiveness and response to therapy. Patients may be scanned on different machines, so quantitative measurements should be reproducible. Reducing SUV variability in PET machines throughout a local network can aid in monitoring patient response to therapy and increase access to clinical trials. Methods: Eighteen female patients with advanced or metastatic breast cancer underwent paired FDG PET/CT test-retest studies with 1-15 days between scans, and without interim change in treatment. Ten patients were studied in the same scanner and 8 patients were studied in 2 different scanners. Five different PET/CT scanners were used (2 GE DSTE, 2 Siemens (BioGraph 6 and mCT), 1 Philips Ingenuity TF). Each PET/CT scanner was calibrated using NIST-traceable reference sources to characterize and reduce variability. All of the images were interpreted by two separate reviewers. SUVmax values in lesions, corresponding normal tissue, and normal liver were collected. Linear mixed models with random intercept (patient effects) were fitted to compare differences in log(|SUVmax % difference|+.01) in multiple lesions per patient. Results: SUVmax was assessed in a total of 130 lesions (75 bone). The median number of lesions per patient was 5 (range 1-17). Average SUVmax ranged from 1.0 to 18.2 (mean±SD = 6.0±3.2). The median SUVmax difference was 0.4 (8%) for 47 lesions imaged twice in the same scanner, and was 0.6 (13%) for 83 lesions imaged in two different scanners. In a multivariable linear mixed effects model, SUVmax for different scanners within the same institution did not differ more than for the same scanner (p=0.39), but repeat scans with different scanners and site personnel at had an average of 78% greater percentage difference in SUVmax than for the same scanner (p=0.009). In the same model, the average percent difference in SUVmax for bone lesions was estimated as 30% lower than for other sites (p=0.06, 95% confidence interval 0-50%). Examining normal liver uptake, the median SUVmean was 2.5 (range 1.9-3.1) with an median 6.5% difference between measurements (range 1.1%-23.7%) that did not appear to differ based on scanners used for repeat measurements (p=0.47). Conclusions: The variability in quantitative FDG SUVmax between scans is modest, suggesting reliable reproducibility in appropriately calibrated settings. In our study, bone lesions had somewhat higher fidelity than other tumor sites. Additional studies will address variability in other cancer types. Careful calibration and monitoring of PET/CT scanners, and consistent imaging protocols are necessary in clinical trials that utilize quantitative PET/CT imaging in order to confidently interpret results. Research Support: NIH grant U01-CA148131 and NCI-SAIC Contract 24XS036-004. Citation Format: Linden HM, Peterson LM, Kurland B, Roberts T, Specht J, Shields AT, Novakova A, Christopfel R, Byrd D, Muzi M, Mankoff DA, Kinahan P. Test-retest fidelity of FDG SUVmax in bone and non-boney metastatic breast cancer lesions in local area network PET/CT scanners [abstract]. In: Proceedings of the 2016 San Antonio Breast Cancer Symposium; 2016 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2017;77(4 Suppl):Abstract nr P4-02-05.

TU-F-CAMPUS-J-04: Impact of Voxel Anisotropy On Statistic Texture Features of Oncologic PET: A Simulation Study

Purpose: Texture metrics extracted from oncologic PET have been investigated with respect to their usefulness as definitive indicants for prognosis in a variety of cancer. Metric calculation is often based on cubic voxels. Most commonly used PET scanners, however, produce rectangular voxels, which may change texture metrics. The objective of this study was to examine the variability of PET texture feature metrics resulting from voxel anisotropy. Methods: Sinograms of NEMA NU-2 phantom for 18F-FDG were simulated using the ASIM simulation tool. The obtained projection data was reconstructed (3D-OSEM) on grids of cubic and rectangular voxels, producing PET images of resolution of 2.73x2.73x3.27mm3 and 3.27x3.27x3.27mm3, respectively. An interpolated dataset obtained from resampling the rectangular voxel data for isotropic voxel dimension (3.27mm) was also considered. For each image dataset, 28 texture parameters based on grey-level co-occurrence matrices (GLCOM), intensity histograms (GLIH), neighborhood difference matrices (GLNDM), and zone size matrices (GLZSM) were evaluated within lesions of diameter of 33, 28, 22, and 17mm. Results: In reference to the isotopic image data, texture features appearing on the rectangular voxel data varied with a range of -34-10% for GLCOM based, -31-39% for GLIH based, -80 -161% for GLNDM based, and −6–45% for GLZSM based while varied with a range of -35-23% for GLCOM based, -27-35% for GLIH based, -65-86% for GLNDM based, and -22 -18% for GLZSM based for the interpolated image data. For the anisotropic data, GLNDM_cplx exhibited the largest extent of variation (161%) while GLZSM_zp showed the least (<1%). As to the interpolated data, GLNDM_busy varied the most (86%) while GLIH_engy varied the least (<1%). Conclusion: Variability of texture appearance on oncologic PET with respect to voxel representation is substantial and feature-dependent. It necessitates consideration of standardized voxel representation for inter-institution studies attempting to validate prognostic values of PET texture features in cancer treatment.