Observer study-based evaluation of a stochastic and physics-based method to generate oncological PET images
Ziping Liu, Richard Laforest, Joyce Mhlanga, Tyler J. Fraum, Malak Itani, Farrokh Dehdashti, Barry A. Siegel, Abhinav K. Jha
TThis manuscript has been accepted to SPIE Medical Imaging, February 15-19, 2021. Please use the followingreference when citing the manuscript.Liu, Z., Laforest, R., Mhlanga, J., Fraum, T. J., Itani, M., Dehdashti, F., Siegel, B. A., and Jha, A. K.,“Observer study-based evaluation of a stochastic and physics-based method to generate oncological PET images”,Proc. SPIE Medical Imaging, 2021. a r X i v : . [ phy s i c s . m e d - ph ] F e b bserver study-based evaluation of a stochastic andphysics-based method to generate oncological PET images Ziping Liu a , Richard Laforest b , Joyce Mhlanga b , Tyler J. Fraum b , Malak Itani b , FarrokhDehdashti b , Barry A. Siegel b , and Abhinav K. Jha a,ba Department of Biomedical Engineering, Washington University in St. Louis, St. Louis, MO,USA b Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis,MO, USA
ABSTRACT
Objective evaluation of new and improved methods for PET imaging requires access to images with ground truth,as can be obtained through simulation studies. However, for these studies to be clinically relevant, it is importantthat the simulated images are clinically realistic. In this study, we develop a stochastic and physics-based methodto generate realistic oncological two-dimensional (2-D) PET images, where the ground-truth tumor propertiesare known. The developed method extends upon a previously proposed approach . The approach captures theobserved variabilities in tumor properties from actual patient population. Further, we extend that approach tomodel intra-tumor heterogeneity using a lumpy object model. To quantitatively evaluate the clinical realism ofthe simulated images, we conducted a human-observer study. This was a two-alternative forced-choice (2AFC)study with trained readers (five PET physicians and one PET physicist). Our results showed that the readershad an average of ∼
50% accuracy in the 2AFC study. Further, the developed simulation method was able togenerate wide varieties of clinically observed tumor types. These results provide evidence for the application ofthis method to 2-D PET imaging applications, and motivate development of this method to generate 3-D PETimages.
Keywords: positron emission tomography, lung cancer, simulation, observer study, image quality assessment
1. INTRODUCTION
Positron emission tomography (PET) is a widely used imaging modality with multiple clinical applications, inparticular for diagnosis and assessment of treatment response of cancers . Thus, several new and improvedmethods have been developed for oncological PET image reconstruction , segmentation , and quantification .Objective evaluation and optimization of these methods typically requires knowledge of the corresponding groundtruth. For instance, evaluation of segmentation methods typically requires knowledge of the ground-truth tumorboundaries. Similarly, optimization of PET imaging methods for clinical tasks using objective assessment ofimage quality (OAIQ) studies typically requires knowledge of the ground truth. Simulation studies provide amechanism for such evaluation since the ground truth is known in these studies. Further, these studies providethe ability to model in vivo anatomical and physiological properties of the patient, incorporate patient-populationvariability, model imaging-system physics, and generate multiple scan realizations of the same patient to evaluaterepeatability. Even more importantly, this is all done in silico , which is inexpensive and enables optimizing themethod before conducting clinical studies. However, for these evaluation and optimization studies to be clinicallyrelevant, the simulated PET images must be clinically realistic.In simulation-based studies to evaluate oncological PET methods, one set of studies uses synthetic phantoms,such as the NEMA phantom . However, these phantoms have limited ability to model patient anatomy andphysiology. Thus, to improve clinical realism, anthropomorphic phantom-based studies, such as using the XCATphantom , have been conducted . However, recent studies suggest that anthropomorphic phantoms may havelimitations in modeling patient physiology . Another limitation of existing simulation studies is that the tumor is Corresponding author: Abhinav K. Jha ([email protected]) ypically modeled as a spherically shaped structure with limited incorporation of intra-tumor heterogeneity
7, 9, 10 .To incorporate variabilities in patient population and tumor models for more clinically relevant evaluation, Leunget al. developed a simulation-based strategy that uses patient images as backgrounds and generates a widevariety of clinically observed tumor types. However, this strategy had limitations in modeling variabilities in intra-tumor heterogeneity. This heterogeneity is often observed in tumors, and methods to quantify this heterogeneityfrom PET images and evaluate its clinical predictive and prognostic value is a topic of intense research
11, 12 . Tomodel this intra-tumor heterogeneity more realistically, in this manuscript, we propose a lumpy model-basedapproach. Using this approach in conjunction with the simulation-based strategy, we develop a stochastic andphysics-based method to generate clinically realistic PET images.The second contribution of this manuscript is in providing a theoretical premise for an observer-study-basedframework to quantitatively evaluate the clinical realism of simulated images. An important goal of evaluatingthe clinical realism is that the simulation studies should accurately capture the variabilities in patient population.Further, ideally, the distribution of simulated images should match that of real images. Human-observer studieshave been applied to evaluate the clinical realism of simulated images
13, 14 . These studies account for the roleof end users such as radiologists in clinical tasks. Human-observer studies are typically conducted using eitherrating-based methods or forced-choice-based methods. In this study, we consider specifically a forced-choice-based method, namely the two-alternative forced-choice (2AFC) study. We first provide a theoretical premisefor this study to evaluate the realism of simulated images. We then apply the study to quantitatively evaluatethe realism of the simulated PET images generated using our developed simulation method.
2. METHODS
In this section, we first describe the developed stochastic and physics-based method to generate simulated PETimages. The theoretical background and methods for conducting the 2AFC study are subsequently provided.
This study was conducted in the context of simulating the primary tumor in [ F]fluorodeoxyglucose (FDG)-PETimages of patients with lung cancer. The study was retrospective, used clinical imaging data, was approved byour institutional review board, and was HIPAA-compliant with a waiver of informed consent.The method is summarized in Fig. 1. In the first step, a realistic high-resolution tumor model was developedto capture the observed variabilities in tumor properties from an actual patient population (Fig. 1a). For thispurpose, we advanced on a simulation-based strategy proposed by Leung et al. . Briefly, tumor descriptors,including first- and second-order statistics for the shape, size, and tumor-to-background intensity ratio, werefirst extracted from clinical FDG-PET images of patients with lung cancer. Tumor shape was quantified byfive harmonic elliptical Fourier shape descriptors , and tumor size was quantified by diameter and volume. Thedistribution of each tumor descriptor was defined using kernel density estimation. The kernel distribution of eachdescriptor was then sampled, and from the sampled parameters, simulated tumors were generated. In Leung etal. , necrosis within the tumor was modeled by assigning a lower intensity to the tumor core than the rim. Weadvanced on that approach to model intra-tumor heterogeneity more realistically. We used the observation thatthe tracer-uptake patterns within tumors can be modeled as a combination of lumps, where the lump locations,amplitudes, and sizes are random variables. More generally, it is suitable to characterize the tracer uptakewithin a tumor as a random process . Thus, the intra-tumor heterogeneity was modeled using a stochasticlumpy object model. This lumpy object model was inspired by the original lumpy background model , but withsome adaptations to account for intra-tumor heterogeneity. Our lumpy object model was given by f ( r ) = s ( r ) N (cid:88) n =1 Λ( r − c n | a n , σ n ) = s ( r ) N (cid:88) n =1 a n πσ n exp (cid:18) − | r − c n | σ n (cid:19) , (1)where s ( r ) denotes the support for the tumor, N denotes the total number of lumps, Λ( · ) denotes the lumpfunction, r denotes the spatial coordinate in two dimensions, and c n , a n , and σ n denote the center, magnitude,and width of the n th lump function, respectively. To model the tracer uptake as a random process, c n wasniformly distributed within the support of tumor, and a n and σ n were uniformly distributed within a pre-defined range but appropriately scaled based on the clinically extracted values of the tumor-to-backgroundintensity ratios.Through this strategy, high-resolution simulated tumors with known ground-truth properties were generated.Note that the ground truth was not needed for the background. To ensure the clinical realism of tumor back-ground and model inter-patient variability, existing patient images containing lung cavities but with no tumorpresent were selected as tumor background. To ensure that the simulated tumors only appear at visually realisticlocations within the lung cavities, tumor locations were manually identified in advance. The simulated tumorswere then randomly generated and placed at these locations.In the second step (Fig. 1b), forward projections for the simulated tumor and patient background weregenerated using a PET simulation software . The high-resolution simulated tumor and low-resolution patientbackground were passed through corresponding projection models to obtain the projection data. Similar to Maet al. , adding the data in the projection space and then performing reconstruction helped incorporate theimpact of noise texture on the tumor appearance in the reconstructed image. The reconstruction was performedusing a 2-D ordered subset expectation maximization (OSEM) algorithm. Detailed simulation and reconstructionparameters are provided in Table 1. shapesize tumor-to-background ratio Extract tumor descriptors from clinical FDG-PET images of patients with lung cancer Define the kernel distribution of each tumor descriptor using kernel density estimation
Tumor descriptor F r equen cy Model intra-tumor heterogeneity using a stochastic lumpy object model (a) Generating high-resolution realistic tumor models(b) Generating simulated PET images
Patient image with no tumor presentHigh-resolution tumor Forward simulation of the simulated tumor and patient image using the corresponding projection model Reconstructed PET imageData added in the projection space
Figure 1: Description of the developed method to first (a) generate high-resolution realistic tumor models withtumor properties extracted from clinical data and intra-tumor heterogeneity modeled using a stochastic lumpyobject model, and then (b) generate simulated PET images using the patient images and tumor model.able 1: Technical acquisition and reconstruction parameters of the PET system. (FWHM:full-width-half-maximum)Parameters ValuesTransaxial field of view 684 mmPixel size 4.07 mm × To quantitatively evaluate the clinical realism of the simulated PET images, we conducted a 2AFC study. We firstprovide the theoretical background for conducting this study. It is well known that computing the probability ofan observer making a correct assignment in the 2AFC study is the same as computing the AUC of that observer.We first consider a hypothetical scenario, where an ideal observer can be constructed to discriminate betweenthe real and simulated images. Following the treatment in Barrett et al.
18, 19 , but in the context of the task ofdiscriminating between real and simulated images, we show that an ideal-observer AUC of 0 . Denote the sets of simulated and real PET images by ˆf and ˆf (cid:48) , each in M-dimensional space. Consider twohypotheses H and H , where H and H refer to the class of simulated and real PET images, respectively.Denote the conditional probability distribution of the observed data ˆf under hypothesis j by pr( ˆf | H j ). Forconvenience in notation, we define q j ( ˆf ) ≡ pr( ˆf | H j ). In the 2AFC study, an observer is presented with two setsof images ˆf and ˆf (cid:48) such that ˆf is sampled from q ( ˆf ) and ˆf (cid:48) is sampled from q ( ˆf (cid:48) ). The observer is then askedto select the image that they think is the real PET image.The observer computes two test statistics θ ( ˆf ) and θ ( ˆf (cid:48) ) and assigns the image that has the higher teststatistic to H . The assignment is correct if θ ( ˆf (cid:48) ) > θ ( ˆf ). Thus, the probability of a correct assignment can becomputed as follows: pr (cid:104) θ ( ˆf (cid:48) ) > θ ( ˆf ) (cid:105) = (cid:90) ∞ d M ˆf q ( ˆf ) (cid:90) ∞ d M ˆf (cid:48) q ( ˆf (cid:48) ) step (cid:104) θ ( ˆf (cid:48) ) − θ ( ˆf ) (cid:105) , (2)where step( · ) denotes the step function. As shown in Barrett and Myers , Eq. (2) is the same as the equationto compute AUC for an arbitrary test statistic with unknown probability law. Thus, the percentage of timesthat an observer correctly identifies the real PET image is equivalent to the AUC of that observer.We now consider the special case of an ideal observer. An ideal observer can be defined as a decision strategythat computes the likelihood ratio q ( ˆf ) q ( ˆf ) and compares it to a threshold. This likelihood ratio is considered asthe optimal discriminant function and is a sufficient statistic that contains all the information needed to performthe discrimination task. For this ideal observer, we can use the concept of the likelihood-generating function to derive the relationship between the AUC and q ( ˆf ) and q ( ˆf ). Essentially, this likelihood-generating function,defined as G ( · ), provides a lower bound for the AUC as follows:AUC ≥ −
12 exp (cid:20) − G (0) (cid:21) , (3)where G (0) is the likelihood-generating function evaluated at origin. G (0) can further be expressed in terms of q ( ˆf ) and q ( ˆf ), i.e. G (0) = − (cid:20)(cid:90) ∞ d M ˆf (cid:113) q ( ˆf ) q ( ˆf ) (cid:21) . (4)rom Eq. (3), we see that a lower bound of AUC = 0 . G (0) = 0. From Eq. (4), G (0) = 0 is achieved when q ( ˆf ) = q ( ˆf ). Thus, an ideal-observer AUC of 0 . .
5, we may infer that the distribution of the simulated images is close tothat of the real images. Thus, our 2AFC study provides a rigorous and practical mechanism to validate theclinical realism of the simulated images. We next describe the design of this observer study to evaluate theclinical realism of the PET images generated using our simulation method.
To conduct the 2AFC study, we developed a web-based app (Fig. 2). During the study, the trained readerswere shown two images side-by-side at a time, one a real patient image sampled from q ( ˆf ) and the other asimulated PET image sampled from q ( ˆf ) using our simulation method. As described in Sec. 2.2.1, the readerswere instructed that the task was to identify the real PET image that they thought had the real tumor. Thetumor location was shown in the images to ensure that the readers were focusing on the tumor-realism task, andnot implicitly treating this as a tumor-detection task. To facilitate a robust observer study, the app incorporatedfunctionalities provided by clinical software, including the option to invert the image intensities and adjustthe image contrast. Further, the app used MySQL to manage the readers’ records, easing data collection andanalysis. Home
WashU Jha Lab
Select Select
Figure 2: Interface of the web-based application presented to readers in the 2AFC study.Six trained readers, which included five board-certified radiologists with specialization in nuclear medicineand many years of experience in PET (B.A.S., F.D., J.C.M., T.J.F., M.I.) and one experienced nuclear-medicinephysicist (R.L.), participated as readers in this study. The readers performed this test for 50 image pairs. Wecomputed the fraction of times that each reader correctly identified the patient image. As shown in Sec. 2.2.1,a percent accuracy close to 50% for a well-trained reader suggests a high similarity between the distributions ofthe real and simulated images.
3. RESULTS
Fig. 3 shows the representative simulated images generated using our simulation method (Sec. 2.1). These imagesdemonstrate that the method can generate a wide variety of clinically observed tumor types, including (a) smalltumors, (b) tumors with multiple hot spots, and (c) tumors with necrotic cores. c) Tumors with necrotic core(b) Tumors with multiple hot spots(a) Small tumors
Figure 3: Representative high-resolution tumor models with different tumor types and the corresponding recon-structed PET images. Tumor locations in the reconstructed images are marked by the arrows.Table 2 lists the results of the 2AFC study. Each trained reader identified the real images accurately in onlyapproximately 50% of the cases, suggesting that the distribution of the simulated images closely matches thatof the real images (Sec. 2.2.1). Examples of simulated images that were incorrectly identified by at least half ofthe readers are shown in Fig. 4. Overall, these results demonstrate that the simulated images generated usingthe developed simulation method are highly realistic.Table 2: Percent accuracy for each trained reader participating in the 2AFC test. (NM: nuclear medicine)Percent accuracyNM physician 1 44%NM physician 2 50%NM physician 3 58%NM physician 4 58%NM physician 5 44%NM physicist 58%Figure 4: Representative simulated images incorrectly identified by at least 3 readers. Tumor locations aremarked by the red arrows. . DISCUSSIONS
In this manuscript, we first developed a stochastic and physics-based method to generate 2-D oncological PETimages. Generation of realistic images is valuable for evaluating imaging methods for clinical tasks. For thispurpose, several techniques have been developed to generate synthetic medical images. These include traditionaldata augmentation techniques, such as translation, scaling, shear, and rotation. However, these techniquesfundamentally produce highly correlated images and do not show an improvement in training performance incertain tasks . Another approach to image generation is the use of generative adversarial network (GAN) .GAN-based image generation has shown promise in multiple imaging modalities
20, 22 . However, such techniquesuffers from limitations of training instability and requirement of large-scale training data. In addition, inthe context of lung cancer, definition of ground-truth tumor properties is not well defined unlike our simulationmethod. Further, GAN-based techniques do not directly exploit the imaging physics and do not incorporate thevariability in instrumentation . In contrast, our method does not suffer from these limitations.High realism of our simulated PET images, as quantitatively validated using the 2AFC study, motivates theapplication of our method to a broader range of quantitative evaluation studies. These include evaluation ofimaging methods for segmentation tasks. For example, Liu et al.
23, 24 developed a deep-learning-based estimationapproach to PET segmentation that estimates the tumor-fraction area within each pixel in a 2-D PET image. Oursimulation method provides a mechanism to objectively evaluate the performance of this segmentation approachusing clinically realistic simulation studies, where the ground-truth tumor boundaries are known. Our methodcan also be used to evaluate other imaging methods for metric quantification. In this context, simulation studieshave been used to evaluate the performance of PVE correction techniques in PET. Existing simulation-basedevaluation studies typically assume simplistic tumor models such as spherically shaped tumors, and thus donot incorporate variability in actual patient population. The ability of our method to generate wide varietiesof clinically observed tumor types will make the evaluation studies more clinically relevant. Additionally, themethod can be used to evaluate imaging methods for OAIQ-based studies in detection
27, 28 and quantification tasks. Further, our proposed lumpy model-based approach to model intra-tumor heterogeneity could be usedto evaluate image-reconstruction methods, as well as methods to quantify intra-tumor heterogeneity from PETimages
7, 12, 33 . In all these studies, access to clinically realistic tumor models will make the studies even moreclinically relevant.To conduct the 2AFC study, we developed a web-based app. The goal was to provide a mechanism to increasethe flexibility of conducting this study. The web-based app eliminates the need to have the readers participatein the study on site. In addition, this app provides functionalities to create a more familiar user interface designas observed in common clinical software. All these features increase the rigor and clinical relevance of the 2AFCstudy. This web app design can be naturally extended to conduct the 2AFC study for other image-simulationmethods such as GAN-based approaches and is generalizable for other imaging modalities. Pending necessarypermissions, we will publish this app on GitHub for wider usage by the image-science community.Limitations of our study include the fact that the developed simulation method generates 2-D tumor modelson transaxial image slices. While the method is less computationally expensive in 2-D tumor modeling, developing3-D tumor models is important for incorporating the whole tumor features and thus is an important research area.Another area of future work is to simulate the PET physics and system instrumentation even more accurately,using approaches such as GATE . One limitation in the observer study design is that the readers are typicallytrained on the task of detecting the tumor and not of discriminating the simulated tumor from the real tumor.One strategy to address this issue is to present examples of simulated images and real images to the readersprior to the 2AFC study and thus train them on this discrimination task. Another limitation of our study isthat the choice of parameters for the lumpy object model to generate intra-tumor heterogeneity was based onvisual inspection and not quantitatively obtained. In this context, several methods have been developed to fitstatistical models of object based on image data
35, 36 . Thus, extending our method to statistically fit lumpyobject model using patient data provides a mechanism to address this limitation.
5. CONCLUSION
In this manuscript, we quantitatively evaluated a stochastic and physics-based method to generate 2-D oncologi-cal PET images with known ground-truth tumor properties. A trained-reader-based observer study demonstratedhat the method yielded highly realistic simulated images. In addition, the method demonstrates the ability togenerate a wide variety of clinically observed tumor types, including tumors with complex intra-tumor hetero-geneity. These results motivate the application of the method to a broader range of clinically relevant quantitativeevaluation studies. Further, the theoretical premise for the observer study provides a foundation for the use ofsuch observer-based studies to evaluate the clinical realism of images generated using other simulation-basedapproaches.
ACKNOWLEDGEMENTS
Financial support for this work was provided by the Department of Biomedical Engineering and the MallinckrodtInstitute of Radiology at Washington University in St. Louis and an NVIDIA GPU grant. We also thank QiyeTan for the help with developing the web-based app for the observer study.
REFERENCES [1] Leung, K. H., Marashdeh, W., Wray, R., Ashrafinia, S., Pomper, M. G., Rahmim, A., and Jha, A. K., “Aphysics-guided modular deep-learning based automated framework for tumor segmentation in PET,”
Phys.Med. Biol. (2020).[2] Zhu, A., Lee, D., and Shim, H., “Metabolic Positron Emission Tomography imaging in cancer detection andtherapy response,” in [
Seminars in Oncology ], (1), 55–69, Elsevier (2011).[3] Tong, S., Alessio, A. M., and Kinahan, P. E., “Image reconstruction for PET/CT scanners: past achieve-ments and future challenges,” Imaging in medicine (5), 529 (2010).[4] Foster, B., Bagci, U., Mansoor, A., Xu, Z., and Mollura, D. J., “A review on segmentation of PositronEmission Tomography images,” Comput. Biol. Med. , 76–96 (2014).[5] Tomasi, G., Turkheimer, F., and Aboagye, E., “Importance of quantification for the analysis of PET data inOncology: review of current methods and trends for the future,” Mol. Imaging Biol. (2), 131–146 (2012).[6] Barrett, H. H., Kupinski, M. A., M¨ueller, S., Halpern, H. J., Morris III, J. C., and Dwyer, R., “Objectiveassessment of image quality VI: Imaging in radiation therapy,” Phys. Med. Biol. (22), 8197 (2013).[7] Nyflot, M. J., Yang, F., Byrd, D., Bowen, S. R., Sandison, G. A., and Kinahan, P. E., “Quantitativeradiomics: impact of stochastic effects on textural feature analysis implies the need for standards,” J. Med.Imaging (4), 041002 (2015).[8] Segars, W. P., Sturgeon, G., Mendonca, S., Grimes, J., and Tsui, B. M., “4D XCAT phantom for multi-modality imaging research,” Med. Phys. (9), 4902–4915 (2010).[9] Lorsakul, A., Li, Q., Trott, C. M., Hoog, C., Petibon, Y., Ouyang, J., Laine, A. F., and El Fakhri, G., “4Dnumerical observer for lesion detection in respiratory-gated PET,” Med. Phys. (10), 102504 (2014).[10] Cysouw, M. C., Kramer, G. M., Hoekstra, O. S., Frings, V., De Langen, A. J., Smit, E. F., van denEertwegh, A. J., Oprea-Lager, D. E., and Boellaard, R., “Accuracy and precision of partial-volume correctionin oncological PET/CT studies,” J. Nucl. Med (10), 1642–1649 (2016).[11] Cook, G. J., Azad, G., Owczarczyk, K., Siddique, M., and Goh, V., “Challenges and promises of PETradiomics,” Int. J. Radiat. Oncol. Biol. Phys. (4), 1083–1089 (2018).[12] Mena, E., Sheikhbahaei, S., Taghipour, M., Jha, A. K., Vicente, E., Xiao, J., and Subramaniam, R. M.,“18F-FDG PET/CT metabolic tumor volume and intra-tumoral heterogeneity in pancreatic adenocarcino-mas: Impact of dual-time-point and segmentation methods,”
Clin. Nucl. Med. (1), e16 (2017).[13] Chen, B., Ma, C., Leng, S., Fidler, J. L., Sheedy, S. P., McCollough, C. H., Fletcher, J. G., and Yu,L., “Validation of a projection-domain insertion of liver lesions into CT images,” Acad. Radiol. (10),1221–1229 (2016).[14] Ma, C., Chen, B., Yu, L., Koo, C. W., Leng, S., Takahashi, E. A., Fletcher, J. G., Levin, D. L., Kuzo,R. S., Viers, L. D., et al., “Evaluation of a projection-domain lung nodule insertion technique in thoracicComputed Tomography,” J. Med. Imaging (1), 013510 (2017).[15] Kuhl, F. P. and Giardina, C. R., “Elliptic Fourier features of a closed contour,” Comput. Gr. Image Pro-cess. (3), 236–258 (1982).16] Henscheid, N., Clarkson, E., Myers, K. J., and Barrett, H. H., “Physiological random processes in precisioncancer therapy,” PLoS One (6), e0199823 (2018).[17] Rolland, J. and Barrett, H. H., “Effect of random background inhomogeneity on observer detection perfor-mance,” J. Opt. Soc. Am. A (5), 649–658 (1992).[18] Barrett, H. H., Abbey, C. K., and Clarkson, E., “Objective assessment of image quality. iii. roc metrics,ideal observers, and likelihood-generating functions,” JOSA A (6), 1520–1535 (1998).[19] Barrett, H. H. and Myers, K. J., [ Foundations of Image Science ], John Wiley & Sons (2013).[20] Shin, H.-C., Tenenholtz, N. A., Rogers, J. K., Schwarz, C. G., Senjem, M. L., Gunter, J. L., Andriole, K. P.,and Michalski, M., “Medical image synthesis for data augmentation and anonymization using generativeadversarial networks,” in [
International workshop on simulation and synthesis in medical imaging ], 1–11,Springer (2018).[21] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio,Y., “Generative adversarial nets,” in [
Advances in neural information processing systems ], 2672–2680 (2014).[22] Yi, X., Walia, E., and Babyn, P., “Generative adversarial network in medical imaging: A review,”
MedImage Anal , 101552 (2019).[23] Liu, Z., Laforest, R., Moon, H. S., Mhlanga, J., Fraum, T., Itani, M., Mintz, A., Dehdashti, F., Siegel, B.,and Jha, A., “An estimation-based segmentation method to delineate tumors in PET images.,” J. Nucl.Med. (supplement 1), 447–447 (2020).[24] Liu, Z., Mhlanga, J. C., Laforest, R., Derenoncourt, P.-R., Siegel, B. A., and Jha, A. K., “An estimation-based approach to tumor segmentation in oncological PET,” In review (2021).[25] Bettinardi, V., Castiglioni, I., De Bernardi, E., and Gilardi, M., “PET quantification: strategies for partialvolume correction,”
Clin. Transl. Imaging (3), 199–218 (2014).[26] Cysouw, M. C., Kramer, G. M., Schoonmade, L. J., Boellaard, R., De Vet, H. C., and Hoekstra, O. S.,“Impact of partial-volume correction in oncological PET studies: a systematic review and meta-analysis,” EJNMMI (12), 2105–2116 (2017).[27] Gifford, H., “Efficient visual-search model observers for PET,” Br. J. Radiol. (1039), 20140017 (2014).[28] Yu, Z., Rahman, M. A., Schindler, T., Gropler, R., Laforest, R., Wahl, R., and Jha, A., “AI-based methodsfor nuclear-medicine imaging: Need for objective task-specific evaluation,” J. Nucl. Med (supplement 1),575–575 (2020).[29] Jha, A. K., Caffo, B., and Frey, E. C., “A no-gold-standard technique for objective assessment of quantitativenuclear-medicine imaging methods,” Phys. Med. Biol. (7), 2780 (2016).[30] Jha, A. K., Mena, E., Caffo, B. S., Ashrafinia, S., Rahmim, A., Frey, E. C., and Subramaniam, R. M.,“Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesionsegmentation in positron emission tomography,” J. Med. Imaging (1), 011011 (2017).[31] Liu, J., Liu, Z., Moon, H. S., Mhlanga, J., and Jha, A., “A no-gold-standard technique for objectiveevaluation of quantitative nuclear-medicine imaging methods in the presence of correlated noise,” J. Nucl.Med. (supplement 1), 523–523 (2020).[32] Jha, A., Wray, R., Marcus, C., Subramaniam, R., and Frey, E., “A no-gold-standard framework to evaluateFDG-PET tumor-segmentation methods on the task of measuring prognostic biomarkers for lung-cancertreatment,” J. Nucl. Med. (supplement 1), 112–112 (2017).[33] Mena, E., Taghipour, M., Sheikhbahaei, S., Jha, A. K., Rahmim, A., Solnes, L., and Subramaniam, R. M.,“Value of intra-tumoral metabolic heterogeneity and quantitative 18F-FDG PET/CT parameters to predictprognosis, in patients with HPV-positive primary oropharyngeal squamous cell carcinoma,” Clin. Nucl.Med. (5), e227 (2017).[34] Jan, S., Santin, G., Strul, D., Staelens, S., Assie, K., Autret, D., Avner, S., Barbier, R., Bardies, M.,Bloomfield, P., et al., “GATE: a simulation toolkit for PET and SPECT,” Phys. Med. Biol. (19), 4543(2004).[35] Kupinski, M. A., Clarkson, E., Hoppin, J. W., Chen, L., and Barrett, H. H., “Experimental determinationof object statistics from noisy images,” JOSA A (3), 421–429 (2003).[36] Samei, E., Hoye, J., Zheng, Y., Solomon, J. B., and Marin, D., “Design and fabrication of heterogeneouslung nodule phantoms for assessing the accuracy and variability of measured texture radiomics features inCT,” J. Med. Imaging6