Neural Network for 3D ICF Shell Reconstruction from Single Radiographs
Bradley T. Wolfe, Zhizhong Han, Jonathan S. Ben-Benjamin, John L. Kline, David S. Montgomery, Elizabeth C. Merritt, Paul A. Keiter, Eric Loomis, Brian M. Patterson, Lindsey Kuettner, Zhehui Wang
NNeural Network for 3D ICF Shell Reconstruction from Single Radiographs a) Bradley T. Wolfe, Zhizhong Han, Jonathan S. Ben-Benjamin, John L. Kline, David S. Montgomery, Elizabeth C. Merritt, Paul A. Keiter, Eric Loomis, Brian M. Patterson, Lindsey Kuettner, and Zhehui Wang Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA Department of Computer Science, University of Maryland, College Park, Maryland, 20742,USA (Dated: 13 January 2021)
In inertial confinement fusion (ICF), X-ray radiography is a critical diagnostic for measuring implosion dy-namics, which contains rich 3D information. Traditional methods for reconstructing 3D volumes from 2Dradiographs, such as filtered backprojection, require radiographs from at least two different angles or lines ofsight (LOS). In ICF experiments, space for diagnostics is limited and cameras that can operate on the fasttimescales are expensive to implement, limiting the number of projections that can be acquired. To improvethe imaging quality as a result of this limitation, convolutional neural networks (CNN) have recently beenshown to be capable of producing 3D models from visible light images or medical X-ray images rendered byvolumetric computed tomography LOS (SLOS). We propose a CNN to reconstruct 3D ICF spherical shellsfrom single radiographs. We also examine sensitivity of the 3D reconstruction to different illumination modelsusing preprocessing techniques such as pseudo-flat fielding. To resolve the issue of the lack of 3D supervision,we show that training the CNN utilizing synthetic radiographs produced by known simulation methods allowsfor reconstruction of experimental data as long as the experimental data is similar to the synthetic data. Wealso show that the CNN allows for 3D reconstruction of shells that possess low mode asymmetries. Furthercomparisons of the 3D reconstructions with direct multiple LOS measurements are justified.Keywords: Machine Learning, Neural Networks, 3D Reconstruction, X-Ray Imaging
I. INTRODUCTION
In ICF, the drive asymmetry during implosion hasbeen known to reduce neutron yield when compared to1D simulations . In double shells, low mode asymme-tries from the ablator become exacerbated due to hydro-dynamic instabilities such as Rayleigh–Taylor, Kelvin-Helmholtz, and Richtmyer–Meshkov. One technique todiagnose this asymmetry is by utilizing X-ray radio-graphic imaging during the experiment. This providesboth challenges and opportunities in data interpretationwhich involve inverse problems. Asymmetry in the implo-sion can have three dimensional (3D) components whilea radiograph is only a @D projection of this 3D informa-tion. 3D tomographic reconstruction is a well developedtechnique and is utilized in technologies such as X-raycomputed tomography (CT) instruments. For this tech-nique, a series of radiographs are collected from multipleangles which are then reconstructed using a variety of al-gorithms including filtered backprojection and iterativemethods such as algebraic reconstruction techniques .Furthermore, 3D reconstructions of time integrated im-plosions of deuterium filled capsules have been producedusing a small number of views . In ICF environments,room for diagnostics including X-ray imaging is scarceand the detectors that can operate on nanosecond to a) Contributed paper to the Proceedings of the 23rd Topical Con-ference on High-Temperature Plasma Diagnostics, Santa Fe, NM,USA, May 31 - June 4, 2020. Rescheduled online, Dec. 14-17, 2020.Correspondence: (B.W.) [email protected], (Z.W.) [email protected]. picosecond time scales are expensive to produce. Con-volutional neural networks have been shown to effec-tively perform tasks such as segmentation, denoising, andfeature recognition, whereas algorithms commonly usedin libraries such as OpenCV and scikit-image rely onsmall parameter spaces that are manually selected. Weutilize a convolutional neural network in order to recon-struct 3D spherical shell objects from X-ray radiographsgenerated from a single line of sight (SLOS). To resolvethe issue of the lack of 3D supervision, synthetic data isgenerated to compensate for the very limited experimen-tal data sets to successfully tune the network parameters.Data-driven 3D reconstruction from SLOS is a new con-cept for ICF applications. It will potentially allow us toreconstruct 3D objects using their 2D projections fromany view angle, and therefore be able to compare withmeasurements directly, or to compare with simulationsin unprecedented details.This paper is divided into the following sections. Wefirst describe the experimental setup used to produce theX-ray radiographs in Sec. II. The architecture of the pro-posed convolutional neural network used is detailed inSec. III. In Sec. IV the procedure for producing synthetic2D-3D training samples is described. In Sec. V the neu-ral network training process and associated hardware aredescribed. Sec. VI shows results generate using the neu-ral network. First, the 2D images rendered from bothclean and noisy synthetic 3D shells are shown. Next,cross-sections of reconstructions from experimental dataare shown. Then Fourier analysis is used to describe theeffects of the neural network in frequency space. Finally,contours are extracted from cross-sections to show theutility of this network for Legendre mode analysis. a r X i v : . [ phy s i c s . d a t a - a n ] J a n Shot Number Inner Shell Outer Shell Capsule FillN170222-003 40 µ m Si O µ m Al N/AN170322-001 N/A 106 µ m Al 35 mg/ cm foamN171016-001 40 µ m Si O µ m Al deuteriumN180321-003 N/A 106 µ m Al N/AN180522-002 N/A 120 µ m Al N/AN180731-002 40 µ m Si O (Ge doped) 106 µ m Al deuteriumN180918-001 40 µ m Si O (Ge doped) 106 µ m Al deuterium TABLE I. Configurations of materials in targets. The shellsthat contain a doping are 1.5% germanium.FIG. 1. Schematic of the experimental setup. This includesthe backlighter foil (far left), the hohlraum-capsule assembly(center left), and pinhole imager (right). The laser whichimpacts the foil to generate X-rays is not pictured and comesfrom the left.
II. EXPERIMENTAL SETUP
Imaging data of double and single shell targetsfrom NIF implosions are produced by using a Zr foilbacklighter and a gated X-ray framing camera . Thefoil backlighter produces 16.3 keV X-rays which passthrough a window in the hohlraum which eventually pro-ceeds to the pinhole camera setup . This pinhole cam-era utilizes microchannel plate strips, followed by a phos-phor screen that is recorded by a CCD camera. The en-tire source-object-detector geometry of the experimentalsetup is shown in Fig. 1. The spherical shell targets con-sist of an outer shell ablator and may contain an innershell . Table I shows the materials contained in thesetargets. Fig. 2 shows a zoomed in rendering of the 3Dreconstruction of the capsule used in shot N180918-001.The image was collected using a lab-based, micro-scale,X-ray CT instrument in which 1261 radiographs werecollected while the sample was rotated 360 ◦ which werereconstructed and rendered. III. NEURAL NETWORK ARCHITECTURE
The convolutional neural network proposed for thistask is based upon the encoder structure from trans-formable bottleneck networks (TBN). TBN’s areused in the task of novel view synthesis for objects givenmultiple input views, while generating an internal vol-umetric render. In the case of X-ray radiography, thevolumetric reconstruction is the more important piece of
FIG. 2. A high resolution 3D rendering from the recon-structed X-ray CT image of a double shell capsule. At thishigh resolution (1.5 µ m voxel size) features such as the jointfeature (shown in box) are visible. information, since producing a different projection of theobject is rather simple. This means that the decoderstructure of the TBN can be neglected. Furthermore,the use of the resampling layer is unnecessary since onlya SLOS is being used. This leaves a 2D encoder, a re-shaping layer, and a 3D decoder. In total these modulescontain 11,962,313 trainable parameters. Fig 3 showsthe shape of an image as it is passed through the con-volutional neural network. The 832 dimension is due tothe fact that the 2D encoder’s final convolutional layercontains 832 filters. For different input image sizes thenumber of features can be changed so that the number isdivisible by the 3D array dimensions. FIG. 3. The architecture of the network given a 256-by-256input and a 64-by-64-by-64 output. This can be amendedto higher input and output sizes by changing the number ofconvolutional filters used in the final layer of the 2D encoder.
IV. SYNTHETIC DATA GENERATION
To train the model, ground truth 3D volumetric ob-jects are required along with the corresponding 2D pro-jections. However, producing large sets of images in anexperimental setting is prohibitively expensive and an ac-curate 3D ground truth is unknown. The inverse of thisproblem however is much better understood. Given a3D object with known material properties, a well definedsource-detector geometry, and well known source proper-ties, projections can be easily produced. This is due tothe Beer-Lambert law which calculates the transmittanceof light through an object as: T := II = e − (cid:82) µds (1)Where µ is the linear attenuation of the material whichcan vary in different regions of the object and the integralis a line integral from the source to a position on thedetector. The linear attenuation is calculated by usingthe mass attenuation coefficients of the materials and thematerial density of the object at room temperature. X-ray mass attenuation coefficients for different materialsare tabulated on the NIST Standard reference Database8 .While the value of µ is related to material propertiesand the energy of the incident photons, the line inte-gral is determined by the geometry of the object and thesource-detector geometry. Since this operation must beperformed for each pixel of the detector, a GPU based to-mography library is used to speed up these calculations.The Tomographic Iterative GPU-based ReconstructionToolbox (TIGRE) provides this functionality and al-lows for a variety of source-detector-object geometries.Given a model for attenuation and the ability to calcu-late the line integral through the object, synthetic SLOSradiographic images are automatically generated by ran-domly sampling source parameters and 3D object prop-erties. Furthermore, the outer radius of the object isselected from a range of 0.15 mm and 1.5 mm, the outershell thickness is drawn from a range of 1%-30% of theouter shell radius, and a skew transform is applied toadd a variable amount of asymmetry to the targets. Us-ing TIGRE, synthetic SLOS radiographic images can begenerated as the projections of these objects, which areregarded as the 3D ground truth, to train the CNN. Bymultiplying the calculated transmission array by a con-stant intensity profile radiographs can be produced underdifferent illumination conditions. Fig. 4 shows an exam-ple of the generated data from the synthetic model.We use an Additive Gaussian Noise Model where thespread of the distribution varies and the Gaussian is sam-pled over integers. This is done since the image is treatedas counts detected with some variation due to noise. Firstthe standard deviation σ was selected using a uniformlydistributed distribution on integers 10 through 30. Thena distribution is defined between the values -100 and 100where the probability of selecting any given integer isgiven by a Gaussian distribution with standard devia-tion σ . Giving a maximum and minimum value for thenoise offset restricts the number of counts which makesthe noisy generated data well behaved. V. TRAINING
The CNN is trained using stochastic gradient descenton the mean squared error (MSE) between the output ofthe network ( f ( X )) and the 3D ground truth ( Y ), wherethe average is taken over the number ( N ) of voxels (3Dpixels). M SE = 1 N || f ( X ) − Y || (2) FIG. 4. A) A projection of an automatically generated objectusing TIGRE. B) The automatically generated object in 3D.A quarter of the object has been removed to aid the readerin visualizing the interior structure. Colors indicate variousmaterials.FIG. 5. A 3D reconstruction from synthetic data using anetwork trained on data without noise. The reconstructionshows the presence of an inner shell and outer shell. In thisplot background voxel values were set to zero by using a simplethreshold. Colors are used to indicate various materials. Thepurple voxels found on the shells and off to the right havevalues that are above the used threshold.
Generated datasets with two thousand pairs of projec-tions and 3D objects are used as a training set to optimizethe parameters of the network. Two separate datasets,one with a noise background and one without. Thesemodels are trained for a large number of epochs ( 300epochs). The optimization was done using a batch sizeof 10 and a learning rate of 0.01. The model efficacyduring training is measured using an independently gen-erated set of 300 pairs.
VI. RESULTS
Figs. 5 and 6 visualize the reconstruction producedfrom the two trained models described in the prior sec-tion. In both cases the reconstruction contains both aninner and outer shell. These reconstructions are bothreasonable since they share a common geometry with theoriginal objects and are accurate when compared numer-ically. The noisy data is shown as cross-sections sincethe background voxels are more difficult to extract from
FIG. 6. Cross-sections of a reconstructed 3D spherical shellfrom synthetic data using a network trained on images withnoise. Images from left to right are cross-sections of the re-construction taken from the center on orthogonal planes. Thetop row shows cross-sections of the reconstruction producedby the network. The bottom row shows the ground truth fromthe synthetic X-ray CT images. the object. The network is able to reconstruct 3D shellsregardless of the size of the object and on varied sourceenergies and intensities.While the synthetic data is similar to the experimentaldata in the fact that the experimental data possesses aninner shell and an outer shell, there are characteristicsthat the experimental images possess that the syntheticimages do not. One example of this is the illuminationof the object. All of the synthetic data used to train thenetworks have a constant illumination across the image.In the experimental settings the backlighter provides anon-uniform illumination which causes regions of imagesto be brighter or dimmer without the density of the ob-ject materials contributing to this effect. This causes anissue since convolutional neural networks perform betteron inputs that are more similar to the training data. Ei-ther the synthetic data needs to account for this or theexperimental data can be preprocessed. In image pro-cessing this illumination problem is typically solved byusing a flat field correction. This is done typically by di-viding the image by a flat field image. In the case of ICFimaging a true flat field cannot be obtained since thiswould require taking an image of the backlighter underthe same conditions without the object and these con-ditions would be difficult to reproduce. We instead usea pseudo-flat field correction to preprocess the exper-imental images. In pseudo-flat fielding, the flat field isproduced by a median filter of the image followed by aGaussian filter. For the median filter a large disk filteris used and for the Gaussian filter a large sigma value isused. This captures larger trends in the illumination ofthe image.Since the experimentally produced images containnoise, the neural network trained on noisy synthetic datais used for reconstruction. Fig. 7 shows the results ofapplication of the network on the preprocessed experi-mental image and raw experimental image. In the caseof the raw image the non-uniform illumination in the im- FIG. 7. (Top) Cross-sections of the reconstruction fromthe experimental image (right). (Bottom) Cross-sections ofthe reconstruction from the pseudo-flatfield corrected image(right). ages causes the reconstruction to miss the key featuresin the experimental image such as the shells. By flatten-ing the image using pseudo-flatfielding the network thenpicks up the key details in the image. The reconstructioneven contains an inner shell which is typically difficult todetect in these images. Similarly to the noisy syntheticdata, the background voxels are difficult to subtract sincethe resulting array is less sharp when compared to thenoiseless synthetic data.While convolutional neural networks are able to pro-duce impressive results, understanding these models canprove to be quite difficult. Since convolutional neural net-works are known to identify structures in images throughspatial filters, Fourier analysis provides a manner to ana-lyze these results. The two dimensional Fourier trans-form produces frequencies that are present in images.The transformed image is typically displayed in the formˆ I [ i, j ] = A [ i, j ] e iφ [ i,j ] (3)where A is the magnitude spectrum and φ is the phasespectrum which are both real valued and can thus berepresented as images themselves. Values closer to thecenter of these images correspond to smaller frequencyvalues, while values further away correspond to higherfrequency values. In Fig. 8 the Fourier transforms ofdifferent types of images used in this work are shown.The phase specta found in the cross-section of the recon-struction possess more structure than the phase spectrathat originate from the experimental images. In imagesthe phase provides information of features such as edges.The improved signal-to-noise of the cross-sectional im-age allows for visualization of this structured phase. Inthe magnitude spectra of cross-section, pixels closer thecenter have higher average values relative to those awayfrom the center. This is due to the fact that the im-ages are low noise and smooth which correspond to lowfrequency components. Therefore, one benefit of the net-work, which is that it produces low noise reconstructionfrom noisy inputs, are now easier to analyze.Since the cross-sectional images are low in noise, theshells are easier to extract using contour based algo-rithms. From thes images, Legendre mode analysis canbe completed. A marching squares algorithm found FIG. 8. (Top) Image analyzed using Fourier Transform.(Middle) Magnitude spectrums from cross-sections of recon-structions have more less high frequency contributions whencompared to the experimental images. (Bottom)The phasespectra from cross-sectional reconstructions are more struc-tured than the experimental counterparts.FIG. 9. Legendre mode fitting on cross-sections. The cross-section in the bottom row is of the same object as the top rowbut at a later time. (Left) Contour detection of cross-section,(Middle Left) Contour radius w.r.t cosθ , (Middle Right) Leg-endre Curve fit using N=15. (Right) coefficients of low modecoeffients, Even modes are prominent while odd modes con-tribute less. The radius r ( θ ) is also lower in the later cross-section. in scikit-image is used to extract the image contours.By locating the center of the object, this contour wasparametrized as a function of angle r ( θ ). A fit is appliedto the data using Legendre polynomials, r ( θ ) = n = N (cid:88) n =0 c n P n ( cosθ ) (4)where P n is the n th degree Legendre polynomial. Fig. 9shows the process of contour detection being applied ona cross-section of the reconstruction in order to measureLegendre coefficients. VII. CONCLUSION
We demonstrated the use of a convolutional neural net-work for 3D reconstruction of ICF capsules from singleline of sight radiographs. This technique shows the ben-efit of leveraging synthetic radiographs in order accountfor the unknown shape of the capsule during implosionand the small amount of data that can be producedin experimental conditions. Furthermore this techniqueis able to be used on asymmetric objects which com-monly appear in ICF environments. This neural networkmodel is able to capture internal structure of the cap-sules such as the inner shell which is normally difficultto detect using conventional means. Further expansionon the method is warranted to generate higher resolu-tion objects, synthetic models which better represent ex-perimental conditions, and support multiple line-of-sightmeasurements.
VIII. ACKNOWLEDGMENT
The authors would like to thank Pawel Kozlowski (LosAlamos National Laboratory) for providing code for thepseudo-flatfielding preprocessing.
REFERENCES Y. Li, Matter and Radiation at Extremes , 69 (2017), specialIssue on Laser Fusion (II). G. A. Kyrala, Laser and Particle Beams , 187–192 (2005). J. G. Colsher, Computer Graphics and Image Processing , 513(1977). M. J. Willemink and P. B.No¨el., European Radiology , 2185(2019). P. L. Volegov, Journal of Applied Physics , 175901 (2017),https://doi.org/10.1063/1.4986652. P. L. Volegov, Journal of Applied Physics , 205903 (2015),https://doi.org/10.1063/1.4936319. The OpenCV Reference Manual , Itseez, 2nd ed. (2014). V. der Walt, PeerJ , e453 (2014). M. Barrios, High Energy Density Physics , 626 (2013). E. C. Merritt, Physics of Plasmas , 052702 (2019). J. A. Oertel, International Society for Optics and Photonics(SPIE, 2004) pp. 214 – 222. J. R. Rygg, Phys. Rev. Lett. , 195001 (2014). E. N. Loomis, Physics of Plasmas , 072708 (2018),https://doi.org/10.1063/1.5040995. T. Cardenas, Fusion Science and Technology , 344. K. Olszewski, “Transformable bottleneck networks,” (2019),arXiv:1904.06458 [cs.CV]. K. Olszewski, “Transformable bottleneck networks,”https://github.com/kyleolsz/TB-Networks. M. J. Berger, “Xcom: Photon cross sections database,” (2010). A. Biguri, Biomedical Physics & Engineering Express , 055010(2016). A. Biguri, Journal of Parallel and Distributed Computing ,52 (2020). Pawel Kozlowski , “Pseudo-flatfielding python code,” (privatecommunication). J. Field, Review of Scientific Instruments , 11E503 (2014),https://doi.org/10.1063/1.4890395. W. S. Varnum, Phys. Rev. Lett.84