Deep learning for intensity mapping observations: Component extraction
Kana Moriwaki, Nina Filippova, Masato Shirasaki, Naoki Yoshida
MMon. Not. R. Astron. Soc. , 1–6 (0000) Printed 21 May 2020 (MN L A TEX style file v2.2)
Deep learning for intensity mapping observations:Component extraction
Kana Moriwaki (cid:63) , Nina Filippova , , Masato Shirasaki , Naoki Yoshida , , , Department of Physics, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan Department of Physics, Princeton University, Princeton, NJ 08544, USA National Astronomical Observatory of Japan (NAOJ), Mitaka, Tokyo 181-8588, Japan Kavli Institute for the Physics and Mathematics of the Universe (WPI), UT Institutes for Advanced Study,The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8583, Japan Research Center for the Early Universe, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan Institute for Physics of Intelligence, School of Science, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan
ABSTRACT
Line intensity mapping (LIM) is an emerging observational method to study the large-scale structure of the Universe and its evolution. LIM does not resolve individualsources but probes the fluctuations of integrated line emissions. A serious limitationwith LIM is that contributions of different emission lines from sources at differentredshifts are all confused at an observed wavelength. We propose a deep learningapplication to solve this problem. We use conditional generative adversarial networksto extract designated information from LIM. We consider a simple case with twopopulations of emission line galaxies; H α emitting galaxies at z = 1 . iii ] emitters at z = 2 . µ m. Our networkstrained with 30 ,
000 mock observation maps are able to extract the total intensityand the spatial distribution of H α emitting galaxies at z = 1 .
3. The intensity peaksare successfully located with 74% precision. The precision increases to 91% when wecombine 5 networks. The mean intensity and the power spectrum are reconstructedwith an accuracy of ∼ Key words: galaxies: high-redshift; cosmology: observations; large-scale structure ofUniverse
Line intensity mapping (LIM) is a promising observationaltechnique for next-generation cosmology. LIM probes thelarge-scale structure of the Universe at a wide range of red-shift and thus enables us to study cosmology as well asgalaxy formation and evolution (Kovetz et al. 2017). Fluc-tuations of the integrated intensity of emission lines such asLyman- α , H α , [C ii ], and CO lines trace the distributions ofthe underlying galaxies, while hydrogen 21-cm line is usedto study the distribution and ionization state of the inter-galactic medium in the early Universe (Pritchard & Loeb2012).A number of LIM observation programmes have beenproposed and are planned (see Kovetz et al. 2017). LIMmeasures the integrated emission from all the sources red-shifted to a wavelength bin. While it provide rich infor- (cid:63) E-mail: [email protected] mation on the sources and their large-scale distribution inprinciple, the confusion of sources or contamination fromforeground/background emission is an inevitable problem inpractice. Fonseca et al. (2017) show that multiple emissionlines from galaxies often contribute roughly equally to thetotal intensity at a certain observed wavelength. There area few methods to infer the contribution from a designatedredshift. One is to perform cross-correlation analysis withother known tracers of galaxies or of the matter distribu-tion at the same redshift (e.g. Visbal & Loeb 2010). Morepractical methods such as masking brightest pixels allow todetect subdominant signals (Gong et al. 2014; Silva et al.2018). It is also possible to distinguish signals from differ-ent redshifts using the anisotropic power spectrum shapes(e.g. Cheng et al. 2016). These methods are aimed at esti-mating the statistical quantities, but do not generate directimages of the intensity distribution. It would be more in-formative and useful if contaminants are removed from animage to show explicitly the intensity distribution at an ar- c (cid:13) a r X i v : . [ a s t r o - ph . GA ] M a y K. Moriwaki et al. bitrary redshift. Here, we propose to use deep learning toseparate/extract information from intensity maps.Convolutional neural networks (CNNs) are a popularand promising tool for image processing including problemsrelated to LIM. Recent studies propose to use CNNs to anal-yse hydrogen 21-cm line signals from the epoch of reioniza-tion (Hassan et al. 2019; Hassan, Andrianomena & Doughty2019; Gillet et al. 2019; Zamudio-Fernandez et al. 2019) orto estimate the line luminosity function from a CO intensitymap (Pfeffer, Breysse & Stein 2019). Shirasaki, Yoshida &Ikeda (2019) use conditional generative adversarial networks(cGANs, e.g., Isola et al. 2016) to de-noise observed weak-lensing mass maps. A cGAN consists of a pair of CNNs thatlearn an image-to-image translation in an adversarial way,and is able to generate fine and complicated images.In this letter, we apply cGANs to intensity maps toreconstruct the intensity distribution and basic statisticsof galaxy distribution. We aim at decoding cosmologicalinformation from future intensity map observations usingground-based and space-borne telescopes. We show that ournetworks, after appropriately trained with a large set ofmock observations, can generate accurately the intensity dis-tribution from a single source population. Throughout thisletter, we adopt ΛCDM cosmology with Ω M = 0 . , Ω Λ =0 . , h = 0 .
673 (Planck Collaboration VI 2018).
We consider the line intensity observed at wavelength of1 . µ m. Planned or proposed near-infrared LIM projects in-clude the Spectrophotometer for the History of the Universe,Epoch of Reionization, and Ice Explorer (SPHEREx, Dor´eet al. 2016) and the Cosmic Dawn Intensity Mapper (CDIM,Cooray et al. 2019). Emission lines from galaxies at z ∼ − α line from z = 1 . iii ] 5007˚A linefrom z = 2 .
0. Observational noises and other contaminantssuch as [O ii ] 3727˚A are to be considered in a forthcomingpaper (Moriwaki et al. in preparation). We generate a number of mock intensity maps for trainingand testing in the following manner. First, we populate acubic volume of 280 h − Mpc with dark matter haloes us-ing the publicly available pinocchio code (Monaco et al.2013). We set the minimum halo mass of the catalog to be3 × h − M (cid:12) . We have tested and confirmed that themap properties such as the total line intensity are not sig-nificantly affected by this choice of the minimum halo mass.We derive the halo mass-luminosity relation usingthe outputs of the cosmological hydrodynamics simulationIllustris-TNG (Nelson et al. 2019). We use the TNG300-1 dataset which has a simulated volume of V box =(302 . . We compute the line luminosity from a sim-ulated galaxy as L line = 10 − A line / . C line ( Z ) SFR , (1)where A line accounts for attenuation by dust. We adopt A H α = 1 . A [Oiii] = 1 .
35 mag. We use the pho-toionization simulation code cloudy (Ferland et al. 2017)to compute the coefficient C line ( Z ) as a function of the meanmetallicity of the galaxy. The cloudy computation is donein the same manner as in Moriwaki et al. (2018) except thatwe adopt typical values of the electron density n = 100 cm − and the ionization parameter U = 0 .
01. We compute the lu-minosity of a simulated halo by summing up the luminositiesof its member galaxies. The halo mass-luminosity relation,i.e., the mean L i and the variance σ i within each halo massbin, is then obtained from the Illustris output. To generatean emissivity field from a pinocchio halo catalogue, we as-sume a Gaussian distribution with a mean L i and a variance σ i and assign luminosities to the haloes in i -th mass bin.We perform the above procedure for H α and [O iii ]indenpendently at the respective redshift. We generatetwo-dimensional intensity maps by projecting the three-dimensional emissivity fields along one direction. The totalarea subtended by a map is (3 . , and we assume aspectral resolution R = 40 that corresponds to the expectedresolution of SPHEREx. We find that the relative contri-bution from the [O iii ] emission (at z = 2 .
0) is ∼
60% ofthe H α map (at = 1 . . are generated by projecting along random direction. Weobtain 30,000 training data in total. In this way, we ob-tain training maps with various mean intensities. Eachmap has 256 ×
256 pixels, corresponding to a pixel size of(0 . . For the test data set, we produce another1,000 halo catalogs and generate 1000 independent maps.We smooth the training and test maps with a Gaussian beamwith σ = 1 . We construct cGANs based on pix2pix by Isola et al.(2016). We train the networks so that they reconstruct bothH α and [O iii ] images from an observed image. This kinds ofone-to-many image translation networks are studied by, forinstance, Lee, Yang & Oh (2018) for separating transparentand reflection scenes.We have two pairs of adversarial convolutional networkscalled generator and discriminator. They are denoted by( G , D ) for H α map and by ( G , D ) for [O iii ] map. Thegenerators try to reconstruct H α and [O iii ] maps from anobserved map X obs , whereas the discriminators try to dis-tinguish the true maps X true , i and the reconstructed maps G i ( X obs ). In other words, for an input ( X obs , X ) with X being either X true , i or G i ( X obs ), the discriminator returns aprobability that X is X true , i . Here, X true ,i ( i = 1 ,
2) denotethe true maps of H α and [O iii ], respectively. The generator We have performed the same analysis in the present paper tothe ”raw” data without smoothing. We have found the perfor-mance of the cGANs is somewhat degraded when the networksare trained with the unsmoothed data populated with a numberof discrete sources. https://github.com/yenchenlin/pix2pix-tensorflowc (cid:13)000
2) denotethe true maps of H α and [O iii ], respectively. The generator We have performed the same analysis in the present paper tothe ”raw” data without smoothing. We have found the perfor-mance of the cGANs is somewhat degraded when the networksare trained with the unsmoothed data populated with a numberof discrete sources. https://github.com/yenchenlin/pix2pix-tensorflowc (cid:13)000 , 1–6 eep learning for intensity mapping Figure 1.
An observed map (top-left) is contributed by H α (top-center) and [O iii ] (top-right) emission. The reconstructed H α and [O iii ]maps and the sum of them are shown in bottom. The shown area is 1 . − cm − sr − .Note the relative difference in intensity for H α and [O iii ] (colour bars). Our network reconstructs even the fainter [O iii ] component. consists of 8 convolution layers followed by 8 de-convolutionlayers and the discriminator consists of 4 convolution lay-ers. Two generators G and G share the first 8 convolutionlayers. The kernel size of the convolutions is 5 ×
5. In eachlayer, batch normalization, dropout, and skip connectionare also performed (see Isola et al. 2016, for more details).During the training phase, the performance of the gen-erators and the discriminators are evaluated by a linear com-bination of the cross-entropy losses and the mean L1 norms: L = (cid:88) i =1 , [ L cGAN ( G i , D i ) + λ i L L1 ( G i )]+ λ tot L L1 , tot ( G , G ) , (2)where L cGAN ( G i , D i ) = log D i ( X obs , X true ,i )+ log[1 − D i ( X obs , G i ( X obs ))] , (3) L L1 ( G i ) = 1 N pix (cid:88) map | X true ,i − G i ( X obs ) | , (4) L L1 , tot ( G , G ) = 1 N pix (cid:88) map | X obs − G ( X obs ) − G ( X obs ) | , (5)where N pix = (256) . In each training set, the generators(discriminators) are updated to decrease (increase) the lossfunction L averaged over a mini batch. We adopt λ = λ = During test phase, we set is training = False in batch nor-malization to use fixed normalization parameters. Mini batch is a randomly selected set of training data, { X obs ,i , X true ,i } n b i =0 , where n b is batch size. In training phase,the networks pass through all the training data without dupli- λ tot = 100 and a batch size of 4. The networks are trainedfor 8 epochs. We use the Adam optimizer (Kingma & Ba2014) with learning rate 0.0002, and decay rate parameters β = 0 . β = 0 .
999 for updating the parameters.
We study the performance of our networks with 1000 testdata. Fig. 1 shows an example of true and reconstructedmaps. In our fiducial case of λ obs = 1 . µ m, the contribu-tion from H α is larger than [O iii ], and then outstandingstructures in the observed map mostly originate from theH α emission at z = 1 .
3. It is thus remarkable that not onlythe H α distribution but also the weaker [O iii ] intensity isreproduced well.It is important to study whether statistical quantitiesare also reproduced accurately. We first examine the peaks inour intensity maps. We select as ”peaks” local maxima withheights greater than 3 σ . We find 24089 (18859) and 24800(17631) peaks in the true and the reconstructed H α ([O iii ])maps over our 1000 test data sets. Among them, 18262(5095) peaks are matched correctly. This means that 76%(27%) of the true peaks are reproduced, and 74% (29%) ofthe reconstructed peaks are true.If our purpose is to study individual peaks or other indi-vidual structures, we may require much higher accuracy forreconstructed structures. Previous studies developed cGANs cation. When we set the number of epochs n e >
1, this passingthrough is repeated for n e times. For n d training data, updatesare performed for n d n e /n b times in total.c (cid:13) , 1–6 K. Moriwaki et al.
Figure 2.
The mean intensities of the reconstructed mapsagainst the mean of the true maps of H α (upper) and [O iii ] (bot-tom) for our 1000 test data set. that also learn the reliability of reconstructed maps (Lee,Yang & Oh 2018; Kendall & Gal 2017). In principle, we canuse these methods to quantify the reliability of the outputs.Another promising idea is to combine multiple networks. Totest this idea, we use 5 networks that have an identical setof convolutional layers but are trained with different setsof data. Intensity maps reconstructed by the 5 trained net-works are similar, but not exactly the same. We find that itis generally difficult to reproduce the true intensity in por-tions where these networks commonly fail. For H α ([O iii ])maps, the number of peaks detected by all the 5 networksis 14332 (895). Among them, 13018 (539) peaks are true,which means a 91% (60%) confidence level for our peak de-tection. We note that if we take the average or the median ofthe reconstructed maps by multiple networks on a pixel-by-pixel basis, dark structures in void regions and small-scalestructures are smoothed out. Summary statistics such as the mean intensity and the powerspectrum are primary tools to study the distribution and theproperties of the emission-line galaxies. These can then beused for galaxy population studies or for cosmological pa-rameter inference. In this section, we examine how well themean intensity and the power spectrum are reconstructed. −2−1012 ( P p r e d − P t r u e ) / σ t r u e −1 k [arcmin −1 ] −17 −16 −15 π k P ( k )[ e r g / s / c m / s r ] true Hα w/o smoothingtrue [OIII] w/o smoothingtrue Hαtrue [OIII]reconstructed Hαreconstructed [OIII] Figure 3.
The two-dimensional power spectra of the recon-structed maps. The error bars and the shaded regions in thebottom panel show the 1 σ variance of the power spectrum ofthe reconstructed and the true maps over 1000 test data, respec-tively. The light-coloured regions show the 1 σ variance of truemaps without smoothing. In the upper panel, we show the dif-ference between the reconstructed and the true power spectranormalized by the variance of the true power spectrum. We take medians of the reconstructed statistics by 5 differ-ent networks and compare them with true ones.Fig. 2 shows the correspondence of the true and thereconstructed mean intensities. The mean intensities arewidely distributed because of the cosmological variance ofthe underlying density field. We see clear correlations be-tween the true and the reconstructed mean intensities. Fig. 2shows that the mean H α ([O iii ]) intensity of each (1 . map can be estimated with ∼
10% ( ∼ and thus the estimatedmean intensities would have a much smaller statistical un-certainty.We test if our networks generate accurate images (in-tensity maps) if the input observed map is significantly dif-ferent from the training data. To this end, we input inten-sity maps with the mean differing as much as 20 %. Someof these samples have mean intensities below or above therange plotted in Fig. 2. We find that the networks recon-struct the H α and [O iii ] intensities with accuracy similar tothose shown in Fig. 2 when both maps are scaled with thesame factor. However, we find systematic offset when onlyone map is scaled more than 10% while the other being un-changed. In order to reconstruct these outliers accurately,we need to consider a wide variety of training data, and/orto combine multiple networks trained with maps in differentmean intensity ranges.Another important statistic is two-dimensional power c (cid:13) , 1–6 eep learning for intensity mapping spectrum. Fig. 3 shows the variation of the true (shaded re-gions) and the reconstructed (error bars) power spectra. Forreference, we also show the power spectra of unsmoothedtrue maps. Clearly, our networks learn the clustering ofgalaxies even though we do not explicitly teach that galax-ies at different redshifts have different clustering amplitudes.The top panel of Fig. 3 shows the difference between thetrue and the reconstructed power spectra normalized by thesquare root of the variance of the true power spectra σ true .We note that the variance of the training data is also σ true .For H α map, the difference is typically less than σ true atlarge scales; our network is able to recover the power spec-trum of H α at z = 1 . ∼
10% from aconfused map.
We have shown, for the first time, that cGANs can sepa-rate desired signals confused in an intensity map. We canalso locate intensity peaks where emission line galaxies areclustered at the target redshift. Combining the distributionof the peaks and other information from follow-up obser-vations of individual galaxies would allow us to study theenvironmental dependence of the galaxy formation.A promising approach is to combine our deep learn-ing method with other conventional method such as cross-correlation analysis. From the statistical information suchas the power spectrum and the mean intensity of the re-constructed intensity maps (galaxy distributions) at a widerange of redshift, we can infer cosmological parameters andcan also learn about the evolution of galaxy populations.In this letter, we have presented the results from ourfirst attempt, and there is much room for improvement. Inorder for our method to be applied to real LIM observa-tions, the networks need to be trained with observationalnoises and other contaminants. For cosmology studies, itwould be important to train the networks with a variety ofastrophysical/cosmological models and parameters to im-prove robustness. Quantifying the uncertainty or the re-liability of a reconstructed map is also important. Whenan input map is quite different from the training dataset,the networks should ideally return such information to-gether with the reconstructed map(s). Methods shown inKendall & Gal (2017) can be used for these purposes. Toimprove the ability of the networks, we can utilize the three-dimensional information or train the networks with a largersurvey area/volume. Our networks can also be applied toobservations in different wavelength such as sub-millimeterLIM. We choose two emission lines with relatively close red-shifts having similar structures in this study. If one focus ontwo redshifts with larger separation, it could be easier forthe networks to learn the difference and reproduce the mapswell. We continue exploring the deep learning approach to,for instance, de-noise intensity maps or to extract designatedinformation from a map with more than two components. Power spectrum of the unsmoothed map P ( k ) can be re-covered from that of the smoothed map P sm ( k ) by P ( k ) =exp( k σ ) P sm ( k ), where σ is the smoothing scale of the Gaus-sian beam. ACKNOWLEDGEMENTS
We thank the anonymous referee for providing us usefulcomments. We thank Yasuhiro Imoto for useful discus-sion. KM is supported by JSPS KAKENHI Grant Num-ber 19J21379. NF’s visit at the University of Tokyo wassupported by the Princeton-UTokyo strategic partnershipgrant. MS is supported by JSPS Overseas Research Fellow-ships. NY acknowledges financial support from JST CREST(JPMHCR1414). A part of our computations in this studyis carried out on Cray XC50 at Center for ComputationalAstrophysics, National Astronomical Observatory of Japan.
REFERENCES
Cheng Y.-T., Chang T.-C., Bock J., Bradford C. M.,Cooray A., 2016, ApJ, 832, 165Cooray A. et al., 2019, arXiv e-prints, arXiv:1903.03144Dor´e O. et al., 2016, arXiv e-prints, arXiv:1606.07039Ferland G. J. et al., 2017, Rev. Mexicana Astron. Astrofis.,53, 385Fonseca J., Silva M. B., Santos M. G., Cooray A., 2017,MNRAS, 464, 1948Gillet N., Mesinger A., Greig B., Liu A., Ucci G., 2019,MNRAS, 484, 282Gong Y., Silva M., Cooray A., Santos M. G., 2014, ApJ,785, 72Hassan S., Andrianomena S., Doughty C., 2019, arXiv e-prints, arXiv:1907.07787Hassan S., Liu A., Kohn S., La Plante P., 2019, MNRAS,483, 2524Isola P., Zhu J., Zhou T., Efros A. A., 2016, CoRR,abs/1611.07004Kendall A., Gal Y., 2017, arXiv e-prints, arXiv:1703.04977Khostovan A. A., Sobral D., Mobasher B., Best P. N., SmailI., Stott J. P., Hemmati S., Nayyeri H., 2015, MNRAS,452, 3948Kingma D. P., Ba J., 2014, arXiv e-prints, arXiv:1412.6980Kovetz E. D. et al., 2017, arXiv e-prints, arXiv:1709.09066Lee D., Yang M.-H., Oh S., 2018, arXiv e-prints,arXiv:1801.04102Monaco P., Sefusatti E., Borgani S., Crocce M., Fosalba P.,Sheth R. K., Theuns T., 2013, MNRAS, 433, 2389Moriwaki K. et al., 2018, MNRAS, 481, L84Nelson D. et al., 2019, Computational Astrophysics andCosmology, 6, 2Pfeffer D. N., Breysse P. C., Stein G., 2019, arXiv e-prints,arXiv:1905.10376Planck Collaboration VI, 2018, arXiv e-prints,arXiv:1807.06209Pritchard J. R., Loeb A., 2012, Reports on Progress inPhysics, 75, 086901Shirasaki M., Yoshida N., Ikeda S., 2019, Phys. Rev. D,100, 043527Silva B. M., Zaroubi S., Kooistra R., Cooray A., 2018, MN-RAS, 475, 1587Sobral D., Smail I., Best P. N., Geach J. E., Matsuda Y.,Stott J. P., Cirasuolo M., Kurk J., 2013, MNRAS, 428,1128Visbal E., Loeb A., 2010, J. Cosmology Astropart. Phys.,11, 016 c (cid:13) , 1–6 K. Moriwaki et al.
Zamudio-Fernandez J., Okan A., Villaescusa-Navarro F.,Bilaloglu S., Derin Cengiz A., He S., Perreault LevasseurL., Ho S., 2019, arXiv e-prints, arXiv:1904.12846 c (cid:13)000