Ensemble photometric redshifts
Nikhil Padmanabhan, Martin White, Tzu-Ching Chang, J.D. Cohn, Olivier Dore, Gil Holder
MMon. Not. R. Astron. Soc. , 1–5 (0000) Printed 6 March 2019 (MN L A TEX style file v2.2)
Ensemble photometric redshifts
Nikhil Padmanabhan , Martin White , , Tzu-Ching Chang , , J.D. Cohn , Olivier Dor´e , ,Gil Holder , Departments of Physics and Astronomy, Yale University, New Haven, CT 06511, USA Departments of Physics and Astronomy, University of California, Berkeley, CA 94720, USA Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA Academia Sinica Institute of Astronomy and Astrophysics, 11F of ASMAB, AS/NTU, 1 Roosevelt Rd Sec. 4, Taipei, 10617, Taiwan Jet Propulsion Laboratory, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, CA 91109, USA Space Sciences Laboratory and Theoretical Astrophysics Center, University of California, Berkeley, CA 94720, USA California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USA University of Illinois, Departments of Physics and Astronomy, Urbana IL 61801
ABSTRACT
Upcoming imaging surveys, such as LSST, will provide an unprecedented view of the Uni-verse, but with limited resolution along the line-of-sight. Common ways to increase resolutionin the third dimension, and reduce misclassifications, include observing a wider wavelengthrange and/or combining the broad-band imaging with higher spectral resolution data. Thechallenge with these approaches is matching the depth of these ancillary data with the orig-inal imaging survey. However, while a full 3D map is required for some science, there aremany situations where only the statistical distribution of objects ( dN / dz ) in the line-of-sightdirection is needed. In such situations, there is no need to measure the fluxes of individual ob-jects in all of the surveys. Rather a stacking procedure can be used to perform an “ensemblephoto- z ”. We show how a shallow, higher spectral resolution survey can be used to measure dN / dz for stacks of galaxies which coincide in a deeper, lower resolution survey. The galaxiesin the deeper survey do not even need to appear individually in the shallow survey. We givea toy model example to illustrate tradeoffs and considerations for applying this method. Thisapproach will allow deep imaging surveys to leverage the high resolution of spectroscopic andnarrow/medium band surveys underway, even when the latter do not have the same reach tohigh redshift. Key words: methods:data analysis; methods:statistical; galaxies:distances and redshifts
The Large Synoptic Survey Telescope (LSST) will be one of thekey astronomical facilities of the next decade. It will allow us tomap large areas of sky with unprecedented depth in six optical passbands ( ugrizY ). The science enabled by this facility will be revo-lutionary (e.g. LSST Science Collaboration et al. 2009). Fully ex-ploiting these deep sky maps will require information on the red-shifts of the objects. A redshift estimate can be obtained directlyfrom the photometry (a “photo- z ”), but such redshifts are rela-tively poor and can be difficult to obtain for some types of galax-ies (e.g. see Hildebrandt et al. 2010; Dahlen et al. 2013; S´anchezet al. 2014; Rau et al. 2015, for recent reviews). Of particular in-terest here is the use of LSST for studies of large-scale structure,which heavily impacts cosmology and fundamental physics. Forsuch problems the addition of high-quality redshift information iscritical (e.g. Newman et al. 2015).One can seek to obtain redshifts for individual galaxies or theredshift distribution for a particularly interesting subsample. The latter will be the topic of this paper. Knowledge of dN / dz for asample can be used to invert a measured 2D correlation functioninto a 3D correlation function (Limber 1953, 1954) or to inter-pret the results of a cosmic shear experiment (Hoekstra & Jain2008). The most natural method for obtaining dN / dz is to ‘stack’the photo- z s (or the redshift PDFs) of the galaxies making up thesample. Another method is to use the fact that objects which areclose on the sky are also likely to be close in redshift. There isa long history of using such “cross-correlation” techniques to de-termine dN / dz (Seldner & Peebles 1979; Phillipps 1985; Phillipps& Shanks 1987; Padmanabhan et al. 2007; Ho et al. 2008; Erbenet al. 2009; Benjamin et al. 2010, 2013; Newman 2008; Matthews& Newman 2010; Schulz 2010; McQuinn & White 2013; Matthewset al. 2013; M´enard et al. 2013; Schmidt et al. 2013; Rahman et al.2015; Choi et al. 2016) and such methods can perform very well.Unfortunately, degeneracies in color-type-redshift space area notorious problem with traditional photo- z methods, especiallywhen restricted to broad-band optical photometry. In this case,galaxies of different types/redshifts can have the same colors c (cid:13) a r X i v : . [ a s t r o - ph . GA ] M a r Padmanabhan et al (Ben´ıtez 2000) and thus are indistinguishable. These degeneraciesare often easily broken by adding photometry in IR-bands, appro-priately chosen narrow-band imaging or low-redshift spectroscopy.However, it is often challenging to match the depths of these ad-ditional data with the original imaging catalog, especially for wideimaging surveys. The key idea in this paper is the realization thatin order to determine redshift distributions, it is not necessary todetect individual sources in these additional data. For large enoughsamples, a “stacked” measurement can constrain the redshift dis-tribution, even if the individual galaxies are all below the detectionthreshold. We dub this idea “ensemble photo- z ’s”.This idea is of particular interest since a number of narrow-band imaging/low resolution spectroscopy large-scale surveysare independently motivated. For instance, SPHEREx (Spectro-Photometer for the History of the Universe, Epoch of Reioniza-tion, and Ices Explorer) is an all-sky spectroscopic survey satellitewhich will obtain R = −
41 spectra for 0 . < λ < . µ mand R = −
130 spectra for 3 . < λ < . µ m, for a total of96 bands, for every 6.2 arc second pixel over the entire-sky (Dor´eet al. 2014). J-PAS (Javalambre Physics of the Accelerating Uni-verse Astrophysical Survey) will cover 8000 deg using 56 narrowband filters in the optical (Benitez et al. 2014). PAUS (Physicsof the Accelerating Universe Survey) will provide a 100 deg < λ < (Advanced Large, Homogeneous Area Medium-Band Red-shift Astronomical Survey) employs 20 contiguous, medium-bandfilters covering 3500 < λ < (Moles et al. 2008;Matute et al. 2012; Molino et al. 2014).At first glance, none of these surveys are deep enough to pro-vide interesting additional bands for surveys like LSST. A typi-cal LSST gold-sample galaxy has an i band magnitude of ∼ . σ de-tection limit per frequency element is ∼ . ∼ > σ spectrum. Similarly, for J-PAS, tak-ing the 5 σ detection limit per frequency element to be magnitude ∼ .
5, stacking (again averaging over 5 adjacent bands) ∼ > σ detection.As we will see, these stacked spectra encode informationabout the underlying redshift distributions of the objects. Further-more, since the LSST gold-sample will have ∼ galaxies, it isplausible that, even after dividing into a number of subsamples, onewould have sufficent numbers of galaxies per subsample to yieldstacked detections in any of these shallower surveys.The next section lays out the simple idea underlying ensem-ble photo- z ’s, and then works through two simplified examples.We then conclude with a discussion of how one might extend thiswork, as well as implications for photometric redshift calibrationsfor LSST. http://spherex.caltech.edu http://alhambrasurvey.com/ We imagine that our data come from two surveys. We assume thatthe first (denoted by P ) is a deep, multi-band imaging survey; theprototypical example is LSST, although one could consider theimaging components of Euclid and WFIRST as well. We imag-ine this survey is augmented by a shallower, low-resolution spec-troscopic survey S . Although we assume spectroscopy below, thiscould be generalized to a second multi-band imaging survey as well(with spectroscopy being the infinitesimal band limit). A key ele-ment here is that the majority of objects of interest in P are notindividually detected in S .Start by considering a sample of N galaxies in P , selected ina small voxel in observed flux space, and compute the photometricredshifts for these galaxies. Since we have, by construction, chosenall of these galaxies to have the same observed fluxes, their esti-mated photometric redshifts will be the same (ignoring, for now,the scatter due to observational errors). The accuracy of these red-shifts will be intrinsically limited by degeneracies in flux space –different templates at different redshifts can produce the same ob-served fluxes. This is particularly true for small numbers of filtersthat span a limited wavelength range. This problem of interloperredshifts is well known in the photometric redshift literature, andthere have been a number of suggested approaches to reduce orquantify this interloper fraction. The simplest approach would beto expand the number of filters (with a limit being a spectrum ofthe galaxy) and/or the wavelength range to break these degenera-cies per object. A different approach is the idea of clustering red-shifts which uses the spatial clustering of galaxies to constrain theredshift distribution of the ensemble.Our idea of ensemble photometric redshifts is intermediate be-tween these two approaches : we will expand the number of “fil-ters”/wavelength range by augmenting our measurements by S , butwe will not assume that the galaxies are individually detected in S . Instead, we use the observation that the average spectrum canbe used to constrain the redshift distribution of the entire sample.Hence “ensemble photometric redshifts” : instead of individuallyfitting a redshift to each object, we fit the stacked spectrum to mea-sure the full redshift distribution.The expected stacked spectrum is just a sum over the N indi-vidual spectra f ( λ ) in S centered on the galaxies identified in P : f av ( λ ) = N (cid:88) i = f i ( λ ) , (1)where λ is in the observer frame. If we imagine that galaxy spec-tra are well described by a relatively small number of templates F α ( λ, z ), we can rewrite the above as f av ( λ ) = N (cid:88) i = A i F α i ( λ, z ) (2)where the normalization A i depends on the luminosity of the object.We can simplify this further using the fact that we selected galaxiesfrom a narrow voxel in observed flux space. This implies that allobjects with the same spectrum F α at the same redshift must havethe same normalization, and we are free to absorb this normaliza-tion into the definition of the galaxy template F α ( λ, z ). The stacked http://sci.esa.int/euclid/ http://wfirst.gsfc.nasa.gov/ c (cid:13) , 1–5 nsemble photo-z spectrum can now be written as f av ( λ ) = (cid:88) α (cid:90) dz (cid:32) dNdz (cid:33) α F α ( λ, z ) (3)where ( dN / dz ) α is the redshift distribution of galaxies of type α .The ensemble photometric redshift problem is now analogous toregular photometric redshifts - we consider maximizing the likeli-hood L ( f av |{ ( dN / dz ) α } ) (or determining the corresponding posteriordistribution).Although the above expression does not explicitly list the de-pendence on the data from the original photometric survey P , thisis implicit in our choice of template spectra and their normaliza-tions. This is possible because we started by selecting galaxies in anarrow voxel in flux-space from P . While this is a useful simplifi-cation, it is possible to extend this to more complicated selectionsby augmenting the likelihood to L ( D P , f av |{ z i , α i , ( dN / dz ) α } ), where D P represents the data from P and we now estimate the individual z i and α i jointly with the redshift distributions.While we chose to work with a stacked spectrum above, onecould imagine directly fitting the observed fluxes of the individualgalaxies in S ; this would be the optimal approach if one had an ac-curate model of the errors in the fluxes. On the other hand, workingwith the stacked fluxes allows one to e.g. diagnose template mis-matches or an inadequate error model. Again, the choice of usinga stacked spectrum is not essential to this discussion but is a usefulmental model of the idea of an ensemble photo- z ; the key idea is tojointly fit all observations to constrain the redshift distributions. We start with a simple example that demonstrates how a stackedspectrum can break degeneracies in redshift distributions. We imag-ine a sample of galaxies in P that are drawn from two popula-tions : spiral and irregular galaxies (we use the Scd B2004a and
Im B2004a templates in Ben´ıtez et al. (2004)). These galaxies areobserved through two simple top-hat filters at 4450 Å and 6580 Å(similar to B and R filters). Fig. 1 shows the B − R color tracks forthese galaxies as a function of redshift. For galaxies with an ob-served B − R color of 0.7 (shown by the dotted line) and a unit (inarbitrary units) B -band flux, we observe a three-fold redshift/typedegeneracy - the irregular galaxy could be at redshifts ∼ . ∼ .
7, while the spiral galaxy could be at ∼ .
9. Photometric errorswould naturally broaden these distributions.We now imagine observing this population of galaxies with alow-resolution spectroscopic survey. We follow our example above,selecting galaxies with a fixed B − R = . R -bandflux. Fig. 2 plots the predicted spectrum for different admixturesof types and redshifts. For definiteness, we assume that the spiralgalaxies form the dominant population, with the irregulars beingcontaminants. For this particular example, we find that the λ > µ m part of the spectrum can determine the overall contamination frac-tion. The differences between contaminants at different redshifts(here z ∼ . .
7) are smaller, although with enough S/N, onecan clearly start to distinguish these cases. The details are clearlyspecific to the case we have chosen here, but this demonstrates thatan averaged spectrum can break degeneracies in photometric red-shift distributions.Furthermore, this figure allows us to schematically understandthe depth requirements for the spectroscopic survey - given two de-generate (in P ) populations of galaxies, one needs to be able to dis-tinguish between the stacked spectra in S . As a numerical example, z B - R ScdIm
Figure 1.
The B − R color of a spiral (Scd) and irregular (Im) galaxy,highlighting the color-redshift degeneracies. The galaxy templates are fromBen´ıtez et al. (2004) and are normalized to have the same observed flux.We approximate the B and R filters as tophat filters centered at 4450 Å and6580 Å with widths of 700 Å and 1000 Å respectively. An example color-redshift degeneracy is highlighted by the dotted line at B − R = .
7; thiscolor is consistent with the irregular galaxy at z ∼ . ∼ . z ∼ . Wavelength (Angstroms) F l u x [ a r b . un i t s ] B R
Figure 2.
A demonstration that a stacked spectrum can be used to break de-generacies in photometric redshifts. From top to bottom, the lines show theexpected stacked spectrum of spiral (Scd) galaxies with 10%, 20% and 30%contamination from a population of irregular (Im) galaxies. In all cases, thespiral galaxies are at a redshift of ∼ z ∼ .
7, while the dashed (red) lines have the irregulargalaxies at z ∼ .
2. By construction, all of these galaxies have the same B − R color, and the same R -band magnitude. Also plotted for reference [dotted]is the irregular galaxy spectrum at z = . B and R bands we use. we suppose that stacking 10 LSST galaxies in S yields a 10% de-tection of flux per frequency element and that S has ∼ − i <
25) isexpected to contain ∼ × galaxies, which suggests that onecould conceptually break the sample into ∼ voxels in magni-tude space, each with sufficient galaxies to stack in the spectro-scopic survey. The above estimates are just meant to be illustrativeand to demonstrate that such an approach is feasible in principle. c (cid:13) , 1–5 Padmanabhan et al
The previous section demonstrated how the stacked spectrum couldbreak degeneracies in photometric redshifts. We extend this idea tomeasuring the redshift distribution here. As in the previous section,we consider a mixture of spiral and irregular galaxies using thesame templates used previously. The assumed redshift distributionsare shown in Fig. 3. The forms we chose reflect the color-redshiftdegeneracies seen in Fig. 1; in particular, note the “contamination”of low-redshift spirals and irregulars. We imagine these galaxiesare selected to have the same R -band flux, they have perfectly mea-sured B − R colors, and that this selection yields 10 galaxies.We now combine the above redshift distributions (for spiralsand irregulars) into a single B − R color distribution. We then splitthe sample into ∆ ( B − R ) = . B − R = .
3. Foreach of these color bins, we compute the stacked spectrum of all thegalaxies in the bin. Fig. 1 shows that, in general, these stacked spec-tra are the combination of spiral and irregular galaxies at two dif-ferent redshifts. We assume that stacking 10 galaxies yields a 10%measurement of flux per frequency element in the spectroscopicsurvey, and we scale this error by (cid:112) / N where N is the numberof galaxies in the B − R bin under consideration. We also assumethat the stacked spectrum is measured from 5000Å to 14000Å with R ∼ ∼
70 frequency elements. The inputs to ouralgorithm are the B − R color distribution and the stacked spectrumin each ∆ ( B − R ) color bin. As with template-based photo- z codes,we assume we have a complete set of spectral templates (in thiscase, the templates for spiral and irregular galaxies).We parametrize the redshift distributions of each individualpopulation (spiral or irregular) by step-wise constant distributionsin z with 100 bins from z = z = .
5. Given a bin in B − R ,Fig. 1 shows that only a small fraction of these redshift bins willhave a consistent B − R color. These redshift bins are the input vari-ables to a least-squares fit to the observed (stacked) spectrum usingEq. 3. We impose additional constraints that ( dN / dz ) α, b (cid:62) (cid:80) z ( dN / dz ) α, b =
1, where b indexes the color bin, and the lat-ter constraint normalizes the redshift distribution. After computingthese redshift distributions over the individual color bins, we com-bine these using (cid:32) dNdz (cid:33) α = (cid:88) b N b (cid:32) dNdz (cid:33) α, b , (4)where N b is the number of galaxies in the b th color bin.Fig. 3 shows the redshift distribution recovered using theabove procedure, averaging over the results of 50 simulations. Al-though we estimate the redshift distribution over 100 bins, the val-ues between adjacent bins are highly covariant (since the spectrado not have the S/N to distinguish between small changes in red-shift). We therefore average neighbouring bins to produce the figureshown. We also compress our results to the fractions of spiral andirregular galaxies, to mimic the case where the shape of the redshiftdistributions might be known (or well-constrained). We see that thissimple procedure recovers the correct fractions of spiral and irreg-ular galaxies as well as their redshift distributions. While this is atoy example, it illuminates the utility of these stacked spectra. We introduce the idea of “ensemble” photometric redshifts as a toolto constrain photometric redshift distributions. The idea is a sim-ple extension of current template-based photometric redshift codes, z d N / d z f Scd = 0 . ± . . f Im = 0 . ± . . Figure 3.
A test showing the recovered redshift distribution using stackedspectra. The dashed lines are the input redshift distributions for spiral (blue)and irregular (green) redshifts. The points show the recovered redshift dis-tributions (averaged over 50 simulations), while the errorbars show the un-certainty expected for a single realization. Also shown are the fractions ofspiral and irregular galaxies with the input values in parentheses. See thetext for more details on the exact simulations. and uses the fact that the shape of a stacked spectrum encodes infor-mation about the redshift distribution of the galaxies being stacked.The advantage of this approach is that the individual galaxies nolonger need to be detected in the second survey, opening up the pos-sibilities of using planned low-resolution shallower spectroscopysurveys like J-PAS, PAU and SPHEREx to calibrate deeper surveyslike DES and LSST. An important point is that the next generationof imaging surveys will have samples of ∼ galaxies, which al-lows one to build large numbers of subsamples, each of which havesufficient numbers of galaxies to stack in the shallower survey. Weoutline the idea in this paper, and discuss some simple examplesdemonstrating how these stacked spectra can be used to break pho-tometric redshift degeneracies and measure redshift distributions.These examples are meant to be illustrative; future work will beneeded to understand the signal to noise for realistic galaxy distri-butions for future surveys.An aspect of the ensemble photo- z method is that one simul-taneously fits both the individual photometric redshifts and the red-shift distribution. We outline a simplified algorithm here, wherewe imagine splitting the original sample into voxels in magnitudespace. This problem has also recently been considered by Leistedt,Mortlock & Peiris (2016) who discuss a more general approachto this problem; their algorithm can be easily extended to includeconstraints from the stacked spectrum. We expect that future workwill also consider optimal algorithms for the next generations ofsurveys.Our approach here has been to stack sources to get a detec-tion in the shallower survey. Clearly, if one has well characterizederrors, it is clear that the same information can be recovered byfitting observed fluxes (even if they are individual non-detections).We were however motivated by the fact that stacking the galaxiesallows us to develop better intuition for the process. It also opensup possibilities for detecting mismatches in photometric redshifttemplates used, which could be folded back into photometric red-shift codes. It should be emphasized that this entire process requiresthat one can average down the noise in the shallow spectroscopicsurvey, which will impose requirements on the data reduction andcalibration. c (cid:13) , 1–5 nsemble photo-z Although the examples presented here used simple models forgalaxy populations, our formalism is straightforward to extend tomore complex cases. One such complication is the effect of dust,which will smear a single population at a fixed redshift along a lineof extinction. We can imagine introducing parameters describingthe scatter in extinction/reddening as well as its direction into ourmodel, and then simultaneously fitting/marginalizing these with theredshift distributions. We defer a detailed study of this and otherreal-world complications to future work.The problem of determining the photometric redshifts for thenext generation of surveys is still an open question. It has beenlong recognized that increasing the wavelength coverage can im-prove photometric redshifts; however, it is normally assumed thatthese additional data should be matched in depth to the primary sur-vey. We point out that, for the specific problem of determining theredshift distribution, this is not necessary, opening up the possibil-ity for alternative/easier routes to calibrating photometric redshiftdistributions.NP is supported in part by DOE de-sc0008080. T.-C. C. ac-knowledges support from MoST grant 103-2112-M- 001-002-MY3and the Simons Foundation. This work was begun and completed atthe Aspen Center for Physics, which is supported by National Sci-ence Foundation grant PHY-1066293. This work made extensiveuse of the NASA Astrophysics Data System and of the astro-ph preprint archive at arXiv.org . Part of the research described inthis paper was carried out at the Jet Propulsion Laboratory, Cali-fornia Institute of Technology, under a contract with the NationalAeronautics and Space Administration.
REFERENCES
Ben´ıtez N., 2000, ApJ, 536, 571Benitez N. et al., 2014, ArXiv e-printsBen´ıtez N. et al., 2004, ApJ Suppl., 150, 1Benjamin J., van Waerbeke L., M´enard B., Kilbinger M., 2010,MNRAS, 408, 1168Benjamin J., et al., 2013, MNRAS, 431, 1547Castander F. J. et al., 2012, in Proc. SPIE, Vol. 8446, Ground-based and Airborne Instrumentation for Astronomy IV, p.84466DChoi A. et al., 2016, MNRAS, 463, 3737Dahlen T. et al., 2013, ApJ, 775, 93Dor´e O. et al., 2014, ArXiv e-printsErben T. et al., 2009, A&A, 493, 1197Hildebrandt H. et al., 2010, A&A, 523, A31Ho S., Hirata C., Padmanabhan N., Seljak U., Bahcall N., 2008,Phys. Rev. D, 78, 043519Hoekstra H., Jain B., 2008, Annual Review of Nuclear and Parti-cle Science, 58, 99Leistedt B., Mortlock D. J., Peiris H. V., 2016, ArXiv e-printsLimber D. N., 1953, ApJ, 117, 134Limber D. N., 1954, ApJ, 119, 655LSST Science Collaboration et al., 2009, ArXiv:0912.0201Matthews D. J., Newman J. A., 2010, ApJ, 721, 456Matthews D. J., Newman J. A., Coil A. L., Cooper M. C., GwynS. D. J., 2013, ApJ Suppl., 204, 21Matute I. et al., 2012, A&A, 542, A20McQuinn M., White M., 2013, MNRAS, 433, 2857M´enard B., Scranton R., Schmidt S., Morrison C., Jeong D., Bu-davari T., Rahman M., 2013, ArXiv e-prints Moles M. et al., 2008, AJ, 136, 1325Molino A. et al., 2014, MNRAS, 441, 2891Newman J. A., 2008, ApJ, 684, 88Newman J. A. et al., 2015, Astroparticle Physics, 63, 81Padmanabhan N., et al., 2007, MNRAS, 378, 852Phillipps S., 1985, MNRAS, 212, 657Phillipps S., Shanks T., 1987, MNRAS, 227, 115Rahman M., M´enard B., Scranton R., Schmidt S. J., MorrisonC. B., 2015, MNRAS, 447, 3500Rau M. M., Seitz S., Brimioulle F., Frank E., Friedrich O., GruenD., Hoyle B., 2015, MNRAS, 452, 3710S´anchez C. et al., 2014, MNRAS, 445, 1482Schmidt S. J., M´enard B., Scranton R., Morrison C., McBrideC. K., 2013, MNRASSchulz A. E., 2010, ApJ, 724, 1305Seldner M., Peebles P. J. E., 1979, ApJ, 227, 30 c (cid:13)000