key: cord-0315138-rl8bt6u0 authors: Stanford, Euclid Collaboration S. A.; Masters, D.; Darvish, B.; Stern, D.; Cohen, J. G.; Capak, P.; Hernitschek, N.; Davidzon, I.; Rhodes, J.; Sanders, D. B.; Mobasher, B.; Castander, F. J.; Paltani, S.; Aghanim, N.; Amara, A.; Auricchio, N.; Balestra, A.; Bender, R.; Bodendorf, C.; Bonino, D.; Branchini, E.; Brinchmann, J.; Capobianco, V.; Carbone, C.; Carretero, J.; Casas, R.; Castellano, M.; Cavuoti, S.; Cimatti, A.; Cledassou, R.; Conselice, C. J.; Corcione, L.; Costille, A.; Cropper, M.; Degaudenzi, H.; Douspis, M.; Dubath, F.; Dusini, S.; Fosalba, P.; Frailis, M.; Franceschi, E.; Franzetti, P.; Fumana, M.; Garilli, B.; Giocoli, C.; Grupp, F.; Haugan, S. V. H.; Hoekstra, H.; Holmes, W.; Hormuth, F.; Hudelot, P.; Jahnke, K.; Kiessling, A.; Kilbinger, M.; Kitching, T.; Kubik, B.; Kummel, M.; Kunz, M.; Kurki-Suonio, H.; Laureijs, R.; Ligori, S.; Lilje, P. B.; Lloro, I.; Maiorano, E.; Marggraf, O.; Markovic, K.; Massey, R.; Meneghetti, M.; Meylan, G.; Moscardini, L.; Niemi, S. M.; Padilla, C.; Pasian, F.; Pedersen, K.; Pettorino, V.; Pires, S.; Poncet, M.; Popa, L.; Pozzetti, L.; Raison, F.; Roncarelli, M.; Rossetti, E.; Saglia, R.; Scaramella, R.; Schneider, P.; Secroun, A.; Seidel, G.; Serrano, S.; Sirignano, C.; Sirri, G.; Taylor, A. N.; Teplitz, H. I.; Tereno, I.; Toledo-Moreo, R.; Valentijn, E. A.; Valenziano, L.; Kleijn, G. A. Verdoes; Wang, Y.; Zamorani, G.; Zoubian, J.; Brescia, M.; Congedo, G.; Conversi, L.; Copin, Y.; Kermiche, S.; Kohley, R.; Medinaceli, E.; Mei, S.; Moresco, M.; Morin, B.; Munari, E.; Polenta, G.; Sureau, F.; Cresp'i, P. Tallada; Vassallo, T.; Zacchei, A.; Andreon, S.; Aussel, H.; Baccigalupi, C.; Balaguera-Antol'inez, A.; Baldi, M.; Bardelli, S.; Biviano, A.; Borsato, E.; Bozzo, E.; Burigana, C.; Cabanac, R.; Camera, S.; Cappi, A.; Carvalho, C. S.; Casas, S.; Castignani, G.; Colodro-Conde, C.; Coupon, J.; Courtois, H. M.; Cuby, J.-G.; Silva, A. Da; Torre, S. de la; Ferdinando, D. Di; Duncan, C. A. J.; Dupac, X.; Fabricius, M.; Farina, M.; Farrens, S.; Ferreira, P. G.; Finelli, F.; Flose-Reimberg, P.; Fotopoulou, S.; Galeotta, S.; Ganga, K.; Gillard, W.; Gozaliasl, G.; Graci'a-Carpio, J.; Keihanen, E.; Kirkpatrick, C. C.; Lindholm, V.; Mainetti, G.; Maino, D.; Martinet, N.; Marulli, F.; Maturi, M.; Maurogordato, S.; Metcalf, R. B.; Nakajima, R.; Neissner, C.; Nightingale, J. W.; Nucita, A. A.; Patrizii, L.; Potter, D.; Renzi, A.; Riccio, G.; Romelli, E.; Sanchez, A. G.; Sapone, D.; Schirmer, M.; Schultheis, M.; Scottez, V.; Stanco, L.; Tenti, M.; Teyssier, R.; Torradeflot, F.; Valiviita, J.; Viel, M.; Whittaker, L.; Zucca, E. title: Euclid Preparation: XIV. The Complete Calibration of the Color-Redshift Relation (C3R2) Survey: Data Release 3 date: 2021-06-21 journal: nan DOI: 10.3847/1538-4365/ac0833 sha: a5a2398b2863d64681ed283851f987c867367817 doc_id: 315138 cord_uid: rl8bt6u0 The Complete Calibration of the Color-Redshift Relation (C3R2) survey is obtaining spectroscopic redshifts in order to map the relation between galaxy color and redshift to a depth of i ~ 24.5 (AB). The primary goal is to enable sufficiently accurate photometric redshifts for Stage IV dark energy projects, particularly Euclid and the Roman Space Telescope, which are designed to constrain cosmological parameters through weak lensing. We present 676 new high-confidence spectroscopic redshifts obtained by the C3R2 survey in the 2017B-2019B semesters using the DEIMOS, LRIS, and MOSFIRE multi-object spectrographs on the Keck telescopes. Combined with the 4454 redshifts previously published by this project, the C3R2 survey has now obtained and published 5130 high-quality galaxy spectra and redshifts. If we restrict consideration to only the 0.2 0.15) to be 0.041, and the bias spectroscopic redshifts were obtained. The true outlier fraction is 0.052, the scatter is 0.033, and after removing outliers the bias is −0.0025 over the entire redshift range. In the left panel those objects which are outliers are located outside of the dashed lines. The dotted line in the right panel is a Gaussian centered at zero with = 0.033. Note that the photometry used to place galaxies on the SOM here is generally deeper than the Euclid wide survey, so these results are not fully representative of what Euclid will achieve. mean (z p − z s ) 1 + z s to be −0.0025, where the scatter and bias are calculated after removing outliers. If we consider redshifts only in the range 0.2 < p < 2.6, corresponding to the epoch that will be used by Euclid for its cosmological weak lensing survey, the bias is −0.0020 (after removing outliers). If we include good quality spectroscopic redshifts ( = 3 or = 3.5) along with the = 4 redshifts from all three C3R2 data releases, then the scatter is 0.024, the true outlier fraction is 0.056, and the bias is −0.0018 (after removing outliers) over the entire redshift range. In Figure 4 , which shows the results for the COSMOS field, we see that there is an increase in the fraction of outliers when all available high-quality spectroscopic redshifts are compared with the SOM photometric redshifts. We dis- Figure 2 , except here we show the performance of the SOM method only in the COSMOS field using all high-quality ( = 4) redshifts from C3R2 combined with all high-quality spectroscopic redshifts available from the literature. For this case, the true outlier fraction is 0.071, the scatter is 0.024, and after removing outliers the bias is −0.0020. The dotted line in the right panel is a Gaussian centered at zero with = 0.024. cuss the outliers in the DR3 sample in more detail in Section 3.3. Discussion of the other samples from the literature in more detail is beyond the scope of this work; we plan to investigate the causes of the photometric and spectroscopic redshift outliers in the future. Once all the outliers are removed, the bias is calculated to be -0.0020 over the entire redshift range. This improvement relative to the case of only the C3R2 redshifts is likely due to the inclusion of brighter galaxies from the literature samples. In M19 we compared C3R2 spectroscopic redshifts to photometric redshifts in the COSMOS field determined with the template fitting code Le Phare (Arnouts et al. 1999; Ilbert et al. 2006 ) using the same photometry as used here to generate the SOM and SOM-based photometric redshifts. Considering the highest quality ( = 4) redshifts from the three merged C3R2 data releases, these Le Phare photometric red-shifts show a scatter NMAD of 0.029, an outlier fraction of 0.038, and a bias (after removing outliers) of −0.013. While the scatter and outlier rate are similar to those from the SOMbased photometric redshifts (Figure 3) , the Le Phare bias is substantially worse. It may be possible to combine a SOMdetermined calibration with the Le Phare method in a two step process to produce photometric redshifts for Euclid, although it is important to remember that the photometry that will be available for calculating Euclid photometric redshifts will not be as good as the existing 30-band photometry in the COSMOS field. Figure 5 presents the SOMs based on all available highconfidence spectroscopic redshifts in C3R2. The fraction of cells with at least one spectroscopic redshift has increased from 49% in M15 to 84%, and these cells account for 91% of the galaxies. If we restrict consideration to only the 0.2 < p < 2.6 range of interest for the Euclid cosmological goals, then the fraction of cells with at least one spectroscopic redshift is now 88%, and these cells account for more than 92% of the galaxies. Figure 6 presents the distributions of the calibrated and uncalibrated cells in the SOM map as a function of magnitude and of redshift. We consider galaxy color cells to be "calibrated" if at least one galaxy in the cell has a high-confidence spectroscopic redshift, while "uncalibrated" cells are those for which no galaxy occupying that cell has a secure redshift measurement. These plots show that most of the uncalibrated cells are faint and located at p < 0.6. The next two sets of plots in Figure 7 further elucidate the nature of these populations. In particular we see that most of the uncalibrated galaxies at faint magnitudes and low photometric redshifts are likely to be passive rather than active, where the restframe colors, estimated with the COSMOS2015 (Laigle et al. 2016 ) data, are used to determine if a galaxy is active or passive. It is worth noting that the photometric redshifts based on Euclid and LSST data for faint, passive galaxies at low redshifts are expected to be robust. In addition to the targeted objects, we also determined = 4 redshifts for 48 serendipitous sources in DR3, and an additional 19 = 3 or = 3.5 redshifts. Some of these serendipitous sources turned out to be in the target catalogs; these are listed in the redshift table with the ID of the matched object and a prefix of "serendip". A few of the serendipitous sources are located very close (e.g., within ∼1 arcsec) to the targeted galaxy. In such situations the catalog photometry is probably affected by blending of the sources within the aperture used on the images to measure brightnesses. This might not be the case for Euclid VIS images, where the detection+segmentation will be used to perform template fitting photometry. The extent to which apparently secure redshifts could affect photometric redshift calibration still needs further investigation. As a final step in the analysis, we identified all sources in DR3 with secure redshifts (i.e., quality flag ≥ 3) where the spectroscopic redshift was discrepant from the SOM-based redshift by at least 15% (i.e., | s − SOM |/(1 + s ) > 0.15). At least two co-authors independently assessed each of the 80 such cases identified. For three galaxies, this reanalysis lowered the spectroscopic quality flag to < 3. For the majority of the remaining cases (41/70; 59%), we revised the assigned spectroscopic redshift based on a reconsideration of the data and the photometric redshift. Most of these cases were one-line redshifts initially assigned quality flag = 3 observed with either MOSFIRE (with its limited spectral range) or LRIS. Typically, a target had a single faint emission line detected, initially assumed to be, for example, H . However, if the line had a different identification, such as [O III], the spectroscopic redshift matched the SOM-based redshift. In all such cases, corroborating lines were searched for, but generally were not found, either due to the limited spectral coverage or strong sky lines. In another two cases, this exercise made us realize that we had misidentified a targeted galaxy with a serendipitous galaxy. In only one case (out of the entire DR3 sample of 635 high-quality redshifts) was a = 4 redshift corrected by this exercise. In that case, both initial spectroscopic redshift assessments matched, though a re-analysis clearly showed that redshift to be incorrect. The remaining outliers were approximately evenly split between three categories. For approximately one-third of the outliers, the spectroscopic redshift was discrepant from the SOM-based photometric redshift but matched the Le Phare photometric redshift, suggesting an issue with the SOMbased redshift (e.g., potentially a SOM cell with a bimodal photometric redshift distribution). In fact, for 2/11 of these cases, we find that the SOM cell of the galaxies is on a clear "caustic" separating low-and high-redshift galaxies with similar Euclid colors, while the others tend to be in cells with intrinsically high redshift dispersion compared with most parts of the color space. It is plausible that the 30-band COSMOS data is capable of resolving these degeneracies, hence the better Le Phare photometric redshift estimates. On the other hand, a detailed examination of the Hubble ACS imaging of these sources showed that a majority seem to be affected by blending to some degree, which may also contribute to the incorrect SOM-based photometric redshifts. Another third of the outliers are due to blends. While the input source catalog is based on ground-based, multi-band photometry, in several cases Hubble imaging clearly resolved the source into two close galaxies, thereby compromising the photometric redshift. While this was easily assessed in the COSMOS field, such higher resolution imaging was not available for most of the other target fields. Finally, the remaining third of targets were deemed true outliers, where the spectroscopic redshift was deemed solid (e.g., generally = 4), but did not match either the SOM-based or the Le Phare photometric redshift. We expect that several of these cases are also unidentified blends where Hubble imaging was not available, highlighting a likely concern for future cosmology projects, both in developing the "gold sample" of calibration photometric redshifts, as well as in assigning pho- tometric redshifts to the large sample of observed objects. It is worth noting that near-Hubble quality imaging will in fact be available for deblending algorithms to use in creating photometric catalogs for Euclid. While we do not expect to spectroscopically confirm all targeted objects, it is useful to examine the possible failure modes that led to not being able to determine a redshift. Neglecting the obvious explanations having to do with poor seeing, clouds, shortened exposures, instrument failure (one LRIS night, primarily targeting Lyman-break galaxies, was conducted without the blue side), there were 32 masks out of the 50 observed which were observed in at least good conditions. These 32 good masks had 1346 targeted objects, of which no secure redshift was obtained for 835 sources. Because the COSMOS catalog has the best photometry and so should have the best predicted exposure times, we restrict our investigation of the failures to only the 597 objects observed in the COSMOS field. The distribution of the good COSMOS masks by instrument is 10/5/1 for MOS-FIRE/DEIMOS/LRIS, respectively. There were 70 objects for which a low-confidence ( = 1 or 2) redshift was determined, 162 with a failure code of −91 (too faint), 39 with a failure code of −92 (reasonable detection but no redshift), and 25 have a code of −93 or −94 meaning there were issues with the data or reductions. The success rates by instrument are fairly similar for DEIMOS (43%) and MOSFIRE (37%) for the good COSMOS masks; there was only one mask done by LRIS in good conditions in the COSMOS field. The success rate for the good LRIS observations in all three fields was lower compared to that obtained with DEIMOS and MOSFIRE. High confidence redshifts were obtained of only 61 of the 413 targeted objects in the 14 good LRIS masks; no secure redshifts were obtained for 347 objects, and the reductions were problematic for five objects. The high confidence redshift success rate for the good LRIS masks was 15%. The vast majority of the targeted objects in the LRIS masks are 1.6 < p < 3 Lyman-break galaxies which often do not have any emission lines. In these cases, redshifts usually need to be determined from rest frame UV absorption features which are much more difficult to detect. This analysis suggests that for the ∼ 2 Lyman-break galaxies we should revise the way that we predict exposure times. Possible solutions to the low success rate with the LRIS observations of such galaxies would be to obtain deeper exposures, or to try observing such targets with MOSFIRE which offers the possibility of detecting emission lines in the rest frame optical. Top left: The distribution of "calibrated" galaxies (i.e., those in SOM cells with at least one spectroscopic redshift) in the plane of redshift and -band magnitude. Top right: The same plot as in the top right panel, but for currently "uncalibrated" galaxies. Bottom left: The relative fractions of passive galaxies in the calibrated set as a function of magnitude and redshift, as estimated from rest-frame UVJ colors. Bottom right: The same plot but for the uncalibrated galaxies. It is clear from these figures that the currently uncalibrated galaxies are fainter and tend to be more quiescent than the calibrated galaxies. In all panels the analysis uses galaxies in the COSMOS2015 sample, and white areas represent no galaxies in a bin. 4. SUMMARY We have presented 676 new high-confidence spectroscopic redshifts obtained by the C3R2 survey. Taken together with the previous two data releases, we have now collected and published 5130 high-confidence spectroscopic redshifts (and associated publicly released spectra) covering a wide range in galaxy type and redshift. 2 While good progress has been made in covering the Euclid color space with spectroscopic redshifts, and the prospects for using these redshifts to calibrate photo-z calculations for Euclid are promising, challenges remain. More work remains to be done to validate published samples of spectroscopic redshifts before they can also be used for this purpose. Also, the relatively low success rate of the observations (particularly with LRIS) of galaxies without emission lines emphasizes the technical challenges faced by redshift surveys such as C3R2 that are attempting to calibrate photometric redshifts with great precision down to the faint magnitudes of interest to upcoming space missions such as Euclid. A significant portion of the research and manuscript preparation took place during the COVID-19 global pandemic. The authors would like to thank all those who risked their lives as essential workers in order for us to safely continue our work from home. The authors also thank the referee whose comments helped improve the clarity and presentation of our results. Table 4 is published in its entirety in machine-readable format. A portion is shown here for guidance regarding its form and content. More detailed information is available in the header of the machine-readable table. Instrument Design and Performance for Optical/Infrared Ground-based Telescopes Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series The research was carried out in part at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. This work was enabled by a NASA Keck grant. FJC acknowledges support from grant ESP2017-89838 ad H2020 European Commission grant 776247.The data presented herein were obtained at the W. M. Keck Observatory, which is operated as a scientific partnership among the California Institute of Technology, the University of California and the National Aeronautics and Space Administration. The Observatory was made possible by the generous financial support of the W. M. Keck Foundation. The authors wish to recognize and acknowledge the very significant cultural role and reverence that the summit of Maunakea has always had within the indigenous Hawaiian community. We are most fortunate to have the opportunity to conduct observations from this mountain.The Euclid Consortium acknowledges the European Space Agency and a number of agencies and institutes that have supported the development of Euclid, in particular the Academy of Finland, the Agenzia Spaziale Italiana, the Belgian Science Policy, the Canadian Euclid Consortium, the Centre National d'Etudes Spatiales, the Deutsches Zentrum für Luft-und Raumfahrt, the Danish Space Research Institute, the Fundação para a Ciência e a Tecnologia, the Ministerio de Economia y Competitividad, the National Aeronautics and Space Administration, the Netherlandse Onderzoekschool Voor Astronomie, the Norwegian Space Agency, the Romanian Space Agency, the State Secretariat for Education, Research and Innovation (SERI) at the Swiss Space Office (SSO), and the United Kingdom Space Agency. A complete and detailed list is available on the Euclid web site (http://www.euclid-ec.org).