ANNz2: Photometric Redshift and Probability Distribution Function Estimation using Machine Learning

Aims. We aim to use multiband imaging from the Phase-3 Verification Data of the J-PLUS survey to derive accurate photometric redshifts (photo-z) and look for potential new members in the surroundings of the nearby galaxy clusters A2589 (z = 0.0414) & A2593 (z = 0.0440), using redshift probability distribution functions (PDFs). The ultimate goal is to demonstrate the usefulness of a 12-band filter system in the study of largescale structure in the local Universe. Methods. We present an optimized pipeline for the estimation of photometric redshifts in clusters of galaxies. This pipeline includes a PSF-corrected photometry, specific photometric apertures capable of enhancing the integrated signal in the bluest filters, a careful recalibration of the photometric uncertainties and accurate upper-limit estimations for faint detections. To foresee the expected precision of our photo-z beyond the spectroscopic sample, we designed a set of simulations in which real cluster galaxies are modeled and reinjected inside the images at different signal-to-noise ratio (S/N) levels, recomputing their photometry and photo-z estimates. Results. We tested our photo-z pipeline with a sample of 296 spectroscopically confirmed cluster members with an averaged magnitude of ⟨r⟩ = 16.6 and redshift ⟨z⟩ = 0.041. The combination of seven narrow and five broadband filters with a typical photometric-depth of r ~ 21.5 provides δz/(1 + z) = 0.01 photo-z estimates. A precision of δz/(1 + z) = 0.005 is obtained for the 177 galaxies brighter than magnitude r < 17. Based on simulations, a δz/(1 + z) = 0.02 and δz/(1 + z) = 0.03 is expected at magnitudes ⟨r⟩ = 18 and ⟨r⟩ = 22, respectively. Complementarily, we used SDSS/DR12 data to derive photo-z estimates for the same galaxy sample. This exercise demonstrates that the wavelength-resolution of the J-PLUS data can double the precision achieved by SDSS data for galaxies with a high S/N. Based on the Bayesian membership analysis carried out in this work, we find as much as 170 new candidates across the entire field (~5 deg2). The spatial distribution of these galaxies may suggest an overlap between the systems with no evidence of a clear filamentary structure connecting the clusters. This result is supported by X-ray Rosat All-Sky Survey observations suggesting that a hypothetical filament may have low density contrast on diffuse warm gas. Conclusions. We prove that the addition of the seven narrow-band filters make the J-PLUS data deeper in terms of photo-z-depth than other surveys of a similar photometric-depth but using only five broadbands. These preliminary results show the potential of J-PLUS data to revisit membership of groups and clusters from nearby galaxies, important for the determination of luminosity and mass functions and environmental studies at the intermediate and low-mass regime.

Accurate photometric redshift (photo-z) estimation requires support from multiband observational data. However, in the actual process of astronomical observations and data processing, some sources may have missing observational data in certain bands for various reasons. This could greatly affect the accuracy and reliability of photo-z estimation for these sources, and even render some estimation methods unusable. The same situation may exist for the upcoming Chinese Space Station Telescope (CSST). In this study, we employ a deep learning method called generative adversarial imputation networks (GAIN) to impute the missing photometric data in CSST, aiming to reduce the impact of data missing on photo-z estimation and improve estimation accuracy. Our results demonstrate that using the GAIN technique can effectively fill in the missing photometric data in CSST. Particularly, when the data missing rate is below 30 per cent, the imputation of photometric data exhibits high accuracy, with higher accuracy in the g, r, i, z, and y bands compared to the NUV and u bands. After filling in the missing values, the quality of photo-z estimation obtained by the widely used easy and accurate Zphot from Yale (eazy) software is notably enhanced. Evaluation metrics for assessing the quality of photo-z estimation, including the catastrophic outlier fraction (fout), the normalized median absolute deviation ($\rm {\sigma _{NMAD}}$), and the bias of photometric redshift (bias), all show some degree of improvement. Our research will help maximize the utilization of observational data and provide a new method for handling sample missing values for applications that require complete photometry data to produce results.

We present catalogues of calibrated photometry and spectroscopic redshifts in the Extended Groth Strip, intended for studies of photometric redshifts (photo-z’s). The data includes ugriz photometry from Canada–France–Hawaii Telescope Legacy Survey (CFHTLS) and Y-band photometry from the Subaru Suprime camera, as well as spectroscopic redshifts from the DEEP2, DEEP3, and 3D-HST surveys. These catalogues incorporate corrections to produce effectively matched-aperture photometry across all bands, based upon object size information available in the catalogue and Moffat profile point spread function fits. We test this catalogue with a simple machine learning-based photometric redshift algorithm based upon Random Forest regression, and find that the corrected aperture photometry leads to significant improvement in photo-z accuracy compared to the original SExtractor catalogues from CFHTLS and Subaru. The deep ugrizY photometry and spectroscopic redshifts are well suited for empirical tests of photometric redshift algorithms for LSST. The resulting catalogues are publicly available at http://d-scholarship.pitt.edu/36064/. We include a basic summary of the strategy of the DEEP3 Galaxy Redshift Survey to accompany the recent public release of DEEP3 data.

Extraction of the Baryon Acoustic Oscillations (BAO) to per cent level accuracy is challenging and demands an understanding of many potential systematics to an accuracy well below 1 per cent, in order to ensure that they do not combine significantly when compared to statistical error of the BAO measurement. Baryon Oscillation Spectroscopic Survey (BOSS) Data Release 11 (DR11) reaches a distance measurement with ?1 per cent statistical error and this prompts an extensive search for all possible sub-per cent level systematic errors which could previously be safely ignored. In this paper, we analyse the potential systematics in BAO fitting methodology using mocks and data from BOSS DR10 and DR11. We demonstrate the robustness of the fiducial multipole fitting methodology to be at 0.1-0.2 per cent level with a wide range of tests in mock galaxy catalogues pre- and post-reconstruction. We also find the DR10 and DR11 data from BOSS to be robust against changes in methodology at a similar level. This systematic error budget is incorporated into the BOSS DR10 and DR11 BAO measurements. Of the wide range of changes we have investigated, we find that when fitting post-reconstructed data or mocks, the only change which has an effect >0.1 per cent on the best-fitting values of distance measurements is varying the order of the polynomials to describe the broad-band terms (?0.2 per cent). Finally, we compare an alternative methodology denoted as Clustering Wedges with Multipoles, and find that it is consistent with the standard approach.

Photometric redshift (photo-z) estimates are playing an increasingly important role in extragalactic astronomy and cosmology. Crucial to many photo-z applications is the accurate quantification of photometric redshift errors and their distributions, including identification of likely catastrophic failures in photo-z estimates. We consider several methods of estimating photo-z errors and propose new training-set based error estimators based on spectroscopic training set data. Using data from the Sloan Digital Sky Survey and simulations of the Dark Energy Survey as examples, we show that this method provides a robust, relatively unbiased estimate of photo-z errors. We show that culling objects with large, accurately estimated photo-z errors from a sample can reduce the incidence of catastrophic photo-z failures.

We measure the large-scale cross-correlation of quasars with the Lyα forest absorption in redshift space, using ∼ 60000 quasar spectra from Data Release 9 (DR9) of the Baryon Oscillation Spectroscopic Survey (BOSS). The cross-correlation is detected over a wide range of scales, up to comoving separations r of 80 h−1Mpc. For r > 15 h−1Mpc, we show that the cross-correlation is well fitted by the linear theory prediction for the mean overdensity around a quasar host halo in the standard ΛCDM model, with the redshift distortions indicative of gravitational evolution detected at high confidence. Using previous determinations of the Lyα forest bias factor obtained from the Lyα autocorrelation, we infer the quasar bias factor to be bq = 3.64+0.13−0.15 at a mean redshift z = 2.38, in agreement with previous measurements from the quasar auto-correlation. We also obtain a new estimate of the Lyα forest redshift distortion factor, βF = 1.1±0.15, slightly larger than but consistent with the previous measurement from the Lyα forest autocorrelation. The simple linear model we use fails at separations r < 15h−1Mpc, and we show that this may reasonably be due to the enhanced ionization due to radiation from the quasars. We also provide the expected correction that the mass overdensity around the quasar implies for measurements of the ionizing radiation background from the line-of-sight proximity effect.

Large imaging surveys will rely on photometric redshifts (photo-z's), which are typically estimated through machine-learning methods. Currently planned spectroscopic surveys will not be deep enough to produce a representative training sample for Legacy Survey of Space and Time (LSST), so we seek methods to improve the photo-z estimates that arise from nonrepresentative training samples. Spectroscopic training samples for photo-z's are biased toward redder, brighter galaxies, which also tend to be at lower redshift than the typical galaxy observed by LSST, leading to poor photo-z estimates with outlier fractions nearly 4 times larger than for a representative training sample. In this Letter, we apply the concept of training sample augmentation, where we augment simulated nonrepresentative training samples with simulated galaxies possessing otherwise unrepresented features. When we select simulated galaxies with (g-z) color, i-band magnitude, and redshift outside the range of the original training sample, we are able to reduce the outlier fraction of the photo-z estimates for simulated LSST data by nearly 50% and the normalized median absolute deviation (NMAD) by 56%. When compared to a fully representative training sample, augmentation can recover nearly 70% of the degradation in the outlier fraction and 80% of the degradation in NMAD. Training sample augmentation is a simple and effective way to improve training samples for photo-z's without requiring additional spectroscopic samples.

We perform a tomographic baryon acoustic oscillations (BAO) analysis using the monopole, quadrupole and hexadecapole of the redshift-space galaxy power spectrum measured from the pre-reconstructed combined galaxy sample of the completed Sloan Digital Sky Survey (SDSS-III) Baryon Oscillation Spectroscopic Survey (BOSS) Data Release (DR)12 covering the redshift range of $0.20<z<0.75$. By allowing for overlap between neighbouring redshift slices, we successfully obtained the isotropic and anisotropic BAO distance measurements within nine redshift slices to a precision of $1.5\%-3.4\%$ for $D_V/r_d$, $1.8\% -4.2\%$ for $D_A/r_d$ and $3.7\% - 7.5\%$ for $H \ r_d$, depending on effective redshifts. We provide our BAO measurement of $D_A/r_d$ and $H \ r_d$ with the full covariance matrix, which can be used for cosmological implications. Our measurements are consistent with those presented in \citet{Acacia}, in which the BAO distances are measured at three effective redshifts. We constrain dark energy parameters using our measurements, and find an improvement of the Figure-of-Merit of dark energy in general due to the temporal BAO information resolved. This paper is part of a set that analyses the final galaxy clustering dataset from BOSS.

Marked correlation functions, which are sensitive to the clustering of galaxies in different environments, have been proposed as constraints on modified gravity models. We present measurements of the marked correlation functions of galaxies in redshift space using 361,761 LOWZ ($z_{\rm eff} = 0.32$) galaxies from the Sloan Digital Sky Survey III (SDSS III) Baryon Oscillation Spectroscopic Survey (BOSS) Data Release 12 (DR12) and compare them to $\Lambda$CDM+General Relativity simulations. We apply mass cuts to find the best match between the redshift space autocorrelation function of subhaloes in the simulation and in the observations. We then compare the marked correlation functions, finding no significant evidence for deviations of the marked correlation functions of LOWZ galaxies from $\Lambda$CDM on scales $6 \ h^{-1}$Mpc $ \leq s \leq$ $69 \ h^{-1}$Mpc. The constraining power of marked correlation functions in our analysis is limited by our ability to model the autocorrelation function of galaxies on small scales including the effect of redshift distortions. The statistical errors are well below the differences seen between marked correlation functions of $f$(R) gravity models and $\Lambda$CDM in recent publications (Armijo et al., Hern\'{a}ndez-Aguayo et al.) indicating that improved future theoretical analyses should be able to rule out some models definitively.

The ctools open-source software package was developed for the scientific analysis of astronomical data from Imaging Air Cherenkov Telescopes (IACTs), such as H.E.S.S., VERITAS, MAGIC, and the future Cherenkov Telescope Array (CTA). To date, the software has been mainly tested using simulated CTA data; however, upon the public release of a small set of H.E.S.S. observations of the Crab nebula, MSH 15–52, RX J1713.7–3946, and PKS 2155–304 validation using real data is now possible. We analysed the data of the H.E.S.S. public data release using ctools version 1.6 and compared our results to those published by the H.E.S.S. Collaboration for the respective sources. We developed a parametric background model that satisfactorily describes the expected background rate as a function of reconstructed energy and direction for each observation. We used that model, and tested all analysis methods that are supported by ctools, including novel unbinned and joint or stacked binned analyses of the measured event energies and reconstructed directions, and classical On-Off analysis methods that are comparable to those used by the H.E.S.S. Collaboration. For all analysis methods, we found a good agreement between the ctools results and the H.E.S.S. Collaboration publications considering that they are not always directly comparable due to differences in the datatsets and event processing software. We also performed a joint analysis of H.E.S.S. and Fermi-LAT data of the Crab nebula, illustrating the multi-wavelength capacity of ctools. The joint Crab nebula spectrum is compatible with published literature values within the systematic uncertainties. We conclude that the ctools software is mature for the analysis of data from existing IACTs, as well as from the upcoming CTA.

Accurate photometric redshift (photo- z ) estimation is a key challenge in cosmology, as uncertainties in photo- z directly limit the scientific return of large-scale structure and weak lensing studies, especially in upcoming Stage IV surveys. The problem is particularly severe for faint galaxies with sparse spectroscopic training data. In this work, we introduce nflow- z , a novel photo- z estimation method using the powerful machine learning technique of normalizing flow. nflow- z explicitly models the redshift probability distribution conditioned on the observables such as fluxes and colors. We build two nflow- z implementations, dubbed cINN and cNSF, and compare their performance. We demonstrate the effectiveness of nflow- z on several datasets, including a CSST mock, the COSMOS2020 catalog, and samples from Dark Energy Survey (DES) Y1, the Sloan Digital Sky Survey, and DESCaLS. Our evaluation against state-of-the-art algorithms shows that nflow- z performs favorably. For instance, cNSF surpasses random forest, multilayer perceptron, and convolutional neural network on the CSST mock test. We also achieve a 30% improvement over official results for the faint DESCaLS sample and outperform conditional generative adversarial network and mixture density network methods on the DES Y1 dataset test. Furthermore, nflow- z is computationally efficient, requiring only a fraction of the computing time of some of the competing algorithms. Our algorithm is particularly effective for the faint sample with sparse training data, making it highly suitable for upcoming Stage IV surveys.

We present a cosmic void catalog using the large-scale structure galaxy catalog from the Baryon Oscillation Spectroscopic Survey (BOSS). This galaxy catalog is part of the Sloan Digital Sky Survey (SDSS) Data Release 12 and is the final catalog of SDSS-III. We take into account the survey boundaries, masks, and angular and radial selection functions, and apply the ZOBOV void finding algorithm to the Galaxy catalog. We identify a total of 10,643 voids. After making quality cuts to ensure that the voids represent real underdense regions, we obtain 1,228 voids with effective radii spanning the range 20–100 and with central densities that are, on average, 30% of the mean sample density. We release versions of the catalogs both with and without quality cuts. We discuss the basic statistics of voids, such as their size and redshift distributions, and measure the radial density profile of the voids via a stacking technique. In addition, we construct mock void catalogs from 1000 mock galaxy catalogs, and find that the properties of BOSS voids are in good agreement with those in the mock catalogs. We compare the stellar mass distribution of galaxies living inside and outside of the voids, and find no large difference. These BOSS and mock void catalogs are useful for a number of cosmological and galaxy environment studies.

Context. With its 12 optical filters, the Javalambre-Photometric Local Universe Survey (J-PLUS) provides an unprecedented multicolor view of the local Universe. The third data release (DR3) covers 3192 deg2 and contains 47.4 million objects. However, the classification algorithms currently implemented in the J-PLUS pipeline are deterministic and based solely on the morphology of the sources. Aims. Our goal is to classify the sources identified in the J-PLUS DR3 images as stars, quasi-stellar objects (QSOs), or galaxies. For this task, we present BANNJOS, a machine learning pipeline that utilizes Bayesian neural networks to provide the full probability distribution function (PDF) of the classification. Methods. BANNJOS has been trained on photometric, astrometric, and morphological data from J-PLUS DR3, Gaia DR3, and CatWISE2020, using over 1.2 million objects with spectroscopic classification from SDSS DR18, LAMOST DR9, the DESI Early Data Release, and Gaia DR3. Results were validated on a test set of about 1.4 × 105 objects and cross-checked against theoretical model predictions. Results. BANNJOS outperforms all previous classifiers in terms of accuracy, precision, and completeness across the entire magnitude range. It delivers over 95% accuracy for objects brighter than r = 21.5 mag and ~ 90% accuracy for those up to r = 22 mag, where J-PLUS completeness is ≲ 25%. BANNJOS is also the first object classifier to provide the full PDF of the classification, enabling precise object selection for high purity or completeness, and for identifying objects with complex features, such as active galactic nuclei with resolved host galaxies. Conclusions. BANNJOS effectively classified J-PLUS sources into around 20 million galaxies, one million QSOs, and 26 million stars, with full PDFs for each, which allow for later refinement of the sample. The upcoming J-PAS survey, with its 56 color bands, will further enhance BANNJOS’s ability to detail the nature of each source.

We have designed, developed, and applied a convolutional neural network (CNN) architecture using multi-task learning to search for and characterize strong HI Lya absorption in quasar spectra. Without any explicit modeling of the quasar continuum nor application of the predicted line-profile for Lya from quantum mechanics, our algorithm predicts the presence of strong HI absorption and estimates the corresponding redshift zabs and HI column density NHI, with emphasis on damped Lya systems (DLAs, absorbers with log NHI > 20.3). We tuned the CNN model using a custom training set of DLAs injected into DLA-free quasar spectra from the Sloan Digital Sky Survey (SDSS), data release 5 (DR5). Testing on a held-back validation set demonstrates a high incidence of DLAs recovered by the algorithm (97.4% as DLAs and 99% as an HI absorber with log NHI > 19.5) and excellent estimates for zabs and NHI. Similar results are obtained against a human-generated survey of the SDSS DR5 dataset. The algorithm yields a low incidence of false positives and negatives but is challenged by overlapping DLAs and/or very high NHI systems. We have applied this CNN model to the quasar spectra of SDSS-DR7 and the Baryonic Oscillation Spectroscopic Survey (BOSS, data release 12) and provide catalogs of 4,913 and 50,969 DLAs respectively (including 1,659 and 9,230 high-confidence DLAs that were previously unpublished). This work validates the application of deep learning techniques to astronomical spectra for both classification and quantitative measurements.

We analyse the anisotropic clustering of the Sloan Digital Sky Survey-IV Extended Baryon Oscillation Spectroscopic Survey (eBOSS) Luminous Red Galaxy Data Release 14 (DR14) sample combined with Baryon Oscillation Spectroscopic Survey (BOSS) CMASS sample of galaxies in the redshift range 0.6 &amp;lt; z &amp;lt; 1.0, which consists of 80 118 galaxies from eBOSS and 46 439 galaxies from the BOSS-CMASS sample. The eBOSS-CMASS Luminous Red Galaxy sample has a sky coverage of 1844 deg2, with an effective volume of 0.9 Gpc3. The analysis was made in configuration space using a Legendre multipole expansion. The Redshift Space Distortion signal is modelled as a combination of the Convolution Lagrangian Perturbation Model and the Gaussian Streaming Model. We constrain the logarithmic growth of structure times the amplitude of dark matter density fluctuations, f(zeff)σ8(zeff) = 0.454 ± 0.134, and the Alcock-Paczynski dilation scales which constraints the angular diameter distance $D_A(z_{\mathrm{ eff}})=1466.5 \pm 133.2 (r_s/r_s^{\rm fid})$ and $H(z_{\rm eff})=105.8 \pm 15.7 (r_s^{\rm fid}/r_s) \mathrm{km\, s^{-1}\, Mpc^{-1}}$, where rs is the sound horizon at the end of the baryon drag epoch and $r_s^{\rm fid}$ is its value in the fiducial cosmology at an effective redshift zeff = 0.72. These results are in full agreement with the current Λ-Cold Dark Matter (Λ-CDM) cosmological model inferred from Planck measurements. This study is the first eBOSS LRG full-shape analysis i.e. including Redshift Space Distortions simultaneously with the Alcock-Paczynski effect and the Baryon Acoustic Oscillation scale.