Strong lens candidates in the DESI DECam Legacy Survey
Huang et al. (2019)
But this is not completely the end of the story...
$\Longrightarrow$ At the rate of LSST, this would mean needing human inspection for 20 million candidates.
Inference at the Image Level
Inferring photometric redshifts from images
Pasquet, Bertin, Treyer, Arnouts, Fouchez (2019)
Inferring surface brightness profile parameters
Tucillo, Huertas-Company, et al. (2017)
Estimated Sersic index by CNN before adaptation (left),
after adaptation (right)
Inference at the Image Level
Reiman & Göhre (2018)
High Dimensional Bayesian Inverse Problems
Adam et al. (2022)
$\Longrightarrow$ generates samples from the full bayesian posterior
$$p(x | y) = \underbrace{p(y|x)}_{\mbox{known likelihood}} \qquad \underbrace{p(x)}_{\mbox{learned prior}}$$
takeways
Is Deep Learning Really Changing the Game in Interpreting Survey
Data?
For Detection, Classification, Cleaning tasks:
Yes, within limits
$\Longrightarrow$ As long as we don't need to understand
precisely the model response/selection function.
For infering physical properties needed in downstream
analysis: Not really...
In general the exact response is not known, and very
non-linear.
Not addressing the core question of representativeness of
training data.
For solving challenging data restoration problems: getting there!
Still need to be careful about the representativeness of training data.
But the problem is well defined, likelihood is physically motivated, and the response is well understood.
Accelerating Cosmological Simulations with Deep Learning
are these Deep Learning models a real game-changer?
The Limitations of Black-Box Large Deep Learning Approaches
There is a risk that a large Deep Learning model can
silently fail.
$\Rightarrow$
How can we build confidence in the output of the
neural network?
Training these models can require a very large number of
simulations. $\Rightarrow$
Do they bring a net computational benefit?
In case of the super-resolution model of
Li et al. (2021), only 16 full-resolution
$512^3$ N-body were necessary.
In many cases, the accuracy (not the quantity) of
simulations will be the main bottleneck.
$\Rightarrow$
What new science are these deep learning models
enabling?
In the case of cosmological SBI, they do not help us
resolve the uncertainty on the simulation model.
(Harrington et al. 2021)
What Would be Desirable Properties of robust ML-based Emulation
Methods?
Make use of known symmetries and physical
constraints.
Modeling residuals to an approximate physical model
Minimally parametric
Can be trained with a very small number of simulations
Could potentially be
inferred from data!
Learning effective physical laws for generating cosmological
hydrodynamics with Lagrangian Deep Learning
Dai & Seljak (2021)
The Lagrangian Deep Learning approach:
Run an approximate Particle-Mesh DM simulation (about 10
steps)
Introduce a displacement $\mathbf{S}$ of particles:
\begin{equation} \mathbf{S}=-\alpha\nabla \hat{O}_{G}
f(\delta) \end{equation} where $\hat{O}_{g}$ is a
parametric Fourier-space filter, $f$ is a parametric
function of $\delta$. $\Rightarrow$ Respects
translational and rotational symmetries.
Apply a non-linear function on the resulting density
field $\delta^\prime$: $$F(x) = \mathrm{ReLu}(b_1
f(\delta^\prime) - b_0)$$
Measure the ellipticity $\epsilon = \epsilon_i + \gamma$ of
all galaxies
$\Longrightarrow$ Noisy tracer of the weak lensing shear
$\gamma$
Compute summary statistics based on 2pt
functions, e.g. the power spectrum
Run an MCMC to recover a posterior on model parameters,
using an analytic likelihood
$$ p(\theta | x ) \propto \underbrace{p(x |
\theta)}_{\mathrm{likelihood}} \
\underbrace{p(\theta)}_{\mathrm{prior}}$$
Main limitation: the need for an explicit likelihood
We can only compute the likelihood for
simple summary statistics and on
large scales
$\Longrightarrow$ We are dismissing a significant fraction of
the information!
Full-Field Simulation-Based Inference
Instead of trying to analytically evaluate the likelihood of
sub-optimal summary statistics, let us build a forward model of the full observables.
$\Longrightarrow$ The simulator becomes the physical model.
Each component of the model is now tractable, but at the
cost of a large number of latent variables.
Benefits of a forward modeling approach
Fully exploits the information content of the data
(aka "full field inference").
Easy to incorporate systematic effects.
Easy to combine multiple cosmological probes by joint simulations.
(Porqueres et al. 2021)
...so why is this not mainstream?
The Challenge of Simulation-Based Inference
$$ p(x|\theta) = \int p(x, z | \theta) dz = \int p(x | z,
\theta) p(z | \theta) dz $$ Where $z$ are
stochastic latent variables of the simulator.
$\Longrightarrow$ This
marginal likelihood is intractable! Hence
the phrase "Likelihood-Free Inference"
Neural Compressor: Graph Convolutional Neural Network on the Sphere Trained by Fisher information maximization.
Cosmological constraints from HSC survey first-year data using deep learning
Lu, Haiman, Li (2023)
How much usable information is there beyond the power spectrum?
Chisari et al. (2018)
Ratio of power spectrum in hydrodynamical simulations vs. N-body simulations
Secco et al. (2021)
DES Y3 Cosmic Shear data vector
$\Longrightarrow$ Can we find non-Gaussian information that is not affected by baryons?
Example of unforeseen impact of shortcuts in simulations
Gatti, Jeffrey, Whiteway et al. (2023)
Is it ok to distribute lensing source galaxies randomly in simulations, or should they be clustered?
$\Longrightarrow$ An SBI analysis could be biased by this effect and you would never know it!
takeways
Likelihood-Free Inference automatizes inference over
numerical simulators.
Turns both summary extraction and inference problems into an
optimization problems
Deep learning allows us to solve that problem!
In the context of upcoming surveys, this techniques provides
many advantages:
Amortized inference: near instantaneous parameter
inference, extremely useful for time-domain.
Optimal information extraction: no longer need for
restrictive modeling assumptions needed to obtain tractable
likelihoods.
Will we be able to exploit all of the information content of LSST,
Euclid, DESI?
$\Longrightarrow$ Not rightaway, but it is not the fault of Deep
Learning!
Deep Learning has redefined the limits of our statistical
tools, creating
additional demand on the accuracy of simulations far
beyond the power spectrum.
Neural compression methods have the downside of being opaque.
It is much harder to detect unknown systematics.
We will need a significant number of
large volume, high resolution simulations.
Conclusion
Deep Learning 8 years later: Revolution or Incremental Science?
The most impactful applications so far are low level
“out-of-the-box” data reduction applications (incremental science).
In some ways, the statistical tools are now beyond our physical
modeling capabilities, they are no longer a limiting factor.
Where are we going next?
Foundation Models for feature extraction:
More information can be extracted from data by self-supervised learning across surveys and data modalities
Generative Models for Data-Driven modeling:
Generative models can be used to learn part of the physical processes that generate the data.
Will Foundation Models make their way to astrophysics?