In the rest of this talk

Implicit Distributions as Priors in
Inverse Problems

Linear inverse problems

Deconvolution

Inpainting

Denoising

Getting started with Deep Priors: deep denoising example

• Let us assume we have access to examples of ${\color{SkyBlue} x}$ without noise.


• We learn the distribution of noiseless data $\log p_\theta(x)$ from samples using a deep generative model.


• The solution should lie on the realistic data manifold, symbolized by the two-moons distribution.

  We want to solve for the Maximum A Posterior solution:

  $$\arg \max - \frac{1}{2} \parallel {\color{Orchid} y} - {\color{SkyBlue} x} \parallel_2^2 + \log p_\theta({\color{SkyBlue} x})$$ This can be done by gradient descent as long as one has access to the score function $\frac{\color{orange} d \color{orange}\log \color{orange}p\color{orange}(\color{orange}x\color{orange})}{\color{orange} d \color{orange}x}$.

Data-driven priors for astronomical inverse problems

Work in collaboration with
Peter Melchior, Fred Moolekamp, Remy Joseph

The Scarlet algorithm: deblending as an optimization problem

Where for a $K$ component blend:

• $P$ is the convolution with the instrumental response


• $A_i$ are channel-wise galaxy SEDs, $S_i$ are the morphology models


• $\mathbf{\Sigma}$ is the noise covariance


• $\log p_\theta$ is a PixelCNN prior


• $f_i$ and $g_i$ are arbitrary additional non-smooth consraints, e.g. positivity, monotonicity...

PixelCNN: Likelihood-based Autoregressive generative model

Models the probability $p(x)$ of an image $x$ as: $$p_{\theta}(x) = \prod_{i=0}^{n} p_{\theta}(x_i | x_{i-1} \ldots x_0)$$

• $p_\theta(x)$ is explicit! We get a number out.


• We can train the model to learn a distribution of isolated galaxy images.


• We can then evaluate its gradient $\frac{\color{orange} d \color{orange}\log \color{orange}p\color{orange}(\color{orange}x\color{orange})}{\color{orange} d \color{orange}x}$.

van den Oord et al. 2016

Training the morphology prior

Postage stamps of isolated COSMOS galaxies used for training, at Roman resolution and fixed fiducial PSF

isolated galaxy $\log p_\theta(x) = 3293.7$

artificial blend $\log p_\theta(x) = 3100.5$

What Would a Bayesian Do?

• $p(y | x)$ is the data likelihood, which contains the physics
  


• $p(x)$ is our prior knowledge on the solution.

Second Realization: Annealing is everything!

• Even with knowledge of the score, sampling in high number of dimensions is difficult!
  



• Convolving a target distribution $p$ with a noise kernel, makes $p_\sigma(x) = \int \mathcal{N}(x; x^\prime, \sigma^2) (x^\prime) d x^{\prime}$ it much better behaved
  $$\sigma_1 > \sigma_2 > \sigma_3 > \sigma_4$$

Score-Based Generative Modeling Song et al. (2021)

• The SDE defines a marginal distribution $p_t(x)$ as the convolution of the target distribution $p(x)$ with a noise kernel $p_{t|s}(\cdot | x_s)$: $$p_t(x) = \int p(x_s) p_{t|s}(x | x_s) d x_s$$
• For a given forward SDE that evolves $p(x)$ to $p_T(x)$, there exists a reverse SDE that evolves $p_T(x)$ back into $p(x)$. It involves having access to the marginal score $\nabla_x \log_t p(x)$.

High-Dimensional Bayesian Inference for Inverse Problems With Neural Score Estimation

Work in collaboration with:
Benjamin Remy, Zaccharie Ramzi

$\Longrightarrow$ Learn complex priors by Neural Score Estimation and sample from posterior with gradient-based MCMC.

Writing down the convergence map log posterior

• The likelihood term is known analytically, given to us by the physics of gravitational lensing.
• There is no close form expression for the prior on dark matter maps $\kappa$.
  However:
  • We do have access to samples of full implicit prior through simulations: $X = \{x_0, x_1, \ldots, x_n \}$ with $x_i \sim \mathbb{P}$

Uncertainty quantification in Magnetic Resonance Imaging (MRI)

Ramzi, Remy, Lanusse et al. 2020

$\Longrightarrow$ We can see which parts of the image are well constrained by data, and which regions are uncertain.

Inference over hybrid physical/data-driven models

Complications specific to astronomical images: spot the differences!

CelebA

HSC PDR-2 wide

Flow-VAE samples

Variational Inference over Hybrid Hierarchical Bayesian Models

Work led by Benjamin Remy

$\Longrightarrow$ Eliminate model bias in shear inference by using data-driven morphology priors.

Impact of galaxy morphology on shape measurement

Mandelbaum, et al. (2013), Mandelbaum, et al. (2014)

Conclusion

Conclusion

Thank you !