publications
2025
- ICLRImproving the evaluation of samplers on multi-modal targetsLouis Grenioux*, Maxence Noble*, and Marylou GabriéIn Frontiers in Probabilistic Inference: Learning meets Sampling, 2025
Addressing multi-modality constitutes one of the major challenges of sampling. In this reflection paper, we advocate for a more systematic evaluation of samplers towards two sources of difficulty that are mode separation and dimension. For this, we propose a synthetic experimental setting that we illustrate on a selection of samplers, focusing on the challenging criterion of recovery of the mode relative importance. These evaluations are crucial to diagnose the potential of samplers to handle multi-modality and therefore to drive progress in the field.
@inproceedings{grenioux2025improving, title = {Improving the evaluation of samplers on multi-modal targets}, author = {Grenioux, Louis and Noble, Maxence and Gabri{\'e}, Marylou}, year = {2025}, booktitle = {Frontiers in Probabilistic Inference: Learning meets Sampling}, url = {https://openreview.net/forum?id=d91E9RhVFU}, } - ICLRLearned Reference-based Diffusion Sampler for multi-modal distributionsIn The Thirteenth International Conference on Learning Representations, 2025
Over the past few years, several approaches utilizing score-based diffusion have been proposed to sample from probability distributions, that is without having access to exact samples and relying solely on evaluations of unnormalized densities. The resulting samplers approximate the time-reversal of a noising diffusion process, bridging the target distribution to an easy-to-sample base distribution. In practice, the performance of these methods heavily depends on key hyperparameters that require ground truth samples to be accurately tuned. Our work aims to highlight and address this fundamental issue, focusing in particular on multi-modal distributions, which pose significant challenges for existing sampling methods. Building on existing approaches, we introduce Learned Reference-based Diffusion Sampler (LRDS), a methodology specifically designed to leverage prior knowledge on the location of the target modes in order to bypass the obstacle of hyperparameter tuning. LRDS proceeds in two steps by (i) learning a reference diffusion model on samples located in high-density space regions and tailored for multimodality, and (ii) using this reference model to foster the training of a diffusion-based sampler. We experimentally demonstrate that LRDS best exploits prior knowledge on the target distribution compared to competing algorithms on a variety of challenging distributions.
@inproceedings{noble2025learned, title = {Learned Reference-based Diffusion Sampler for multi-modal distributions}, author = {Noble, Maxence and Grenioux, Louis and Gabri{\'e}, Marylou and Durmus, Alain Oliviero}, year = {2025}, booktitle = {The Thirteenth International Conference on Learning Representations}, url = {https://openreview.net/forum?id=fmJUYgmMbL}, }
2024
- ICMLStochastic Localization via Iterative Posterior SamplingIn Proceedings of the 41st International Conference on Machine Learning, 2024
This paper has been selected as a spotlight-designated paper at the conference. Top 3.5% acceptance rate.
Building upon score-based learning, new interest in stochastic localization techniques has recently emerged. In these models, one seeks to noise a sample from the data distribution through a stochastic process, called observation process, and progressively learns a denoiser associated to this dynamics. Apart from specific applications, the use of stochastic localization for the problem of sampling from an unnormalized target density has not been explored extensively. This work contributes to fill this gap. We consider a general stochastic localization framework and introduce an explicit class of observation processes, associated with flexible denoising schedules. We provide a complete methodology, Stochastic Localization via Iterative Posterior Sampling (SLIPS), to obtain approximate samples of these dynamics, and as a by-product, samples from the target distribution. Our scheme is based on a Markov chain Monte Carlo estimation of the denoiser and comes with detailed practical guidelines. We illustrate the benefits and applicability of SLIPS on several benchmarks of multi-modal distributions, including Gaussian mixtures in increasing dimensions, Bayesian logistic regression and a high-dimensional field system from statistical-mechanics.
@inproceedings{grenioux2024stochastic, title = {Stochastic Localization via Iterative Posterior Sampling}, author = {Grenioux, Louis and Noble, Maxence and Gabri\'{e}, Marylou and Oliviero Durmus, Alain}, year = {2024}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, publisher = {PMLR}, series = {Proceedings of Machine Learning Research}, volume = {235}, pages = {16337--16376}, url = {https://proceedings.mlr.press/v235/grenioux24a.html}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, }
2023
- ICMLBalanced Training of Energy-Based Models with Adaptive Flow SamplingLouis Grenioux, Éric Moulines, and Marylou GabriéWorkshop on Structured Probabilistic Inference & Generative Modeling, 2023
Energy-based models (EBMs) are versatile density estimation models that directly parameterize an unnormalized log density. Although very flexible, EBMs lack a specified normalization constant of the model, making the likelihood of the model computationally intractable. Several approximate samplers and variational inference techniques have been proposed to estimate the likelihood gradients for training. These techniques have shown promising results in generating samples, but little attention has been paid to the statistical accuracy of the estimated density, such as determining the relative importance of different classes in a dataset. In this work, we propose a new maximum likelihood training algorithm for EBMs that uses a different type of generative model, normalizing flows (NF), which have recently been proposed to facilitate sampling. Our method fits an NF to an EBM during training so that an NF-assisted sampling scheme provides an accurate gradient for the EBMs at all times, ultimately leading to a fast sampler for generating new data.
@article{grenioux2023balanced, title = {Balanced Training of Energy-Based Models with Adaptive Flow Sampling}, author = {Grenioux, Louis and Moulines, Éric and Gabrié, Marylou}, year = {2023}, journal = {Workshop on Structured Probabilistic Inference & Generative Modeling}, } - ICMLOn Sampling with Approximate Transport MapsLouis Grenioux, Alain Oliviero Durmus, Eric Moulines, and 1 more authorIn Proceedings of the 40th International Conference on Machine Learning, 2023
Transport maps can ease the sampling of distributions with non-trivial geometries by transforming them into distributions that are easier to handle. The potential of this approach has risen with the development of Normalizing Flows (NF) which are maps parameterized with deep neural networks trained to push a reference distribution towards a target. NF-enhanced samplers recently proposed blend (Markov chain) Monte Carlo methods with either (i) proposal draws from the flow or (ii) a flow-based reparametrization. In both cases, the quality of the learned transport conditions performance. The present work clarifies for the first time the relative strengths and weaknesses of these two approaches. Our study concludes that multimodal targets can be reliably handled with flow-based proposals up to moderately high dimensions. In contrast, methods relying on reparametrization struggle with multimodality but are more robust otherwise in high-dimensional settings and under poor training. To further illustrate the influence of target-proposal adequacy, we also derive a new quantitative bound for the mixing time of the Independent Metropolis-Hastings sampler.
@inproceedings{grenioux2023sampling, title = {On Sampling with Approximate Transport Maps}, author = {Grenioux, Louis and Oliviero Durmus, Alain and Moulines, Eric and Gabri\'{e}, Marylou}, year = {2023}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, publisher = {PMLR}, series = {Proceedings of Machine Learning Research}, volume = {202}, pages = {11698--11733}, url = {https://proceedings.mlr.press/v202/grenioux23a.html}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, }