Agentic Bayesian Inference Debugging

Abstract

Probabilistic programming provides a flexible framework for modeling uncertain real-world systems, but Bayesian inference for such models is often computationally expensive and difficult to debug. In particular, convergence of approximate inference algorithms such as Hamiltonian Monte Carlo cannot be directly verified without access to the true posterior, making non-convergence hard to detect in practice. Further, resolving any detected issues often requires deep knowledge of the interplay between a probabilistic model and a chosen Bayesian inference algorithm. To make Bayesian inference debugging faster, more practical, and more accessible, this thesis presents an online approach to debugging Bayesian inference, implemented in a debugger for probabilistic programs, together with a benchmark for evaluating debugging methods and a fully automated agentic extension of the proposed approach. When evaluated in a user study with 18 participants, our user-facing debugger significantly reduced debugging time and increased the number of issues resolved. For the fully automated agentic debugger, we found that the online approach improves issue resolution on our new benchmark for Bayesian inference debugging by 8 percentage points compared to a baseline agentic system. Ultimately, these tools pave the way for more robust and accessible probabilistic programming.