Posterior predictive model assessment using formal methods in a spatio-temporal model

Abstract

We propose an interdisciplinary framework, Bayesian formal predictive model assessment. It combines Bayesian predictive inference, a well established tool in statistics, with formal verification methods rooting in the computer science community. Bayesian predictive inference allows for coherently incorporating uncertainty about unknown quantities by making use of methods or models that produce predictive distributions, which in turn inform decision problems. By formalizing these problems and the corresponding properties, we can use spatio-temporal reach and escape logic to formulate and probabilistically assess their satisfaction. This way, competing models can directly be compared based on their ability to predict the property satisfaction a posteriori. The approach is illustrated on an urban mobility application, where the crowdedness in the center of Milan is proxied by aggregated mobile phone traffic data. We specify several desirable spatio-temporal properties related to city crowdedness such as a fault-tolerant network or the reachability of hospitals. After verifying these properties on draws from the posterior predictive distributions, we compare several spatio-temporal Bayesian models based on their overall and property-based predictive performance.