Errors and uncertainty in machine learning models

Abstract

Machine learning models have become very successful for chemical applications, such as the prediction of molecular or reaction properties, or as surrogate models of the interatomic potential. But how certain is a specific prediction? Can an uncertainty estimation via mean-variance prediction or ensembling really relate to the actual model error? In this work, we present general rules how to identify aleatoric and epistemic contributions to the uncertainty of a model, further divide epistemic error into model bias and variance, and discuss means to critically assess the applicability of common uncertainty metrics to a range of different prediction tasks on various datasets.