The prediction and control of river sediment yield (SY) are critical but challenging tasks. Erosion and sediment transfer in river catchments are controlled by different processes, whose relative importance varies in space and time. We thus put forward that SY can be estimated more efficiently by using explicitly the information contained in the similarity within groups. To test this hypothesis, we developed a novel Bayesian hierarchical model, applied it to a sample of heterogeneous river catchments and compared its fixed-effects and mixed-effects performance incorporating different group levels, namely gauges, rivers, basins and clusters of catchments. With a parsimonious linear model consisting of four variables (specific and extreme discharge, elevation and retention coefficient), we achieved good performance criteria in the calibration (NSE: 0.79–0.85) and in the crossvalidation for temporal and spatial prediction (NSE: 0.71 and 0.72, respectively). These results support the promising potential of this technique.