Applying a Standardized Benchmarking Framework to Evaluate AI Methods for Precipitation Downscaling over Australia

Abstract

Downscaling techniques are essential for refining coarse-resolution climate projections to scales relevant for local and regional impact assessments, with artificial intelligence (AI) emerging as a promising approach for this task. However, a standardized benchmarking framework for evaluating these AI-based downscaling methods has been lacking. This study presents the first evaluation of AI-based downscaling methods using established performance expectations within a standardized benchmarking framework. Three machine learning (ML) models, including a generative diffusion model, a vision transformer, and a recurrent neural network, are assessed against observational data and compared with 24 simulations by regional climate models (RCMs). The evaluation employs minimum standard metrics focused on four fundamental rainfall characteristics across Australia: total precipitation, spatial distribution, seasonal cycle, and temporal trends. Results show that all three ML models and ten RCMs meet the minimum performance benchmarks, with rankings varying depending on the rainfall characteristic and region assessed. ML models demonstrate comparable performance to RCMs while offering substantial computational advantages. This highlights the potential of ML models to supplement traditional downscaled simulations, thereby enhancing climate projection ensembles and improving uncertainty quantification. Such an approach aligns with recommendations advocating for diverse modeling methodologies in national assessments. By addressing a critical gap through a standardized evaluation framework, this work provides a comprehensive benchmark dataset comprising ML and RCM outputs for Australian precipitation, facilitating the evaluation of emerging AI downscaling approaches and contributing to the standardization of regional climate modeling practices