Reproducible Query Processing and Data Citation of in Situ Soil Moisture Data

Abstract

Data in today's dynamic world undergoes constant change and evolution, spanning various formats such as text, websites, tweets, and sensor readings. Storing and referencing these diverse data types pose significant challenges due to data movement, changes in content or structure, and limited availability. Efficient data identification is crucial for speeding up scientific discovery and result validation, especially when data accessibility is guaranteed. Recent years have witnessed progress in data citation practices, with conferences mandating the inclusion of utilized and generated data. However, existing solutions primarily cater to static datasets, rendering them ineffective for dynamically evolving ones. This paper addresses this gap by providing a tailored dynamic data citation prototype for the International Soil Moisture Network, one of the largest scientific in situ soil moisture databases. Our work encompasses the implementation and evaluation of different data versioning strategies and a query store architecture that enables the citation, reproducibility, and verification of large sets of SQL queries to recreate data requests by users. By applying the RDA Dynamic Data Citation Guidelines, we assess the necessary needs for such a system and further measure the performance and storage impact of our proposed approaches.