moduli: A Disaggregated Data Management Architecture for Data-Intensive Workflows

Abstract

As companies store, process, and analyse bigger and bigger volumes of highly heterogeneous data, novel research and technological challenges are emerging. Traditional and rigid data integration and processing techniques become inadequate for a new class of data-intensive applications. There is a need for new architectural, software, and hardware solutions that are capable of providing dynamic data integration, assuring high data quality, and offering safety and security mechanisms, while facilitating online data analysis. In this context, we propose moduli, a novel disaggregated data management reference architecture for data-intensive applications that organizes data processing in various zones. Working on moduli allowed us also to identify open research and technological challenges.