Quality Dimensions and Evaluation Framework for Machine-Actionable DMPs

Abstract

Data Management Plans (DMPs) describe how research data is managed, stored, and preserved. While machine-actionable DMPs (maDMPs) enable structured metadata for applications, their review remains a manual, labor-intensive process. This article introduces a conceptual framework for the automated evaluation of DMPs, focusing on FAIR assessment and funder compliance. Key contributions include the collection of requirements for automated DMP evaluation based on prior work on maDMPs and community input, as well as the development of a taxonomy of evaluation goals and dimensions with corresponding metrics. The article proposes the DMP Quality Vocabulary (DMPQV) for standardized communication of DMP quality measurements and introduces a mechanism for representing contextual information for maDMPs. A prototype implementation, validated through a case study using the Science Europe Practical Guide, extends the Research Data Alliance maDMP standard and supports funder-independent metrics such as completeness, feasibility, quality of actions, and compliance. Results demonstrate the framework's ability to generate standardized quality measurements and reports, aligning with manual assessments, while emphasizing the importance of clear guidelines for accurate evaluation.