A Framework and Practical Guidelines for Sharing Open Benchmark Datasets in Cartographic User Research Utilizing Neuroscientific Methods

Abstract

This paper presents a structured approach to creating benchmark datasets and proposes guidelines for open-data sharing in the field of cartographic user research using neuroscientific methods including but not limited to eye tracking, EEG, and fMRI. The unique complexities introduced by geospatial data and maps make geospatial tasks fundamentally different from those encountered in the experimental psychology or cognitive visualization domains. We argue that datasets capable of addressing specific cartographic problems possess significant value and hold the potential to become benchmarks. For instance, studying the cognitive load and strategies employed by map users during various map tasks can provide valuable insights for map design and serve as benchmarks in developing complexity algorithms for cartography. We emphasize that benchmarks should be tailored to specific scientific issues rather than solely focusing on data standards. Such benchmarks not only contribute to map usability research but also play a pivotal role in developing predictive models that consider the visual attention and map use capabilities of users. Researchers across domains bear the responsibility of actively seeking concrete methods to encourage the open sharing of experimental data, complemented by high-quality metadata. By fostering the creation of benchmark datasets and promoting open-data sharing, collaboration is enhanced, cartographic research advances, and the scientific community is empowered to effectively address cartographic challenges.