Fries, J., Seelam, N., Altay, G., Weber, L., Kang, M., Datta, D., Su, R., Garda, S., Wang, B., Ott, S., Samwald, M., & Kusa, W. (2022). Dataset Debt in Biomedical Language Modeling. In Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models (pp. 137–145). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.bigscience-1.10
Proceedings of BigScience Episode #5 -- Workshop on Challenges & Perspectives in Creating Large Language Models
-
ISBN:
978-1-955917-26-1
-
Date (published):
May-2022
-
Event name:
60th Annual Meeting of the Association for Computational Linguistics
en
Event date:
22-May-2022 - 27-May-2022
-
Event place:
Dublin, Ireland
-
Number of Pages:
9
-
Publisher:
Association for Computational Linguistics
-
Keywords:
biomedical NLP; language modelling; dataset debt; biomedical datasets; biomedical language modelling
en
Abstract:
Large-scale language modeling and natural language prompting have demonstrated exciting capabilities for few and zero shot learning in NLP. However, translating these successes to specialized domains such as biomedicine remains challenging, due in part to biomedical NLP’s significant dataset debt – the technical costs associated with data that are not consistently documented or easily incorporated into popular machine learning frameworks at scale. To assess this debt, we crowdsourced curation of datasheets for 167 biomedical datasets. We find that only 13% of datasets are available via programmatic access and 30% lack any documentation on licensing and permitted reuse. Our dataset catalog is available at: https://tinyurl.com/bigbio22.
en
Project title:
Domänen-spezifische Systeme für Informationsextraktion und -suche: 860721 (European Commission)
CC BY 4.0
