Record link: 
http://hdl.handle.net/20.500.12708/210281
-
Title: 
Extended overview of the CLEF 2024 LongEval Lab on Longitudinal Evaluation of Model Performance
en
Citation: 
Alkhalifa, R., Borkakoty, H., Deveaud, R., El-Ebshihy, A., Espinosa-Anke, L., Fink, T., Galuščáková, P., Gonzalez-Saez, G., Goeuriot, L., Iommi, D., Liakata, M., Tayyar Madabushi, H., Medina-Alias, P., Mulhem, P., Piroi, F., Popel, M., & Zubiaga, A. (2024). Extended overview of the CLEF 2024 LongEval Lab on Longitudinal Evaluation of Model Performance. In Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2024) (pp. 2267–2289). http://hdl.handle.net/20.500.12708/210281
-
Publication Type: 
Inproceedings - Full-Paper Contribution
en
Language: 
English
-
Authors: 
Alkhalifa, Rabab 
Borkakoty, Hsuvas 
Deveaud, Romain 
El-Ebshihy, Alaa 
Espinosa-Anke, Luis 
Fink, Tobias 
Galuščáková, Petra 
Gonzalez-Saez, Gabriela 
Goeuriot, Lorraine 
Iommi, David 
Liakata, Maria 
Tayyar Madabushi, Harish 
Medina-Alias, Pablo 
Mulhem, Philippe 
Piroi, Florina 
Popel, Martin 
Zubiaga, Arkaitz 
-
Organisational Unit: 
E194-04 - Forschungsbereich Data Science 
E058-06 - Fachbereich Zentrum für Forschungsdatenmanagement 
-
Published in: 
Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2024) 
-
Volume: 
3740
-
Date (published): 
5-Aug-2024
-
Event name: 
Conference and Labs of the Evaluation Forum (CLEF 2024) : Information Access Evaluation meets Multilinguality, Multimodality, and Visualization 
-
Event date: 
9-Sep-2024 - 12-Sep-2024
-
Event place: 
Grenoble, France
-
Number of Pages: 
23
-
Peer reviewed: 
Yes
-
Keywords: 
Evaluation; Temporal Persistence; Temporal Generalisability; Information Retrieval; Text Classification
en
Abstract: 
We describe the second edition of the LongEval CLEF 2024 shared task. This lab evaluates the temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 requires IR systems to run on corpora acquired at several timestamps, and evaluates the drop in system quality (NDCG) along these timestamps. Task 2 tackles binary sentiment classification at different points in time, and evaluates the performance drop for different temporal gaps. Overall, 37 teams registered for Task 1 and 25 for Task 2. Ultimately, 14 and 4 teams participated in Task 1 and Task 2, respectively.
en
Research Areas: 
Information Systems Engineering: 20%
Computer Science Foundations: 80%
-
Science Branch: 
1020 - Informatik: 90%
5020 - Wirtschaftswissenschaften: 10%
-
Appears in Collections:
Conference Paper

Show full item record

Page view(s)

247
checked on Jan 30, 2025

Google ScholarTM

Check