Record link: 
http://hdl.handle.net/20.500.12708/210200
-
Title: 
Overview of the CLEF 2024 LongEval Lab on Longitudinal Evaluation of Model Performance
en
Citation: 
Alkhalifa, R., Borkakoty, H., Deveaud, R., El-Ebshihy, A., Espinosa-Anke, L., Fink, T., Galuščáková, P., Gonzalez-Saez, G., Goeuriot, L., Iommi, D., Liakata, M., Madabushi, H. T., Medina-Alias, P., Mulhem, P., Piroi, F., Popel, M., & Zubiaga, A. (2024). Overview of the CLEF 2024 LongEval Lab on Longitudinal Evaluation of Model Performance. In Experimental IR Meets Multilinguality, Multimodality, and Interaction : 15th International Conference of the CLEF Association, CLEF 2024, Grenoble, France, September 9–12, 2024, Proceedings, Part II (pp. 208–230). Springer. https://doi.org/10.1007/978-3-031-71908-0_10
-
Publisher DOI: 
10.1007/978-3-031-71908-0_10
-
Publication Type: 
Inproceedings - Full-Paper Contribution
en
Language: 
English
-
Authors: 
Alkhalifa, Rabab 
Borkakoty, Hsuvas 
Deveaud, Romain 
El-Ebshihy, Alaa 
Espinosa-Anke, Luis 
Fink, Tobias 
Galuščáková, Petra 
Gonzalez-Saez, Gabriela 
Goeuriot, Lorraine 
Iommi, David 
Liakata, Maria 
Madabushi, Harish Tayyar 
Medina-Alias, Pablo 
Mulhem, Philippe 
Piroi, Florina 
Popel, Martin 
Zubiaga, Arkaitz 
-
Organisational Unit: 
E194-04 - Forschungsbereich Data Science 
E058-06 - Fachbereich Zentrum für Forschungsdatenmanagement 
-
Published in: 
Experimental IR Meets Multilinguality, Multimodality, and Interaction : 15th International Conference of the CLEF Association, CLEF 2024, Grenoble, France, September 9–12, 2024, Proceedings, Part II 
-
ISBN: 
978-3-031-71908-0
-
Volume: 
14959
-
DOI of the book: 
10.1007/978-3-031-71908-0
-
Date (published): 
19-Sep-2024
-
Event name: 
Conference and Labs of the Evaluation Forum (CLEF 2024) : Information Access Evaluation meets Multilinguality, Multimodality, and Visualization 
en
Event date: 
9-Sep-2024 - 12-Sep-2024
-
Event place: 
Grenoble, France
-
Number of Pages: 
23
-
Publisher: 
Springer
-
Peer reviewed: 
Yes
-
Keywords: 
Evaluation; Information Retrieval; Temporal Generalisability; Temporal Persistence; Text Classification
en
Abstract: 
We describe the second edition of the LongEval CLEF 2024 shared task. This lab evaluates the temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 requires IR systems to run on corpora acquired at several timestamps, and evaluates the drop in system quality (NDCG) along these timestamps. Task 2 tackles binary sentiment classification at different points in time, and evaluates the performance drop for different temporal gaps. Overall, 37 teams registered for Task 1 and 25 for Task 2. Ultimately, 14 and 4 teams participated in Task 1 and Task 2, respectively.
en
Research Areas: 
Information Systems Engineering: 10%
Computer Science Foundations: 90%
-
Science Branch: 
1020 - Informatik: 90%
5020 - Wirtschaftswissenschaften: 10%
-
Appears in Collections:
Conference Paper

Show full item record

Page view(s)

12
checked on Jan 30, 2025

Google ScholarTM

Check