<div class="csl-bib-body">
<div class="csl-entry">Sertkan, M., Althammer, S., Hofstätter, S., Knees, P., & Neidhardt, J. (2023). Exploring Effect-Size-Based Meta-Analysis for Multi-Dataset Evaluation. In <i>Proceedings of the 3rd Workshop Perspectives on the Evaluation of Recommender Systems 2023 co-located with the 17th ACM Conference on Recommender Systems (RecSys 2023)</i>. PERSPECTIVES 2023 - Perspectives on the Evaluation of Recommender Systems Workshop co-located with the 17th ACM Conference on Recommender Systems, Singapore, Singapore. CEUR-WS.org. https://doi.org/10.34726/5352</div>
</div>
-
dc.identifier.uri
http://hdl.handle.net/20.500.12708/191697
-
dc.identifier.uri
https://doi.org/10.34726/5352
-
dc.description.abstract
In this paper, we address the essential yet complex task of evaluating Recommender Systems (RecSys) across multiple datasets. This is critical for gauging their overall performance and applicability in various contexts. Owing to the unique characteristics of each dataset and the variability in algorithm performance, we propose the adoption of effect-size-based meta-analysis, a proven tool in comparative research. This approach enables us to compare a “treatment model” and a “control model” across multiple datasets, offering a comprehensive evaluation of their performance. Through two case studies, we highlight the flexibility and effectiveness of this method in multi-dataset evaluations, irrespective of the metric utilized. The power of forest plots in providing an intuitive and concise summarization of our analysis is also demonstrated, which significantly aids in the communication of research findings. Our work provides valuable insights into leveraging these methodologies to draw more reliable and validated conclusions on the generalizability and robustness of RecSys models.
en
dc.description.sponsorship
Christian Doppler Forschungsgesells
-
dc.language.iso
en
-
dc.relation.ispartofseries
CEUR Workshop Proceedings
-
dc.rights.uri
http://creativecommons.org/licenses/by/4.0/
-
dc.subject
recommender systems
en
dc.subject
evaluation
en
dc.subject
effect-size
en
dc.subject
meta-analysis
en
dc.subject
forest plots
en
dc.title
Exploring Effect-Size-Based Meta-Analysis for Multi-Dataset Evaluation
Proceedings of the 3rd Workshop Perspectives on the Evaluation of Recommender Systems 2023 co-located with the 17th ACM Conference on Recommender Systems (RecSys 2023)
-
tuw.container.volume
3476
-
tuw.peerreviewed
true
-
tuw.book.ispartofseries
CEUR Workshop Proceedings
-
tuw.relation.publisher
CEUR-WS.org
-
tuw.project.title
Christian Doppler Labor für Weiterentwicklung des State-of-the-Art von Recommender-Systemen in mehreren Domänen
-
tuw.researchTopic.id
I4
-
tuw.researchTopic.name
Information Systems Engineering
-
tuw.researchTopic.value
100
-
tuw.publication.orgunit
E194-04 - Forschungsbereich Data Science
-
dc.identifier.libraryid
AC17202101
-
dc.description.numberOfPages
10
-
tuw.author.orcid
0000-0003-0984-5221
-
tuw.author.orcid
0009-0006-1229-2612
-
tuw.author.orcid
0000-0003-3906-1292
-
tuw.author.orcid
0000-0001-7184-1841
-
dc.rights.identifier
CC BY 4.0
de
dc.rights.identifier
CC BY 4.0
en
tuw.event.name
PERSPECTIVES 2023 - Perspectives on the Evaluation of Recommender Systems Workshop co-located with the 17th ACM Conference on Recommender Systems
en
tuw.event.startdate
19-09-2023
-
tuw.event.enddate
19-09-2023
-
tuw.event.online
Hybrid
-
tuw.event.type
Event for scientific audience
-
tuw.event.place
Singapore
-
tuw.event.country
SG
-
tuw.event.presenter
Sertkan, Mete
-
tuw.presentation.online
Online
-
tuw.event.track
Single Track
-
wb.sciencebranch
Informatik
-
wb.sciencebranch.oefos
1020
-
wb.sciencebranch.value
100
-
item.openaccessfulltext
Open Access
-
item.cerifentitytype
Publications
-
item.languageiso639-1
en
-
item.openairecristype
http://purl.org/coar/resource_type/c_5794
-
item.grantfulltext
open
-
item.fulltext
with Fulltext
-
item.mimetype
application/pdf
-
item.openairetype
conference paper
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.orcid
0000-0003-0984-5221
-
crisitem.author.orcid
0000-0003-3906-1292
-
crisitem.author.orcid
0000-0001-7184-1841
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering