Title: Tuning retrieval system effectiveness via the retrievability bias
Language: English
Authors: Kascha, Michael Alexander 
Qualification level: Diploma
Keywords: Information Retrieval; Retrievability; Effektivität; Leistung; Relevanz; Gini-Koeffizient
information retrieval; retrievability; effectiveness; performance; relevance judgments; gini coefficient
Advisor: Rauber, Andreas
Issue Date: 2012
Number of Pages: 117
Qualification level: Diploma
The performance of an information retrieval system depends on the document collection it has to work on and can be influenced to a certain degree by tuning the systems parameter values. There are several different measures to evaluate the effectiveness of retrieval systems.
All of them have in common that they require human relevance judgments to determine if search results are satisfying the information need of the queries. However, their creation is both very expensive and time consuming. There have been three fundamentally different approaches to create performance rankings without the need of these relevance judgments. This thesis first introduces relevant background information and then takes a closer look at the approach that shows the most potential in determining the top performers. It employs an unsupervised measure called retrievability which expresses how accessible documents are for the system. Experiments are conducted to compare various effectiveness measures to the retrievability bias at different system parameter values. This is done with three different retrieval methods on four different collections. Results show that while there is usually always a general correlation of high effectiveness and low retrievability bias observable, it largely depends on the length and/or quality of the user queries issued. In some cases a minimal retrievability bias does not indicate a good parameter setting for maximizing the effectiveness values. There might be several reasons for this behavior which still require further research.
URI: https://resolver.obvsg.at/urn:nbn:at:at-ubtuw:1-54359
Library ID: AC07813272
Organisation: E188 - Institut für Softwaretechnik und Interaktive Systeme 
Publication Type: Thesis
Appears in Collections:Thesis

Files in this item:

Show full item record

Page view(s)

checked on Feb 18, 2021


checked on Feb 18, 2021

Google ScholarTM


Items in reposiTUm are protected by copyright, with all rights reserved, unless otherwise indicated.