<div class="csl-bib-body">
<div class="csl-entry">Hofstätter, S. (2022). <i>Optimizing the tradeoff between cost and effectiveness in neural ranking</i> [Dissertation, Technische Universität Wien]. reposiTUm. https://doi.org/10.34726/hss.2023.109581</div>
</div>
-
dc.identifier.uri
https://doi.org/10.34726/hss.2023.109581
-
dc.identifier.uri
http://hdl.handle.net/20.500.12708/148084
-
dc.description.abstract
The tradeoff between cost and effectiveness has always been an integral part of Information Retrieval. Since the early beginnings of IR, researchers have been concerned with creating the highest quality search system while abiding by the available hardware, scalability, or simply cost constraints. In recent years, the query to document text-retrieval task underwent an all-encompassing paradigm shift towards learned neural network based approaches. These -- usually large -- networks provide a great effectiveness boost over traditional indexing methods. However, these quality improvements come at large hardware and time costs and challenges to their scalability. This thesis contributes fundamental insights and approaches to retrieval and re- ranking using large scale language models. We developed a family of retrieval specific ranking models called the Transformer-Kernel architecture. The core of this architecture uses lightweight Transformer layers to contextualize text and Kernel- pooling for transparent matching. We propose several variants, specific for different sub-tasks of text retrieval, that expand the Pareto frontier between cost and effectiveness in each area. Our models also offer novel interpretability components to highlight contextualized term scores. We study the interaction between common ranking models and common retrieval datasets and highlight problems and provide improved dataset augmentations based on them. Furthermore, we propose widely applicable knowledge distillation procedures between different ranking architectures. The target architectures include both re-ranking and dense retrieval methods, that can replace traditional inverted indices. Our training processes, which use a trained teacher model to train a smaller or more efficient student model, again improve on the Pareto frontier -- as we increase the effectiveness of the efficient architectures, without changing their cost aspects at all. We conclude this thesis with a summary of our education initiatives that complement our research activities, by offering remote and open-source lectures, as well as proposing future directions based on the work in this thesis. We bring novel neural advances to the largest possible audience, both by lowering costs with acceptable effectiveness levels and providing the materials to lower the learning curve for this paradigm shift.
en
dc.language
English
-
dc.language.iso
en
-
dc.rights.uri
http://rightsstatements.org/vocab/InC/1.0/
-
dc.subject
neural retrieval
en
dc.subject
information retrieval
en
dc.subject
efficient search
en
dc.title
Optimizing the tradeoff between cost and effectiveness in neural ranking
en
dc.type
Thesis
en
dc.type
Hochschulschrift
de
dc.rights.license
In Copyright
en
dc.rights.license
Urheberrechtsschutz
de
dc.identifier.doi
10.34726/hss.2023.109581
-
dc.contributor.affiliation
TU Wien, Österreich
-
dc.rights.holder
Sebastian Hofstätter
-
dc.publisher.place
Wien
-
tuw.version
vor
-
tuw.thesisinformation
Technische Universität Wien
-
tuw.publication.orgunit
E194 - Institut für Information Systems Engineering
-
dc.type.qualificationlevel
Doctoral
-
dc.identifier.libraryid
AC16760934
-
dc.description.numberOfPages
156
-
dc.thesistype
Dissertation
de
dc.thesistype
Dissertation
en
dc.rights.identifier
In Copyright
en
dc.rights.identifier
Urheberrechtsschutz
de
tuw.advisor.staffStatus
staff
-
tuw.advisor.orcid
0000-0002-7149-5843
-
item.openairetype
doctoral thesis
-
item.openairecristype
http://purl.org/coar/resource_type/c_db06
-
item.openaccessfulltext
Open Access
-
item.languageiso639-1
en
-
item.fulltext
with Fulltext
-
item.cerifentitytype
Publications
-
item.grantfulltext
open
-
item.mimetype
application/pdf
-
crisitem.author.dept
E194-04 - Forschungsbereich E-Commerce
-
crisitem.author.parentorg
E194 - Institut für Information Systems Engineering