TimIR: Time-Traveling Through IR History

Abstract

In live systems, where the underlying document corpora evolve frequently, a query executed at two different points in time can yield two different result sets. Although not important in traditional web search settings, domains, such as patent retrieval or systematic literature reviews, rely on the time of execution to obtain the relevant result set. To ensure reproducibility and auditability of their search results researchers in these fields usually rely on Boolean Retrieval. This is because sparse and dense retrieval methods do not satisfy this requirement, as sparse retrieval relies on global term and document statistics, and dense retrieval relies on document embeddings. These values and vectors are subject to change if the document corpora are updated and therefore change the ranking as well. In this paper we present TimIR (https://timir.ds-ifs.tuwien.ac.at), a dashboard to explore the evolution of Information Retrieval publications over time, while showcasing a hybrid retrieval system that allows researchers to recreate sparse rankings for historical states of the document corpora. Having search result lists that can be recreated, with a system like TimIR for example, makes it possible to cite such sets of data, especially when the results are used for further down-the-stream research, without the impracticality of additionally storing the search results themselves. Furthermore, it possible to compare rankings for a particular query over time, or to explore the literature available during a specific time period, for example the time when Karen Spärck Jones introduced TF-IDF.