Towards a new paradigm for patent experimentation: WPI+

Abstract

We enhance the WPI patent research collection, which is publicly accessible and free of charge, to facilitate more comparable, transparent, and reproducible experiments. This is accomplished through what we call “soft standardization” advocating the adoption of consistent methods in using the test collection. We offer data statistics, predefined collection subsets, ground-truth data for additional tasks, and open-source tools for using the collection, all on a public GitHub repository. These resources not only relieve researchers from performing essential collection analysis tasks but also implicitly guide them toward sound methods for conducting experiments with the collection. Our initiative is primarily motivated by the goal of enhancing comparability and reproducibility of patent research. This is achieved through the development of a carefully designed resource that will be continuously expanded and maintained. Our work is also driven by the observation that highly integrated Information Retrieval experiment platforms for large scale evaluation are not widely adopted by researchers. We provide examples of how the WPI+ resource/collection can be used for research on multiple patent specific tasks, including prior-art search, patent classification, and summarization. Overall, our work shows that the traditional concept of a test collection—limited to just a corpus, topics, and relevance assessments—can be broadened to support more efficient and reliable scientific experimentation.