Exploring the Benefits of Iterative Retrieval-Augmented Generation for Risk Mitigation in LLM Responses

Abstract

The correctness of provided information is essential for LLMs to be used in real life scenarios. However, they suffer from knowledge cut-offs as well as hallucinations. Retrieval-augmented generation (RAG) aims at solving these problems by providing on-demand, real-time information, usable as context for addressing queries. An issue with RAG based systems is the potential of low quality retrievals, that provide wrong or irrelevant context to the LLM. To mitigate this risk, we implemented Iter-RAG, an evolution of standard RAG which utilizes multi-iteration document retrieval based on missing information. Using Iter-RAG, the correctness of answers for the TriviaQA dataset is increased compared to standard RAG. Additionally, we implemented a chatbot that can correctly present information about an energy provider, admit when information is missing in its database, and refuse to answer adversarial queries, even when the requested information is present.