Talk by Jian-Yun Nie

Large language models are increasingly used to generate answers to questions across different domains. When these models are used on their own, hallucinations or incorrect answers are often observed. Retrieval-Augmented Generation (RAG) can offer a partial solution to this problem.

Current approaches treat the language model as a black box: the retrieved documents are provided to the model as part of the prompt, and it is up to the model to use them according to its internal parameters, in particular via attention to the words in the documents. It has difficulty distinguishing the relevance of documents and assessing their usefulness.

In this talk, we propose an open-box approach, OpenDecoder. We use signals about the quality of the retrieved documents to influence the language model’s attention. Our hope is that the model will pay more attention to relevant and useful information during generation, and ignore non-relevant information. Our experiments on several test collections confirm that these document-quality signals help improve answer quality, and that the model is more robust to noise in the retrieved documents.

Biography:
Jian-Yun Nie is a professor at the Université de Montréal. He obtained his PhD from the Université de Grenoble. His research focuses on information retrieval and natural language processing. Among other contributions, he has worked on information retrieval models, cross-lingual information retrieval approaches, and more recently, information retrieval methods that use large language models. He was elected to the SIGIR Academy in 2022. He also holds a Canada Research Chair in Natural Language Information Processing.