Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders - Sorbonne Université
Conference Papers Year : 2024

Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders

Abstract

With the recent addition of Retrieval-Augmented Generation (RAG), the scope and importance of Information Retrieval (IR) has expanded. As a result, the importance of a deeper understanding of IR models also increases. However, interpretability in IR remains under-explored, especially when it comes to the models' inner mechanisms. In this paper, we explore the possibility of adapting Integrated Gradient-based methods in an IR context to identify the role of individual neurons within the model. In particular, we provide new insights into the role of what we call "relevance" neurons, as well as how they deal with unseen data. Finally, we carry out an in-depth pruning study to validate our findings.

Dates and versions

hal-04668348 , version 1 (06-08-2024)

Identifiers

Cite

Mathias Vast, Basile van Cooten, Laure Soulier, Benjamin Piwowarski. Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders. ICTIR '24: The 2024 ACM SIGIR International Conference on the Theory of Information Retrieval, Jul 2024, Washington DC, United States. pp.133-143, ⟨10.1145/3664190.3672528⟩. ⟨hal-04668348⟩
24 View
0 Download

Altmetric

Share

More