Simple Domain Adaptation for Sparse Retrievers - Sorbonne Université Access content directly
Book Sections Year : 2024

Simple Domain Adaptation for Sparse Retrievers

Adaptation de Domaine Simple pour la Recherche Parcimonieuse

Abstract

n Information Retrieval, and more generally in Natural Language Processing, adapting models to specific domains is conducted through fine-tuning. Despite the successes achieved by this method and its versa- tility, the need for human-curated and labeled data makes it impractical to transfer to new tasks, domains, and/or languages when training data doesn’t exist. Using the model without training (zero-shot) is another option that however suffers an effectiveness cost, especially in the case of first-stage retrievers. Numerous research directions have emerged to tackle these issues, most of them in the context of adapting to a task or a language. However, the literature is scarcer for domain (or topic) adaptation. In this paper, we address this issue of cross-topic discrepancy for a sparse first-stage retriever by transposing a method initially designed for language adaptation. By leveraging pre-training on the target data to learn domain-specific knowledge, this technique alleviates the need for annotated data and expands the scope of domain adaptation. Despite their relatively good generalization ability, we show that even sparse retrievers can benefit from our simple domain adaptation method.
Dans le cadre de la Recherche d'Information (RI), l'apprentissage des modèles repose fortement sur l'approche "pré-entraîner puis affiner". Malgré ses très bons résultats, cette méthode nécessite d'avoir accès à un jeu de données labellisées ce qui complique son application à de nouveaux domaines ou langues, en particulier si ceux-ci sont faiblement fournis. Cet article propose une solution simple au transfert d'un modèle de recherche parcimonieux, SPLADE, vers des domaines sans données labellisées.
Embargoed file
Embargoed file
0 5 5
Year Month Jours
Avant la publication
Monday, September 23, 2024
Embargoed file
Monday, September 23, 2024
Please log in to request access to the document

Dates and versions

hal-04517668 , version 1 (22-03-2024)

Identifiers

Cite

Mathias Vast, Yuxuan Zong, Benjamin Piwowarski, Laure Soulier. Simple Domain Adaptation for Sparse Retrievers. Advances in Information Retrieval, 14610, Springer Nature Switzerland, pp.403-412, 2024, Lecture Notes in Computer Science, ⟨10.1007/978-3-031-56063-7_32⟩. ⟨hal-04517668⟩
1 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More