Simple Domain Adaptation for Sparse Retrievers
Adaptation de Domaine Simple pour la Recherche Parcimonieuse
Abstract
n Information Retrieval, and more generally in Natural Language Processing, adapting models to specific domains is conducted through fine-tuning. Despite the successes achieved by this method and its versa-
tility, the need for human-curated and labeled data makes it impractical to transfer to new tasks, domains, and/or languages when training data doesn’t exist. Using the model without training (zero-shot) is another
option that however suffers an effectiveness cost, especially in the case of first-stage retrievers. Numerous research directions have emerged to tackle these issues, most of them in the context of adapting to a task or a language. However, the literature is scarcer for domain (or topic) adaptation. In this paper, we address this issue of cross-topic discrepancy for a sparse first-stage retriever by transposing a method initially designed for language adaptation. By leveraging pre-training on the target data to learn domain-specific knowledge, this technique alleviates the need for annotated data and expands the scope of domain adaptation. Despite their relatively good generalization ability, we show that even sparse retrievers can benefit from our simple domain adaptation method.
Dans le cadre de la Recherche d'Information (RI), l'apprentissage des modèles repose fortement sur l'approche "pré-entraîner puis affiner". Malgré ses très bons résultats, cette méthode nécessite d'avoir accès à un jeu de données labellisées ce qui complique son application à de nouveaux domaines ou langues, en particulier si ceux-ci sont faiblement fournis.
Cet article propose une solution simple au transfert d'un modèle de recherche parcimonieux, SPLADE, vers des domaines sans données labellisées.
Origin | Files produced by the author(s) |
---|