Skip to Main content Skip to Navigation
Conference papers

Architecture basée sur les mécanismes d'attention: le cas de la génération de questions neuronales

Abstract : Neural architectures based on self-attention, such as Transformers, recently attracted interest from the research community, and obtained significant improvements over the state of the art in several tasks. Adapting Transformers to Neural Question Generation is not straight- forward as data is relatively scarce in this task. We hence explore how Transformers can be adapted, and, in particular, study the effect of copy mechanisms, placeholders, and contex- tual word embeddings. Those mechanisms are particularly useful for the treatment of out-of- vocabulary words, which are more likely to affect performance in tasks with relatively smaller data available. The experiments reported show encouraging results in the answer-aware scenario (for which the target answer is known), while improvements over the state-of-the-art systems are obtained in the answer-agnostic setup.
Complete list of metadatas

https://hal.sorbonne-universite.fr/hal-02351017
Contributor : Benjamin Piwowarski <>
Submitted on : Wednesday, November 6, 2019 - 11:26:52 AM
Last modification on : Friday, November 8, 2019 - 1:37:46 AM

Identifiers

Citation

Thomas Scialom, Benjamin Piwowarski, Jacopo Staiano. Architecture basée sur les mécanismes d'attention: le cas de la génération de questions neuronales. COnférence en Recherche d'Informations et Applications - CORIA 2019, 16th French Information Retrieval Conference, May 2019, Lyon, France. ⟨10.24348/coria.2019.CORIA_2019_paper_11⟩. ⟨hal-02351017⟩

Share

Metrics

Record views

37