Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation - Sorbonne Université
Conference Papers Year : 2021

Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation

Abstract

QUESTEVAL is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QUESTEVAL's code and models available for reproducibility purpose, as part of the QUESTEVAL project. 1
Fichier principal
Vignette du fichier
2104.07555.pdf (380.03 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03479905 , version 1 (14-12-2021)

Identifiers

Cite

Clément Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier, et al.. Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation. 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Nov 2021, Punta Cana, Dominican Republic. pp.8029-8036, ⟨10.18653/v1/2021.emnlp-main.633⟩. ⟨hal-03479905⟩
136 View
75 Download

Altmetric

Share

More