Contextualized Topic Coherence Metrics - Sorbonne Université Access content directly
Conference Papers Year : 2024

Contextualized Topic Coherence Metrics

Abstract

This article proposes a new family of LLM-based topic coherence metrics called Contextualized Topic Coherence (CTC) and inspired by standard human topic evaluation methods. CTC metrics simulate human-centered coherence evaluation while maintaining the efficiency of other automated methods. We compare the performance of our CTC metrics and five other baseline metrics on seven topic models and show that CTC metrics better reflect human judgment, particularly for topics extracted from short text collections by avoiding highly scored topics that are meaningless to humans.
No file

Dates and versions

hal-04517597 , version 1 (22-03-2024)

Identifiers

  • HAL Id : hal-04517597 , version 1

Cite

Hamed Rahimi, Jacob Louis Hoover, David Mimno, Hubert Naacke, Camelia Constantin, et al.. Contextualized Topic Coherence Metrics. EACL 2024 - The 18th Conference of the European Chapter of the Association for Computational Linguistics, Mar 2024, St. Julian's, Malta. pp.1760--1773. ⟨hal-04517597⟩
0 View
0 Download

Share

Gmail Facebook X LinkedIn More