Diffusion models for virtual agent facial expression generation in Motivational interviewing - Sorbonne Université
Communication Dans Un Congrès Année : 2024

Diffusion models for virtual agent facial expression generation in Motivational interviewing

Résumé

Motivational interviewing (MI) is a client-centered counseling style that addresses (the client) user’s motivation for behavior change. In this paper, we present a behavior generation model for Socially Interactive Agents (SIA) and apply it to an SIA acting as a virtual therapist in (MI). MI defines different types of dialogue acts for therapist and client. It has been shown that therapist builds rapport with their client by adapting their verbal and nonverbal behaviors. Based on the analysis of a human-human MI dataset (AnnoMI), we found co-occurrences between facial expressions and dialogue acts for both therapist and client. Moreover, the therapist adapts their behavior to their client’s behavior to favor rapport. Our behavior generation model embeds these co-occurrences as well as such behavior adaptation. To this aim, we build an observation-to-action framework based on a conditional diffusion approach trained on the AnnoMI corpus. Our model learns to generate the virtual therapist’s facial expressions conditioned by MI dialogue acts and the client’s nonverbal behaviors. We aim to make SIAs more effective in therapy-like interactions, by using user’s behaviors in addition to contextual information (i.e. dialogue acts and nonverbal behaviors of both user and agent) to drive the SIA behavior.
Fichier principal
Vignette du fichier
Younsi24_DiffusionModel_AVI.pdf (1.09 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04604194 , version 1 (06-06-2024)

Identifiants

  • HAL Id : hal-04604194 , version 1

Citer

Nezih Younsi, Catherine Pelachaud, Laurence Chaby. Diffusion models for virtual agent facial expression generation in Motivational interviewing. International Conference on Advanced Visual Interfaces, Jun 2024, Genoa Italy, Italy. ⟨hal-04604194⟩
201 Consultations
64 Téléchargements

Partager

More