Diffusion models for virtual agent facial expression generation in Motivational interviewing - Sorbonne Université Access content directly
Conference Papers Year : 2024

Diffusion models for virtual agent facial expression generation in Motivational interviewing

Abstract

Motivational interviewing (MI) is a client-centered counseling style that addresses (the client) user’s motivation for behavior change. In this paper, we present a behavior generation model for Socially Interactive Agents (SIA) and apply it to an SIA acting as a virtual therapist in (MI). MI defines different types of dialogue acts for therapist and client. It has been shown that therapist builds rapport with their client by adapting their verbal and nonverbal behaviors. Based on the analysis of a human-human MI dataset (AnnoMI), we found co-occurrences between facial expressions and dialogue acts for both therapist and client. Moreover, the therapist adapts their behavior to their client’s behavior to favor rapport. Our behavior generation model embeds these co-occurrences as well as such behavior adaptation. To this aim, we build an observation-to-action framework based on a conditional diffusion approach trained on the AnnoMI corpus. Our model learns to generate the virtual therapist’s facial expressions conditioned by MI dialogue acts and the client’s nonverbal behaviors. We aim to make SIAs more effective in therapy-like interactions, by using user’s behaviors in addition to contextual information (i.e. dialogue acts and nonverbal behaviors of both user and agent) to drive the SIA behavior.
Fichier principal
Vignette du fichier
Younsi24_DiffusionModel_AVI.pdf (1.09 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04604194 , version 1 (06-06-2024)

Identifiers

  • HAL Id : hal-04604194 , version 1

Cite

Nezih Younsi, Catherine Pelachaud, Laurence Chaby. Diffusion models for virtual agent facial expression generation in Motivational interviewing. International Conference on Advanced Visual Interfaces, Jun 2024, Genoa Italy, Italy. ⟨hal-04604194⟩
0 View
0 Download

Share

Gmail Mastodon Facebook X LinkedIn More