Cinematographic Camera Diffusion Model - Laboratoire d'informatique de l'X (LIX)
Article Dans Une Revue Computer Graphics Forum Année : 2024

Cinematographic Camera Diffusion Model

Cinematographic Camera Diffusion Model

Hongda Jiang
  • Fonction : Auteur
  • PersonId : 1091027
Marc Christie
Baoquan Chen
  • Fonction : Auteur correspondant

Résumé

Designing effective camera trajectories in virtual 3D environments is a challenging task even for experienced animators. Despite an elaborate film grammar, forged through years of experience, that enables the specification of camera motions through cinematographic properties (framing, shots sizes, angles, motions), there are endless possibilities in deciding how to place and move cameras with characters. Dealing with these possibilities is part of the complexity of the problem. While numerous techniques have been proposed in the literature (optimization-based solving, encoding of empirical rules, learning from real examples, etc.), the results either lack variety or ease of control. In this paper, we propose a cinematographic camera diffusion model using a transformer-based architecture to handle temporality and exploit the stochasticity of diffusion models to generate diverse and qualitative trajectories conditioned by high-level textual descriptions. We extend the work by integrating keyframing constraints and the ability to blend naturally between motions using latent interpolation, in a way to augment the degree of control of the designers. We demonstrate the strengths of this text-to-camera motion approach through qualitative and quantitative experiments and gather feedback from professional artists.
Fichier principal
Vignette du fichier
CineCamDiff_AuthorVer.pdf (43.04 Mo) Télécharger le fichier
teaser.jpeg (5.05 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04826479 , version 1 (13-12-2024)

Licence

Identifiants

Citer

Hongda Jiang, Xi Wang, Marc Christie, Libin Liu, Baoquan Chen. Cinematographic Camera Diffusion Model. Computer Graphics Forum, 2024, 43 (2), pp.1-14. ⟨10.1111/cgf.15055⟩. ⟨hal-04826479⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More