Collaborating Foundation Models for Domain Generalized Semantic Segmentation - Laboratoire d'informatique de l'X (LIX)
Communication Dans Un Congrès Année : 2024

Collaborating Foundation Models for Domain Generalized Semantic Segmentation

Résumé

Domain Generalized Semantic Segmentation (DGSS) deals with training a model on a labeled source domain with the aim of generalizing to unseen domains during inference. Existing DGSS methods typically effectuate robust features by means of Domain Randomization (DR). Such an approach is often limited as it can only account for style diversification and not content. In this work, we take an orthogonal approach to DGSS and propose to use an assembly of CoLlaborative FOUndation models for Domain Generalized Semantic Segmentation (CLOUDS). In detail, CLOUDS is a framework that integrates FMs of various kinds: (i) CLIP backbone for its robust feature representation, (ii) generative models to diversify the content, thereby covering various modes of the possible target distribution, and (iii) Segment Anything Model (SAM) for iteratively refining the predictions of the segmentation model. Extensive experiments show that our CLOUDS excels in adapting from synthetic to real DGSS benchmarks and under varying weather conditions, notably outperforming prior methods by 5.6% and 6.7% on averaged miou, respectively. The code is available at : https://github.com/yasserben/CLOUDS
Fichier principal
Vignette du fichier
Benigmim_Collaborating_Foundation_Models_for_Domain_Generalized_Semantic_Segmentation_CVPR_2024_paper.pdf (3.44 Mo) Télécharger le fichier
Vignette du fichier
thumbnail.png (206.76 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
licence
Format Figure, Image
licence

Dates et versions

hal-04611902 , version 1 (06-12-2024)

Licence

Identifiants

Citer

Yasser Benigmim, Subhankar Roy, Slim Essid, Vicky Kalogeiton, Stéphane Lathuilière. Collaborating Foundation Models for Domain Generalized Semantic Segmentation. The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024, Jun 2024, Seattle, WA, United States. pp.3108-3119, ⟨10.1109/CVPR52733.2024.00300⟩. ⟨hal-04611902⟩
0 Consultations
1 Téléchargements

Altmetric

Partager

More