Gaze Latent Support Vector Machine for Image Classification Improved by Weakly Supervised Region Selection

Xin Wang 1, * Nicolas Thome 1 Matthieu Cord 1
* Auteur correspondant
1 MLIA - Machine Learning and Information Access
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : This paper deals with Weakly Supervised Learning (WSL), i.e. performing image classification by leveraging local information with models trained from global image labels. We propose a new WSL method which incorporates gaze features collected by an eye-tracker to guide the region selection policy. Our approach presents two appealing advantages: gaze features are cheap to collect, e.g. one order of magnitude faster than bounding boxes, and our method only requires gaze features during training, while being gaze free at test time. For this purpose, the training objective is enriched with a gaze loss, from which we derive a concave-convex upper bound, leading to an off-the-shelf CCCP optimization scheme.  Extensive experiments are conducted to validate the effectiveness of the approach for WSL image classification on two public datasets with gaze annotation, i.e. PASCAL VOC 2012 action and POET. In addition, we publicly release a new food-related dataset, the Gaze-based UPMC Food dataset (UPMC-G20), which covers 20 food categories and 2,000 images. This dataset intends to promote the research in the food-related computer vision community.
Type de document :
Article dans une revue
Pattern Recognition, Elsevier, 2017, 72, pp.59-71. 〈10.1016/j.patcog.2017.07.001〉
Liste complète des métadonnées

Littérature citée [68 références]  Voir  Masquer  Télécharger

https://hal.sorbonne-universite.fr/hal-01557368
Contributeur : Gestionnaire Hal-Upmc <>
Soumis le : jeudi 6 juillet 2017 - 11:58:06
Dernière modification le : dimanche 9 décembre 2018 - 01:26:18
Document(s) archivé(s) le : mercredi 24 janvier 2018 - 12:19:18

Fichier

Wang_Gaze_Latent_Support.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Xin Wang, Nicolas Thome, Matthieu Cord. Gaze Latent Support Vector Machine for Image Classification Improved by Weakly Supervised Region Selection. Pattern Recognition, Elsevier, 2017, 72, pp.59-71. 〈10.1016/j.patcog.2017.07.001〉. 〈hal-01557368〉

Partager

Métriques

Consultations de la notice

162

Téléchargements de fichiers

51