Learning Representations by Predicting Bags of Visual Words - Archive ouverte HAL Access content directly
Conference Papers Year :

Learning Representations by Predicting Bags of Visual Words

Spyros Gidaris
  • Function : Author
  • PersonId : 974024
Andrei Bursuc
Nikos Komodakis
  • Function : Author
  • PersonId : 945099
Patrick Pérez
  • Function : Author
  • PersonId : 1022281


Self-supervised representation learning targets to learn convnet-based image representations from unlabeled data. Inspired by the success of NLP methods in this area, in this work we propose a self-supervised approach based on spatially dense image descriptions that encode discrete visual concepts, here called visual words. To build such discrete representations, we quantize the feature maps of a first pre-trained self-supervised convnet, over a k-means based vocabulary. Then, as a self-supervised task, we train another convnet to predict the histogram of visual words of an image (i.e., its Bag-of-Words representation) given as input a perturbed version of that image. The proposed task forces the convnet to learn perturbation-invariant and context-aware image features, useful for downstream image understanding tasks. We extensively evaluate our method and demonstrate very strong empirical results, e.g., our pre-trained self-supervised representations transfer better on detection task and similarly on classification over classes "unseen" during pre-training, when compared to the supervised case. This also shows that the process of image discretization into visual words can provide the basis for very powerful self-supervised approaches in the image domain, thus allowing further connections to be made to related methods from the NLP domain that have been extremely successful so far.
Fichier principal
Vignette du fichier
2002.12247.pdf (1.67 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02505058 , version 1 (11-03-2020)


  • HAL Id : hal-02505058 , version 1


Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, Matthieu Cord. Learning Representations by Predicting Bags of Visual Words. CVPR2020 - Conference on Computer Vision and Pattern Recognition, Jun 2020, Seattle, WA, United States. ⟨hal-02505058⟩
70 View
64 Download


Gmail Facebook Twitter LinkedIn More