Skip to Main content Skip to Navigation
Conference papers

Learning Representations by Predicting Bags of Visual Words

Abstract : Self-supervised representation learning targets to learn convnet-based image representations from unlabeled data. Inspired by the success of NLP methods in this area, in this work we propose a self-supervised approach based on spatially dense image descriptions that encode discrete visual concepts, here called visual words. To build such discrete representations, we quantize the feature maps of a first pre-trained self-supervised convnet, over a k-means based vocabulary. Then, as a self-supervised task, we train another convnet to predict the histogram of visual words of an image (i.e., its Bag-of-Words representation) given as input a perturbed version of that image. The proposed task forces the convnet to learn perturbation-invariant and context-aware image features, useful for downstream image understanding tasks. We extensively evaluate our method and demonstrate very strong empirical results, e.g., our pre-trained self-supervised representations transfer better on detection task and similarly on classification over classes "unseen" during pre-training, when compared to the supervised case. This also shows that the process of image discretization into visual words can provide the basis for very powerful self-supervised approaches in the image domain, thus allowing further connections to be made to related methods from the NLP domain that have been extremely successful so far.
Complete list of metadata

Cited literature [79 references]  Display  Hide  Download

https://hal.sorbonne-universite.fr/hal-02505058
Contributor : Matthieu Cord <>
Submitted on : Wednesday, March 11, 2020 - 11:05:05 AM
Last modification on : Tuesday, March 23, 2021 - 9:28:03 AM
Long-term archiving on: : Friday, June 12, 2020 - 2:52:53 PM

File

2002.12247.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02505058, version 1

Citation

Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, Matthieu Cord. Learning Representations by Predicting Bags of Visual Words. CVPR2020 - Conference on Computer Vision and Pattern Recognition, Jun 2020, Seattle, WA, United States. ⟨hal-02505058⟩

Share

Metrics

Record views

119

Files downloads

115