Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection - Sorbonne Université Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection

Résumé

Multi-Task Learning (MTL) is appealing for deep learning regularization. In this paper, we tackle a specific MTL context denoted as primary MTL, where the ultimate goal is to improve the performance of a given primary task by leveraging several other auxiliary tasks. Our main methodological contribution is to introduce ROCK, a new generic multi-modal fusion block for deep learning tailored to the primary MTL context. ROCK architecture is based on a residual connection, which makes forward prediction explicitly impacted by the intermediate auxiliary representations. The auxiliary predictor's architecture is also specifically designed to our primary MTL context, by incorporating intensive pooling operators for maximizing complementarity of intermediate representations. Extensive experiments on NYUv2 dataset (object detection with scene classification, depth prediction, and surface normal estimation as auxiliary tasks) validate the relevance of the approach and its superiority to flat MTL approaches. Our method outperforms state-of-the-art object detection models on NYUv2 by a large margin, and is also able to handle large-scale heterogeneous inputs (real and synthetic images) with missing annotation modalities.
Fichier principal
Vignette du fichier
ROCK.pdf (3.67 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01922291 , version 1 (14-11-2018)
hal-01922291 , version 2 (27-11-2018)
hal-01922291 , version 3 (20-12-2018)

Identifiants

  • HAL Id : hal-01922291 , version 3

Citer

Taylor Mordan, Nicolas Thome, Gilles Henaff, Matthieu Cord. Revisiting Multi-Task Learning with ROCK: a Deep Residual Auxiliary Block for Visual Detection. 32nd Conference on Neural Information Processing Systems (NeurIPS), Dec 2018, Montréal, Canada. ⟨hal-01922291v3⟩
349 Consultations
127 Téléchargements

Partager

Gmail Facebook X LinkedIn More