Deep unsupervised network for multimodal perception, representation and classification

Abstract : In this paper, we tackle the problem of multimodal learning for autonomous robots.Autonomous robots interacting with humans in an evolving environment need the ability to acquire knowledge from their multiple perceptual channels in an unsupervised way.Most of the approaches in the literature exploit engineered methods to process each perceptual modality. In contrast, robots should be able to acquire their own features from the raw sensors, leveraging the information elicited by interaction with their environment: learning from their sensorimotor experience would result in a more efficient strategy in a life-long perspective.To this end, we propose an architecture based on deep networks, which is used by the humanoid robot iCub to learn a task from multiple perceptual modalities (proprioception, vision, audition).By structuring high-dimensional, multimodal information into a set of distinct sub-manifolds in a fully unsupervised way, it performs a substantial dimensionality reduction by providing both a symbolic representation of data and a fine discrimination between two similar stimuli. Moreover, the proposed network is able to exploit multimodal correlations to improve the representation of each modality alone.
Complete list of metadatas

Cited literature [89 references]  Display  Hide  Download

https://hal.sorbonne-universite.fr/hal-01083521
Contributor : Alain Droniou <>
Submitted on : Monday, November 17, 2014 - 1:55:47 PM
Last modification on : Friday, May 24, 2019 - 5:30:38 PM
Long-term archiving on : Friday, April 14, 2017 - 1:58:37 PM

File

hal_version.pdf
Files produced by the author(s)

Identifiers

Citation

Alain Droniou, Serena Ivaldi, Olivier Sigaud. Deep unsupervised network for multimodal perception, representation and classification. Robotics and Autonomous Systems, Elsevier, 2015, 71, pp.83-98. ⟨10.1016/j.robot.2014.11.005⟩. ⟨hal-01083521⟩

Share

Metrics

Record views

984

Files downloads

2489