Skip to Main content Skip to Navigation
Journal articles

Deep unsupervised network for multimodal perception, representation and classification

Alain Droniou 1, 2 Serena Ivaldi 1, 3, 4 Olivier Sigaud 1, 2
ISIR - Institut des Systèmes Intelligents et de Robotique
4 LARSEN - Lifelong Autonomy and interaction skills for Robots in a Sensing ENvironment
Inria Nancy - Grand Est, LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : In this paper, we tackle the problem of multimodal learning for autonomous robots.Autonomous robots interacting with humans in an evolving environment need the ability to acquire knowledge from their multiple perceptual channels in an unsupervised way.Most of the approaches in the literature exploit engineered methods to process each perceptual modality. In contrast, robots should be able to acquire their own features from the raw sensors, leveraging the information elicited by interaction with their environment: learning from their sensorimotor experience would result in a more efficient strategy in a life-long perspective.To this end, we propose an architecture based on deep networks, which is used by the humanoid robot iCub to learn a task from multiple perceptual modalities (proprioception, vision, audition).By structuring high-dimensional, multimodal information into a set of distinct sub-manifolds in a fully unsupervised way, it performs a substantial dimensionality reduction by providing both a symbolic representation of data and a fine discrimination between two similar stimuli. Moreover, the proposed network is able to exploit multimodal correlations to improve the representation of each modality alone.
Complete list of metadata

Cited literature [89 references]  Display  Hide  Download
Contributor : Alain Droniou Connect in order to contact the contributor
Submitted on : Monday, November 17, 2014 - 1:55:47 PM
Last modification on : Saturday, January 15, 2022 - 3:47:19 AM
Long-term archiving on: : Friday, April 14, 2017 - 1:58:37 PM


Files produced by the author(s)



Alain Droniou, Serena Ivaldi, Olivier Sigaud. Deep unsupervised network for multimodal perception, representation and classification. Robotics and Autonomous Systems, Elsevier, 2015, 71, pp.83-98. ⟨10.1016/j.robot.2014.11.005⟩. ⟨hal-01083521⟩



Les métriques sont temporairement indisponibles