Using Memristors for Robust Local Learning of Hardware Restricted Boltzmann Machines - Sorbonne Université Accéder directement au contenu
Article Dans Une Revue Scientific Reports Année : 2019

Using Memristors for Robust Local Learning of Hardware Restricted Boltzmann Machines

Résumé

One of the biggest stakes in nanoelectronics today is to meet the needs of Artificial Intelligence by designing hardware neural networks which, by fusing computation and memory, process and learn from data with limited energy. For this purpose, memristive devices are excellent candidates to emulate synapses. A challenge, however, is to map existing learning algorithms onto a chip: for a physical implementation, a learning rule should ideally be tolerant to the typical intrinsic imperfections of such memristive devices, and local. Restricted Boltzmann Machines (RBM), for their local learning rule and inherent tolerance to stochasticity, comply with both of these constraints and constitute a highly attractive algorithm towards achieving memristor-based Deep Learning. On simulation grounds, this work gives insights into designing simple memristive devices programming protocols to train on chip Boltzmann Machines. Among other RBM-based neural networks, we advocate using a Discriminative RBM, with two hardware-oriented adaptations. We propose a pulse width selection scheme based on the sign of two successive weight updates, and show that it removes the constraint to precisely tune the initial programming pulse width as a hyperparameter. We also propose to evaluate the weight update requested by the algorithm across several samples and stochastic realizations. We show that this strategy brings a partial immunity against the most severe memristive device imperfections such as the non-linearity and the stochasticity of the conductance updates, as well as device-to-device variability.
Fichier principal
Vignette du fichier
s41598-018-38181-3.pdf (2.95 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-02046570 , version 1 (22-02-2019)

Identifiants

Citer

Maxence Ernoult, Julie Grollier, Damien Querlioz. Using Memristors for Robust Local Learning of Hardware Restricted Boltzmann Machines. Scientific Reports, 2019, 9, pp.1851. ⟨10.1038/s41598-018-38181-3⟩. ⟨hal-02046570⟩
171 Consultations
105 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More