Skip to Main content Skip to Navigation
Journal articles

Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning

Guillaume Viejo 1, 2, * Mehdi Khamassi 1, 2 Andrea Brovelli 3 Benoît Girard 1, 2
* Corresponding author
ISIR - Institut des Systèmes Intelligents et de Robotique
Abstract : Current learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus–response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal.
Document type :
Journal articles
Complete list of metadata

Cited literature [54 references]  Display  Hide  Download
Contributor : Gestionnaire Hal-Upmc <>
Submitted on : Wednesday, October 14, 2015 - 10:52:38 AM
Last modification on : Friday, March 19, 2021 - 3:32:54 AM
Long-term archiving on: : Friday, May 5, 2017 - 1:32:41 PM


Publication funded by an institution


Distributed under a Creative Commons Attribution 4.0 International License



Guillaume Viejo, Mehdi Khamassi, Andrea Brovelli, Benoît Girard. Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning. Frontiers in Behavioral Neuroscience, Frontiers, 2015, 9, pp.225. ⟨10.3389/fnbeh.2015.00225⟩. ⟨hal-01215419⟩



Record views


Files downloads