Skip to Main content Skip to Navigation
Journal articles

Modeling awake hippocampal reactivations with model-based bidirectional search

Mehdi Khamassi 1 Benoît Girard 1
1 AMAC - Architectures et modèles d'Adptation et de la cognition
ISIR - Institut des Systèmes Intelligents et de Robotique
Abstract : Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into state-action values for decision-making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reacti-vations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.
Document type :
Journal articles
Complete list of metadatas

Cited literature [72 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02504897
Contributor : Mehdi Khamassi <>
Submitted on : Wednesday, March 11, 2020 - 10:15:15 AM
Last modification on : Thursday, March 26, 2020 - 3:21:54 PM
Document(s) archivé(s) le : Friday, June 12, 2020 - 2:13:39 PM

File

MBBPreplay2020R3.pdf
Files produced by the author(s)

Identifiers

Citation

Mehdi Khamassi, Benoît Girard. Modeling awake hippocampal reactivations with model-based bidirectional search. Biological Cybernetics (Modeling), Springer Verlag, 2020, ⟨10.1007/s00422-020-00817-x⟩. ⟨hal-02504897⟩

Share

Metrics

Record views

199

Files downloads

555