Skip to Main content Skip to Navigation
Journal articles

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

Abstract : In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.
Document type :
Journal articles
Complete list of metadata

Cited literature [57 references]  Display  Hide  Download

https://hal.sorbonne-universite.fr/hal-01927184
Contributor : Gestionnaire Hal-Upmc <>
Submitted on : Monday, November 19, 2018 - 4:43:40 PM
Last modification on : Wednesday, December 9, 2020 - 3:05:15 PM
Long-term archiving on: : Wednesday, February 20, 2019 - 4:08:42 PM

File

s41467-018-06781-2.pdf
Publication funded by an institution

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Sophie Bavard, Maël Lebreton, Mehdi Khamassi, Giorgio Coricelli, Stefano Palminteri. Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences. Nature Communications, Nature Publishing Group, 2018, 9, pp.4503. ⟨10.1038/s41467-018-06781-2⟩. ⟨hal-01927184⟩

Share

Metrics

Record views

73

Files downloads

165