Memory Development with Heteroskedastic Bayesian Last Layer Probabilistic Deep Neural Networks - Sorbonne Université
Conference Papers Year : 2023

Memory Development with Heteroskedastic Bayesian Last Layer Probabilistic Deep Neural Networks

Abstract

Learning world models in Model-Based Reinforcement Learning (MBRL) facilitates sample-efficient learning while mitigating model-bias by employing uncertainty estimates in the transition dynamics. This approach also enables the utilization of Model Predictive Control (MPC) and Recurrent Neural Networks (RNN) for effective action planning. However, the current methodologies do not offer uncertainty estimates for the prediction (transition) step in a closed form, which could be instrumental for analytically estimating changes in the dynamics function. In this paper, we introduce a hybrid approach designed to capture both aleatoric (data) uncertainty through Probabilistic Deep Neural Networks (PrDNN) and epistemic (model) uncertainty via a Bayesian Last Layer. By adopting an analytical approach, we can identify alterations in dynamics necessitating the learning and memorization of diverse world models. We present a memory schema and demonstrate its efficacy in swiftly adapting to changes in dynamic functions by retrieving previously learned models.
Fichier principal
Vignette du fichier
Velentzas_et_al_2023_final.pdf (380.75 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04249890 , version 1 (24-10-2023)

Identifiers

  • HAL Id : hal-04249890 , version 1

Cite

George Velentzas, Costas S Tzafestas, Mehdi Khamassi. Memory Development with Heteroskedastic Bayesian Last Layer Probabilistic Deep Neural Networks. Workshop on World Models and Predictive Coding in Cognitive Robotics at 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023), Oct 2023, Detroit (Michigan), United States. ⟨hal-04249890⟩
56 View
98 Download

Share

More