Reinforcement Planning for effective ε-optimal policies in dense time with discontinuities - Sorbonne Université
Communication Dans Un Congrès Année : 2023

Reinforcement Planning for effective ε-optimal policies in dense time with discontinuities

Résumé

Lately, the model of (Decision) Stochastic Timed Automata (DSTA) has been proposed, to model those Cyber Physical Systems displaying dense time (physical part), discrete actions and discontinuities such as timeouts (cyber part). The state of the art results on controlling DSTAs are however not ideal: in the case of infinite horizon, optimal controllers do not exist, while for timed bounded behaviors, we do not know how to build such controllers, even ε-optimal ones. In this paper, we develop a theory of Reinforcement Planning in the setting of DSTAs, for discounted infinite horizon objectives. We show that optimal controllers do exist in general. Further, for DSTAs with 1 clock (which already generalize Continuous Time MDPs with e.g. timeouts), we provide an effective procedure to compute ε-optimal controllers. It is worth noting that we do not rely on the discretization of the time space, but consider symbolic representations instead. Evaluation on a DSTA shows that this method can be more efficient. Last, we show on a counterexample that this is the furthest this construction can go, as it cannot be extended to 2 or more clocks. 2012 ACM Subject Classification Theory of computation → Timed and hybrid models Keywords and phrases reinforcement planning, timed automata, planning Digital Object Identifier 10.4230/LIPIcs...

Domaines

Informatique
Fichier principal
Vignette du fichier
HGD23.pdf (815.77 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence

Dates et versions

hal-04280462 , version 1 (11-11-2023)

Licence

Identifiants

Citer

Léo Henry, Blaise Genest, Alexandre Drewery. Reinforcement Planning for effective ε-optimal policies in dense time with discontinuities. 43rd IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science, Dec 2023, hyderabad, India. ⟨10.4230/LIPIcs⟩. ⟨hal-04280462⟩
50 Consultations
40 Téléchargements

Altmetric

Partager

More