Click here to flash read.
Safety is one of the biggest concerns to applying reinforcement learning (RL)
to the physical world. In its core part, it is challenging to ensure RL agents
persistently satisfy a hard state constraint without white-box or black-box
dynamics models. This paper presents an integrated model learning and safe
control framework to safeguard any agent, where its dynamics are learned as
Gaussian processes. The proposed theory provides (i) a novel method to
construct an offline dataset for model learning that best achieves safety
requirements; (ii) a parameterization rule for safety index to ensure the
existence of safe control; (iii) a safety guarantee in terms of probabilistic
forward invariance when the model is learned using the aforementioned dataset.
Simulation results show that our framework guarantees almost zero safety
violation on various continuous control tasks.
