- Tytuł:
- Self-improving Q-learning based controller for a class of dynamical processes
- Autorzy:
-
Musial, Jakub
Stebel, Krzysztof
Czeczot, Jacek - Powiązania:
- https://bibliotekanauki.pl/articles/1845515.pdf
- Data publikacji:
- 2021
- Wydawca:
- Polska Akademia Nauk. Czytelnia Czasopism PAN
- Tematy:
-
process control
Q-learning algorithm
reinforcement learning
intelligent control
on-line learning - Opis:
- This paper presents how Q-learning algorithm can be applied as a general-purpose self-improving controller for use in industrial automation as a substitute for conventional PI controller implemented without proper tuning. Traditional Q-learning approach is redefined to better fit the applications in practical control loops, including new definition of the goal state by the closed loop reference trajectory and discretization of state space and accessible actions (manipulating variables). Properties of Q-learning algorithm are investigated in terms of practical applicability with a special emphasis on initializing of Q-matrix based only on preliminary PI tunings to ensure bumpless switching between existing controller and replacing Q-learning algorithm. A general approach for design of Q-matrix and learning policy is suggested and the concept is systematically validated by simulation in the application to control two examples of processes exhibiting first order dynamics and oscillatory second order dynamics. Results show that online learning using interaction with controlled process is possible and it ensures significant improvement in control performance compared to arbitrarily tuned PI controller.
- Źródło:
-
Archives of Control Sciences; 2021, 31, 3; 527-551
1230-2384 - Pojawia się w:
- Archives of Control Sciences
- Dostawca treści:
- Biblioteka Nauki
Artykuł