Reinforcement Learning-Based Tap-Operations of a Regulated Distribution Transformer for Autonomous Voltage Control

Leschek Kopczynski, Oliver Wolf and Roland Zeise (University of Applied Sciences Düsseldorf, Germany); Holger Hirsch (University of Duisburg-Essen, Germany)

This paper addresses the voltage control problem in low-voltage distribution networks. The objective is to find an optimal policy to determine the tap position for a regulated distribution transformer (RDT) using a deep reinforcement learning (DRL) approach. A deep deterministic policy gradient (DDPG) agent with a continuous action space is used. During the offline training process, the DDPG algorithm learns the best control actions of the RDT for different system states. The main focus of the described approach is to stay in between given voltage boundaries while reducing the number of tap-operations. This is important in a practical manner, since tap-operations can shorten the interval until the next revision and therefore increase maintenance costs. To address this, an additional slope over a window of past observations is calculated and considered within the state space and the reward function. In a detailed simulation environment that explicitly takes into account voltage dependencies, it is shown that this approach results in a reduced number of tap-operations.

Journal: International Journal of Simulation- Systems, Science and Technology- IJSSST V23

Published: no date/time given

DOI: 10.5013/IJSSST.a.23.02.01