Reinforcement Learning Implementation In The Control System For The Unmanned, Compound Helicopter

Abstract

The paper deals with the development of an automatic control system for an unmanned, small-scale compound helicopter, equipped with an additional pusher propeller located at the end of the tail boom. A linear quadratic regulator controls the classical part of the helicopter, while the pitch angle of the pusher propeller, at a given constant angular velocity, is controlled by two interchangeable controllers: a nonlinear controller based on the TD3 reinforcement algorithm and a linear, proportional controller. The latter is used to compare the performance of the control system with the nonlinear controller against the traditional, linear approach. Besides developing the control system, the paper demonstrates the possibility of applying the reinforcement learning to the control problem. Simulation results show that the preset forward velocity cannot be achieved without incorporating the thrust force of the pusher propeller. Furthermore, at higher flight velocities, the nonlinear controller better minimizes the velocity deviation by adjusting the pitch angle of the pusher propeller according to the current flight conditions.