Investigating the Robustness and Generalizability of Deep Reinforcement Learning Based Optimal Trade Execution Systems

Abstract

Recently, researchers from both academia and the financial industry have attempted to leverage the cutting edge techniques of Deep Reinforcement Learning (DRL) to develop autonomous trade execution systems. They desire a system that could learn from the trade data and develop trade execution strategies to optimize the execution costs. While several researchers have reported success in developing such autonomous trade execution systems, none of them have investigated the robustness of the systems. Despite the powerfulness of DRL, the overfitting and generalization remain a challenge in DRL. For real-world applications, especially in financial trading, the DRL system’s robustness is critical, as the cost of wrong decisions could be devastating. In our experiment, we investigate the robustness of the DRL based autonomous trade execution systems by applying the policies learned from one stock to other stocks directly without any refining or training. As different stocks have different dynamics, they might serve as suitable environments to investigate the systems’ capability to generalize. The result suggests that the DRL systems have generalized well on other stocks without further refining or training on the specific stock’s historical trade data. The result is significant from two perspectives: 1) It suggests that the DRL based trade execution systems can generalize well, which gives us and potential users more confidence in their performances; 2) It also presents an opportunity to develop a more sample efficient system.