POMDPStressTesting.jl: Adaptive Stress Testing for Black-Box Systems

Abstract

POMDPStressTesting.jl is a package that uses reinforcement learning and stochastic optimization to find likely failures in black-box systems through a technique called adaptive stress testing (Lee et al., 2019). Adaptive stress testing (AST) has been used to find failures in safety-critical systems such as aircraft collision avoidance systems (Lee et al., 2015), flight management systems (Moss et al., 2020), and autonomous vehicles (Koren et al., 2018). The POMDPStressTesting.jl package is written in Julia (Bezanson et al., 2017) and is part of the wider POMDPs.jl ecosystem (Egorov et al., 2017), which provides access to simulation tools, policies, visualizations, and—most importantly—solvers. We provide different solver variants including online planning algorithms such as Monte Carlo tree search (Coulom, 2006) and deep reinforcement learning algorithms such as trust region policy optimization (TRPO) (Schulman et al., 2015) and proximal policy optimization (PPO) (Schulman et al., 2017). Stochastic optimization solvers such as the cross-entropy method (Rubinstein, 1999) are also available and random search is provided as a baseline. Additional solvers can easily be added by adhering to the POMDPs.jl interface.