SMT solver testing with type and grammar based mutation

Abstract

Satisfiability Modulo Theories (SMT) solvers are at the core of many software advances such as program analysis and verification which are highly safety-critical. Hence, to ensure the correctness of the solvers, there have been multiple fuzzing campaigns targeting different logics since 2009. In this paper, we propose a generative type-aware mutation strategy, which is a generalization of a type-aware operator mutation. We have realized the generative type-aware mutation and reported 158 bugs in Z3 and CVC4 including bugs from the versions released as early as 2016 in five months.