Robustness verification of $\\boldsymbol~K$-NN classifiers via constraint relaxation and randomized smoothing

Abstract

We study the robustness verification problem for $K$-NN classifiers. The objective of formal robustness verification is to find the exact minimal adversarial perturbation or a guaranteed lower bound of the perturbation. We find that the robustness verification of $K$-NN classifiers could be formalized as a series of quadratic programming problems. Solving these quadratic programming problems is not possible in general because the number of problems grows exponentially with respect to $K$. The constraint relaxation method is proposed to compute the lower bound of the minimal adversarial perturbation in polynomial time. However, we find that the resulting lower bound tends to be extremely loose when $K$ is large; hence, $K$-NN with a large $K$ being less robust is counterintuitive. To tackle this issue, we propose to employ the randomized smoothing method to verify the robustness of $K$-NN classifiers. By exploiting the resistance of $K$-NN to random Gaussian noise, the randomized smoothing method achieves high performance in verification. Our experiments on benchmark datasets show that the smoothed $K$-NN classifier is more verifiably robust than state-of-the-art robust neural networks.