Adversarial Training for Solving Inverse Problems in Image Processing

Abstract

Inverse problems are a group of important mathematical problems that aim at estimating source data <inline-formula> <tex-math notation= LaTeX >$x$ </tex-math></inline-formula> and operation parameters <inline-formula> <tex-math notation= LaTeX >$z$ </tex-math></inline-formula> from inadequate observations <inline-formula> <tex-math notation= LaTeX >$y$ </tex-math></inline-formula>. In the image processing field, most recent deep learning-based methods simply deal with such problems under a pixel-wise regression framework (from <inline-formula> <tex-math notation= LaTeX >$y$ </tex-math></inline-formula> to <inline-formula> <tex-math notation= LaTeX >$x$ </tex-math></inline-formula>) while ignoring the physics behind. In this paper, we re-examine these problems under a different viewpoint and propose a novel framework for solving certain types of inverse problems in image processing. Instead of predicting <inline-formula> <tex-math notation= LaTeX >$x$ </tex-math></inline-formula> directly from <inline-formula> <tex-math notation= LaTeX >$y$ </tex-math></inline-formula>, we train a deep neural network to estimate the degradation parameters <inline-formula> <tex-math notation= LaTeX >$z$ </tex-math></inline-formula> under an adversarial training paradigm. We show that if the degradation behind satisfies some certain assumptions, the solution to the problem can be improved by introducing additional adversarial constraints to the parameter space and the training may not even require pair-wise supervision. In our experiment, we apply our method to a variety of real-world problems, including image denoising, image deraining, image shadow removal, non-uniform illumination correction, and underdetermined blind source separation of images or speech signals. The results on multiple tasks demonstrate the effectiveness of our method.