E-ComSupResNet: Enhanced Face Super-Resolution Through Compact Network

Abstract

Practical systems such as in surveillance applications capture Low-Resolution (LR) face images due to the wider angle of imaging or longer stand-off distance to the camera. However, face recognition applications demand high resolution face images for effective feature extraction and comparison. To meet the requirement of face recognition system despite the low resolution face captured, a number of works have been proposed to super-resolve the LR face images. Here, we propose a compact and computationally efficient Convolutional Neural Network (CNN) to increase the spatial resolution of an LR face image to obtain a High-Resolution (HR) face image with an upscaling factor of up to <inline-formula> <tex-math notation= LaTeX >$\\times 8$ </tex-math></inline-formula> by improving our previous approach. The newly proposed approach, which we refer to as <italic>E-ComSupResNet</italic>, is an enhanced architecture of our preliminary approach ComSupResNet with multiple enhancements. Contrary to other earlier works, the proposed architecture with a compact network focuses on extracting both low-frequency and high-frequency features along with a reconstruction module. With the introduction of Residual Block (ResBlock), a novel upscaling network is provided, to super-resolve the feature maps for the factor of <inline-formula> <tex-math notation= LaTeX >$\\times 2$ </tex-math></inline-formula>, a Global Residual Learning (GRL) and two newly customized loss functions imposing similarity in the new network architecture we obtain superior SR images compared to our earlier network and competing state-of-art approaches. The newly proposed architecture further has <inline-formula> <tex-math notation= LaTeX >$approx$ </tex-math></inline-formula>. <inline-formula> <tex-math notation= LaTeX >$1.5~M$ </tex-math></inline-formula> number of parameters as compared to similar earlier work which has more than <inline-formula> <tex-math notation= LaTeX >$57 ~M$ </tex-math></inline-formula> parameters. We further attempt to make the approach generalizable by learning the network in a cross-database setting and training the network on <italic>CASIA WebFace</italic> dataset while evaluating it both on <italic>CelebA</italic> and <italic>LFW</italic> datasets. Through empirical evaluations, we demonstrate the gain in high fidelity reconstruction in terms of Structural Similarity Index Metric (SSIM) and Peak-Signal-to-Noise Ratio (PSNR) measures despite the compactness of the model. As another new contribution, we conduct a series of experiments to measure the applicability of obtained SR images using the proposed network through conducting face recognition experiments using a commercial and open source Face Recognition System (FRS) with a clear gain in verification performance over the other existing techniques.