A Non-Local Means Approach for Gaussian Noise Removal from Images using a Modified Weighting Kernel
Mojtaba Kazemi, Ehsan Mohammadi.P, Parichehr shahidi sadeghi, Mohamad B. Menhaj
22017 25 th Iranian Conference on Electrical Engineering (ICEE)
A Non-Local Means Approach for Gaussian Noise Removal from Images using a Modified Weighting Kernel
Mojtaba Kazemi department of electrical, biomedical and mechatronics engineering, Qazvin Islamic Azad Qazvin, Iran [email protected] Ehsan Mohammadi.P department of electrical, biomedical and mechatronics engineering, Qazvin Islamic Azad Qazvin, Iran [email protected] Parichehr shahidi sadeghi department of electrical, biomedical and mechatronics engineering, Qazvin Islamic Azad Qazvin, Iran [email protected] Mohamad B. Menhaj Electrical engineering department Amir Kabir university of Technology * Tehran, Iran [email protected]
Abstract — Gaussian noise removal is an interesting area in digital image processing not only to improve the visual quality, but for its impact on other post-processing algorithms like image registration or segmentation. Many presented state-of-the-art denoising methods are based on the self-similarity or patch-based image processing. Specifically, Non-Local Means (NLM) as a patch-based filter has gained increasing attention in recent years. Essentially, this filter tends to obtain the noise-less signal value by computing the Gaussian-weighted Euclidean distance between the patch under-processing and other patches inside the image. However, the NLM filter is sensitive to the outliers (pixels that their intensity values are far away from other pixels) inside the patch, meaning that the pixels with the symmetric locations in the patch are assigned the same weight. This can lead to sub-optimal denoising performance when the destructive nature of noise generates some outliers inside patches. In this paper, we propose a new weighting approach to modify the Gaussian kernel of the NLM filter. Our approach employs the geometric distance between image intensities to come up with new weights for each pixel of a patch, lowering the impact of outliers on the denoising performance. Experiments on a set of standard images and different noise levels show that our proposed method outperforms the other compared denoising filters.
Keywords- Denoising, Gaussian Noise, Self Similarities, Non Local Means (NLM) I. I NTRODUCTION
Digital images are usually degraded by noise produced during image acquisition, recording, and transmission. The presence of noise can decrease the visual quality of the acquired image and also can negatively affect the performance of other computerized post-processing approaches such as image registration and segmentation [1,2]. Amongst different noise model used for various natural images, the Gaussian noise model is the most prevalent one [3]. Hence, many works have been dedicated to study different characteristics of the Gaussian noise and provide relevant filtering methods to suppress the noise disturbance. There have been numerous denoising methods to restore the underlying noise-free signal from the given noisy image. A suitable image denoising algorithm should properly remove the noise trace while producing sharp images and preserving the fine details [4,5]. Tomasi et al , [6] proposed the bilateral filter, where the authors developed a denoising method based on the SUSAN filter [7] as an extension of the Yaroslavky filter [8]. This method utilized the weighted average of pixels with similar intensity within a local neighborhood to restore the noise-less signal value. However, this filter is not efficient when the noise power is high. Takeda et al , [9] proposed an image denoising method using signal-dependent steering kernel regression (SKR) framework, obtaining more robust performance under strong noise condition. Based on the concept of the pixel-based bilateral filter, the well-known Non-Local Means (NLM) method as a patch-based filter approach was proposed in [3], where the concept of locality was extended to the entire image. Kerverran et al , [10] developed a denoising method based on the NLM filter that best shows the potential of this method. Using discrete cosine transform and existing data redundancy in the data, BM3D [11] suggests a hybrid approach for grouping similar patches for enoising purposes. In [12,13], a patch-based paradigm for global filtering based on the spectral decomposition was proposed, where each pixel is estimated from all pixels in the image. A large number of speech and image denoising literature is devoted to transform domain methods (e.g., DCT, Fourier, and wavelet) [14-17]. These filtering methods aim to separate image and noise components in the transform domain, performing denoising on the shrinkage of the transform coefficients. Chang et al , [15] employed a spatially adaptive threshold parameter together with the wavelet basis for image denoising purposes. In [16] a transform denoising filter was proposed by modeling the wavelet coefficients of images as mixtures of Gaussians. Luisier et al , [17] proposed a denoising method to reduce the mean-squared error (MSE) by wavelet thresholding, reaching a suitable filtering performance. Dictionary-based learning is another category of image denoising developed based on this notion that similar patches share similar sub-dictionaries; such sub-dictionaries could then be used for better image modeling. Using the dictionary learning basis and sparse representation, Elad et al , [18] * His also a part time faculty member of department of electrical, biomedical and mechatronics engineering, Qazvin Islamic Azad University, [email protected]
017 25 th Iranian Conference on Electrical Engineering (ICEE) (a) (b) Fig.1. (a) Left side show the Gaussian weighting scheme given by the NLM filter. As seen in the provided patch the weights assigned to the non-central pixels are all equal to 0.08. (b) Right side depicts the proposed geometric distance-based approach that assigned a different weight to each pixel based on the geometric distance of its intensity with respect to other pixels inside the patch. extended the KSVD approach for image denoising purposes. A nonlocal sparse model (NLSM) was proposed in [19] to improve the performance of the KSVD [20] framework. Note that, the dictionary-based methods essentially provide implicit modeling for natural images. However, Joshi et al , [21] focused on the color image denoising by explicitly modeling each pixel as a combination of two colors, where the basis colors are estimated within a local neighborhood. Moreover, in [22], Markov Random Fields (MRFs) as a field of experts (FOE) were applied for image denoising, where the parameters for the model are learned from example images. In this paper, a modified version of the NLM filter for denoising of the additive Gaussian noise is presented. Essentially, NLM is an efficient filter to deal with Gaussian noise; however, the Gaussian kernel used with this method is sensitive to the outliers (pixels that their intensity values are far away from other pixels) inside each patch, meaning that the pixels with the symmetric locations inside the patch are assigned the same weight. This can lead to sub-optimal denoising performance in case of high noise power, where the destructive nature of noise can generate many outliers in the image. Here, we aim to provide a new weighting scheme to modify the Gaussian kernel of the NLM method. Our approach employs the geometric distance of the image intensities to assign a new weight for each pixel inside the patch. In our experiments, we observed that the geometric distance is a reliable measure in detecting the outliers inside a set of almost similar intensities. Using this new proposed weighting scheme helps lower the impact of outliers on the denoising performance, improving the filtering capabilities. The rest of this paper is organized as follows: Section II provides the theory of the proposed method. Section III illustrates the experiments and results. Finally, conclusions and some remarks are given in Section IV. II. M ETHODOLOGY
In this paper, we aim to modify the Gaussian kernel of the NLM filter such that it can suppress the weight of outliers inside each patch under consideration. Hence, the main properties of the NLM filter are briefly reviewed here. A. Theory
The NLM filter is proposed based on the Yaroslavsky filter [8], which used to remove noise trace using the average distance between intensities of the image. To find the underlying signal value, the NLM filter evaluates the patch-based similarity between all pixels of the image [3]. Such a patch-based measure is intrinsically more robust than the pixel-based one given by [8], leading to higher denoising performance [3, 23,24]. Given a noisy image Y , using the NLM filter the noise-free signal value at a point p is calculated as a weighted average of all the pixels in the image. So, we have:
YqYq qpwqpwts qYqpwpYNLM where, p and q respectively represent the pixel under-processing and other pixels in the image. The weights w ( p , q ) define the similarity between two patches N p and N q (i.e., a square neighborhood of size 3 around each pixel) based on a Gaussian kernel function. It is given by:
22 ),(2 ),( )()(),( (2) 1),( qpp h qpdq h qpd
NYNYGqpd eeqpw where, h is a constant called smoothing parameter and G p is the normalized Gaussian weighting function with zero mean and a specific standard deviation (usually set to 1). The NLM filter has shown remarkable performance in cancelling Gaussian noise. Note that, the main principle behind this approach is employing the existing redundancy of information within the natural images. So, instead of using the pixels inside a local neighborhood, it tends to find the similar pixels throughout the image based on its Gaussian weighting kernel. Then, these weighted pixels are put together to obtain the noise-less underlying signal value. B. Modified Kernel Non-Local Means ( MK-NLM ) In this paper, we propose a new weighting function to improve the filtering performance of the NLM method. As mentioned earlier, the NLM method assigns same weights to the pixels with the same geometric location inside a patch (see Fig. (1-a)). Essentially, such a weight assignment scheme is not sensitive to the outliers inside the patch, meaning that the outliers are taken as important as other pixels. This issue can be problematic in the presence of heavy noise levels, where the destructive nature of noise may generate many outliers inside the image. 017 25 th Iranian Conference on Electrical Engineering (ICEE) Wiener NLM GNLM Proposed O r i g i n a l F ilt e r e d N o i s y R e s i du a l Wiener
NLM
GNLM
Proposed O r i g i n a l F ilt e r e d N o i s y R e s i du a l Fig. 2. Visual comparison of filters with 10 % Gaussian noise. In each case, the residual image is shown below the corresponding filtered image. As seen, the filtered image by the proposed method shows remarkable performance in removing noise impact while there is no considerable information and image structure on the residual image.
To address the aforementioned drawback, we propose to modify the weighting scheme given by the NLM filter to reduce the effect of the outliers or singular intensities, decreasing their weights in the calculations. In other words, we consider not only the location of each pixel inside the patch but the self-similarity of pixels as well, decreasing the impact of the outliers in calculating the similarity measure. We employ the geometric distance to isolate the outliers, as it was shown to be a reliable measure in detecting the singular values within a set of almost similar intensities. This is done by calculating the weights between the intensity of each pixel and other existing intensities inside a patch. Fig. (1-b) illustrates our proposed weighting scheme. Thus, our scheme is given by: (3) ,1 )(1 1)( Njiaaaweight
Nj jii where, a represents the intensity of each pixel of a patch with the size equal to N . Note that, the calculated geometric distance (i.e., the phrase under the square root) is added by 1 and inverted to calculate the corresponding weight for each pixel. This will ensure that each single weight is normalized in the range of [0,1]. Moreover, it implies that the weights of outliers are set close to zero. Using the weights calculated by Eq. (3), the Gaussian weighting kernel of the NLM filter is modified for each patch as follows: .(4))()( )()()()(),(_ q qp qqppp NwNw NYNwNYNwGqpNLMMK where, w ( N p ) and w ( N q ) are respectively the calculated weights for the patch under processing ( N p ) and other patches ( N q ). The denominator of Eq. (4) works like a normalization factor to ensure that the new weights are 0 ≤ w ( N p ), w ( N q ) ≤ 1. III. E XPERIMENTS AND R ESULTS
In this section, we compare our proposed method against some existing denoising filters used for Gaussian noise removal purposes. A. Quantitative Metrics
Two well-known quantitative measures are used to evaluate the quality of the filtered image. The first one is Root Mean Squared Error (RMSE) which computes the distance between the ground-truth ( A ) and filtered image ( A ') of the same size N as follows: 017 25 th Iranian Conference on Electrical Engineering (ICEE) T ABLE I. Q UANTITATIVE COMPARISON OF DIFFERENT DENOISING FILTERS WITH G AUSSIAN NOISE LEVEL OF TO AND DIFFERENT IMAGES . T HE BEST VALUE FOR THE
RMSE
AND
SSIM
QUALITY MEASURES IS HIGHLIGHTED IN EACH CASE .
5% 10% 15% 20% RMSE SSIM RMSE SSIM RMSE SSIM RMSE SSIM L e n a NLM[3] 3.058 0.951 4.250 0.901 5.421 0.847 6.756 0.785 GNLM[12] C a m e r a m a n NLM[3] 3.650 0.958 5.483 0.907 6.508 0.843 7.764 0.786 GNLM[12] 3.615 0.952 5.120 0.919 6.320 0.865 7.557 0.821 MK-NLM P e pp e r s NLM[3] 2.511 0.965 4.205 0.926 5.610 0.875 7.004 0.820 GNLM[12] P a rr o t NLM[3] 2.810 0.969 4.386 0.928 5.802 0.872 7.182 0.814 GNLM[12] 2.852 0.958 4.222 0.933 5.608 0.892 6.885 0.849 MK-NLM H ou s e NLM[3] 3.010 0.9445 4.073 0.897 5.283 0.840 6.534 0.803 GNLM[12] 2.922 (5) .1
21 ' Ni xx
AANRMSE
We also use the structural similarity (SSIM) index [25] which reveals the perceptual similarity between two images and proved to be more consistent with human visual system. SSIM is given by: (6) .))(( )2)(2(1)',(
1, 222122 21 Nyx xxyx xyyx cc ccNAASSIM where, x and y are the local mean values of images A and A ', x and y are the respective standard deviations, xy is the covariance value and c and c are two constants. B. Filtering Performance
We evaluate the denoising ability of filters in different noise conditions to study the performance of methods with different noise power. In all cases, the parameters of the denoising methods are set to the best values given by the authors, providing the best performance for each filter. The quantitative results for a wide range of additive Gaussian noise (5-20 percent of the maximum gray level) are reported in Table I. As seen, regardless of the noise power and image in use, the proposed denoising approach returns the best results in most cases. An interesting comparison between methods arises when the noise power is high. As given in Table I, the proposed filter outperforms the other filters when the noise power increases.
This shows the higher restoration performance of the proposed filter against other compared filters when the destructive nature of noise is high. For instance, the average RMSE and SSIM measures of the proposed MK-NLM for all employed images are respectively 6.45 and 0.85 as compared to 6.78 and 0.83 given by the GNLM for the noise level 20%. But, the quantitative measures for the same experiments are respectively (2.86 and 0.96) and (2.9 and 0.96) in noise level equal to 5%. Fig. 2 provides the visual comparison between different denoising methods as well as the proposed approach for 10% of the Gaussian noise level. In addition to the filtered images, the residual images (i.e., the difference between the filtered and the noise-less images) are shown for each filter. Note that, a suitable denoising approach should remove the noise trace as much as possible while maintaining the fine structures of images. Moreover, less information on the residual images is another proper characteristic for the reliable denoising approach. As shown in Fig. 2, the proposed approach shows remarkable capability in removing noise. The results of the GNLM are also comparable to those of the proposed filter. But, on average, it shows weaker performance with the quantitative measures. Furthermore, the residual images obtained by the proposed method mostly include the random characteristic of noise and show no considerable information of the ground-truth. IV. C ONCLUSIONS
This paper proposed a Gaussian noise removal approach based on the well-known nonlocal means (NLM) filter. Intrinsically, the NLM approach provides a patch-based similarity measure based on a Gaussian weighted similarity measure. However, it does not consider the homogeneity of intensities inside the patch under consideration and assigns the same weight to pixels located on the same geometric position. In contrast, we presented a new filtering approach by modifying the weighting scheme of the NLM filter. As a consequence, our proposal takes into account not only the nonlocal similarity among patches, but considers the similarity of pixels intensity inside each patch based on the extracted weighting formulation. 017 25 th Iranian Conference on Electrical Engineering (ICEE) We carried out various experiments on the standard images with different noise levels. The proposed method was compared against several state-of-the-art denoising filters developed to deal with Gaussian noise model. The quantitative results show the superiority of the presented filter in almost all cases, specifically in the high noise levels when the destructive nature of noise destroys the underlying image drastically. An interesting extension to this work is applying the modified kernel scheme on other kinds of the noise models by customizing the weighting scheme to the noise distribution. R
EFERENCES [1]
P. Milanfar, ―Symmetrizing smoothing filters,‖
SIAM J. Imag. Sci. , vol. 6, no. 1, pp. 263–284, 2012. [2]
P. Chatterjee and P. Milanfar, ―Practical bounds on image denoising: From estimation to information,‖
IEEE Trans. Image Process. , vol. 20, no. 5, pp. 1221–1233, May 2011. [3]
A. Buades, B. Coll, and J. M. Morel, ―A review of image denoising algorithms, with a new one,‖ Multiscale Model. Simulation, vol. 4, no. 2, pp. 490-530, 2005. [4]
H. M. Golshan, R. P.R. Hasanzadeh, and S. C. Yousefzadeh, ―An MRI denoising method using image data redundancy and local SNR estimation,‖ Mag. Reson. Imag., vol. 31, no. 7, pp. 1206-1217, Sep. 2013. [5]
H. M. Golshan, Reza PR Hasanzadeh,―A modified Rician LMMSE estimator for the restoration of magnitude MR images,‖ Int. J. Light Electron Opt. vol. 124, no. 16, pp. 2387-2392, Aug. 2013. [6]
C. Tomasi and R. Manduchi, ―Bilateral filtering for gray and color images,‖ in
Proc. Int. Conf. Comput. Vis. ,Washington, DC, Jan. 1998, pp. 839–846. [7]
S. M. Smith and J. M. Brady, ―SUSAN—A new approach to low level image processing,‖
Int. J. Comput. Vis. , vol. 23, no. 1, pp. 45–78, May. 1997. [8]
L. P. Yaroslavsky , Digital Picture Processing . Secaucus, NJ: Springer-Verlag, 1985. [9]
H. Takeda, S. Farsiu, and P. Milanfar, ―Kernel regression for image processing and reconstruction,‖
IEEE Trans. Image Process. , vol. 16, no. 2, pp. 349–366, Feb. 2007. [10]
C. Kervrann and J. Boulanger, ―Optimal spatial adaptation for patchbased image denoising,‖
IEEE Trans. Image Process. , vol. 15, no. 10, pp. 2866–2878, Oct. 2006. [11]
K. Dabov, A. Foi, V. Katkovnik, and K. O. Egiazarian, ―Image denoising by sparse 3-D transform-domain collaborative filtering,‖
IEEE
Trans. Image Process. , vol. 16, no. 8, pp. 2080–2095, Aug. 2007. [12]
Hossein Talebi, and Peyman Milanfar, ―Nonlocal image editing,‖ IEEE Trans, Image Process., vol. 23, no. 10, pp. 4460-4473, Oct. 2014. [13]
Priyam Chatterjee, and Peyman Milanfar, ―Patch-based near-optimal image denoising,‖ IEEE Trans. Image Process., vol. 12, no. 4, pp. 1635-1649. Apr. 2012. [14]
M. Najafian, S. Safavi, A. Hanani, and M. Russell, ―Acoustic model selection using limited data for accent robust speech recognition,‖ ( EUSIPCO ), pp. 1786-1790, 2014. [15]
S. G. Chang, B. Yu, and M. Vetterli, ―Spatially adaptive wavelet thresholding with context modeling for image denoising,‖
IEEE Trans. Image
Process. , vol. 9, no. 9, pp. 1522–1531, Sep. 2000. [16]
J. Portilla, V. Strela, M. J.Wainwright, and E. P. Simoncelli, ―Image denoising using a scale mixture of Gaussians in thewavelet domain,‖
IEEE
Trans. Image Process. , vol. 12, no. 11, pp. 1338–1351, Nov. 2003. [17]
F. Luisier, T. Blu, and M. Unser, ―A new SURE approach to image denoising: Interscale orthonormal wavelet thresholding,‖
IEEE Trans.
Image Process. , vol. 16, no. 3, pp. 593–606, Mar. 2007. [18]
M. Aharon, M. Elad, and A. Bruckstein, ―K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,‖
IEEE
Trans. Signal Process. , vol. 54, no. 11, pp. 4311–4322, Nov. 2006. [19]
J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, ―Non-local sparse models for image restoration,‖ in
Proc. IEEE Int. Conf. Comput.
Vis. , Kyoto, Japan, Sep./Oct. 2009, pp. 2272–2279. [20]
M. Elad and M. Aharon, ―Image denoising via sparse and redundant representations over learned dictionaries,‖
IEEE Trans. Image Process. , vol. 15, no. 12, pp. 3736–3745, Dec. 2006. [21]
N. Joshi, C. L. Zitnick, R. Szeliski, and D. Kriegman, ―Image deblurring and denoising using color priors,‖ in
Proc. IEEE Conf. Comput.
Vis. Pattern Recog. , Miami, FL, Jun. 2009, pp. 1550–1557. [22]
S. Roth and M. J. Black, ―Fields of experts,‖
Int. J. Comput. Vis. , vol. 82, no. 2, pp. 205–229, Apr. 2009. [23]
H. M. Golshan, Reza PR Hasanzadeh ,―An optimized LMMSE based method for 3D MRI denoising,‖ IEEE/ACM Trans on Computational Biology and Bioinformatics., vol. 12, no. 4, pp. 861-870, Aug. 2014. [24]
H. M. Golshan, Reza PR Hasanzadeh ,―A non-local Rician noise reduction approach for 3-D magnitude magnetic resonance images,‖ Conference on Machine Vision and Image Processing (MVIP)., pp. 1-5, Nov. 2011. [25]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simonceli, ―Image quality assessment: From error visibility to structural similarity,‖
IEEE Trans. Image Process. , vol. 13, no. 4, pp. 600–612, Apr. 2004., vol. 13, no. 4, pp. 600–612, Apr. 2004.