Patch-based Contour Prior Image Denoising for Salt and Pepper Noise
PPatch-based Contour Prior Image Denoising forSalt and Pepper Noise
Abstract —The salt and pepper noise brings a significant challenge to image denoising technology, i.e. how toremoval the noise clearly and retain the details effectively? In this paper, we propose a patch-based contour priordenoising approach for salt and pepper noise. First, noisy image is cut into patches as basic representation unit, adiscrete total variation model is designed to extract contour structures; Second, a weighted Euclidean distance isdesigned to search the most similar patches, then, corresponding contour stencils are extracted from these similarpatches; At the last, we build filter from contour stencils in the framework of regression. Numerical results illustratethat the proposed method is competitive with the state-of-the-art methods in terms of the peak signal-to-noise (PSNR)and visual effects.
Index Terms — image denoising, Patch directional prior, salt and pepper noise, total variation
1. INTRODUCTION
Images are always polluted by noise during image acquisition and transmission because of imperfections in theimaging and capturing process. Removing noise while preserving image details and textures (edge detection,segmentation, etc.) is one of the most important and fundamental issues in image processing. As an importantpre-processing step, image denoising algorithms are also widely used in computer vision, pattern recognition andmedical image analysis fields. At different stages of imaging, the image may be polluted by corresponding type ofnoises, for example Poisson noise, Gaussian noise, etc.Gaussian noise is an additive noise, this kind of noise generally appears during the acquisition of an image.There are huge literature solely dedicated to Gaussian denoising, some of them achieved remarkable results forexample famous non-local means(NLM)[1] which exploit the self-similarity property of natural images to performdenoising by aggregation of similar patches. Block-matching and 3D filtering(BM3D)[2] uses 3-D similar patchesto perform denoising in transform domain. Another very effective method is the K-SVD algorithm [3], of which thekey is how to get optimal dictionary of image patches adapted for the observed noisy data.Impulse noise is another common type of image noise, it is generally due to data loss happening.Unfortunately, denoising methods for Gaussian noise are totally useless for impulse noise. The main reason is thatdistributions of two kind of noise are completely different. Two models of impulse noise are generally studied inthe literatures, one model is the so-called random-valued impulse noise, which each pixel is replaced withprobability p by a random value in the set {0, 1, . . . , M}, M is set to 255. Another type of image noise is theso-called salt and pepper noise, which each pixel is replaced with a given probability by [0, δ) or (255-δ, 255], δ isgenerally defined as a very small value like 0,1,2 and 3.In this paper, we will focus on the second model of impulse noise i.e. salt and pepper noise because it usually
Bo Fu , XiaoYang Zhao ,Yi Li , XiangHai Wang
1. College of Computer and Information Technology, Liaoning Normal University, Dalian, China
E-mail: [email protected] ring more obvious visual interference (black or white pixel). For the removal of salt and pepper noise problem,finding noise and repairing it while preserving details and texture information is of great importance. Let U asoriginal image, size M*N. Let V as observed noisy image, ( i , j ) be the coordinates of a pixel, f (i,j) is the gray valueof pixel (i,j). If pixel (i,j) is noise, the noise model is then defined as: ( , ) ( , ) p pi j s si j p s N if f Tf N if t TV if t T T K (1)In the past decade, a number of denoising algorithms have been proposed. One well-known algorithm familyis nonlinear filters such as median filters [4,5]. These kinds of methods adopt the median value of the pixel value inthe local area as the basis for restoration. Switching filter and its extension introduce a switching templet todecrease noise’s interference during repairing step[6-12]. Though some achievements have been achieved, thesemethods mentioned above only take local region into account in analysis and repair stage. Too little informationleads to inaccurate identify noise and instable repair work.Buades firstly proposed non local concept, and design a non-local means for Gaussian noise[1]. Based non-localidea, famous BM3D was born [2,13], it constructs non local collaborative filtering in frequency domain. Drawingon this idea, some denoising algorithms for salt and pepper have been proposed forward. Nasri et al. improvedclassical non-local means by introducing a switching filter to reduce the interference of impulse noise [14],Varghese J proposed an adaptive version[15]. All methods mentioned above have two common points. First, theyare all patches based methods. Second, the computation of similarity takes place in a non-local scope. The abovecharacteristic brings both advantages and challenges. Patch contain more information(texture, edge, etc.) thansingle pixel, but it is more difficult to estimate and mine effective information under high noise density condition. Alarge number of patches will pose a new challenge in precision measurement between patches especially thesepatches are disturbed by noise. In view of these problems, scholars in the field of CV have launched some attempts.Delon designed a statistical estimator to estimate patch and compute distance between patches [16]. Meanwhile,some clusters that consist of a few patches are abandoned to suppress noise[17-20]. Some low-rank methods andpatch retrieval methods are also be proposed[21-24,30-33]. But they generally did not dig the texture information ofthe patch meanwhile restrain the noise.Given the above problems, the idea of this paper is relies on two contributions. First, we introduce TotalVariation(TV) to describe sharp trend of texture in each patch. Second, for each patch we will generate a geometricstructure according patches’ texture, and build regression restoration in non-local range. The rest of this paper isorganized as follows: In section 2, we define a basic form of TV model, and compute a direction template toapproximate function meanwhile resisting noise. In section 3, using generated a geometric structure in section 2, webuild a Non local filtering for patches regression. Section 4 shows numerous experimental results and comparisonswith several related algorithms on several types of nature images.
2. P
ATCH CONTOURS ESTIMATION BASED ON TOTAL VARIATION
TV model is proposed by Rudin firstly [25], a significant advantage of TV model is that it allows discontinuitypoints exist in variational function space
BV( ) (it leads to an underlying sparse solution).Set R , ( ) U L is a bounded open collection, it is usually assumed to be a Lipschitz domain. Assume U can be expressed as function u (i,j), and u is smooth. The TV express of u is: BV( ) min TV[ ]= | |d d =( , ) x yu u u x y u u u , (2)ere, u satisfies the following constraint conditions: ( , ) = ( , )d d u x y dxdy u x y x y , u x y u x y x y (3)Further, the document [26] defines 8 kinds of contour templates to discretize the contour stencils(CS)information of the integral region, CS is defined as: * , ( [ ])( ) : ( , ) k m k nm n u k m n u u R S S (4)For each pixel, they find aimed contour stencils which it’s ( [ ])( )
S u k is minimum. But, these templates onlycover 8 directions and they are under the assumption that the function is smooth.So in our work, we firstly choose median value to change salt and pepper noise to avoid noise interferencebefore generating contour stencils. And then we extend 8 directions to 3 types of directional templates includinghorizontal direction, vertical direction, and diagonal direction. Each direction including 8 contour stencils andcorresponding discretization templates. The specific definitions are given in Table 1. Table 1 Multi-angle template with3 directions
Horizontal (1)1 S (1)2 S (1)3 S (1)4 S (1)5 S (1)6 S (1)7 S (1)8 S DirectionAngle
1- +arctan 2 Discretizationmatrix vertical (2)1 S (2)2 S (2)3 S (2)4 S (2)5 S (2)6 S (2)7 S (2)8 S DirectionAngle - +arctan 2 -arctan 2 38 -arctan 2 arctan 2 38 Discretizationmatrix diagonal (3)1 S (3)2 S (3)3 S (3)4 S (3)5 S (3)6 S (3)7 S (3)8 S DirectionAngle
6- 8
2- 8
4- 8 iscretizationmatrix We denotes ( ) d dk S as our extended contour templates, and upper corner mark d represents 3 kinds ofdirectional classes(i=1, 2, 3), under corner mark k represents 8 directional contour templates(k=1,2,…8). Where d ( ) ( , ) ( . [ ]) dk i j u S is seen a discrete estimate value i.e. (i j) ( ) ( ) k TV( ) ( . [ ]) || || di i d dk C u u , S . We can generate the correspondingcontour stencil by minimum value of d ( ) ( , ) ( . [ ]) dk i j u S . For a patch p, we denote corresponding contour stencils as cs .
3. P
ROPOSED M ETHOD
We introduce the proposed patch-based contour prior approach denoising algorithm for salt and pepper noise. Inthis approach, we firstly design a step of generating contour templets pixel by pixel. For each patch, there is acorresponding contour structure. Second, we use nearest-neighbors (NN) to find similar patches. Then, ouralgorithm use a regression method to repair central noise in patches. The basic flow of our algorithm is as follows:
Patch-based contour prior denoising algorithm input : noisy image V,output: denoised image V’,cut V into M*N patches for q from 1 to M*N do if the central pixel of patch P q is noise; for q1 form 1 to 28 do Compute the Equ.4;Return the minim value;Generate the contour stencils cs q . for q2 form 1 to M*Nuse the nearest-neighbors to find mm the most similar contour stencils;use regression method to repair the central noise in P q Return V’3.1 weighted Euclidean distance
In order to restore noise in the target contour structure, we need search for enough similar contour stencils. Itis well known that l distance is good measurement method between patches. But in this stance, all patchesgenerally contain large number of salt and pepper noise(maximum and minimum value), these noise will causegreat interference to the process of measurement. In order to avoid this problem, we use a probabilistic approachto compute distance avoid noise interference. The specific way is as follows:For a target patch P, we firstly compute P’s similarities with other patches Q in the whole noisy image scope.Suppose that the target P and Q contain n pixels, calculate the distance between two patches pixel by pixel, andthen summing up these distances with a weight w . The specific weights of w are generated by the use 0-1distribution. We can estimate roughly estimate the number of noise n n with classical identifier, and get therobability p of 0-1 distribution by n n /n. Denotes the distance of each pair of points as D p (P kk ,Q kk ),kk=1,2,..n. Wesorted the distances of D p (P kk ,Q kk ) from large to small. Obviously, the greater the distance, the more likely it isnoise. So, the first n n weithts’s value in the sequence are 0, and the others are 1/( n - n n ). Then we can compute theweighted Euclidean distance as follows: ( , ) n kk kk kkkk D P Q w P Q (5) According to the distance D(
P,Q ), we can use nearest-neighbors methods to find a set of similar patches R inthe global scope of noisy image. For a corrupted pixel (i,j) in patch P, we can get a set of most similar patches Q. We denotes the contourstencil of P as CS p , the contour stencil of Q as CS q . We adopt contour stencils to build filter because they containsmore priori information. In the regression framework, we can use these contour stencils to construct non localfiltering. Let the restored gray value of the corrupted pixel be ( f i,j) % , which is obtained by weighted averaging asfollows: ( ) (CS , ) ( )( ) kk p CSq kkq R f w f % (6) Here, any CS q belong to the similar patches R, (CS , ) p CSq w is the weight according to the contour stencil CS p andCS q . Weight (CS , ) p CSq w is calculated using the similarity (CS , ) p CSq s between each reference contour stencil and thetarget contour stencil, the corresponding formula is as follows: (CS , )(CS , ) (CS , )1 p CSqp CSq mm p CSq sw s (7)Here, mm is the number of similar contour stencils. The specific definition of (CS , ) p CSq s is as follow: (CS , ) CS p CSq p CSq s e (8) Here, is used to control strength of the filtering corrosion.
4. E
XPERIMENTS
Dataset
We choose some classical nature images as experimental dataset, it contains 11 commonly used images fordenoising, e.g., “Lena”, “Baboon” and “Pepper”, “Bark” etc.
Baseline algorithm
We use four existing denoising algorithms as baselines, including adaptive median filter (AMF),decision based algorithm (DBA[6]), method based on pixel density filter(BPDF)[27], Decision basedUnsymmetrical Trimmed Variants (DBUTVF)[28], Adaptive Center Weighted Median Filter(ACWMF)[32], andPatch-based Approach to Remove Impulse-Gaussian Noise(PARIGI[16]). For fair comparison, all methods’ codeimplementations are their publicly available versions and their parameters are set following the guidelines inriginal articles. The peak signal-to-noise ratio (PSNR) is adopted to measure the objective performance of ouralgorithm.
Evaluation Metric
The peak signal-to-noise ratio (PSNR) is adopted to measure the objective performance of ouralgorithm. PSNR is defined as follows.
MSE (9) Experiments were carried out on corrupted images with various noise densities (10%–90%). The denoisingresults of Image Lena are compared in Table 2.Table 2 PSNR scores result
Level AMF DBA BPDF ACWMF DBUTVF PARIGI Our algorithm
10% 38.61 41.50 40.04 39.19 41.50 37.65
20% 36.10 37.47 35.84 30.99 37.48 35.83
30% 33.79 34.81 33.09 24.08 34.85 33.93
40% 32.22 32.30 30.59 19.06 34.23 32.32
50% 30.39 30.11 28.13 15.28 30.14 29.91
60% 28.81 28.08 25.89 12.27 28.26 27.45
It can be seen that our algorithm obtain great superiority in low noise density condition. It not obtain the bestscores in high noise density condition because TV model no longer works.At the same time, visual comparisons are also provided in Figure 2, some details are enlarged in lower rightcorner of each picture.
10% noise level 30% noise level 50% noise level 70% noise levelesult for 10% noisy image Result for 30% noisy image Result for 50% noisy image Result for 70% noisy image
Fig. 2 Visual quality results of Image Lena
It can be seen that our algorithm obtains good visual effect. Due to TV model, our algorithm can protect edgeinformation better.
5. C
ONCLUSION
In this paper, an image denoising algorithm based on patch-based contour prior for salt and pepper noise is proposed.First, a discrete total variation model is introduced to extract contour structures. Second, a weighted Euclideandistance is designed to search the most similar patches, then, corresponding contour stencils are extracted fromthese similar patches; At the last, we build filter from contour stencils in the framework of regression. Numericalresults illustrate that the proposed method is competitive with the state-of-the-art methods in terms of the peaksignal-to-noise (PSNR) and visual effects.
ACKNOWLEDGEMENTS
This work is supported by the National Natural Science Foundation of China (NSFC) Grant No. 61702246, 41671439,Liaoning Province of China General Project of Scientific Research No. L2015285, Doctoral Start-up Foundation ofLiaoning Province No. 201601243. R EFERENCES [1] A. Buades, B. Coll, J.M. Morel,A non-local algorithm for image denoising, in Proc. IEEE Conf. Computer Vision and PatternRecognition, vol. 2, pp. 60-65. Oct. 2005[2] H. Hwang, R.A. Hadded, Adaptive median filter: new algorithms and results, IEEE Trans. Image Process. 4(4), (1995), 499-502.[3] M. Elad, M. Aharon,Image denoising via sparse and redundant representations over learned dictionaries, IEEE Trans. ImageProcess., vol. 15, no. 12, pp. 3736 – ––