Blind image separation based on exponentiated transmuted Weibull distribution
BBlind image separation based on exponentiated transmuted Weibull distribution
A. M. Adam Department of Mathematics, Faculty of Science, Zagazig University, P.O. Box Zagazig, Egypt. R. M. Farouk Department of Mathematics, Faculty of Science, Zagazig University, P.O. Box Zagazig, Egypt. M. E. Abd El-aziz Department of Mathematics, Faculty of Science, Zagazig University, P.O. Box Zagazig, Egypt
Abstract -In recent years the processing of blind image separation has been investigated. As a result, a number of feature extraction algorithms for direct application of such image structures have been developed. For example, separation of mixed fingerprints found in any crime scene, in which a mixture of two or more fingerprints may be obtained, for identification, we have to separate them. In this paper, we have proposed a new technique for separating a multiple mixed images based on exponentiated transmuted Weibull distribution. To adaptively estimate the parameters of such score functions, an efficient method based on maximum likelihood and genetic algorithm will be used. We also calculate the accuracy of this proposed distribution and compare the algorithmic performance using the efficient approach with other previous generalized distributions. We find from the numerical results that the proposed distribution has flexibility and an efficient result. Keywords- Blind image separation, Exponentiated transmuted Weibull distribution, Maximum likelihood, Genetic algorithm, Source separation, FastICA. I. I NTRODUCTION
Recently the blind source separation (BSS) has more attention because it can be considered as an advanced image/signal processing technique and has many applications such as: speech sound, image, communication, and biomedicine [1β4]. BSS aims to recover source (images/signals) from a mixture with little known information. There are many BSS algorithms that have been discussed from various viewpoints, including principle component analysis (PCA) [9], maximum likelihood [7], mutual information minimization [6], tensors [8], non-Gaussianity [5], and neural networks [10-12]. Regarding to BSS, the separation and optimization methods play the most important roles. Separation step is used as the measurement of separability and optimization step is used to get the optimum solution for the objective function which we get from separation mechanism. Using generalized distributions usually gives good results of blind separation due to the variant properties of its sub-models. In the independent component analysis (ICA) framework, accurately estimates the statistical model of the sources is still an open and challenging problem [2]. Practical BSS scenarios employ difficult source distributions and even situations where many sources with variant probability density functions (pdf) mixed together. Towards this direction, many parametric density models have been made available in recent literature. For examples of such models, the generalized Gaussian density
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016434 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
GGD) [13], the generalized gamma density (GGD) [14], and even combinations and generalizations such as super and generalized Gaussian mixture model (GMM) [15], the Pearson family of distributions [16], the generalized alfa-beta distribution (AB-divergences) [17] and even the so-called extended generalized lambda distribution (EGLD) [18] which is an extended parameterizations of the aforementioned generalized lambda distribution (GLD) and generalized beta distribution (GBD) models [19]. In this paper, we have presented the exponentiated transmuted Weibull distribution (ETWD) which is a generalization of the Weibull distribution. We have evaluated the accuracy of our proposed ETWD and compare the algorithmic performance using many different previous distributions. The numerical results, shows that the ETWD give a good results comparing with many different cases. The rest of this paper is organized as follows: In section 2, we present the BSS model. In section 3, we will discuss the ETWD. In section 4, we will use maximum likelihood to estimate the parameters of ETWD based on genetic algorithm. Finally, we will present the computational efficient performance of our proposed technique. II. B LIND SOURCE SEPARATION (BSS)
MODEL
Let
S(t) = [s (t), s (t), . . . , s N (t)] T (t = 1, 2, . . . , l) denote independent source image vector that comes from N image sources. We can get observed mixtures X(t) = [x (t), x (t), . . . , x K (t)] T (N = K) under the circumstances of instantaneous linear mixture. X(t) = AS(t), (1) where
π is a N Γ N mixing matrix. The task of the BSS algorithm is to recover the sources from mixtures x(t) by using
U(t) = WX(t), (2) where
π is a N Γ N separation matrix and
U(t) = [u (t), u (t), . . . , u N (t)] T is the estimate of N sources. Often sources are assumed to be zero-mean and unit-variance signals with at most one having a Gaussian distribution. To solve the problem of source estimation the un-mixing matrix W must be determined. In general, the majority of BSS approaches perform ICA, by essentially optimizing the negative log-likelihood (objective) function with respect to the un-mixing matrix W such that L(u, W) = β E[log p ul (u l )] β log|det(W)| Nl=1 , (3) where
E[. ] represents the expectation operator and p u1 (u ) is the model for the marginal pdf of u l , for all l = 1,2, β¦ , N . In effect, when correctly hypothesizing upon the distribution of the sources, the maximum likelihood (ML) principle leads to estimating functions, which in fact are the score functions of the sources International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016435 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 l (u l ) = β ddu l log p ul (u l ) (4) In principle, the separation criterion in (3) can be optimized by any suitable ICA algorithm where contrasts are utilized (see; e.g., [2]). The FastICA [3], based on W k+1 = W k + D(E[Ο(u)u T ] β diag(E[Ο l (u l )u l ]))W k , (5) where, as defined in [4], D = diag ( 1E[Ο l (u l )u l ] β E[Ο lβ² (u l )]) , (6) where Ο(t) = [Ο (u ), Ο (u ), β¦ , Ο n (u n )] T , valid for all l = 1, 2, β¦ , n . In the following section, we propose ETWD for image modeling. III. E XPONENTIATED TRANSMUTED W EIBULL DISTRIBUTION (ETWD) Following [20] ETWD is a new generalization of the two parameters Weibull distribution. The pdf of ETWD is defined as: π(π₯) = ππ½πΌ (π₯ π πΌ ) π½β1 π β(π₯ π πΌ ) π½ [1 β π + 2ππ β(π₯ π πΌ ) π½ ] Γ [1 + (π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ ] πβ1 (7) cumulative distribution function of ETWD is given by: πΉ(π₯) = {1 + (π β 1)π β(π₯πΌ) π½ β ππ β2(π₯πΌ) π½ } π π₯ β₯ 0 , (8) where Ξ±, Ξ² > 0 , and |Ξ»| β€ 1 are the scale, shape and transmuted parameters, respectively. It is clear that the ETWD is very flexible. This is so since there are many several other distributions that can be considered as special cases of ETW, by selecting the appropriate values of the parameters. These special cases include eleven distributions as shown in Table (I). In Figure (1-4) there are several distributions generated from ETWD by changing the parameters. IV. E STIMATION OF THE PARAMETERS
To estimate the parameters of ETWD, the maximum likelihood is used. Let X , X β¦ , X n be a sample of size N from an ETWD. Then the log-likelihood function ( β ) is given by: β = log β = log (β [ππ½πΌ (π₯ π πΌ ) π½β1 π β(π₯ π πΌ ) π½ [1 β π + 2ππ β(π₯ π πΌ ) π½ ] Γ [1 + (π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ ] πβ1 ] ππ=1 ) (9) International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016436 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 herefore, maximum likelihood estimation of Ξ±, Ξ², Ξ» and Ξ½ are derived from the derivatives of β . They should satisfy the following equations: πβππΌ = 0, βββΞ» = 0 , βββΞ² = 0 , βββΞ½ = 0 πβππΌ = β ππ½πΌ + π½ β (π₯ π πΌ ) π½β1ππ=1 + β 2ππ β(π₯ π πΌ ) π½ π½ (π₯ π πΌ ) π½β1 ( π₯ π πΌ )2ππ β(π₯ π πΌ ) π½ β π + 1 ππ=1 + (π β 1)Γ β (π β 1)π β(π₯ π πΌ ) π½ π½ (π₯ π πΌ ) π½β1 ( π₯ π πΌ ) β 2ππ β2(π₯ π πΌ ) π½ π½ (π₯ π πΌ ) π½β1 ( π₯ π πΌ )(π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ + 1 ππ=1 (10) πβππ½ = ππ½ + β log(π₯ π ) ππ=1 β π log πΌ β β (π₯ π πΌ ) π½ log (π₯ π πΌ ) ππ=1 + β β2ππ β(π₯πΌ) π½ (π₯πΌ) π½ log (π₯ π πΌ )2ππ β(π₯ π πΌ ) π½ β π + 1 ππ=1 + (π β 1)Γ β β(π β 1)π β(π₯ π πΌ ) π½ (π₯ π πΌ ) π½ log (π₯ π πΌ ) + 2ππ β2(π₯ π πΌ ) π½ (π₯ π πΌ ) π½ log (π₯ π πΌ )(π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ + 1 ππ=1 (11) πβππ = β 2π β(π₯ π πΌ ) π½ β 12ππ β(π₯ π πΌ ) π½ β π + 1 ππ=1 + (π β 1) Γ β π β(π₯ π πΌ ) π½ β π β2(π₯ π πΌ ) π½ (π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ + 1 ππ=1 (12) πβππ = β log [(π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ + 1] ππ=1 + ππ (13) To estimate the value of parameters, the system of equations (10-13) must be solved. However, it is difficult to solve this system so, the genetic algorithm (GA) [21-22] will be used as an alternative numerical method to estimate the parameters. The appeal of the GA optimization technique lies in the fact that it can minimize the negative of the log-likelihood objective function in (3), essentially without depending on any derivative information.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016437 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 able I
The ETWD sub-models, shows the specific values of the parameters used to generate the above mentioned eleven special cases, Where Ξ± > 0, Ξ² >0, Ξ½>0, |Ξ»|β€1 Ξ² = 2 Ξ² = 1 Ξ½ = 1 Ξ² = 2 , Ξ½ = 1
Exponentiated transmuted Rayleigh (ETR) Exponentiated transmuted exponential (ETE) Transmuted Weibull (TW) Transmuted Rayleigh (TR) Ξ² = 1 , Ξ½ = 1 Ξ» = 0 Ξ² = 2 , Ξ» = 0 Ξ² = 1 , Ξ» = 0
Transmuted exponential (TE) Exponentiated Weibull (EW) Exponentiated Rayleigh (ER) Exponentiated exponential (EE) Ξ» = 0 , Ξ½ = 1 Ξ² = 2, Ξ» = 0, Ξ½ = 1 Ξ² = 1, Ξ» = 0, Ξ½ = 1
Weibull (W) Rayleigh (R) Exponential (E)
Figure 1. The ETWD with fixed Ξ±=3.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016438 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 2. The ETWD with fixed π =2. Figure 3. The ETWD with fixed Ξ»=0.5.
Figure 4. The ETWD with fixed Ξ½=2.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016439 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 . N UMERICAL RESULTS
Numerical experiments have shown that the GA method can converge to an acceptably accurate solution with substantially fewer function evaluations. We have generated random number from ETWD with parameters Ξ±, Ξ², Ξ½ and Ξ» . By performing GA, we obtain best estimation of parameters as in table (II). Applications of ETWD for BSS
We resolve to FastICA algorithm for blind signal separation (BSS). This algorithm depends on the estimated parameters and an un-mixing matrix W which estimated by FastICA algorithm. By substituting (7) into (4) for the source estimates u l , l = 1, 2, . . . , n , it quickly becomes clear that the proposed score function inherits a generalized parametric structure, which can be attributed to the highly flexible ETWD parent model. So, a simple calculus yields the flexible BSS score function π π (π’ π ) = β πππ’ π log ππ½πΌ (π₯ π πΌ ) π½β1 π β(π₯ π πΌ ) π½ [1 β π + 2ππ β(π₯ π πΌ ) π½ ] Γ [1 + (π β 1)π β(π₯ π πΌ ) π½ β ππ β2(π₯ π πΌ ) π½ ] πβ1 (14) In principle Ο l (u l |ΞΈ) is capable of modeling a large number of signals as well as various other types of challenging heavy- and light-tailed distributions. Experiments were done to investigate the performance of our method through three applications (two in source separation and one in image denoising) when impulsive noise is presented. In all experiments, the performance of our method is compared with generalized gamma [14], tanh, skew, pow3 [23], and Gauss [15]. Our performance is measured by the peak-signal-to- noise ratio (PSNR), defined as: ππππ = 20 πππ ( 255πππΈ) (15) Table II Parameter estimation by using GA π π π π πΜ πΜ πΜ πΜ Err X1 0.5 2 3 4 0.59 1.86 2.97 4.11 0.02 X2 1 2.5 5.2 6.8 1.16 2.42 5.27 6.80 0.06 X3 3 5.7 1.9 8.2 2.98 5.63 1.98 8.12 0.006
Example We have run the algorithm using natural images taken from [24]. We selected 4 noise-free natural images with 512Γ512 pixels. Further, to reduce the dimension of input image data, the data set X is centered and whitened by principal component analysis (PCA) method. Then, using the updating rules of W defined in (5), the objective function given in (14) is minimized. Where Figure (5-6) show the original, mixed and separated images by Gauss, International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016440 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 ow3, skew, tanh, generalized gamma, and ETWD algorithms. Also, Table (III) illustrates the performance of these algorithms. From this table and Figure (5-6), the ETWD is higher performance than other algorithms.
Table III Image separation PSNR Distribution / PSNR First Image Second Image Third Image Forth Image Elapsed time (in seconds) MSE PSNR MSE PSNR MSE PSNR MSE PSNR Gauss 0.1176 57.4255 0.2972 53.4009 0.1773 55.6426 0.1314 56.9444 8.757703 Pow3 0.1375 56.7477 0.2130 54.8477 0.1736 55.7363 0.1259 57.1320 24.921161 Skew 0.0044 71.7366 0.0177 65.6481 0.2340 54.4378 0.2193 54.7209 5.788523 Tanh 0.1179 57.4172 0.1647 55.9628 0.1810 55.5538 0.0741 59.4309 6.852007 Generalized Gamma 0.1341 56.8571 0.2659 53.8840 0.1865 55.4237 0.1305 56.9746 4.333974 ETWD 0.0011 77.6298 0.0159 66.1132 0.0026 73.9429 0.0015 76.2714 4.285013
Example In this example, we illustrate the performance of our algorithm to denoise medical images taken from [25]. Where Figure (7-12) show the original images, noised images, and denoised images by different algorithms. After applying algorithms of Gauss, pow3, skew, tanh, generalized gamma and, our algorithm ETWD, the results are illustrated in Figure (7- 12), also Table (IV) illustrates the performance of these algorithms. From table (IV) and Figure (7-12), the ETWD is higher performance than other algorithms.
Table IV Denoising PSNR Distribution / PSNR First Image (Medical) Second Image (Medical) Elapsed time (in seconds) MSE PSNR MSE PSNR Gauss 0.0092 68.4753
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016441 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 5. A original images, B mixed images, C Gauss separated images, and D pow3 separated images.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016442 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 6. E skew separated images, F tanh separated images, G generalized gamma separated images, and H ETWD separated images.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016443 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 7. Medical image denoising using Gauss filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
Figure 8. Medical image denoising using pow3 filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016444 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 9. Medical image denoising using Skew filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
Figure 10. Medical image denoising using tanh filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016445 https://sites.google.com/site/ijcsis/ ISSN 1947-5500
Figure 11. Medical image denoising using generalized gamma filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
Figure 12. Medical image denoising using ETWD filter: A, D are the source images, B, E are the noised images, C, F are the denoised images.
International Journal of Computer Science and Information Security (IJCSIS), Vol. 14, No. 3, March 2016446 https://sites.google.com/site/ijcsis/ ISSN 1947-5500 I. C ONCLUSION
In this paper, we introduced a new technique for blind image separation and image denoise based on exponentiated transmuted Weibull distribution. Our proposed technique outperforms existing solutions in terms of separation quality and computational cost. When the GA is used to estimate the parameters of ETWD and it gives small error. Also the results of ETWD are better than other algorithms. R
EFERENCES [1]
Y. Zhang and Y. Zhao, 2013. Modulation domain blind speech separation in noisy environments, Speech Communication, vol. 55, no. 10, pp. 1081β1099. [2]
M. T. Β¨ Ozgen, E. E. KuruoΛglu, and D. Herranz, 2009. Astrophysical image separation by blind time-frequency source separation methods, Digital Signal Processing, vol. 19, no. 2, pp. 360β369. [3]
Ikhlef, K. Abed-Meraim, and D. Le Guennec, 2010. Blind signal separation and equalization with controlled delay for MIMO convolutive systems, Signal Processing, vol. 90, no. 9, pp. 2655β 2666. [4]
R. Romo VΒ΄azquez, H. VΒ΄elez-PΒ΄erez, R. Ranta, V. Louis Dorr, D. Maquin, and L. Maillard, Blind source separation, 2012. Wavelet denoising and discriminant analysis for EEG artifacts and noise cancelling, Biomedical Signal Processing and Control, vol. 7, no. 4, pp. 389β400. [5]
M. Kuraya, A. Uchida, S. Yoshimori, and K. Umeno, 2008. Blind source separation of chaotic laser signals by independent component analysis, Optics Express, vol. 16, no. 2, pp. 725β730. [6]
M. Babaie-Zadeh and C. Jutten, 2005. A general approach formutual information minimization and its application to blind source separation, Signal Processing, vol. 85, no. 5, pp. 975β995. [7]
K. Todros and J. Tabrikian, 2007. Blind separation of independent sources using Gaussian mixture model, IEEE Transactions on Signal Processing, vol. 55, no. 7, pp. 3645β3658. [8]
P. Comon, 2014. Tensors: a brief introduction, IEEE Signal Processing Magazine, vol. 31, no. 2, pp. 44β53. [9]
E. Oja and M. Plumbley, April 2003. Blind separation of positive sources using nonnegative PCA, in Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation (ICA β03), Nara, Japan, pp. 11β16. [10]
W. L. Woo and S. S. Dlay, 2005. Neural network approach to blind signal separation of mono-nonlinearly mixed sources, IEEE Transactions on Circuits and Systems I, vol. 52, no. 6, pp. 1236β1247. [11]
Cichocki and R. Unbehauen, 1996. Robust neural networks with on-line learning for blind identification and blind separation of sources, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, vol. 43, no. 11, pp. 894β906. [12]
S.-I. Amari, T.-P. Chen, and A. Cichocki, 1997. Stability analysis of learning algorithms for blind source separation, Neural Networks, vol. 10, no. 8, pp. 1345β1351. [13]
K. Kokkinakis and A. K. Nandi, 2005. Exponent parameter estimation for generalized Gaussian probability density functions with application to speech modeling, Signal Processing, vol. 85, no. 9, pp. 1852β1858. [14]
E. W. Stacy, 1962. A generalization of the gamma distribution, Annals of Mathematical Statistics, vol. 33, no. 3, pp. 1187β1192. [15]
J. A. Palmer, K. Kreutz-Delgado, and S. Makeig, March 2006. Super-Gaussian mixture source model for ICA, in Proceedings of the International Conference on Independent Component Analysis and Blind Signal Separation, Charleston, SC, USA pp. 854β861. [16]
J. Eriksson, J. Karvanen, and V. Koivunen, 2002. Blind separation methods based on Pearson system and its extensions, Signal Processing, vol. 82, no. 4, pp. 663β673. [17]
Sarmiento, I. DurΓ‘n-DΓaz, A. Cichocki, and S. Cruces, 2015. A contrast based on generalized divergences for solving the permutation problem of convolved speech mixtures, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 23, no. 11, pp. 1713-1726. [18]
J. Karvanen, J. Eriksson, and V. Koivunen, 2002. Adaptive Score Functions for Maximum Likelihood ICA, The Journal of VLSI Signal Processing, Vol. 32, no 1-2, PP 83-92. [19]
J. Karvanen, J. Eriksson, and V. Koivunen, June 2000. Source distribution adaptive maximum likelihood estimation of ICA model, in Proceedings of the 2nd International Conference on ICA and BSS, Helsinki, Finland, pp. 227β 232. [20]
A. N. Ebraheim, 2014. Exponentiated transmuted Weibull distribution a generalization of the Weibull distribution, International Journal of Mathematical, Computational, Physical and Quantum Engineering Vol. 8 No. 6. [21]
M. Li and J. Mao, June 2004. A new algorithm of evolutional blind source separation based on genetic algorithm, in Proceedings of the 5th World Congress on Intelligent Control and Automation, Hangzhou, Zhejiang, China, pp. 2240β2244. [22]
S. Mavaddaty and A. Ebrahimzadeh, December 2009. Evaluation of performance of genetic algorithm for speech signals separation, in Proceedings of the International Conference on Advances in Computing, Control and Telecommunication Technologies (ACTβ09), Trivandrum, Kerala, India, pp. 681β683. [23]
A. Hyvarinen, J. Karhunen, and E. Oja, 2001. Independent Component Analsysis, JohnWiley & Sons. [24]