Feasibility study on the least square method for fitting non-Gaussian noise data
aa r X i v : . [ s t a t . O T ] J un Feasibility study on the least square method for fittingnon-Gaussian noise data
Wei Xu, Wen Chen, Yingjie LiangInstitute of Soft Matter Mechanics, College of Mechanics and Materials, HohaiUniversity, Nanjing, ChinaState Key Laboratory of Hydrology-Water Resources and HydraulicEngineering
Corresponding authors :Wen Chen, Email address: [email protected] Liang, Email address: [email protected]
Abstract:
This study is to investigate the feasibility of least square methodin fitting non-Gaussian noise data. We add different levels of the two typicalnon-Gaussian noises, L´evy and stretched Gaussian noises, to exact value of theselected functions including linear equations, polynomial and exponential equa-tions, and the maximum absolute and the mean square errors are calculated forthe different cases. L´evy and stretched Gaussian distributions have many appli-cations in fractional and fractal calculus. It is observed that the non-Gaussiannoises are less accurately fitted than the Gaussian noise, but the stretched Gaus-sian cases appear to perform better than the L´evy noise cases. It is stressedthat the least-squares method is inapplicable to the non-Gaussian noise caseswhen the noise level is larger than 5%.
Keywords:
Least square method, non-Gaussian noise, L´evy distribution, stretchedGaussian distribution, least square fitting
1. Introduction
Non-Gaussian noise is universal in nature and engineering. − In recentdecades, non-Gaussian noise has widely been studied, especially in signal de-tection and processing, − theoretical model analysis, and error statistics. Itis known that the Gaussian distribution is the mathematical precondition to usethe least square method. However, it is often directly used to process such non-Gaussian noise data, which may give wrong estimation. Thus, this study is toquantitatively examine the applicability of the least square method to analyzenon-Gaussian noise data.Generally, non-Gaussian noise has detrimental influence on the stability ofpower system, and also can stimulate systems to generate ordered patterns. − To our best knowledge, L´evy and stretched Gaussian noises are two kinds oftypical non-Gaussian noise, which are frequently used in fractional and fractalcalculus. − L´evy noise has been observed in many complex systems, such1s turbulent fluid flows, signal processing, financial times series, − neuralnetworks. We also note that the parameters estimation for stochastic differen-tial equations driven by small L´evy noise were investigated. Compared to theL´evy noise the stretched Gaussian noise is less studied, but its correspondingstretched Gaussian distribution has been explored, such as in the motion offlagellate protozoa, SoL interchange turbulence simulation, anomalous diffu-sion of particles with external force, not mentioned too much. It also shouldbe pointed that processing of non-sinusoidal signals or sound textures has be-come an important research topic, and the derived algorithms significantlyimprove the perceptual quality of stretched noise signals. It is well known that the least square method is a standard regression ap-proach to approximate the solutions of over determined systems, which is mostfrequently used in data fitting and estimation. The core concept of the leastsquare method is to identify the best match for the system by minimizing thesquare error. Supposed that the data points are ( x , y ) , ( x , y ) ( x , y ) , ..., ( x , y ) , where x represents the independent variable and y is the dependent variable.The fitting error d characterizes the distance between y and the estimatedcurve f ( x ), i.e., d = y − f ( x ) , ..., d n = y n − f ( x n). The best fitting curve isto minimize the square error δ = d + d + ... + d n = n P i =1 [ y i − f ( x i )] , wherethe errors d i are usually modeled by Gaussian distribution. Field data are often polluted by noise and the Gaussian noise is the classicalone, whose probability density function obeys Gaussian distribution. We havementioned above that several types of noise data obey non-Gaussian distribution. To examine the feasibility of the least square method in fitting non-Gaussiannoise data, we generate the non-Gaussian random numbers as the noise, andthen add different levels of the noise to the exact values of the selected func-tions including linear equations, polynomial and exponential equations as theobserved values. By using the least square method, the maximum absoluteand mean square errors are calculated and compared in the Gaussian and non-Gaussian applications.The rest of the paper is organized as follows. In Section 2, we introduce theGaussian distribution, L´evy distribution, stretched Gaussian distribution, andthe methods we use to analyze the noise data. In Section 3, we give the resultsand discussion. Finally, a brief summary is provided.
2. Theory and methods
Gaussian distribution is also called as normal distribution, which is oftenencountered in mathematics, physics and engineering. The probability densityfunction of Gaussian distribution is: f ( x ; µ, σ ) = 1 √ πσ exp − ( x − µ ) σ ! , (1)2here µ and σ respectively represent mean and standard deviation. When µ = 0and σ = 1, it degenerates into the standard Gaussian distribution. Figure1 givesfour different cases of Gaussian density function. −5 0 500.10.20.30.40.50.60.70.8 x p r ob a b ilit y d e n s it y f un c ti on f( x ) µ =0 σ =0.2 µ =0 σ =1 µ =0 σ =5 µ =−2 σ =0.5 Figure 1. The probability density functions of Gaussian distribution.
L´evy distribution, named after Paul L´evy, is a rich class of probability dis-tributions. The Gaussian and Cauchy distributions are its special cases. It isusually defined by its characteristic function φ α,β ( k ; µ, σ ). φ α,β ( k ; µ, σ ) = F { f α,β ( k ; µ, σ ) } = exp (cid:20) iµk − σ α | k | α (cid:18) − iβ k | k | ω ( k, α ) (cid:19)(cid:21) , (2)where ω ( k, α ) = (cid:26) tan πα ( α = 1 , < α < − π ln | k | ( α = 1) , (3)stability index 0 < α ≤
2, skewness parameter − ≤ β ≤
1, scale parameter σ >
0, and location parameter µ ∈ ℜ . α and β respectively determine theproperties of asymptotic decay and symmetry. The standard L´evy distributioncan be obtained by the following transformation. p α,β ( k ; µ, σ ) = 1 σ p α,β ( x − µσ ; 0 , . (4)When β = 1 , µ = 0, the probability density function of the L´evy distributionis stated as: f ( x ; µ, c ) = r c π (cid:18) − e − c x x . (cid:19) , (5)where x ≥ µ is the location parameter and c is the scale parameter. Differentcases of Eq.(5) are illustrated in Figure 2.3 x p r ob a b ilit y d e n s it y f un c ti on f( x ) c=0.5c=1c=2c=4c=8 Figure 2. The probability density functions of L´evy distribution.
The stretched Gaussian distribution has widely been used to describe anoma-lous diffusion and turbulence, especially in the porous media with fractal structure. The solution to the fractal derivative equation in characterizing the fractal me-dia has the form of stretched Gaussian distribution, whose probability densityfunction is defined as: f β ( x ) = β /β Γ (1 /β ) σ exp − (cid:12)(cid:12)(cid:12)(cid:12) x − aσ (cid:12)(cid:12)(cid:12)(cid:12) β ! , ( −∞ < x < ∞ , β > . (6)where β is the stretched exponent. When β = 2 and a = 0, it becomes tothe standard Gaussian distribution. Figure 3 shows three cases of stretchedGaussian density function. −15 −10 −5 0 5 10 1510 −35 −30 −25 −20 −15 −10 −5 x p r ob a b ilit y d e n s it y f un c ti on f( x ) β =1.5 β =2.0 β =2.5 Figure 3. The probability density functions of stretched Gaussiandistribution.
4n this study, the noise data are obtained based on the above mentionedGaussian and non-Gaussian random variables, which can be generated by usingthe inverse function method and the selection method. Specifically, Cham-bers, Mallows, Stuck proposed the CMS method in L´evy random variablessimulation, which is the fastest and most accuracy method. By using theCMS method, some variables need to be defined first. V = π (cid:18) U − (cid:19) ,W = − ln U ,L = n β tan (cid:16) πα (cid:17)o / α ,θ = 1 α arctan (cid:16) β tan (cid:16) πα (cid:17)(cid:17) , where U and U are two independent uniform distribution on interval (0 , α = 1, the L´evy random number is X = L sin { α ( V + θ ) }{ cos ( V ) } /α . (cid:20) cos ( V − α ( V + θ )) W (cid:21) (1 − α ) /α . (7)When α = 1, X = 2 π (cid:26)(cid:16) π βV (cid:17) tan V − β ln (cid:18) π W cos V π + βV (cid:19)(cid:27) . (8)The general L´evy random numbers can be obtained based on some knownproperties. For the stretched Gaussian distribution, we use the acceptance rejectionmethod to generate its random numbers. Both linear and nonlinear functions estimation are considered by using theleast square method, in which the model function f : R m → R is estimated as y i = f ( x i ) + ε i ( i = 1 , , ..., n ) , (9)where n is the number of observations, y i ∈ R the response variable, x i ∈ R m the explanatory variable, the noise ε i = rand × a % ( a = 1 , , , , . (10)In this study, we consider the case n = 200, then the observed values y , y , ..., y can be constructed by adding the values of the random numbers to the exact5alues of the selected functions including linear equations, polynomial and ex-ponential equations, finally the maximum absolute error and the mean squareerror are calculated for the above different cases in conjunction with the leastsquare method.We give the following abbreviations in noise date processing for convenience:FA: Gaussian noise least square error fitting, µ = 5 , σ = 0 . . FB: L´evy noise least square error fitting, α = 1 . , β = 0 , µ = 1 , σ = 0 . FC: Stretched Gaussian noise least square error fitting, β = 2 . , a = 1 , σ = 3 . Rerr1: Maximum absolute error: max | F ( x i ) − f ( x i ) | .Rerr2: Mean square error: s n P i =1 { F ( x i ) − f ( x i ) } n .
3. Results and discussion
In this section, we apply the least square method to fit various noise-polluteddata by adding different levels of Gaussian and non-Gaussian noise to exactvalues of the selected functions including linear equations, polynomial and ex-ponential equations, and give a brief discussion.a) The simplest typical model is the linear function. f ( x ) = ax + b, ( a = 0) . (11)Here we select the following case as example: f ( x ) = 5 x. (12)Tables 1 to 5 give the estimated parameters and the errors for five differentlevels of noise in the linear case. We can observe that the Gaussian noise fit-ting data maximum absolute error is in the range of (0 . , . . e − , . e − Rerr
F A ) < Rerr F C ) < Rerr F B ) , (13) Rerr
F A ) < Rerr F C ) < Rerr F B ) . (14)The corresponding fitting curves are depicted in Figures 4-8. We can findthat the results of Gaussian noise fitting have the best accuracy, and the stretchedGaussian noise fitting curves are closer to those of the Gaussian noise comparedwith the results of L´evy noise data fitting.Table 1. The estimated results for 1% noise in the linear case. function a b Rerr1 Rerr2f FA . . . . e − FB . − . . . e − FC . − . . . e − .048 0.0485 0.049 0.0495 0.05 0.05050.2360.2380.240.2420.2440.2460.2480.250.2520.2540.256 X Y fFAFBFC Figure 4. Linear model fitting to 1% noise.
Table 2. The estimated results for 5% noise in the linear case. function a b
Rerr1 Rerr2f FA . − . . . e − FB . . . . e − FC . − . . . e − y fFAFBFC Figure 5. Linear model fitting to 5% noise.
Table 3. The estimated results for 10% noise in the linear case. function a b
Rerr1 Rerr2f FA . − . . . e − FB . . . . e − FC . − . . . e − .04 0.042 0.044 0.046 0.048 0.050.180.190.20.210.220.230.240.250.26 x y fFAFBFC Figure 6. Linear model fitting to 10% noise.
Table 4. The estimated results for 15% noise in the linear case.function a b
Rerr1 Rerr2f FA . . . . e − FB . . . . e − FC . − . . . e − y fFAFBFC Figure 7. Linear model fitting to 15% noise.
Table 5. The estimated results for 20% noise in the linear case. function a b
Rerr1 Rerr2f FA . . . . e − FB . . . . e − FC . − . . . e − y fFAFBFC Figure 8. Linear model fitting to 20% noise. b) A polynomial can be constructed by means of addition, multiplicationand exponentiation to a non-negative power, which is usually written as thefollowing form with a single variable x , f ( x ) = a n x n + a n − x n − + ... + a x + a x + a , (15)where a , a , ..., a n − , a n are constants. We select three parameters polynomialfunction. F ( x ) = ax + bx + c ( a = 0) . (16)Here the following case is used as example: y = 4 x + 3 x + 2 ( a = 0) . (17)Tables 6 to10 give the estimated parameters and the errors for five differ-ent levels of noise in the polynomial case. The corresponding fitting curvesare depicted in Figures 9-13. Gaussian noise fitting maximum absolute error isin the range of (0 . , . . e − , . e − function a b c Rerr1 Rerr2f FA . . . . . e − FB . . . . . e − FC . . . . . e − .1055 0.106 0.1065 0.107 0.1075 0.108 0.10853.2583.263.2623.2643.2663.268 x y fFAFBFC Figure 9. Polynomial model fitting to 1%noise.
Table 7. The estimated results for 5% noise in the polynomial case. function a b c
Rerr1 Rerr2f FA . . . . . e − FB . . . . . e − FC . . . . . e − y fFAFBFC Figure 10. Polynomial model fitting to 5%noise.
Table 8. The estimated results for 10% noise in the polynomial case. function a b c
Rerr1 Rerr2f FA . . . . . e − FB . . . . . e − FC . . . . . e − .06 0.07 0.08 0.09 0.13.123.143.163.183.23.223.243.263.283.3 x y fFAFBFC Figure 11. Polynomial model fitting to 10%noise.
Table 9. The estimated results for 15% noise in the polynomial case. function a b c
Rerr1 Rerr2f FA . . . . . e − FB . . . . . e − FC . . . . . e − y fFAFBFC Figure 12. Polynomial model fitting to 15%noise.
Table 10. The estimated results for 20% noise in the polynomial case. function a b c
Rerr1 Rerr2f FA . . . . . e − FB . . . . . e − FC . . . . . e − .08 0.1 0.12 0.14 0.163.13.153.23.253.33.353.43.45 x y fFAFBFC Figure 13. Polynomial model fitting to 20%noise. c) Non-linear equations can be divided into two categories, one is polynomialequation, and the other is non-polynomial equation. In this part, we select thefour parameters exponential function. F ( x ) = ae x + be − t − ct + d ( a = 0) . (18)Here the following case is used as example: f ( x ) = 0 . e x + 0 . e − t − t + 13 . (19)Tables 11 to 15 give the estimated parameters and the errors for five differ-ent levels of noise in the exponential function case. The corresponding fittingcurves are shown in Figures 14-18. The Gaussian noise fitting data maximumabsolute error is in the range of (0 . , . . e − , . e − function a b c d Rerr1 Rerr2f . . / FA . . . − . . . e − FB . . . . . . e − FC . . . . . . e − .206 0.208 0.21 0.212 0.214 0.2161.2221.2241.2261.2281.231.2321.2341.2361.2381.241.242 x y fFAFBFC Figure 14. Exponential model fitting to 1% noise data.
Table 12. The estimated results for 5% noise in the exponential case. function a b c d
Rerr1 Rerr2f . . / FA . . − . − . .
002 2 . e − FB . − . . . . . e − FC . − . . . . . e − x y fFAFBFC Figure 15. Exponential model fitting to 5% noise data.
Table 13 The estimated results for 10% noise in the exponential case. function a b c d
Rerr1 Rerr2f . . / FA . − . . . . . e − FB . . − . − . . . e − FC . − . . . . . e − .08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.241.051.11.151.21.251.31.35 x y fFAFBFC Figure 16. Exponential model fitting to 10% noise.
Table 14. The estimated results for 15% noise in the exponential case. function a b c d
Rerr1 Rerr2f . . / FA . . − . − . . . e − FB . − . . . . . e − FC . . − . − . . . e − x y fFAFBFC Figure17. Exponential model fitting to 15% noise.
Table 15 The estimated results for 20% noise in the exponential case. function a b c d
Rerr1 Rerr2f . . / FA . . − . − . . FB . . − . − . . . e − FC . − . . . . . e − .05 0.1 0.15 0.20.90.9511.051.11.151.21.251.31.351.4 x y fFAFBFC Figure 18. Exponential model fitting to 20% noise.
To summarize all the above results, we can find that the maximum absoluteand the mean square errors for the Gaussian noise cases are the smallest, butthe values for the L´evy noise cases are the biggest, i.e.,
Rerr
F A ) < Rerr F C ) < Rerr F B ) , (20) Rerr
F A ) < Rerr F C ) < Rerr F B ) . (21)It can be observed from Figures 4 to 18, that the results of Gaussian noisefitting have the best accuracy, and the stretched Gaussian noise fitting curvesare closer to those of the Gaussian noise compared with the results of L´evy noisedata fitting. Thus, the least square method is less accurate when it is applied tothe non-Gaussian noise data fitting compared with the cases of Gaussian noise,especially when the noise level is larger than 5%.This study mainly verifies the least square method is inapplicable to non-Gaussian noise when the noise level is high. To extend the results in morecomplicated systems, a mathematical proof to the conclusion should be derivedin future study.The second goal of our further work is to modify the least square methodin fitting non-Gaussian noises. Actually the core concept of the least squaremethod is to minimize the square error δ = n P i =1 [ y i − f ( x i )] , i.e., in the linearcase, δ = n X i =1 [ yi − ax − b ] , (22)to compute the minimum value of Eq. (22), the main task is to set the first-orderderivatives of the parameters to be zero. (cid:26) ∂δ∂a = 0 ∂δ∂b = 0 . (23)The solutions of Eq. (23) are the target values of the parameters a and b .Combining our previous work on fractional and fractal derivatives, , we canemploy the fractional and fractal derivatives to generalize Eq. (23), and the15orresponding fitting errors can be defined by using the following power lawtransform: ˆ x = x β . (24)
4. Conclusions
This study examines the feasibility of least square method in fitting variousnoise data polluted by adding different levels of Gaussian and non-Gaussiannoise to exact values of the selected functions including linear equations, poly-nomial and exponential equations. The maximum absolute error and the meansquare error are calculated and compared for the different cases. Based on theforegoing results and discussions, the following conclusions can be drawn:1. The fitting results for the non-Gaussian noise are less accurate than thoseof the Gaussian noise, but the stretched Gaussian cases appear to perform betterthan the L´evy noise cases.2. The least-squares method is inapplicable to the non-Gaussian noise datawhen the noise level is larger than 5%.3. A theoretical proof and improved least mean square methods for non-Gaussian noise data are under intense study.
Acknowledgments
This paper was supported by the National Science Funds for DistinguishedYoung Scholars of China (Grant No. 11125208) and the 111 project (Grant No.B12032).
References
1. D. Middleton. Non-Gaussian noise models in signal processing for telecom-munications: new methods an results for class A and class B noise models.
IEEETransactions on Information Theory
CommunicationsIEEE Transactions on
IEEE Transactions on Vehicular Technology
IEEE Trans. Signal Processing
Chinese Physics B: English
IEEE Transactions on SignalProcessing
IEEE Proceedings Communications
Geophysical Journal International 1985 ; 82(3): 339-374.9. A. Nasri, R. Schober. Performance of BICM-SC and BICM-OFDM sys-tems with diversity reception in non-Gaussian noise and interference.
IEEETransactions on Communications
Nature Reviews Neuroscience
Chaos Solitons & Fractals
Physical Review Letters ohn Wiley & Sons, Inc.
Phys. Rev. E.
Phys. Rev. Lett . 2002;88: 11-18.18. H. Long, C. Ma, Y. Shimizu. Least squares estimators for stochasticdifferential equations driven by small L´evy noises.
Stochastic Processes & TheirApplications
Proc Int Conf on Digital Audio Effects
Plos One
Journal of the Physical Society of Japan
Contents lists available at Science Direct: Physical A
Proc Int Confon Digital Audio Effects
Phys. Rev. Lett . 1925;98(6): 068102-068102.26. X. Chen. Concise history of statistics. Changsha: Hunan EducationPublishing House, 2002.27. H. Xian. Study on ANN Noise Adaptability in Application of IndustryProcess Characteristics Mining.
Advanced Materials Research
IEEE Transactions on Signal Processing
Computers & Mathematics with Applications
ChaosSolitons & Fractals
Journal of Ningxia Teachers Univer-sity
Journal of Chongqing normal university (Natural science) ournal of the American Statistical Association
Handbook of Computational Statistics . Berlin: Springer, 2004.36. W. Chen, L. Ye, H. Su. Fractional diffusion equations by the Kansamethod.