On causal band-limited mean square approximation
aa r X i v : . [ m a t h . O C ] A ug On causal band-limited mean square approximation
Nikolai Dokuchaev
Department of Mathematics & Statistics, Curtin University,GPO Box U1987, Perth, 6845 Western Australia
Web-published: 29 November 2011. Revised: 10 August 2012
Abstract
We study causal dynamic approximation of non-bandlimited processes by band-limitedprocesses such that a part of the historical path of the underlying process is approximatedin L -norm by the trace of a band-limited process. This allows to cover the case of irregularnon-smooth processes. We show that this problem has an unique optimal solution. Theapproximating band-limited process has unique extrapolation on future times and can beinterpreted as a optimal forecast. To accommodate the current flow of observations, theselection of this band-limited process has to be changed dynamically. This can be interpretedas a causal and linear filter that is not time invariant. Key words : band-limited processes, causal filters, sampling, low-pass filters, prediction.AMS 2010 classification : 42A38, 42B30, 93E10PACS 2008 numbers: 02.30.Mv, 02.30.Nw, 02.30.Yy, 07.05.Mh, 07.05.Kf
We study causal dynamic approximation of non-bandlimited processes by band-limited pro-cesses. It is known that it is not possible to find an ideal low-pass causal linear time-invariantfilter. It is also known that the distance of the set of these ideal low-pass time invariant filtersfrom the set of all causal filters is positive [1]. In addition, it is known that optimal approx-imation of the ideal low-pass filter is not feasible in the class of causal linear time-invariant1lters (see, e.g., [3] and references here). In the present paper, we are trying to substitute thesolution of these unsolvable problems by solution of an easier problem where the filter is notnecessary time invariant. Our motivation is that, for some problems, time invariancy for a filteris not crucial. For example, a typical approach to forecasting in finance is to approximate theknown path of the stock price process by a smooth process that has an unique extrapolationand accept this extrapolation as the forecast. This procedure has to be done at current time;it is nor required that the same forecasting rule will be applied at future times. We apply thisapproach with the band-limited processes used as approximating smooth predictable processes.More precisely, we suggest to approximate in L -norm the known historical path of the processby the trace of a band-limited process. In this setting, the approximating curve does not nec-essary match the underlying process at given sampling points. This is different from classicalsampling approach (see, e.g., [7]). Similarly to [4]-[5], our setting allows to cover the case ofirregular non-differentiable or discontinuous processes such as historical stock prices in continu-ous time models. The difference is that [4]-[5] achieves point-wise matching for the underlyingprocess being smoothed by a convolution operator; we consider approximation of the underlyingprocess directly using different methods. In [4]-[5], the estimate of the error norm is given. Inour setting, it is guaranteed that the approximation generates the error of the minimal norm.We show that an unique optimal solution of approximation problem exits. The approximat-ing process is derived in time domain in a form of sinc series. To accommodate the currentflow of observations, the coefficients of these series and the related band-limited processes haveto be changed dynamically. It can be interpreted as a causal and linear filter that is not timeinvariant. We denote by L ( D ) the usual Hilbert space of complex valued square integrable functions x : D → C , where D is a domain.For x ( · ) ∈ L ( R ), we denote by X = F x the function defined on i R as the Fourier transformof x ( · ); X ( iω ) = ( F x )( iω ) = Z ∞−∞ e − iωt x ( t ) dt, ω ∈ R . Here i = √−
1. For x ( · ) ∈ L ( R ), the Fourier transform X is defined as an element of L ( R )2more precisely, X ( i · ) ∈ L ( R )).For a given Ω >
0, let U Ω , ∞ = { X ( iω ) ∈ L ( i R ) : X ( iω ) = 0 for | ω | > Ω } , and let U Ω ,N be the set of all X ∈ U Ω , ∞ such that there exists a sequence { y k } Nk = − N ∈ C N +1 such that X ( iω ) = P Nk = − N y k e ikω/ Ω I {| ω |≤ Ω } , where I is the indicator function.For N = + ∞ and for integers N ≥
0, consider Hilbert spaces Y N such that Y N = C N +1 for N < + ∞ and Y N is the set of all sequences { y k } Nk = − N ∈ C N +1 such that P ∞ k = −∞ | c k | < + ∞ .Let s ∈ R and q < s be given; the case when q = −∞ is not excluded. Consider Hilbertspaces of complex valued functions X = L ( −∞ , + ∞ ) and X − = L ( q, s ).Let Ω > N be given (the case of N = + ∞ is not excluded). Let X Ω ,N be the subsetof X − consisting of functions x | ( q,s ] , where x ∈ X are such that x ( t ) = ( F − X )( t ) for t ∈ [ q, s ]for some X ( iω ) ∈ U Ω ,N . Proposition 2.1
For any x ∈ X Ω ,N , there exists an unique X ∈ U Ω ,N such that x ( t ) =( F − X )( t ) for t ∈ [ q, s ] . For a Hilbert space H , we denote by ( · , · ) H the corresponding inner product. We use notationsinc ( x ) = sin( x ) /x . Let x ∈ X be a process. We assume that the path x ( s ) | s ∈ [ q,s ] represents available historical data.Let Hermitian form F : X Ω ,N × X − → R be defined as F ( b x, x ) = Z sq | b x ( t ) − x ( t ) | dt. Theorem 3.1
For any N ≤ + ∞ , there exists an unique solution b x of the minimization problemMinimize F ( b x, x ) over b x ∈ X Ω ,N . (3.1) Remark 3.1
By Proposition 2.1, there exists an unique extrapolation of the band-limited so-lution b x ( t ) of problem (3.1) on the future time interval ( s, + ∞ ) . It can be interpreted as theoptimal forecast (optimal given Ω and N ). .2 Optimal sinc coefficients To solve problem (3.1) numerically, it is convenient to expand X ( iω ) via Fourier series.For a given Ω >
0, consider the mapping Q : Y N → X Ω ,N such that x = Q y is such that x ( t ) = ( F − X )( t ) for a.e. t ∈ ( q, s ], where X ( iω ) = N X t = − N y t e itω/ Ω I {| ω |≤ Ω } . Clearly, this mapping is linear and continuous.Let Hermitian form G : Y N × X − → R be defined as G ( y, x ) = F ( Q y, x ) = Z sq | b x ( t ) − x ( t ) | dt, b x = Q y. (3.2) Corollary 3.1
For any N ≤ + ∞ , there exists an unique solution y of the minimization problemMinimize G ( y, x ) over y ∈ Y N . (3.3)Problem (3.1) can be solved via problem (3.3); its solution with N < + ∞ can be foundnumerically. Let N be given, let Z be the set of all integers z such that | z | ≤ N if N < + ∞ , and let Z bethe set of all integers if N = + ∞ . Let X ( iω ) = X k ∈ Z y k e ikωπ/ Ω I {| ω |≤ Ω } , where { y k } ∈ Y N . Let b x = F − X . We have that b x ( t ) = 12 π Z Ω − Ω X k ∈ Z y k e ikωπ/ Ω ! e iωt dω = 12 π X k ∈ Z y k Z Ω − Ω e ikωπ/ Ω+ iωt dω = 12 π X k ∈ Z y k e ikπ + i Ω t − e − ikπ − i Ω t ikπ/ Ω + it = Ω π X k ∈ Z y k sinc ( kπ + Ω t ) . emark 3.2 Let t [ k ] = − kπ/ Ω. Clearly, b x = F − X is such that b x ( t [ k ]) = y k · Ω /π , i.e., y k = b x ( t [ k ]) · π/ Ω, and, therefore, b x ( t ) = X k ∈ Z b x ( t [ k ])sinc ( kπ + Ω t ) . It gives celebrated Sampling Theorem; see, e.g., [7].
Remark 3.3
We consider a setting when only the part x ( t ) | t ∈ [ q,s ] of the path of the processis available at current time s < + ∞ . In this setting, sampling theorem is not applicable. Ourapproximation can be considered as a modification of the truncated sinc approximation (see,e.g., [6], [7]). The difference is that the increasing of N is not related to extension the timeinterval [ q, s ] in our setting.We have that G ( y, x ) = Z sq | b x ( t ) − x ( t ) | dt = Z sq (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) Ω π X k ∈ Z y k sinc ( kπ + Ω t ) − x ( t ) (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) dt = ( y, Ry ) Y N − y, rx ) X − + ( ρx, x ) X − . (3.4)Here R : Y N × Y N → Y N is a linear bounded Hermitian operator, r : X − → Y N is a boundedlinear operator, ρ : X − × X − → X − is a linear bounded Hermitian operator.It follows from the definitions that the operator R is non-negatively defined (it suffices tosubstitute x ( t ) ≡ N < + ∞ Up to the end of this paper, we assume that
N < + ∞ . In this case, the space Y N is finitedimensional, it follows that the operator R can be represented via a matrix R = { R km } ∈ C N +1 , N +1 , where R km = ¯ R mk and ( Ry ) k = P Nk = − N R km y m . Theorem 3.2 (i) For any
N < + ∞ , the operator R is positively defined.(ii) Problem (3.3) has a unique solution b y = R − rx .(iii) The components of the matrix R can be found from the equality R km = Ω π Z sq sinc ( mπ + Ω t )sinc ( kπ + Ω t ) dt. (3.5)5 iv) The components of the vector rx = { ( rx ) k } Nk = − N can be found from the equality ( rx ) k = Ω π Z sq sinc ( kπ + Ω t ) x ( t ) dt. (3.6) Corollary 3.2
Let b y = b y ( q, s ) be the vector calculated as in Theorem 3.2, b y = { b y k } Nk = − N . Theprocess b x ( t ) = b x ( t, q, s ) = Ω π X k ∈ Z y k sinc ( kπ + Ω t )represents the output of a causal filter that is linear but not time invariant. In the numerical experiments described below, we have used MATLAB symbolic integrationfor calculation of integrals (3.5) and (3.6) . The experiments show that some eigenvalues of R are quite close to zero. Because of the integration errors, some eigenvalues of the calculatedmatrix R are actually fluctuating around zero despite the fact that, by Theorem 3.2, R > E = k R b y − rx k L ( q,s ) | for the MATLAB solution of the equation R b y = rx does not vanish. This error depends on the error tolerance parameter tol of MATLAB integrationoperator QU AD that was used; the default value is tol = 10 − ; we used tol = 10 − . Further, inour experiments, we found that the error E can be decreased by the replacing R in the equation b x = R − rx by R ε = R + εI , where I is the unit matrix and where ε > ε = 0 . E ( ε ) = k R − ε rx − b y k L ( q,s ) < k R − rx − b y k L ( q,s ) , i.e., theapproximation on [ q, s ] is better for b y = R − ε rx calculated for ε = 0 .
001 than for b y = R − rx calculated for ε = 0.Figures 5.1 and 5.2 show examples of a process x ( t ) and the band-limited process b x ( t ) approx-imating x ( t ) on time intervals ( q, s ] = ( − , −
2] and ( q, s ] = ( − , ε = 0 .
001 for Ω = 4 and N = 30. As expected, the change of the time interval from( q, s ] = ( − , −
2] to ( q, s ] = ( − ,
0] results in the change of the approximating band-limitedprocess.Note that the experiments demonstrate robustness with respect to the changes of N . Thecurves of b x ( t ) will be almost the same if we consider N = 50 instead of N = 30, when all other6arameters are the same. However, the error E is larger for large N = 100, due to accumulatedlarger error of integration.The shape of curves of b x ( t ) depends on the choice Ω. Figure 5.3 shows an example of a process x ( t ) and of the band-limited process b x ( t ) approximating x ( t ) on time interval ( q, s ] = ( − , b x ∈ X Ω ,N on the future time interval ( s, + ∞ )can be interpreted as the optimal forecast (optimal given Ω and N ). Remark 4.1
We have used the procedure of replacement R by R ε = R + εI with small ε > R that is positively definedbut is close to a degenerate matrix. It can be noted that the same replacement could lead to ameaningful setting for the case when ε > G ( y, x ) + ε N X k = − N | y k | over y ∈ Y N . (4.1)The solution restrains the norm of y , and, respectively, the norm of b x .Figure 5.4 illustrates Remark 4.1 with an example of a process x ( t ) and the correspondingband-limited process b x ( t ) calculated via solution of problem (4.1) for ε = 0 .
05, when all otherparameters are the same as for Figure 5.2. This solution was obtained by replacement of R by R ε = R + εI with ε = 0 . Proof of Proposition 2.1 . The statement of this proposition is known in principle. It suffices toprove that if x ( · ) ∈ X Ω ,N is such that x ( t ) = 0 for t ∈ ( q, s ], then x ( t ) ≡
0. For the sake ofcompleteness, we give below a proof. For
C >
0, consider a class M ( C ) of infinitely differentiablefunctions x ( t ) : R → R such that there exists M = M ( x ( · )) > (cid:13)(cid:13)(cid:13)(cid:13) d k xdt k ( · ) (cid:13)(cid:13)(cid:13)(cid:13) L ( R ) ≤ C k M, k = 0 , , , .... Let M = ∪ C> M ( C ). Any x ∈ M is infinitely differentiable and such that there exists C = C ( x ( · )) > M = M ( x ( · )) > t (cid:12)(cid:12)(cid:12)(cid:12) d k xdt k ( t ) (cid:12)(cid:12)(cid:12)(cid:12) ≤ C k M . X Ω ,N ⊂ M . Therefore, any x ( · ) ∈ X Ω ,N is analytic and allows the Taylor seriesexpansion at any point with an arbitrarily large radius of convergence. Consider the Taylorseries expansion at t ∈ ( q, s ). Since all derivatives at this point are equal to zero, the expansionis identically equal to zero. This completes the proof of Proposition 2.1. (cid:3) Proof of Theorem 3.1.
It suffices to prove that X Ω ,N is a closed linear subspace of L ( q, s ).In this case, there exists a unique projection b x of x | [ q,s ] on X Ω ,N , and the theorem is proven.Clearly, for any N ≤ + ∞ , the set U Ω ,N is a closed linear subspace of L ( R ). Consider amapping Q : U Ω ,N → X Ω ,N such that x ( t ) = ( QX )( t ) = ( F − X )( t ) for t ∈ [ q, s ]. It is a linearcontinuous operator. By Proposition 2.1, it is a bijection. Since this mapping is continuous,it follows that the inverse mapping Q − : X Ω ,N → U Ω ,N is also continuous (see Corollary inCh.II.5 [8], p.77). Since the set U Ω ,N is a closed linear subspace of L ( R ), it follows that X Ω ,N is a closed linear subspace of X − . This completes the proof of Theorem 3.1. (cid:3) Proof of Theorem 3.2.
Let us prove statement (i). We know that R ≥
0. Suppose that thereexists ¯ y ∈ C N +1 such that ¯ y = 0 and R ¯ y = 0. Let r ∗ : Y N → X − be the adjoint operator to theoperator r ∗ : X − → Y N . If r ∗ ¯ y = 0 then there exists x ∈ X − such that G (¯ y, x ) <
0, which is notpossible since G ( y, x ) ≥ y, x . Therefore, r ∗ ¯ y = 0, i.e., G (¯ y, x ) = ( ρx, x ) X − . Further, let b y be a solution of problem (3.3). We have that G ( b y, x ) = G ( b y + ¯ y, x ). Hence b y + ¯ y = b y is anothersolution of problem (3.3). This contradicts to Corollary 3.1 that states that this problem hasan unique solution. Statement (ii) follows from (i) and from classical theory of quadratic forms.Statements (iii)-(iv) follow immediately from representation (3.4). This completes the proof ofTheorem 3.2. (cid:3) Acknowledgment
This work was supported by ARC grant of Australia DP120100928 to the author.
References [1] Almira, J.M. and Romero, A.E. (2008). How distant is the ideal filter of being a causalone?
Atlantic Electronic Journal of Mathematics (1) 46–55.[2] Dokuchaev, N. (2010). Predictability on finite horizon for processes with exponential de-crease of energy on higher frequencies, Signal processing (2) (2010) 696–701.83] Dokuchaev, N. (2012). On sub-ideal causal smoothing filters. Signal Processing , iss. 1,219-223.[4] Ferreira P. G. S. G.. (1995). Nonuniform sampling of nonbandlimited signals. IEEE SignalProcessing Letters , Iss. 5, 89–91.[5] Ferreira P. G. S. G.. (1995). Approximating non-band-limited functions by nonuniformsampling series. In: SampTA’95, 1995 Workshop on Sampling Theory and Applications ,276–281.[6] Jagerman, D. (1966). Bounds for truncation error of the sampling expansion.
SIAM Jour-nal on Applied Mathematics , No. 4, 714-723.[7] Jerry, A. (1977). The Shannon sampling theorem - its various extensions and applications:A tutorial review. Proc. IEEE , 11, 1565–1596.[8] Yosida, K. Functional Analysis.
Springer, Berlin Heilderberg New York, 1965. −10 −8 −6 −4 −2 0 2−1.5−1−0.500.511.5 t original processband−limited
Figure 5.1:
Example of x ( t ) and band-limited process b x ( t ) approximating x ( t ) on ( q, s ] = ( − , − ,with Ω = 4 , and N = 30 . Figure 5.2:
Example of x ( t ) and band-limited process b x ( t ) approximating x ( t ) on ( q, s ] = ( − , ,with Ω = 4 , and N = 30 . −8 −6 −4 −2 0 2 4−2−1.5−1−0.500.511.5 t original processband−limited Figure 5.3:
Example of x ( t ) and band-limited process b x ( t ) approximating x ( t ) on ( q, s ] = ( − , ,with Ω = 2 , and N = 30 . Figure 5.4:
Example of x ( t ) and band-limited process b x ( t ) calculated via solution of problem (4.1) for ε = 0 . , ( q, s ] = ( − , , Ω = 4 , and N = 30 ..