Recovering Signals from Lowpass Data
aa r X i v : . [ c s . I T ] J u l RECOVERING SIGNALS FROM LOWPASS DATA 1
Recovering Signals from Lowpass Data
Yonina C. Eldar,
Senior Member, IEEE, and Volker Pohl
Abstract — The problem of recovering a signal from its lowfrequency components occurs often in practical applications dueto the lowpass behavior of many physical systems. Here we studyin detail conditions under which a signal can be determinedfrom its low-frequency content. We focus on signals in shift-invariant spaces generated by multiple generators. For thesesignals, we derive necessary conditions on the cutoff frequency ofthe lowpass filter as well as necessary and sufficient conditionson the generators such that signal recovery is possible. Whenthe lowpass content is not sufficient to determine the signal,we propose appropriate pre-processing that can improve thereconstruction ability. In particular, we show that modulatingthe signal with one or more mixing functions prior to lowpassfiltering, can ensure the recovery of the signal in many cases,and reduces the necessary bandwidth of the filter.
Index Terms — Sampling, shift-invariant spaces, lowpass signals
I. I
NTRODUCTION
Lowpass filters are prevalent in biological, physical andengineering systems. In many scenarios, we do not haveaccess to the entire frequency content of a signal we wishto process, but only to its low frequencies. For example, itis well known that parts of the visual system exhibit lowpassnature: the neurons of the outer retina have strong responseto low frequency stimuli, due to the relatively slow responseof the photoreceptors. Similar behavior is observed also in thecons and rods [1]. Another example is the lowpass nature offree space wave propagation [2]. This limits the resolution ofoptical image reconstruction to half the wave length. Manyengineering systems introduce lowpass filtering as well. Onereason is to allow subsequent sampling and digital signalprocessing at a low rate.Clearly if we have no prior knowledge on the original signal,and we are given a lowpassed version of it, then we cannotrecover the missing frequency content. However, if we haveprior knowledge on the signal structure then it may be possibleto interpolate it from the given data. As an example, considera signal x that lies in a shift-invariant (SI) space generatedby a generator φ , so that x ( t ) = P a n φ ( t − nT ) for some T . Even if x is not bandlimited, it can be recovered from theoutput of a lowpass filter with cutoff frequency π/T as longas the Fourier transform ˆ φ ( ω ) of the generator is not zero forall ω ∈ [ − π/T, π/T ) [3], [4].The goal of this paper is to study in more detail under whatconditions a signal x can be recovered from its low-frequency The authors are with the Department of Electrical Engineering, Technion –Israel Institute of Technology, Haifa 32000, Israel, Phone: +972 4 829 3256,Fax: +972 4 829 5757, e-mail: {yonina,pohl}@ee.technion.ac.il.This work was supported in part by the Israel Science Foundation underGrant no. 1081/07 and by the European Commission in the framework ofthe FP7 Network of Excellence in Wireless COMmunications NEWCOM++(contract no. 216715). V. Pohl acknowledges the support by the GermanResearch Foundation (DFG) under Grant PO 1347/1–1. content. Our focus is on signals that lie in SI spaces, generatedby multiple generators [5], [6], [7]. Following a detailedproblem formulation in Section II, we begin in Section IIIby deriving a necessary condition on the cutoff frequencyof the low pass filter (LPF) and sufficient conditions on thegenerators such that x can be recovered from its lowpassedversion. As expected, there are scenarios in which recoveryis not possible. For example, if the bandwidth of the LPFis too small, or if one of the generators is zero over acertain frequency interval and all of its shifts with period π/T , then recovery cannot be obtained. For cases in whichthe recovery conditions are satisfied, we provide a concretemethod to reconstruct x from the its lowpass frequency contentin Section IV.The next question we address is whether in cases in whichthe recovery conditions are not satisfied, we can improve ourability to determine the signal by appropriate pre-processing.In Section V we show that pre-processing with linear time-invariant (LTI) filters does not help, even if we allow for a bankof LTI filters. As an alternative, in Section VI we considerpre-processing by modulation. Specifically, the signal x ismodulated by multiplying it with a periodic mixing functionprior to lowpass filtering. We then derive conditions on themixing function to ensure perfect recovery. As we show, alarger class of signals can be recovered this way. Moreover,by applying a bank of mixing functions, the necessary cutofffrequency in each channel can be reduced. In Section VII webriefly discuss how the results we developed can be appliedto sampling sparse signals in SI spaces at rates lower thanNyquist. These ideas rely on the recently developed frameworkfor analog compressed sensing [8], [9], [10]. In our setting,they translate to reducing the LPF bandwidth, or the numberof modulators. Finally, Section VIII summarizes and pointsout some open problems.Modulation architectures have been used previously in dif-ferent contexts of sampling. In [11] modulation was usedin order to obtain high-rate sigma-delta converters. Morerecently, modulation has been used in order to sample sparsehigh bandwidth signals at low rates [12], [13]. Our specificchoice of periodic functions is rooted in [13] in which asimilar bank of modulators was used in order to samplemultiband signals at sub-Nyquist rates. Here our focus is onsignals in general SI spaces and our goal is to develop abroad framework that enables pre-processing such as to ensureperfect reconstruction. We treat signals that lie in a predefinedsubspace, in contrast to the union of subspaces assumptionused in the context of sparse signal models. Our results canbe used in practical systems that involve lowpass filtering topre-process the signal so that all its content can be recoveredfrom the received low-frequency signal (without requiring asparse signal model). RECOVERING SIGNALS FROM LOWPASS DATA
II. P
ROBLEM F ORMULATION
A. Notations
We use the following notation: As usual, C N , L , and ℓ denote the N -dimensional Euclidean space, the space ofsquare integrable function on the real line, and the space ofsquare summable sequences, respectively. All these spaces areHilbert spaces with the usual inner products. Throughout thepaper we write ˆ x for the Fourier transform of a function x ∈ L : ˆ x ( ω ) = Z ∞−∞ x ( t ) e − jωt dt , ω ∈ R . The
Paley-Wiener space of functions in L that are bandlimitedto [ − B, B ] will be denoted by P W ( B ) : P W ( B ) = { x ∈ L : ˆ x ( ω ) = 0 for all ω / ∈ [ − B, B ] } , and P B is the orthogonal projection L → P W ( B ) onto P W ( B ) . Clearly, P B is a bounded linear operator on L .We will also need the Paley-Wiener space of functions whoseinverse Fourier transform is supported on a compact interval,i.e. d P W ( B ) = { ˆ x ∈ L : x ( t ) = 0 for all t / ∈ [ − B, B ] } . For any a ∈ R , the shift (or translation) operator S a : L → L is defined by ( S a x )( t ) = x ( t − a ) .If { φ k } k ∈I is a set of functions in L with an arbitraryindex set I then span { φ k : k ∈ I} denotes the closed linearsubspace of L spanned by { φ k } k ∈I . B. Problem Formulation
We consider the problem of recovering a signal x ( t ) , t ∈ R from its low-frequency content. Specifically, suppose that x isfiltered by a LPF with cut off frequency π/T c , as in Fig. 1.We would like to answer the following questions: • What signals x can be recovered from the output y of theLPF? • Can we perform preprocessing of x prior to filtering toensure that x can be recovered from y ? x ( t ) - - y ( t ) − π/T c π/T c Fig. 1. Lowpass filtering of x ( t ) . Filtering a signal x ∈ L with a LPF with cutoff frequency π/T c corresponds to a projection of x onto the Paley Wienerspace P W ( π/T c ) . Therefore we can write y = P π/T c x .Note, that we assume here that the output y ( t ) , t ∈ R isanalog. Since y is a lowpass signal, an equivalent formulationis to sample y with period T s = 1 /f s lower than the Nyquistperiod T c to obtain the sequence of samples { y [ n ] } n ∈ Z . Theproblem is then to recover x ( t ) , t ∈ R from the samples { y [ n ] } n ∈ Z , as in Fig. 2. Since { y [ n ] } n ∈ Z uniquely determines y , the two formulations are equivalent. For concreteness, wefocus here on the problem in which we are given y ( t ) , t ∈ R x ( t ) - - − π/T c π/T c (cid:17)(cid:17)? t = nT s - y [ n ] Fig. 2. Sampling of x ( t ) after lowpass filtering. directly. Thus, our emphasis is not on the sampling rate,but rather on the information content in the lowpass regime,regardless of the sampling rate to follow.Clearly, if x is bandlimited to [ − π/T c , π/T c ] , then it canbe recovered from y . However, we will assume here that x isa general SI signal, not necessarily bandlimited. These signalshave the property that if x ( t ) lies in a given SI space, then sodo all its shifts ( S kT x )( t ) = x ( t − kT ) by integer multiplesof some given T . Bandlimited signals are a special class ofSI signals. Indeed, if x is bandlimited then so are all its shifts S kT x , k ∈ Z for a given T . In fact, bandlimited signals have aneven stronger property that all their shifts S a x by any number a ∈ R are bandlimited. Throughout, we assume that x lies ina generally complex SI space with multiple generators.Let φ = { φ , . . . , φ N } be a given set of functions in L and let T ∈ R be a given real number. Then the shift-invariantspace generated by φ is formally defined as [5], [6], [7]: S T ( φ ) = span { S kT φ n : k ∈ Z , ≤ n ≤ N } . The functions φ n are referred to as the generators of S T ( φ ) .Thus, every function x ∈ S T ( φ ) can be written as x ( t ) = N X n =1 X k ∈ Z a n [ k ] φ n ( t − kT ) , t ∈ R , (1)where for each ≤ n ≤ N , { a n [ k ] } k ∈ Z is an arbitrarysequence in ℓ . Examples of such SI spaces include multibandsignals [14] and spline functions [15], [3]. Expansions of thetype (1) are also encountered in communication systems, whenthe analog signal is produced by pulse amplitude modulation.In order to guarantee a unique and stable representationof any signal in S T ( φ ) by sequences of coefficients { a n [ k ] } ,the generators φ are typically chosen to form a Riesz basis for S T ( φ ) . This means that there exist constants α > and β < ∞ such that α k a k ≤ (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) N X n =1 X k ∈ Z a n [ k ] φ n ( t − kT ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) L ≤ β k a k , (2)where k a k = P Nn =1 P k ∈ Z | a n [ k ] | . Condition (2) impliesthat any x ∈ S T ( φ ) has a unique and stable representation interms of the sequences { a n [ k ] } k ∈ Z . In particular, it guaranteesthat the sequences { a n [ k ] } k ∈ Z can be recovered from x ∈S T ( φ ) by means of a linear bounded operator.By taking Fourier transforms in (2) it can be shown that thegenerators φ form a Riesz basis if and only if [6] α I (cid:22) M φ ( ω ) (cid:22) β I , a.e. ω ∈ [ − π/T, π/T ] . (3) Here and in the sequel, when we say that a set of generators φ form (orgenerate) a basis, we mean that the basis functions are { φ n ( t − kT ) , k ∈ Z , ≤ n ≤ N } . LDAR, POHL 3
Here M φ ( ω ) is called the Grammian of the generators φ = { φ , . . . , φ N } , and is the N × N matrix M φ ( ω ) = R φ φ ( ω ) . . . R φ φ N ( ω ) ... ... ... R φ N φ ( ω ) . . . R φ N φ N ( ω ) , (4)where for any two generators φ i , φ j the function R φ i φ j isgiven by R φ i φ j ( ω ) = X k ∈ Z ˆ φ i ( ω − k πT ) ˆ φ j ( ω − k πT ) . (5)Note that the functions R φ i φ j are π/T -periodic. Therefore,condition (3) is equivalent to α I (cid:22) M φ ( ω − a ) (cid:22) β I for everyarbitrary real number a . We will need in particular the case a = π/T , for which the entries of the matrix M φ ( ω − a ) are R φ i φ j ( ω − πT ) = X k ∈ Z ˆ φ i ( ω − [2 k + 1] πT ) ˆ φ j ( ω − [2 k + 1] πT ) . (6)III. R ECOVERY C ONDITIONS
The first question we address is whether we can recover x ∈ S T ( φ ) of the form (1) from the output y = P π/T c x of aLPF with cutoff frequency π/T c , assuming that the generators φ satisfy (3). We further assume that the generators are notbandlimited to π/T c , namely that they have energy outside thefrequency interval [ − π/T c , π/T c ] . We will provide conditionson the generators φ and on the bandwidth of the LPF such that x can be recovered from y . As we show, even if the generators φ are not bandlimited, x can often be determined from y .First we note that in order to recover x ∈ S T ( φ ) fromthe lowpass signal y = P π/T c x it is sufficient to recover thesequences { a n [ k ] } k ∈ Z , ≤ n ≤ N because the generators φ are assumed to be known. The output of the LPF can bewritten as y ( t ) = ( P π/T c x )( t ) = N X n =1 X k ∈ Z a n [ k ] ψ n ( t − kT ) where ψ n := P π/T c φ n denotes the lowpass filtered generator φ n , and the sum on the right-hand side converges in L since P π/T c is bounded. Therefore, we immediately have thefollowing observation: The sequences { a n [ k ] } k ∈ Z , ≤ n ≤ N can be recovered from y if ψ forms a Riesz basis for S T ( ψ ) .This is equivalent to the following statement. Proposition 1:
Let φ = { φ , . . . , φ N } be a set generators,and let ψ n = P π/T c φ n , ≤ n ≤ N be the lowpass filteredgenerators where π/T c is the bandwidth of the LPF. Thenthe signal x ∈ S T ( φ ) can be recovered from the observations y = P π/T c x if the Grammian M ψ ( ω ) satisfies (3) for some < α ≤ β < ∞ . Example 1:
We consider the case of one generator ( N = 1 ) φ ( t ) = (cid:26) / (2 D ) , t ∈ [ − D, D ]0 , t / ∈ [ − D, D ] (7)for some D > . The Fourier transform of this generator is ˆ φ ( ω ) = sin( ωD ) / ( ωD ) which becomes zero at ω = kπ/D for all k = ± , ± , . . . . We assume that D/T is not an integer. Then one can easily see that this generator satisfies (3), i.e.there exists α, β such that < α ≤ X k ∈ Z (cid:12)(cid:12)(cid:12)(cid:12) sin( ωD − πkD/T ) ωD − πkD/T (cid:12)(cid:12)(cid:12)(cid:12) ≤ β < ∞ (8)for all ω ∈ [ − π/T, π/T ] . The lower bound follows from theassumption that D/T is not an integer, so that all the functionsin the above sum have no common zero in [ − π/T, π/T ] . Theupper bound β follows from X k ∈ Z (cid:12)(cid:12)(cid:12)(cid:12) sin( ωD − πkD/T ) ωD − πkD/T (cid:12)(cid:12)(cid:12)(cid:12) ≤ X k ∈ Z | ωD − πkD/T | ≤ (cid:18) TπD (cid:19) " ∞ X k =1 k − ≤ (cid:18) TπD (cid:19) using that | ω D − πkD/T | ≥ πD/T (2 | k | − for all k = ± , ± , . . . and all ω ∈ [ − π/T, π/T ] .Assume now that the LPF has cutoff frequency π/T c = π/T . Then the Fourier transform ˆ ψ of the filtered generator ψ = P π/T φ will satisfy a relation like (8) only if D ≤ T ,i.e. only if ˆ φ has no zero in [ − π/T, π/T ] . In cases where D > T the cutoff frequency has to be larger in order to allow arecovery of the original signal. One easily sees that the cutofffrequency of the LPF has to lie at least π/T − π/D above π/T in order that ˆ ψ will satisfy a relation similar to (8). Inthis case, the shifts ˆ ψ ( ω ± π/T ) compensate for the zeroof ˆ ψ ( ω ) in the sum (8). Thus for cutoff frequencies π/T c ≥ π/T − π/D a recovery of the signal x from the LPF signal y will be possible.The previous example illustrates that the question whether ψ forms a Riesz basis for S T ( ψ ) depends on the given generators φ and on the bandwidth π/T c of the LPF. The next propositionderives a necessary condition on the required bandwidth π/T c of the LPF such that ψ can be a Riesz basis for S T ( ψ ) . Proposition 2:
Let φ = { φ , . . . , φ N } be a Riesz basis forthe space S T ( φ ) and let ψ n = P π/T c φ n with ≤ n ≤ N .Then a necessary condition for ψ = { ψ , . . . , ψ N } to be aRiesz basis for S T ( ψ ) is that π/T c ≥ N π/T . Proof:
We consider the Grammian M ψ ( ω ) whose entriesare equal to R ψ i ψ j ( ω ) = X | k |≤ ( TTc +1) ˆ ψ i ( ω − k πT ) ˆ ψ j ( ω − k πT ) . All other terms in the generally infinite sum (cf. (5)) areidentically zero since ˆ ψ n ( ω ) is bandlimited to [ − π/T c , π/T c ] .This Grammian can be written as M ψ ( ω ) = Ψ ∗ ( ω ) Ψ ( ω ) with Ψ ( ω ) = ˆ ψ ( ω + [ L + 1] πT ) . . . ˆ ψ N ( ω + [ L + 1] πT )ˆ ψ ( ω + L πT ) . . . ˆ ψ N ( ω + L πT ) ... ... ˆ ψ ( ω ) . . . ˆ ψ N ( ω ) ... ... ˆ ψ ( ω − L πT ) . . . ˆ ψ N ( ω − L πT )ˆ ψ ( ω − [ L + 1] πT ) . . . ˆ ψ N ( ω − [ L + 1] πT ) (9) RECOVERING SIGNALS FROM LOWPASS DATA where L is the largest integer such that L ≤ ( T /T c − / .Since every ˆ ψ n ( ω ) is banded to [ − π/T c , π/T c ] , the first andthe last row of this matrix are identically zero for some ω ∈ [ − π/T, π/T ] . At these ω ’s, the matrix Ψ ( ω ) has effectively L = 2 L +1 rows and N columns, and it holds that L ≤ T /T c .Since M ψ ( ω ) = Ψ ∗ ( ω ) Ψ ( ω ) , the Grammian can have fullrank for every ω ∈ [ − π/T, π/T ] only if L ≥ N , i.e. only if π/T c ≥ N π/T .The necessary condition on the bandwidth of the LPFgiven in the previous proposition is not sufficient, in gen-eral. However, given a bandwidth π/T c which satisfies thenecessary condition of Proposition 2, sufficient conditions onthe generators φ can be derived such that the lowpass filteredgenerators ψ form a Riesz basis for S T ( ψ ) , i.e. such that x can be recovered from y . Proposition 3:
Let φ = { φ , . . . , φ N } be a Riesz basis for S T ( φ ) and let ψ n = P π/T c φ n for ≤ n ≤ N with π/T c ≥ N π/T . Denote by L the largest integer such that L ≤ T /T c .If L = 2 L + 1 is an odd number, then we define the L × N matrix Φ L ( ω ) by Φ L ( ω ) = ˆ φ ( ω + 2 L πT ) . . . ˆ φ N ( ω + 2 L πT ) ... ... ˆ φ ( ω + 2 πT ) . . . ˆ φ N ( ω + 2 πT )ˆ φ ( ω ) . . . ˆ φ N ( ω )ˆ φ ( ω − πT ) . . . ˆ φ N ( ω − πT ) ... ... ˆ φ ( ω − L πT ) . . . ˆ φ N ( ω − L πT ) . (10)For L = 2 L even, we define Φ L ( ω ) = ˆ φ ( ω + [2 L − πT ) . . . ˆ φ N ( ω + [2 L − πT ) ... ... ˆ φ ( ω + πT ) . . . ˆ φ N ( ω + πT )ˆ φ ( ω − πT ) . . . ˆ φ N ( ω − πT ) ... ... ˆ φ ( ω − [2 L − πT ) . . . ˆ φ N ( ω − [2 L − πT ) . (11)If there exists a constant α > such that M L ( ω ) := Φ ∗ L ( ω ) Φ L ( ω ) (cid:23) α I a.e. ω ∈ [ − πT , πT ] (12)then ψ = { ψ , . . . , ψ N } forms a Riesz basis for S T ( ψ ) .Moreover, if T /T c is an integer, then condition (12) is alsonecessary for ψ to be a Riesz basis for S T ( ψ ) .When π/T c → ∞ , i.e. L → ∞ , the matrix M L ( ω ) reducesto M φ ( ω ) of (4), which by definition satisfies (3). However,since for the calculation of the entries of M L ( ω ) we are onlysumming over a partial set of the integers, we are no longerguaranteed that M L ( ω ) satisfies the lower bound of (3).The requirements of Proposition 3 imply that L ≥ N .Consequently, the matrix M L ( ω ) = Φ ∗ L ( ω ) Φ L ( ω ) is positivedefinite for almost all ω ∈ [ − π/T, π/T ] if and only if Φ L ( ω ) has full column rank for almost all ω ∈ [ − π/T, π/T ] . Note that Example 1 shows that (12) is not necessary, ingeneral: With T < D < T and a cutoff frequency of π/T c > π/T − π/D , the corresponding ψ form a Riesz basisfor S T ( ψ ) . However, it can easily be verified that (12) is notsatisfied. Proof:
We consider the case of L being odd. It hasto be shown that the Grammian M ψ ( ω ) satisfies (3). Since N T c ≤ T , the Grammian can be written as M ψ ( ω ) = Ψ ∗ ( ω ) Ψ ( ω ) with Ψ ( ω ) defined by (9). Next Ψ ( ω ) is writtenas Ψ ( ω ) = Ψ L ( ω )+ Ψ ⊥ ( ω ) where Ψ ⊥ ( ω ) is the (2 L + 1) × N matrix whose first and last row coincide with those of Ψ ( ω ) and whose other rows are identically zero. Similarly Ψ L ( ω ) denotes the matrix whose first and last row is identically zeroand whose remaining rows coincide with those of Ψ ( ω ) . Since ψ n ( ω ) = φ n ( ω ) for all ω ∈ [ − π/T c , π/T c ] and for every ≤ n ≤ N , we have that Ψ ∗ L ( ω ) Ψ L ( ω ) = Φ ∗ L ( ω ) Φ L ( ω ) .Therefore, M ψ ( ω ) = Ψ ∗ L ( ω ) Ψ L ( ω ) + Ψ ∗⊥ ( ω ) Ψ ⊥ ( ω )+ Ψ ∗ L ( ω ) Ψ ⊥ ( ω ) + Ψ ∗⊥ ( ω ) Ψ L ( ω )= Φ ∗ L ( ω ) Φ L ( ω ) + Ψ ∗⊥ ( ω ) Ψ ⊥ ( ω ) (13)since by the definition of Ψ L ( ω ) and Ψ ⊥ ( ω ) , we obviouslyhave that Ψ ∗ L ( ω ) Ψ ⊥ ( ω ) ≡ and Ψ ∗⊥ ( ω ) Ψ L ( ω ) ≡ . Nowit follows from (13) that for every x ∈ C N x ∗ M ψ ( ω ) x = k Φ L ( ω ) x k C N + k Ψ ⊥ ( ω ) x k C N ≥ k Φ L ( ω ) x k C N = x ∗ Φ ∗ L ( ω ) Φ L ( ω ) x ≥ α, where the last inequality follows from (12). This shows that theGrammian M ψ ( ω ) is lower bounded as in (3). The existenceof an upper bound for M ψ ( ω ) is trivial since M ψ ( ω ) has finitedimensions.Assume now that T /T c is an (odd) integer. In this case L = ( T /T c − / and it can easily be verified that thematrix Ψ ⊥ ( ω ) is identically zero. From (13), M ψ ( ω ) = Φ ∗ L ( ω ) Φ L ( ω ) = M L ( ω ) which shows that if the Grammian M ψ ( ω ) satisfies (3) then Φ L ( ω ) satisfies (12). This proves that(12) is also necessary for ψ to be a Riesz basis for S T ( ψ ) .The case of L even follows from the same arguments butstarting with expression (6) for the entries of the Grammianinstead of (5). Therefore, the details are omitted. Example 2:
We consider an example with two generators(N=2) which both have the form as in Example 1, withdifferent values for D , i.e. φ i ( t ) = (cid:26) / (2 D i ) , t ∈ [ − D i , D i ]0 , t / ∈ [ − D i , D i ] i = 1 , with Fourier transforms ˆ φ i ( ω ) = sin( ωD i ) / ( ωD i ) . As inExample 1 we assume that D i /T are not integers and that D = D . Under these conditions, the Grammian M φ ( ω ) of φ = { φ , φ } satisfies (3). To see this, we consider thedeterminant of M φ ( ω ) for some arbitrary but fixed ω ∈ LDAR, POHL 5 [ − π/T, π/T ] : det[ M φ ( ω )] = X k ∈ Z (cid:12)(cid:12)(cid:12) ˆ φ ( ω − k πT ) (cid:12)(cid:12)(cid:12) X k ∈ Z (cid:12)(cid:12)(cid:12) ˆ φ ( ω − k πT ) (cid:12)(cid:12)(cid:12) − X k ∈ Z ˆ φ ( ω − k πT ) ˆ φ ( ω − k πT ) ! . (14)We know from Example 1, that the first term on the right handside is lower bounded by some constant α α > . Moreover,the Cauchy-Schwarz inequality shows that the second term onthe right-hand side is always smaller or equal than the firstterm with equality only if the two sequences { ˆ φ i ( ω − k πT ) } k ∈ Z , i = 1 , are linearly dependent. However, since D = D , it is not hardto verify that these two sequences are linearly independent.Consequently det[ M φ ( ω )] > for all ω ∈ [ − π/T, π/T ] which shows that M φ ( ω ) satisfies the lower bound of (3). That M φ ( ω ) satisfies also the upper bound in (3) follows from asimilar calculation as in Example 1 using that | ˆ φ i ( ω ) | deceasesproportional to /ω as | ω | → ∞ .Assume now that the bandwidth of the LPF satisfies π/T ≤ π/T c < π/T . In this case the matrix Φ L ( ω ) ofProposition 3 is given by Φ L ( ω ) = (cid:20) ˆ φ ( ω + πT ) ˆ φ ( ω + πT )ˆ φ ( ω − πT ) ˆ φ ( ω − πT ) (cid:21) , and the determinant of M L ( ω ) := Φ ∗ L ( ω ) Φ L ( ω ) becomes det[ M L ( ω )] = X k = ± (cid:12)(cid:12)(cid:12) ˆ φ ( ω − k πT ) (cid:12)(cid:12)(cid:12) X k = ± (cid:12)(cid:12)(cid:12) ˆ φ ( ω − k πT ) (cid:12)(cid:12)(cid:12) − X k = ± ˆ φ ( ω − k πT ) ˆ φ ( ω − k πT ) ! . (15)This expression is similar to (14) and the same arguments showthat det[ M L ( ω )] > for all ω ∈ [ − π/T, π/T ] . Namely, since D i /T are not integers, the functions ˆ φ i ( ω + π/T ) and ˆ φ i ( ω − π/T ) have no common zero such that the first term on theright hand side of (15) is lower bounded by some α α > .The Cauchy-Schwarz inequality implies that the second termis always smaller than the first one.We conclude that Φ L ( ω ) satisfies the condition of Propo-sition 3, so that the signal x can be recovered from its lowfrequency components y = P π/T c x .If for a certain bandwidth π/T c of the LPF the generators φ satisfy the conditions of Proposition 3 then the signal x canbe recovered from y = P π/T c x . However, if the generators φ do not satisfy these conditions, then there exists in principletwo ways to enable recovery of x : • Increasing the bandwidth of the LPF. • Pre-process x before lowpass filtering, i.e. modify thegenerators φ .It is clear that for a given set φ = { φ , . . . φ N } of generatorsan increase of the LPF can only increase the "likelihood" thatthe matrix Φ L ( ω ) of Proposition 3 will have full column rank.This is because enlarging π/T c increases the number L i.e. it adds additional rows to the matrix which can only enlarge thecolumn rank of Φ L ( ω ) . Pre-processing of x will be discussedin detail in Sections V and VI.IV. R ECOVERY A LGORITHM
We now describe a simple method to reconstruct the desiredsignal x from its low frequency components. This methodis used in later sections to show how pre-processing of thesignal x may facilitate its recovery. Throughout this section,we assume that the bandwidth π/T c of the LPF satisfies thenecessary condition of Proposition 2, and that the generatorssatisfy the sufficient condition of Proposition 3.Taking the Fourier transform of (1), we see that every x ∈S T ( φ ) can be expressed in the Fourier domain as ˆ x ( ω ) = N X n =1 ˆ a n ( e jωT ) ˆ φ n ( ω ) , ω ∈ R (16)where ˆ a n ( e jωT ) = X k ∈ Z a n [ k ] e − jωkT is the π/T -periodic discrete time Fourier transform of thesequence { a n [ k ] } k ∈ Z at frequency ωT . Denoting by ˆ a ( e jωT ) the vector whose n th element is equal to ˆ a n ( e jωT ) and by ˆ φ ( ω ) the vector whose n th element is equal to ˆ φ n ( ω ) we canwrite (16) in vector form as ˆ x ( ω ) = ˆ φ T ( ω ) ˆ a ( e jωT ) . The Fourier transform of the LPF output y = P π/T c x isbandlimited to π/T c , and for all ω ∈ [ − π/T c , π/T c ] we have ˆ y ( ω ) = ˆ x ( ω ) . Therefore ˆ y ( ω ) = ˆ φ T ( ω ) ˆ a ( e jωT ) , ω ∈ [ − πT c , πT c ] . (17)For every ω ∈ [ − π/T c , π/T c ] , (17) describes an equationfor the N unknowns ˆ a n ( e jωT ) . Clearly, one equation is notsufficient to recover the length- N vector ˆ a ( e jωT ) ; we need atleast N equations. However, since according to Proposition 2the bandwidth of the LPF has to be at least π/T c ≥ N π/T , wecan form more equations from the given data by noting that ˆ a is periodic with period π/T , while ˆ φ , and consequently ˆ y , are generally not. Specifically, let ω ∈ [ − π/T, π/T ] bean arbitrary frequency. For any ω k = ω + 2 πk/T with k an integer we have that ˆ a ( e jω k T ) = ˆ a ( e jω T ) . Therefore, byevaluating ˆ y and ˆ φ at frequencies − π/T c ≤ ω k ≤ π/T c , wecan use (17) to generate more equations. To this end, let L be the largest integer for which L ≤ T /T c . Assume first that L = 2 L + 1 for some integer L , so that L is odd. We thengenerate the equations ˆ y k ( ω ) := ˆ y ( ω − k πT ) = N X n =1 ˆ φ n ( ω − k πT ) ˆ a n ( ω − k πT ) for − L ≤ k ≤ L and for ω ∈ [ − π/T, π/T ] . Since byour assumption π/T c ≥ L π/T , all the observations ˆ y k ( ω ) =ˆ y ( ω − k π/T ) are in the passband regime of the LPF. Theabove set of L equations may be written as ˆ y ( ω ) = Φ L ( ω ) ˆ a ( e jωT ) , ω ∈ [ − π/T, π/T ] , (18) RECOVERING SIGNALS FROM LOWPASS DATA where ˆ y ( ω ) = [ˆ y − L ( ω ) , . . . , , . . . , ˆ y L ( ω )] T is a length L vector containing all the different observations ˆ y k of the output ˆ y , and Φ L ( ω ) is the L × N matrix given by (10). In the casewhere L = 2 L is an even number , we generate additionalequations by ˆ y k ( ω ) := N X n =1 ˆ φ n ( ω − [2 k + 1] πT ) ˆ a n ( ω − [2 k + 1] πT ) (19)for − L ≤ k ≤ L − . Here again all the observations in(19) are in the passband regime of the LPF. Therefore, (19)can be written as in (18) where Φ L ( ω ) is now given by (11),and the definition of ˆ a is changed accordingly.If the matrix Φ L ( ω ) satisfies the sufficient conditions ofProposition 3, then the unknown vector ˆ a ( e jωT ) can berecovered from (18) by solving the linear set of equations forall ω ∈ [ − π/T, π/T ] . In particular, there exists a left inverse G ( ω ) of Φ L ( ω ) such that ˆ a ( e jωT ) = G ( ω ) ˆ y ( ω ) . Finally, thedesired sequences { a n [ k ] } k ∈ Z are the Fourier coefficients ofthe π/T periodic functions ˆ a n .V. P REPROCESSING W ITH F ILTERS
When Φ L ( ω ) does not has full column rank for all ω ∈ [ − π/T, π/T ] and if the bandwidth of the LPF can not beincreased, an interesting question is whether we can pre-process x before lowpass filtering in order to ensure that itcan be recovered from the LPF output. In this and in thenext section we consider two types of pre-processing: usinga bank of filters, and using a bank of mixers (modulators),respectively.Suppose we allow pre-processing of x with a set of N filters,as in Fig. 3. The question is whether we can choose the filters g n in the figure so that x can be recovered from the outputs y n of each of the branches under more mild conditions thanthose developed in Section III. - g N ( t ) - - y N ( t ) − π/T c π/T c - g ( t ) - - y ( t ) − π/T c π/T c x ( t ) - ... ... Fig. 3. Preprocessing of x ( t ) by a bank of N LTI filters.
Let ˆ y , ˆ g be the length- N vectors with n th elements givenby ˆ y n , ˆ g n . Then we can immediately verify that ˆ y ( ω ) = ˆ g ( ω ) ˆ φ T ( ω ) ˆ a ( e jωT ) , ω ∈ [ − πT c , πT c ] . (20)Clearly, ˆ a cannot be recovered from this set of equations asall the equations are linearly dependent (they are all multiples In subsequent sections, we will only discuss the case where L is odd. Thenecessary changes for the case of L being even are obvious. of each other). Thus, although we have N equations, onlyone of them provides independent information on ˆ a . We can,as before, use the periodicity of ˆ a if T c is small enough.Following the same reasoning as in Section IV, assumingthat π/T c ≥ L π/T , we can create L − new measurementsusing the same unknowns ˆ a by considering ˆ y ( ω ) for differentfrequencies ω + k π/T . In this case though it is obvious thatthe pre-filtering does not help, since only one equation canbe used from the set of N equations (20) for each frequency.In other words, all the branches in Fig. 3 provide the sameinformation. The resulting equation is the same as in theprevious section up to multiplication by ˆ g n for one index ≤ n ≤ N . Therefore, the recovery conditions reduce tothe same ones as before, and having N branches does notimprove our ability to recover x .VI. P REPROCESSING W ITH M IXERS
We now consider a different approach, which as we willsee leads to greater benefit. In this strategy, instead of usingfilters in each branch, we use periodic mixing functions p n .Each sequence is assumed to be periodic with period equalto T p = T . By choosing the mixing functions appropriately,we can increase the class of functions that can be recoveredfrom the lowpass filtered outputs. A. Single Channel
Let us begin with the case of a single mixing function, asin Fig. 4. Since p is assumed to be periodic with period T , it x ( t ) - l × - - y ( t ) − π/T c π/T c p ( t ) Fig. 4. Mixing prior to lowpass filtering of x ( t ) . can be written as a Fourier series p ( t ) = X k ∈ Z b k e j πkt/T (21)where b k = 1 T Z T/ − T/ p ( t ) e − j πkt/T dt , k ∈ Z (22)are the Fourier coefficients of p . The sum (21) is assumed toconverge in L which implies that the sequence { b k } k ∈ Z is anelement of ℓ . The output y = P π/T c ( p x ) of the LPF is thengiven in the frequency domain by ˆ y ( ω ) = X k ∈ Z b k ˆ x ( ω − k πT ) , ω ∈ [ − πT c , πT c ] . (23)Using (16) and the fact that ˆ a n ( e jωT ) is π/T -periodic, (23)can be written as ˆ y ( ω ) = N X n =1 ˆ a n ( e jωT ) X k ∈ Z b k ˆ φ n ( ω − k πT ) , (24) Note, that we can also choose T p = T/r for an integer r . However, forsimplicity we restrict attention to the case r = 1 . LDAR, POHL 7 for ω ∈ [ − π/T c , π/T c ] . Defining ˆ γ n ( ω ) := X k ∈ Z b k ˆ φ n ( ω − k πT ) , ≤ n ≤ N (25)and denoting by ˆ γ the vector whose n th element is ˆ γ n , wecan express (24) as ˆ y ( ω ) = ˆ γ T ( ω ) ˆ a ( e jωT ) , ω ∈ [ − πT c , πT c ] . (26)Equation (26) is similar to (17) with ˆ γ replacing ˆ φ . There-fore, as in the case in which no pre-processing took place(cf. Section IV), we can create L − additional equationsby evaluating ˆ y ( ω ) at frequencies ω + 2 k π/T as long as π/T c ≥ L π/T . This yields the system of equations ˆ y ( ω ) = Γ L ( ω ) ˆ a ( e jωT ) , ω ∈ [ − πT , πT ] , (27)where ˆ y and ˆ a are defined as in (10) and Γ L ( ω ) = ˆ γ ( ω + L πT ) . . . ˆ γ N ( ω + L πT ) ... ... ˆ γ ( ω ) . . . ˆ γ N ( ω ) ... ... ˆ γ ( ω − L πT ) . . . ˆ γ N ( ω − L πT ) . Consequently, we can recover ˆ a from the given measurementsas long as the matrix Γ L ( ω ) has full column rank for all ω ∈ [ − π/T, π/T ] . To this end it is necessary that π/T c ≥ N π/T ,i.e. that L ≥ N .Due to the mixing of the signal, the coefficient matrix Φ L ( ω ) in (18) is changed to Γ L ( ω ) in (27). This newcoefficient matrix is constructed out of the "new generators" { γ n } Nn =1 in exactly the same way as Φ L ( ω ) is constructedfrom the original generators { φ n } Nn =1 . Equation (25) showsthat the Fourier transform ˆ γ n of each new generator lies in ashift invariant space S πT ( ˆ φ n ) = span { S k πT ˆ φ n : k ∈ Z } spanned by shifts of ˆ φ n . The coefficients { b k } k ∈ Z of themixing sequence are then the "coordinates" of ˆ γ n in S πT ( ˆ φ n ) .We now want to show that the condition of invertibilityof Γ L ( ω ) is in general easier to satisfy then the analogouscondition on the matrix Φ L ( ω ) of (10). To this end, we write Γ L ( ω ) as Γ L ( ω ) = B L Φ ( ω ) , (28)where Φ ( ω ) denotes the matrix consisting of N columns andinfinitely many rows ˆ φ T ( ω + k π/T ) with k ∈ Z . Note that Φ ( ω ) has the form (10) with L → ∞ , i.e. Φ ( ω ) = Φ ∞ ( ω ) .The matrix B L with L = 2 L + 1 rows and infinite columnscontains the Fourier coefficients { b k } k ∈ Z of the mixing se-quence (21) and is given by B L = . . . b L − b L b L +1 . . . ... . . . b b b . . .. . . b − b b . . .. . . b − b − b . . . ... . . . b − L − b − L b − L +1 . . . . (29) Representation (28) follows immediately from the relation ˆ γ n ( ω − ℓ π/T ) = P k ∈ Z b k − ℓ ˆ φ n ( ω − k πT ) for the entriesof the matrix Γ L ( ω ) .The Grammian M φ ( ω ) of the generators φ , defined in (4),can be written as M φ ( ω ) = Φ ∗ ( ω ) Φ ( ω ) . Therefore, under ourassumption (3) on the generators, Φ ( ω ) has full column rankfor all ω ∈ [ − π/T, π/T ] . The question then is whether wecan choose the sequence { b k } k ∈ Z ∈ ℓ , and consequently thefunction p , so that B L Φ ( ω ) has full-column rank i.e. such thatthe matrix Γ ∗ L ( ω ) Γ L ( ω ) = Φ ∗ ( ω ) B ∗ L B L Φ ( ω ) is invertiblefor all ω ∈ [ − π/T, π/T ] .If we choose the mixing sequence p ( t ) ≡ then b = 1 and b k = 0 for all k = 0 . Consequently B L Φ ( ω ) is comprised ofthe first L rows of Φ ( ω ) , so that Γ L ( ω ) = Φ L ( ω ) . However,by allowing for general sequences { b k } k ∈ Z , we have morefreedom in choosing B L such that the product B L Φ ( ω ) mayhave full column-rank, even if Φ L ( ω ) does not.We next give a simple example which demonstrates thatpre-processing by an appropriate mixing function can enablethe recovery of the signal. Example 3:
We continue Example 1 with the single gen-erator φ given by (7). Here we assume that the parameter D satisfies the relation < D/T < / and that the cutofffrequency of the lowpass filter is π/T c = π/T . In this case,recovery of x from its lowpass component y = P π/T c x is notpossible, as discussed in Example 1. However, we will showthat there exist mixing functions p so that x can be recoveredfrom y = P π/T c ( px ) .One possible mixing function is p ( t ) = 1 + 2 sin(2 πt/T ) whose Fourier coefficients (22) are given by b − = − j , b =1 , b = j , and b k = 0 for all | k | ≥ . With this choice, the"new generator" (25) becomes ˆ γ ( ω ) = sin( ωD ) ωD + j (cid:20) sin( ωD − πD/T ) ωD − πD/T − sin( ωD + 2 πD/T ) ωD + 2 πD/T (cid:21) . Since π/T c = π/T , the matrix Γ L ( ω ) reduces to the scalar ˆ γ ( ω ) and we have to show that < | ˆ γ ( ω ) | < ∞ for all ω ∈ [ − π/T, π/T ] . The upper bound is trivial; for the lowerbound, it is sufficient to show that the real and imaginarypart of ˆ γ have no common zero in [ − π/T, π/T ] . This fact iseasily verified by noticing that the only zeros of the real partof ˆ γ ( ω ) are at ω = π/D and ω = − π/D . Evaluating theimaginary part ℑ{ ˆ γ } of ˆ γ at these zeros gives |ℑ{ ˆ γ ( ω , ) }| = 12 π | sin(2 πD/T ) | ( D/T ) − / which is non-zero under the assumption made on D/T .The general question whether for a given set φ = { φ , . . . , φ N } of generators there exists a matrix B L suchthat (28) is invertible for all ω ∈ [ − π/T, π/T ] , or underwhat conditions on the generators φ such a matrix can befound seems to be an open and non-trivial question. The majordifficulty is that according to (28), we look for a constant(independent of ω ) matrix B L such that B L Φ ( ω ) has full RECOVERING SIGNALS FROM LOWPASS DATA column rank for all ω ∈ [ − π/T, π/T ] . Moreover, the matrix B L has to be of the particular form (29) with a sequence { b k } k ∈ Z ∈ ℓ .The next example characterizes a class of generators forwhich a simple (trivial) mixing sequence always exist. Example 4 (generators with compact support):
Weconsider the case of a single generator ( N = 1 ) andassume that π/T c = π/T , i.e. L = N = 1 . Our problemthen reduces to finding a function ˆ γ ∈ S πT ( ˆ φ ) such that ˆ γ ( ω ) = 0 for all ω ∈ [ − π/T, π/T ] .We treat the special case of a generator φ with finitesupport of the form [ − D, D ] for some D ∈ R , i.e. we assumethat φ ( t ) = 0 for all t / ∈ [ − D, D ] . This means that its Fouriertransform ˆ φ is an element of the Paley-Wiener space d P W ( D ) and so are all linear combinations of the shifts S π/T ˆ φ . Itfollows that S ( ˆ φ ) ⊂ d P W ( D ) .Let now ˆ γ ∈ S ( ˆ φ ) be arbitrary and let { ω k } k ∈ Z be theordered sequence of real zeros of ˆ γ with ω n ≤ ω n +1 . Then atheorem of Walker [16] states that sup n ∈ Z | ω n +1 − ω n | > π/D. Thus there exists at least one interval of the real line of length π/D such that ˆ γ has no zeros in this interval. Consequently,if π/D > π/T then there always exists a k ∈ Z such that ˆ γ ( ω − k πT ) = 0 for all ω ∈ [ − πT , πT ] . (30)This holds in particular for the generator ˆ φ itself.We conclude that if the support of the generator φ satisfies supp( φ ) < T / , then there always exists a k ∈ Z such that ˆ γ ( ω ) = ˆ φ ( ω − k π/T ) = 0 for all ω ∈ [ − π/T, π/T ] .The corresponding mixing sequence is given by b k = 1 and b k = 0 for all k = k . B. Multiple Channels
In the single channel case, it was necessary that the cutofffrequency π/T c of the LPF is at least N times larger than thebandwidth of the desired signal ˆ a in order to be able to recoverthe signal. We will now show that using several channels canreduce the cutoff frequency π/T c of the filter in each channel,from which we can still recover the original signal x .Suppose that we have L ≥ N channels, where each channeluses a different mixing sequence, as in Fig. 5. Since L ≥ N ,we expect to be able to reduce the cutoff in each channel.We therefore consider the case in which T c = T . The output y ℓ = P π/T ( p ℓ x ) of the ℓ th channel in the frequency domainis then equal to ˆ y ℓ ( ω ) = ˆ γ Tℓ ( ω )ˆ a ( e jωT ) , ω ∈ [ − πT , πT ] where ˆ γ Tℓ ( ω ) is the vector with n th element [ˆ γ ℓ ( ω )] n = ˆ γ ℓn ( ω ) := X k ∈ Z b ℓk ˆ φ i ( ω + k πT ) , and { b ℓk } k ∈ Z are the Fourier coefficients associated with the ℓ th sequence p ℓ . Defining by ˆ y ( ω ) the vector with ℓ th element ˆ y ℓ ( ω ) we conclude that ˆ y ( ω ) = Γ L ( ω ) ˆ a ( e jωT ) , ω ∈ [ − πT , πT ] - l × - - y L ( t ) p L ( t ) − π/T c π/T c - l × - - y ( t ) p ( t ) − π/T c π/T c x ( t ) - ... ... Fig. 5. Bank of mixing functions. where Γ L ( ω ) is the matrix whose entry in the ℓ th row and n th column is [ Γ ( ω )] ℓ,n = ˆ γ ℓn ( ω ) . Now, all we need is tochoose the L sequences { b ℓk } k ∈ Z ∈ ℓ such that Γ L ( ω ) hasfull column rank. More specifically, as before we can write Γ L ( ω ) = B L Φ ( ω ) , (31)where B L is a matrix with L rows and infinitely many columnswhose ℓ th row is given by the coefficient sequence { b ℓk } k ∈ Z ,i.e. B L = . . . b − b − b b b . . .. . . b − b − b b b . . . ... . . . b L − b L − b L b L b L . . . . By our assumption Φ ( ω ) has full column rank and so itremains to choose B L such that Γ L ( ω ) is invertible for every ω ∈ [ − π/T, π/T ] .It should be noted that we used the same notation as in theprevious subsection although the definition of the particularmatrices and vectors differ slightly in both cases. Nevertheless,the formal approach is very similar. In the previous subsection,we observed the output signal in different frequency channels ≤ ℓ ≤ L whereas in this subsection the channels ≤ ℓ ≤ L are characterized by different mixing sequences .As in the previous subsection, the general question whetherfor a given system φ = { φ , . . . , φ N } of generators therealways exists an appropriate system of mixing sequences p = { p , . . . , p L } such that Γ L ( ω ) has full column rank forall frequencies ω seems to be non-trivial. The formal difficultylies in the fact that we look for a constant (independent of ω ) matrix B L such that (31) has full column rank for each ω ∈ [ − π/T, π/T ] . However, compared with the previoussection, where only one mixing sequence was applied, theproblem of finding an appropriate matrix B L becomes simpler:In the former case B L has to have the special (diagonal)form (29), whereas here its entries can be chosen (almost)arbitrarily. The sequences { b ℓk } k ∈ Z only have to be in ℓ .A special choice of periodic functions that are easy toimplement in practice are binary sequences. This example was In the first case we perform "frequency multiplexing" whereas the secondcase resembles "code multiplexing".
LDAR, POHL 9 studied in [13] in the context of sparse multiband sampling.More specifically, p ℓ , ≤ ℓ ≤ L are chosen to attain thevalues ± over intervals of length T /M where M is a giveninteger. Formally, p ℓ ( t ) = α ℓn , n TM ≤ t < ( n + 1) TM , ≤ n ≤ M − (32)with α ℓn ∈ { +1 , − } , and p ℓ ( t + kT ) = p ℓ ( t ) for every k ∈ Z .In this case, we have b ℓk = 1 T Z T p ℓ ( t ) e − j πT kt dt = 1 T Z T/M M − X n =0 α ℓn e − j πT k ( t + n TM ) dt = 1 T M − X n =0 α ℓn e − j πM nk Z T/M e − j πT kt dt. Evaluating the integral gives b ℓ = 1 M ˆ α ℓ and b ℓk = 1 − e − jω k j πk ˆ α ℓk , k = 0 where ω = 2 π/M , and { ˆ α ℓk } k ∈ Z denotes the discreteFourier transform (DFT) of the sequence { α ℓn } M − n =0 . Note that { ˆ α k } k ∈ Z is M -periodic so that ˆ α k = ˆ α k + M .With these mixing sequences, the infinite matrix B L can bewritten as B L = QF ∗ W , (33)where Q is a matrix with M columns and L rows, whose ℓ throw is given by the sequence { ˆ α ℓn } M − n =0 , F is the M × M Fourier matrix, and W is a matrix with M rows and infinitelymany columns consisting of block diagonal matrices of size M × M whose diagonal values are given by the sequence { w k } k ∈ Z defined by w = 1 /M and w k = − e − jω k j πk for k = 0 . Applying these binary mixing sequences, the problemis now to find a finite L × N matrix Q with values in { +1 , − } such that QF ∗ W Φ ( ω ) has full column rank for every ω ∈ [ − π/T, π/T ] .The next example shows how to select Q in the case ofbandlimited generators. Example 5 (bandlimited generators):
We consider the casewhere each generator φ n is bandlimited to the interval [ − K π/T, K π/T ] for some K ∈ N , and N = 2 K + 1 . Inthis case, Φ ( ω ) = Φ N ( ω ) is essentially an N × N matrix (allother entries are identically zero). This matrix is invertible forevery ω ∈ [ − π/T, π/T ] according to assumption (3).We now apply L = N different mixing sequences { p ℓ } Lℓ =1 having the special structure (32), and choose M = N .According to (31) and (33) the matrix Γ L ( ω ) then becomes Γ L ( ω ) = Q F ∗ W Φ ( ω ) , (34)where Q F ∗ and W Φ ( ω ) are matrices of size N × N . Thematrix W Φ ( ω ) may be considered as the product of theinvertible N × N matrix Φ ( ω ) = Φ N ( ω ) with an N × N diagonal matrix consisting of the central diagonal matrix of W , i.e. W Φ ( ω ) = diag ( w , . . . , w N − ) Φ N ( ω ) . Since this diagonal matrix is invertible also
W Φ N ( ω ) isinvertible for every ω ∈ [ − π/T, π/T ] . Therefore, using thefact that the Fourier matrix F is invertible, Γ L ( ω ) is invertiblefor each ω ∈ [ − π/T, π/T ] if the values { α ℓn } Nn =1 of themixing sequences p ℓ are chosen such that Q is invertible. Thiscan be achieved by choosing Q as a Hadamard matrix of order N . It is known that Hadamard matrices exists at least for allorders up to [17].In the previous example, Φ N ( ω ) was an N × N invertiblematrix for all ω ∈ [ − π/T, π/T ] . According to Proposition 3recovery of the signal x is therefore possible if the bandwidthof the LPF is larger than N π/T . However, the example showsthat pre-processing of x by applying the binary sequences in L = N channels allows recovery of the signal already fromits signal components in the frequency range [ − π/T, π/T ] .For simplicity of the exposition, we assumed throughoutthis subsection that the bandwidth π/T c of the lowpass filteris equal to the signal bandwidth π/T and that the numberof channels L is at least equal to the number of generators N . However, it is clear from the first subsection that in caseswhere L < N , recovery of the signal may still be possible ifthe bandwidth of the LPF is increased.VII. C
ONNECTION WITH S PARSE A NALOG S IGNALS
In this section we depart from the subspace assumptionwhich prevailed until now. Instead, we would like to incor-porate sparsity into the signal model x ( t ) of (1). To this end,we follow the model proposed in [9] to describe sparsity ofanalog signals in SI spaces. Specifically, we assume that only K out of the generators φ n ( t ) are active, so that at most K of the sequences a n [ k ] have positive energy.In [9], it was shown how such signals can be sampledand reconstructed from samples at a low rate of K/T . Thesamples are obtained by pre-processing the signal x ( t ) witha set of K sampling filters, whose outputs are uniformlysampled at a rate of /T . Without the sparsity assumption, atleast N sampling filters are needed where generally N is muchlarger than K . In contrast to this setup, here we are constrainedto sample at the output of a LPF with given bandwidth. Thus,we no longer have the freedom to choose the sampling filtersas we wish. Nonetheless, by exploiting the sparsity of thesignal we expect to be able to reduce the bandwidth neededto recover x ( t ) of the form (1), or in turn, to reduce the numberof branches needed when using a bank of modulators.We have seen that the ability to recover x ( t ) dependson the left invertibility of the matrix Φ L ( ω ) (or Γ L ( ω ) ).With appropriate definitions, our problem becomes that ofrecovering ˆ a ( e jωT ) from the linear set of equations (18) (with Γ L ( ω ) replacing Φ L ( ω ) when preprocessing is used). Ourdefinition of analog sparsity implies that at most K of theFourier transforms ˆ a n ( ω ) have non-zero energy. Therefore,the infinite set of vectors { ˆ a ( e jωT ) , ω ∈ [ − π/T, π/T ] } sharea joint sparsity pattern with at most K rows that are not zero.This in turn allows us to recover { ˆ a ( e jωT ) , ω ∈ [ − π/T, π/T ] } from fewer measurements. Under appropriate conditions, it issufficient that ˆ y ( ω ) has length K , which in general is muchsmaller than N . Thus, fewer measurements are needed with respect to the full model (1). The reduction in the number ofmeasurements corresponds to choosing a smaller bandwidthof the LPF, or reducing the number of modulators.In order to recover the sequences in practice, we rely onthe separation idea advocated in [8]: we first determine thesupport set, namely the active generators. This can be doneby solving a finite dimensional optimization problem under thecondition that Φ L ( ω ) (or Γ L ( ω ) ) are fixed in frequency up toa possible frequency-dependent normalization sequence. Re-covery is then obtained by applying results regarding infinitemeasurement vector (IMV) models to our problem [8]. When Φ L ( ω ) does not satisfy this constraint, we can still convert theproblem to a finite dimensional optimization problem as longas the sequences a k [ n ] are rich [10]. This implies that everyfinite set of vectors share the same frequency support. As ourfocus here is not on the sparse setting, we do not describehere in detail how recovery is obtained. The interested readeris referred to [8], [9], [10] for more details.The main point we want to stress here is that the ideasdeveloped in this paper can also be used to treat the scenarioof recovering a sparse SI signal from its lowpass content.The difference is that now we can relax the requirement forinvertibility of Φ L ( ω ) , Γ L ( ω ) . Instead, it is enough that thesematrices satisfy the known conditions from the compressedsensing literature. This in turn allows in general reductionof the LPF bandwidth, or the number of modulators, incomparison with the non-sparse scenario.VIII. C ONCLUSIONS AND O PEN P ROBLEMS
This paper studied the possibility of recovering signals in SIspaces from their low frequency components. We developednecessary conditions on the minimal bandwidth of the LPFand sufficient conditions on the generators of the SI spacesuch that recovery is possible. We also showed that properpre-processing may facilitate the recovery, and allow to reducethe necessary bandwidth of the LPF. Finally, we discussed howthese ideas can be used to recover sparse SI signals from theoutput of a LPF.An important open problem we leave to future work isthe characterization of the class of generators for which theproposed pre-processing scheme can (or cannot) be applied.To this end, the following question has to be answered. Weformulate it only for the most simple case of one generator(cf. also the discussion in Example 4).
Problem 1:
Let φ ∈ L be an arbitrary function withFourier transform ˆ φ and whose Grammian satisfies (3) .Consider the shift-invariant space spanned by ˆ φ , i.e. S πT ( ˆ φ ) = span n S k πT ˆ φ : k ∈ Z o . For which functions φ ∈ L does there exist a function ˆ γ ∈S ( ˆ φ ) such that ˆ γ ( ω ) = 0 for all ω ∈ [ − π/T, π/T ] .The interesting case is when every function ˆ φ ( ω − k π/T ) , k ∈ Z has at least one zero in the interval [ − π/T, π/T ] . Thenthe question is whether in this case it is still possible to finda linear combination of these functions which has no zero in [ − π/T, π/T ] . R EFERENCES[1] G. K. Hung and K. C. Ciuffreda,
Models of the Visual System . NewYork: Kluwer Academic, 2002.[2] J. W. Goodman,
Introduction to Fourier Optics , 3rd ed. Englewood,CO: Roberts & Co. Publishers, 2005.[3] Y. C. Eldar and T. Michaeli, “Beyond bandlimited sampling: Nonlin-earities, smoothness and sparsity,”
IEEE Signal Process. Mag. , vol. 26,no. 3, pp. 48–68, May 2009.[4] A. Aldroubi and K. Gröchening, “Non-uniform sampling and reconstruc-tion in shift-invariant spaces,”
Siam Review , vol. 43, no. 4, pp. 585–620,2001.[5] C. de Boor, R. DeVore, and A. Ron, “The structure of initely generatedshift-invariant spaces in L ( R d ) ,” J. Funct. Anal. , vol. 119, no. 1, pp.37–78, 1994.[6] J. S. Geronimo, D. P. Hardin, and P. R. Massopust, “Fractal functionsand wavelet expansions based on several scaling functions,”
Journal ofApproximation Theory , vol. 78, no. 3, pp. 373–401, 1994.[7] O. Christensen and Y. C. Eldar, “Generalized shift-invariant systemsand frames for subspaces,”
J. Fourier Analys. Appl. , vol. 11, no. 3, pp.299–313, Jun. 2005.[8] M. Mishali and Y. C. Eldar, “Reduce and Boost: Recovering ArbitrarySets of Jointly Sparse Vectors,”
IEEE Trans. Signal Process. , vol. 56,no. 10, pp. 4692–4702, Oct. 2008.[9] Y. C. Eldar, “Compressed sensing of analog signals in shift-invariantspaces,”
IEEE Trans. Signal Process. , 2009, to appear.[10] ——, “Uncertainty Relations for Analog Signals,”
IEEE Trans. Inf.Theory , 2009, to appear.[11] I. Galton and H. T. Jensen, “Oversampling parallel delta-sigma modula-tor A/D conversion,”
IEEE Trans. Circuits Syst. II , vol. 43, no. 12, pp.801–810, 1996.[12] J. N. Laska, S. Kirolos, M. F. Duarte, T. S. Ragheb, R. G. Baraniuk, andY. Massoud, “Theory and Implementation of an Analog-to-InformationConverter using Random Demodulation,” in
Proc. Intern. Symp. onCircuits and Systems (ISCAS) , New Orleans, USA, May 2007, pp. 1959–1962.[13] M. Mishali and Y. C. Eldar, “From Theory to Practice: Sub-NyquistSampling of Sparse Wideband Analog Signals,”
IEEE J. Sel. TopicsSignal Process. , Feb. 2009, submitted for publ.[14] ——, “Blind multi-band signal reconstruction: Compressed sensing foranalog signals,”
IEEE Trans. Inf. Theory , Sep. 2007, to appear.[15] I. J. Schoenberg,
Cardinal Spline Interpolation . Philadelphia, PA:SIAM, 1973.[16] W. J. Walker, “Zeros of the Fourier transform of a distribution,”
J. Math.Anal. Appl. , vol. 154, no. 1, pp. 77–79, 1991.[17] H. Kharaghani and B. Tayfeh-Rezaie, “A Hadamard matrix of order428,”