Separate Source-Channel Coding for Broadcasting Correlated Gaussians
aa r X i v : . [ c s . I T ] M a y Separate Source-Channel Coding for BroadcastingCorrelated Gaussians
Yang Gao, Ertem Tuncel
University of California, Riverside, CAEmail: [email protected], [email protected]
Abstract —The problem of broadcasting a pair of correlatedGaussian sources using optimal separate source and channelcodes is studied. Considerable performance gains over previouslyknown separate source-channel schemes are observed. Althoughsource-channel separation yields suboptimal performance in gen-eral, it is shown that the proposed scheme is very competitive forany bandwidth compression/expansion scenarios. In particular,for a high channel SNR scenario, it can be shown to achieveoptimal power-distortion tradeoff.
I. I
NTRODUCTION
Consider the problem of transmitting two correlated Gaus-sian sources over a Gaussian broadcast channel with tworeceivers, each of which desires only to recover one of thesources. In [1], it was proven that analog (uncoded) transmis-sion, the simplest possible scheme, is actually optimal whenthe signal-to-noise ratio (SNR) is below a threshold for thecase of matched source and channel bandwidth. To solve theproblem for other cases, various hybrid digital/analog (HDA)schemes have been proposed in [2], [3], [4], and [5]. Infact, the HDA scheme in [5] achieves optimal performancefor matched bandwidth whenever pure analog transmissiondoes not, thereby leading to a complete characterization ofthe achievable power-distortion tradeoff. For the bandwidth-mismatch case, the HDA schemes proposed in [3] and [4]comprise of different combinations of previous schemes usingeither superposition or dirty-paper coding.In all the aforementioned work, authors also comparedachieved performances with that of separate source-channelcoding. Since the channel is degraded, source coding boilsdown to sending a “common” message to both decoders anda “refinement” message to the decoder at the end of the betterchannel. In both of the two source coding schemes proposedin [2], the first source is encoded as the common message, butone scheme encodes (as the refinement message) the secondsource independently, and the other after de-correlating it withthe first source. In [5], on the other hand, the second source isencoded after it is de-correlated with the reconstruction of thefirst source. Although this approach provably yields a betterperformance than the schemes in [2], it is still not optimal.In [6], it was shown that the optimal rate-distortion (RD)tradeoff in this source coding scenario is in fact achieved by ascheme called successive coding, whereby both common andrefinement messages are generated by encoding both sourcesjointly, instead of using any kind of de-correlation. Althoughsuccessive coding is a special case of successive refinement in its general sense, computation of the RD tradeoff, evenfor Gaussians, turned out to be non-trivial. A Shannon-typelower bound derived for the problem was rigorously shownto be tight, yielding an analytical characterization of the RDtradeoff.In this paper, we investigate the performance of sepa-rate source and channel coding for any bandwidth compres-sion/expansion ratio. As discussed in the previous paragraph,the source coding method to be used for optimal performanceis successive coding. We first show that this separate codingscheme achieves the optimal power-distortion tradeoff whenone receiver requires almost lossless recovery, and the otherrequires a small enough distortion. Comparing with best-known schemes and outer bounds, we then show that thisscheme is competitive in other cases as well. Our results implythat with a (sometimes marginal) sacrifice of power-distortionperformance, we can design separate source and channelcodes, and thus enjoy the advantages such as simple extensionto different bandwidth compression/expansion ratios.In Section II, the problem is formally defined. Our mainresults are proved in Section III and the separate codingscheme is compared with other separation-based schemes andhybrid schemes in Section IV.II. P
RELIMINARIES
Encoder Decoder 1Decoder 2( S k , S k ) U n W n W n V n V n ˆ S k ˆ S k Fig. 1. System model.
As depicted in Fig. 1, a pair of correlated Gaussian sources ( S k , S k ) are broadcast to two receivers, and receiver i , i ∈{ , } , is only to reconstruct S ki . Without loss of generality, weassume the source sequences are generated in an i.i.d. fashionby p S S = N (0 , C ) , where C = (cid:20) ρρ (cid:21) and ρ ∈ [0 , . The transmitter encodes the source sequencesto U n and thus can be described mathematically as U n = ( S k , S k ) . We define bandwidth compression/expansion ratio κ = nk with the unit of channel uses per source symbol.The channel also has an average input power constraint,given by n n X j =1 E (cid:2) ( U ( j )) (cid:3) ≤ P .
At receiver i , U n is corrupted by i.i.d. additive Gaussian noise W ni , which satisfies W i ∼ N (0 , N i ) , where we assume that N ≥ N . The channel output V ni is then a Gaussian sequencegiven by V i ( j ) = U ( j ) + W i ( j ) . Decoder reconstructs S k from the channel output V n and can be described as afunction ˆ S k = φ ( V n ) . Analogously, decoder computes ˆ S k = φ ( V n ) . The reconstruction quality is measured withsquared-error distortion, i.e., d ( s k , ˆ s k ) = 1 k k X j =1 ( s j − ˆ s j ) , for any source block s k and reconstruction block ˆ s k . Theproblem is to find the optimal tradeoff between the channelinput power constraint P and the expected distortion pair ( D , D ) achieved at the receivers.In [2], an outer bound to the distortion region is obtained for κ = 1 by assuming full knowledge of S at the second (strong)receiver. In [4], that outer bound is extended to bandwidth-mismatched case, in the form of D ≥ (cid:18) ηP ¯ ηP + N (cid:19) − κ (1) D ≥ (1 − ρ ) (cid:18) ηPN (cid:19) − κ , (2)where η ∈ [0 , and ¯ η = 1 − η .Several separation-based schemes have been previouslyproposed, differing only in their source coding strategy. Inthe first separation-based scheme, termed Scheme A in [2],sources S and S are encoded as if they are independent,resulting in the distortion region given by D ≥ (cid:18) ηP ¯ ηP + N (cid:19) − κ D ≥ (cid:18) ηPN (cid:19) − κ . In Scheme B in [2], the second source is written as S = ρS + E , where S ⊥ E , and S and E are treated as twonew independent sources. Hence we obtain D ≥ (cid:18) ηP ¯ ηP + N (cid:19) − κ D ≥ (1 − ρ ) (cid:18) ηPN (cid:19) − κ + ρ (cid:18) ηP ¯ ηP + N (cid:19) − κ . In the scheme introduced in [5], which we call Scheme C, S is quantized to ˆ S and S is then encoded conditioned on ˆ S . The resultant distortion region becomes D ≥ (cid:18) ηP ¯ ηP + N (cid:19) − κ (3) D ≥ (cid:2) − ρ (1 − D ) (cid:3) (cid:18) ηPN (cid:19) − κ . (4)Of the three, it is obvious that Scheme C achieves the bestperformance. However, it is still not optimal as we will showin Section IV. The optimal strategy is in fact what is calledsuccessive coding in [6], whereby the sources are encodedjointly at both the common and the refinement layers. TheRD tradeoff for successive coding of Gaussian sources withsquared-error distortion was given in [6] parametrically withrespect to α ∈ [0 , as R ( α ) = 12 log 1 − ρ D (1 − ν δ ) − ( ρ − νδ ) R ( α ) = (cid:20)
12 log 1 − ν δD (cid:21) + , where δ = 1 − D , [ x ] + = max { x, } , and ν = (cid:26) ν , if D < − ν δν ∗ , if 1 − ν δ ≤ D < − ρ δ with ν ∗ = q − D δ , and ν is the unique root of f α ( ν ) = (1 − α )( ρ − νδ )(1 − νρ ) − α ( ν − ρ )(1 − ν δ ) in the interval [ ρ, min( ρ , ρδ )] .III. M AIN R ESULTS
We first show the RD region of successive coding can besimplified by eliminating both the parameter α and the needto find the roots of the cubic polynomial f α ( ν ) . Lemma 1:
The achievable source coding rate pair ( R , R ) ,for any distortion pair ( D , D ) , is given by R ( ν ) = 12 log 1 − ρ D (1 − ν δ ) − ( ρ − νδ ) (5) R ( ν ) = (cid:20)
12 log 1 − ν δD (cid:21) + , (6)where ν ∈ h ρ, min( ρ , ρδ , ν ∗ ) i .The proof is deferred to Appendix A.In separate coding, the region of all achievable ( P, D , D ) triplets can be determined using one of two methods. Theconventional method fixes P and searches for the lowerenvelope of all ( D , D ) whose source rate region intersectswith the capacity region given in [7]. Alternatively, we can fix ( D , D ) and search for the minimum P whose correspondingcapacity region intersects with the source rate region given inLemma . We find this alternative both more convenient andmore meaningful. More specifically, it is easier to compare When D > − ρ (1 − D ) , the optimal strategy degenerates into sendingonly a common message and estimating S k solely from ˆ S k . So this trivialcase is excluded from the discussion in the sequel. chemes based on the minimum power they need to achievethe same distortion pair, and the ratio of minimum powersyields a single number as a quality measure.To be able to use this alternative, first we need to find outthe minimum required power for any given source coding ratepair ( R , R ) . Lemma 2:
For any source coding rate pair ( R , R ) , theminimal required power is given by P ( R , R ) = N (cid:16) R /κ − (cid:17) + N (cid:16) R /κ − (cid:17) R /κ . (7) Proof:
For a Gaussian broadcast channel where the betterreceiver is the second one, R > and R > , rates ofcommon and private information, respectively, can be achievedif and only if there exists ≤ η ≤ such that R ≤ κ (cid:18) ηP ¯ ηP + N (cid:19) R ≤ κ (cid:18) ηPN (cid:19) where ¯ η = 1 − η . This, in turn, implies that P is achievableif and only if there exists < ¯ η < − R /κ such that P ≥ max ( N (cid:0) R /κ − (cid:1) − ¯ η R /κ , N (cid:0) R /κ − (cid:1) ¯ η ) . Since the terms in the maximum exhibit opposite monotonicitywith respect to ¯ η with asymptotes at ¯ η = 0 and ¯ η = 2 − R /κ ,the minimum power is achieved when the two terms are equal,that is, when ¯ η = N (2 R /κ − N (2 R /κ −
1) + N (2 R /κ − R /κ , and has the form in (7).By substituting (5) and (6) into (7), we obtain the minimumpower required for the separate coding scheme as a functionof ν : P ( ν ) = N (cid:20) − ρ D (1 − ν δ ) − ( ρ − νδ ) (cid:21) /κ − ! + N "(cid:18) − ν δD (cid:19) /κ − − ρ D (1 − ν δ ) − ( ρ − νδ ) (cid:21) /κ . (8)For bandwidth-matched case, the minimum power of sepa-rate coding P sep = min ν P ( ν ) can actually be found analyti-cally for any ( D , D ) . We omit the details here.The following theorem is our first main result. Theorem 1:
Separate source-channel coding achieves opti-mal power-distortion tradeoff when ( D , D ) satisfies eitherof the following conditions1) D → and D ≤ − ρ ,2) D → and D ≤ − ρ . Proof:
We first find the minimum power the outer bound(1) and (2) requires. Note that when D > − ρ , (2) will holdfor any η ∈ [0 , , and hence the minimum power is obtained solely from (1), whereas when D ≤ − ρ , the minimumpower satisfies equality in both (1) and (2). Combining thetwo cases, we obtain the concise expression P outer bound = N D − /κ "(cid:18) D − ρ (cid:19) − /κ − + + N ( D − /κ − . On the other hand, from (8), we have P ( ρ ) = N D − /κ "(cid:18) D − ρ δ (cid:19) − /κ − + N ( D − /κ − . Since δ = 1 − D , it is easy to see that when D ≤ − ρ P ( ρ ) P outer bound → as D → , and since ν = ρ is feasible, the minimum power of separatecoding satisfies P sep ≤ P ( ρ ) . Therefore the performance ofseparate coding scheme approaches the outer bound, or P sep P outer bound → , when D ≤ − ρ and D → . Similarly, by setting ν = ρδ , we have P (cid:16) ρδ (cid:17) = N " − ρ D (1 − ρ δ ) /κ − + N − ρ δ D ! /κ − " − ρ D (1 − ρ δ ) /κ , and when D → , P ( ρδ ) P outer bound → . Note when (1 − D )(1 − D ) ≥ ρ , min (cid:16) ρ , ρδ , ν ∗ (cid:17) = ρδ , which again implies P sep ≤ P ( ρδ ) , thus proving the second part of the theorem. Remark 1:
Here we proved that the outer bound is tight inthe region of (1 − D )(1 − D ) ≥ ρ when either D or D goes to , and the performance of separate coding approachesthe outer bound. The condition that either D or D goes to translates to infinite channel SNR.In fact, as we show in the following theorem, separatecoding is approximately optimal for the entire region (1 − D )(1 − D ) ≥ ρ , in the sense that the power ratio P sep P outer bound can be upper-bounded universally in ( N , N , D , D ) . Theorem 2:
When (1 − D )(1 − D ) ≥ ρ , P sep P outer bound ≤ min ( ρ ) /κ (cid:2) (1 + ρ ) /κ − (cid:3) (1 + ρ ) /κ − (1 − ρ ) /κ , (cid:18) (1 + ρ ) ρ (cid:19) /κ + (cid:16) (1+ ρ ) ρ (cid:17) /κ − − ρ ) − /κ − ) . roof: The first half is true because P sep P outer bound ( a ) ≤ P ( ρ ) P outer bound = N N h (1 − ρ δ ) /κ − D /κ i + D /κ (1 − D /κ ) N N h (1 − ρ ) /κ − D /κ i + D /κ (1 − D /κ ) ( b ) ≤ (1 − ρ δ ) /κ − ( D D ) /κ (1 − ρ ) /κ − ( D D ) /κ ( c ) ≤ (1 − ρ δ ) /κ − (1 − ρ ) /κ (1 − ρ ) /κ − (1 − ρ ) /κ ( d ) ≤ ρ ) /κ (cid:2) (1 + ρ ) /κ − (cid:3) (1 + ρ ) /κ − (1 − ρ ) /κ , where ( a ) follows since ν = ρ is feasible, ( b ) by N N ≤ , ( c ) follows since D D ≤ (1 − ρ ) , and ( d ) since δ ≥ ρ .By relaxing P sep to P (cid:16) ρ √ δ (cid:17) , the second half of the boundcan be obtained in a similar way. The detailed proof is omittedhere due to lack of space.IV. P ERFORMANCE COMPARISON
A. Separation-based schemes
As illustrated in [5] for κ = 1 , the outer bound in (1) and(2) is not always tight. Nevertheless, we can still compare P sep and P outer bound for any ( D , D ) , which provides anupper bound to the ratio of the minimum separate and jointcoding power levels. To show the optimality of our separatecoding scheme, we also compare our scheme with SchemeC, which provides the best performance among the threeseparation-based schemes mentioned earlier. The minimumrequired power of Scheme C can be obtained from (3) and(4) as P C = N D − /κ "(cid:18) D − ρ δ (cid:19) − /κ − + N ( D − /κ − . Note that P C = P ( ρ ) in the non-trivial distortion regions,so it is immediately clear that the successive coding schemeoutperforms Scheme C.As an example with bandwidth compression, we show thepower ratio between our separate coding scheme and theouter bound in Figure 2(a), and that between Scheme C andour separate coding scheme in Figure 2(b), both in dB. Forreference, the black curves illustrate the different distortionregions for a related problem in [8], where only one receiveris in presence and interested in reconstructing both sources.The lower left corner region is actually (1 − D )(1 − D ) ≥ ρ ,where, in general, small dB differences are observed, asimplied by the two theorems above. As can be seen fromthe figure, even for highly correlated sources, the optimumseparate coding scheme does not require too much extra powerin most of the ( D , D ) plane. Again, since the outer boundis not always tight, the large power difference in some regionsmay be dramatically reduced when the outer bound is replacedby the optimum performance. For smaller ρ values, we observe
10 log P Sep P OB , N = 1 , N = 0 . , ρ = 0 . , κ = 0 . D (a) Power difference between separate coding and the outer bound.
10 log P C P Sep , N = 1 , N = 0 . , ρ = 0 . , κ = 0 . D (b) Power difference between Scheme C and optimal separate coding.Fig. 2. Comparison between the power of outer bound, Scheme C andoptimal separate coding. ρ = 0 . , N = 1 , N = 0 . and κ = 0 . . that the power difference is very small in the entire plane, as anexample illustrates in the next section. This is a natural resultbecause separate coding is optimal for independent sourcesand small ρ value means the sources are not highly dependent.There is also noticeable power difference between ourscheme and Scheme C, and we numerically observe large dBvalues near the point D = 1 − ρ and D = 0 , for which wecan obtain analytically the power ratio. Theorem 3:
When D = 1 − ρ and D → , P C P sep → (1 + ρ ) /κ . The proof uses the fact that the power of our separate codingscheme goes to that of the outer bound when approaching thisoint.When the two receivers have the same noise level, i.e., N = N , it can be shown that our separate coding scheme achievesthe corresponding rate-distortion function R ( D , D ) in [8],whereas none of the three separate coding schemes mentionedearlier has the same performance. B. Hybrid Digital/Analog (HDA) schemes −8 −7 −6 −5 −4 −3 −2 −1−10−9.5−9−8.5−8−7.5−7−6.5−6−5.5−5 10log D l og D P = 1.995 N = 1 N = 0.3162 ρ = 0.2 κ = 2 Outer boundRFZ/HWZSeparate (a) ρ = 0 . −14 −13 −12 −11 −10 −9 −8 −7 −6 −5 −4−10−9−8−7−6−5−4 10log D l og D P = 1.995 N = 1 N = 0.3162 ρ = 0.8 κ = 2 Outer boundRFZ/HWZSeparate (b) ρ = 0 . Fig. 3. Comparison between outer bound, RFZ scheme in [4] and separatecoding. P = 3 dB = 1 . , N = 0 dB = 1 , N = − dB = 0 . and κ = 2 . In [3] and [4], a group of hybrid digital/analog (HDA)schemes were proposed for bandwidth-mismatched case,where analog, digital, and hybrid schemes are layered withsuperposition or dirty-paper-coding. The achievable distortionregion can be found by varying power allocation and scalingcoefficients. In [4], an HDA scheme from [9] for broadcastinga common source with bandwidth expansion was adapted forthe problem of broadcasting correlated sources and is termed the RFZ scheme. In addition, a scheme, termed the HWZscheme, containing an analog layer and two digital layerseach with a Wyner-Ziv coder and a channel coder, was alsoproposed. It is argued by an example in [4] that the HWZscheme performs similar to the RFZ scheme. Here we compareour separate coding scheme with the outer bound and theRFZ/HWZ scheme in Figure 3. For this comparison, werevert to the more familiar ( D , D ) plot for the exact same ( P, ρ, κ, N , N ) as those used in the examples in [4].As seen in Figure 3(a), when ρ is small, the separate codingscheme almost coincides with the outer bound and outperformsRFZ/HWZ schemes. When the sources are highly correlatedas in Figure 3(b), the separate coding scheme is still betterthan the RFZ/HWZ schemes when D is lower than a certainvalue, and also provides competitive performance when it ishigher. We observed similar performance behavior when wecompared the separate coding scheme to the layered schemesin [3] for bandwidth compression.V. C ONCLUSION
The performance of optimum separate source-channel cod-ing scheme for broadcasting two correlated Gaussians is ana-lyzed. The minimum power required for a given distortion pairis used as a tool to compare performances of different schemes.It is illustrated that this separate coding scheme outperformsother known separate schemes, and is competitive in generalin the sense that its minimum required power is close tothe power implied by the outer bound. Also, in a certain“low distortion” region, the power difference is analyticallybounded. In fact, within this region, in the extreme cases ofalmost lossless reconstruction of either source, the separatescheme is provably optimal.A
PPENDIX AP ROOF OF L EMMA The cubic function in [6] is f η ( ν ) = (1 − η )( ρ − νδ )(1 − νρ ) − η ( ν − ρ )(1 − ν δ ) and when f η ( ν ) = 0 , it can be re-written as η = ( ρ − νδ )(1 − νρ ) ν [1 − ρ − δ (1 − νρ + ν )] . It will be shown that varying η in [0 , is equivalent withvarying ν in (cid:20) ρ, min( ρ , ρδ , q − D δ ) (cid:21) , by showing η is amonotonically decreasing function of ν .When δ > ρ , ρ < ρδ < ρ and also note η = 1 and 0 when ν = ρ and ρδ , respectively. We examine h ( ν ) = ρ − νδ − ρ − δ (1 − νρ + ν ) instead of η , and dhdν ∝ − − ρ + δ + 2 νρ − ν δ . The performance of RFZ is plotted to represent both schemes as theircurves almost coincide at least for this set of parameters in [4]. he right hand side is a quadratic function of ν centered at ν = ρδ and the maximum value is − (1 − δ )(1 − ρ δ ) ≤ .(When ν = ρ , the function value is − (1 − ρ )(1 − δ ) .)Similarly, when δ < ρ , we have ρ < ρ < ρδ and in thiscase, η = 1 and 0 when ν = ρ and ρ . We examine h ( ν ) = 1 − νρ − ρ − δ (1 − νρ + ν ) , and thus have dhdν ∝ − ρ (1 − ρ ) − ρδ + 2 νδ − ν ρδ . The right hand side is centered at ν = ρ and the maximumvalue is − (1 − ρ )( ρ − δρ ) ≤ . (cid:4) R EFERENCES[1] S. Bross, A. Lapidoth, and S. Tinguely, “Broadcasting correlated Gaus-sians,”
Proc. IEEE Int. Symp. Inf. Theory (ISIT 2008) , Toronto, ON, July2008.[2] R. Soundararajan and S. Vishwanath, “Hybrid coding for Gaussianbroadcast channels with Gaussian sources,”
Proc. IEEE Int. Symp. Inf.Theory (ISIT 2009) , Seoul, Korea, June 2009.[3] H. Behroozi, F. Alajaji, and T. Linder, “Hybrid digital-analog jointsource-channel coding for broadcasting correlated Gaussian sources,”
Proc. IEEE Int. Symp. Inf. Theory (ISIT 2009) , Seoul, Korea, June 2009.[4] H. Behroozi, F. Alajaji, and T. Linder, “Broadcasting correlated Gaussiansources with bandwidth expansion,”
IEEE Inf. Theory Workshop (ITW2009) , Taormina, Italy, October 2009.[5] C. Tian, S. Diggavi, and S. Shamai, “The achievable distortion region ofbivariate Gaussian source on Gaussian broadcast channel,”
Proc. IEEEInt. Symp. Inf. Theory (ISIT 2010) , Austin, TX, June 2010.[6] J. Nayak and E. Tuncel, “ Successive coding of correlated sources,”
IEEETrans. Inf. Theory , vol. 55, no. 9, pp. 4286-4298, September 2009.[7] P. Bergmans, “A simple converse for broadcast channels with additivewhite Gaussian noise,”
IEEE Trans. Inf. Theory , vol. 20, no. 2, pp. 279-280, May 1974.[8] J. -J. Xiao and Z. -Q. Luo, “Compression of correlated Gaussian sourcesunder individual distortion criteria,”
Proc. 43rd Allerton Conf. Commun.Control Comput. , pp. 438-447, September 2005.[9] Z. Reznic, M. Feder, and R. Zamir, “Distortion bounds for broadcastingwith bandwidth expansion,”