Multiple Access Channel Simulation
Gowtham R. Kurri, Viswanathan Ramachandran, Sibi Raj B. Pillai, Vinod M. Prabhakaran
11 Multiple Access Channel Simulation
Gowtham R. Kurri, Viswanathan Ramachandran, Sibi Raj B. Pillai, Vinod M. Prabhakaran
Abstract
We study the problem of simulating a two-user multiple access channel over a multiple access network of noiseless links. Twoencoders observe independent and identically distributed (i.i.d.) copies of a source random variable each, while a decoder observesi.i.d. copies of a side-information random variable. There are rate-limited noiseless communication links and independent pairwiseshared randomness resources between each encoder and the decoder. The decoder has to output approximately i.i.d. copies ofanother random variable jointly distributed with the two sources and the side information. We are interested in the rate tupleswhich permit this simulation. This setting can be thought of as a multi-terminal generalization of the point-to-point channelsimulation problem studied by Bennett et al. (2002) and Cuff (2013). General inner and outer bounds on the rate region arederived. For the specific case where the sources at the encoders are conditionally independent given the side-information atthe decoder, we completely characterize the rate region. Our bounds recover the existing results on function computation oversuch multi-terminal networks. We then show through an example that an additional independent source of shared randomnessbetween the encoders strictly improves the communication rate requirements, even if the additional randomness is not availableto the decoder. Furthermore, we provide inner and outer bounds for this more general setting with independent pairwise sharedrandomness resources between all the three possible node pairs.
Index Terms
Channel simulation, strong coordination, pairwise shared randomness, multiple access channel, random binning.
I. I
NTRODUCTION
What is the minimum amount of communication required to create correlation remotely? The channel simulation problemseeks to answer this fundamental question. In the point-to-point formulation, an encoder observing an independent andidentically distributed (i.i.d.) source X n with distribution q X sends a message through a noiseless link to a decoder. Thedecoder has to output Y n such that the total variation distance between the joint distribution on ( X n , Y n ) and the i.i.d. jointdistribution induced by passing the source X n through a discrete memoryless channel q Y | X vanishes asymptotically. Thisrequirement that the synthesized joint distribution be close to the desired i.i.d. joint distribution in total variation distance hasbeen termed as strong coordination [1], which is also the focus of this paper. A source of common randomness accessible toboth the encoder and the decoder may assist them in the aforementioned task. This framework was first investigated by Bennettet al. [2] assuming unlimited common randomness, where they established a ‘reverse Shannon theorem’ to synthesize a noisychannel from a noiseless channel . It was shown that the minimum communication rate is nothing but the mutual information I ( X ; Y ) of the joint distribution. Harsha et al. [3] studied the non-asymptotic version of this problem. Winter [4] studied thesetting with limited common randomness, albeit only for a certain extremal operating point. Cuff [5] and Bennett et al. [6]independently determined the entire optimal trade-off between communication and shared randomness rates. Later, Wilde etal. [7] obtained a similar trade-off in the quantum information-theoretic setting, generalizing the above result. Yassaee et al. [8]established a similar trade-off for channel simulation in a point-to-point network with side-information at the decoder. Aweaker form of coordination, namely, empirical coordination , where only the empirical distribution of the sequence of samplesis required to be close to the desired distribution, has also been studied in point-to-point networks [1], [8]–[12].Channel simulation problems may also be thought of as distributed computation of randomized functions. Distributedcomputation of (deterministic) functions has received much attention in the computer science literature (see, e.g., [13],[14] and references therein) and the information theory literature [15]–[30]. Two terminal interactive function computationwas studied in [16], [17], [19], [20], [24]. Distributed multi-terminal function computation in a multiple-access network ofnoiseless links was studied by K¨orner and Marton [15], Han and Kobayashi [18], Kuzuoka and Watanabe [27], Watanabe [30],and Sefidgaran et al. [28]. Function computation in more general graph networks where a single node seeks to compute afunction of the inputs at the other nodes was studied by Appuswamy et al. [25], Kowshik and Kumar [26], and Sefidgaranand Tchamkerten [29]. A related line of work is function computation over multiple-access channels studied by Nazer andGastpar [21], [22], and Gastpar [23], which, in contrast to the above works, exploit the computation performed by the channelitself. This is fundamentally different from computation over a network of noiseless links where the communication channeldoes not perform any computation. G. R. Kurri is with the School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, AZ 85287 USA(email:[email protected]). Viswanathan Ramachandran and Sibi Raj B. Pillai are with the Department of Electrical Engineering, Indian Instituteof Technology Bombay, India (email: { viswanathan,bsraj } @ee.iitb.ac.in. V. M. Prabhakaran is with the School of Technology and Computer Science, TataInstitute of Fundamental Research, Mumbai 400005, India (email: [email protected]). This work was done while Gowtham R. Kurri was at the Tata Instituteof Fundamental Research. This paper was submitted in part to IEEE ISIT 2021. Referring to Shannon’s channel coding theorem as the simulation of a noiseless channel using a noisy channel. a r X i v : . [ c s . I T ] F e b Enc Dec M ∈ [1 : 2 nR ] Y n S ∼ Unif [1 : 2 nR ] S ∼ Unif [1 : 2 nR ] X n Enc X n M ∈ [1 : 2 nR ] Z n Fig. 1. Strong coordination over a multiple-access network. Encoder j ∈ { , } on observing the source X nj and shared randomness S j sends a message M j over a noiseless link to the decoder, which has side-information Z n . Here ( X i , X i , Z i ) , i = 1 , . . . , n , are i.i.d. with q X X Z . Also ( X n , X n , Z n ) , S ,and S are mutually independent. The decoder has to output Y n such that ( X i , X i , Z i , Y i ) , i = 1 , . . . , n , are approximately i.i.d. with q X X ZY . There are relatively few conclusive results on channel simulation in multi-terminal networks where possibly randomizedfunctions of the inputs need to be computed. A natural generalization of the point-to-point network to a multi-terminal setting isthe cascade network [31], [32]. Satpathy and Cuff [31] considered a cascade network under a secrecy requirement and obtainedthe optimal trade-off between communication and common randomness rates. Vellambi et al. [32] obtained the optimal ratesfor specific settings, e.g., when the communication topology matches the coordination structure. Empirical coordination inmulti-terminal networks has been studied in [1], [33], [34].In this paper, we study multiple access channel simulation (randomized function computation) over a multiple access networkof noiseless links. In particular, there are two encoders who share noiseless communication links with a decoder – see Figure 1.The two encoders and the decoder observe i.i.d. copies of the sources X and X , and the side-information Z , respectively, thatare generated according to a given distribution q X X Z . Also, each encoder has access to a source of pairwise shared randomnesswith the decoder, which are independent of each other and also jointly independent of the sources and side information. Notethat this pattern of shared randomness reflects the communication network topology, in that the encoders of a multiple accesschannel (MAC) [35]–[37] usually cannot cooperate with one another. The pairwise shared randomness resources may be thoughtof as being generated by the users via communication at some point in the past, possibly using the same network. In addition,the encoders and the decoder may privately randomize. The encoders transmit messages through noiseless links to the decoder,whose output together with the input sources and decoder side-information should be approximately i.i.d. with q X X ZY .Furthermore, we study a generalization of this setting to allow for encoder cooperation using privately shared randomnessbetween the encoders – see Figure 2. One possible motivation for this configuration of shared randomness resources is thesetting of multiple access channels with partially cooperating encoders [38] – see also [39], [40]. However, the encoders arenot allowed to cooperate here after observing the sources.The problem of finding the optimal communication rates for computing possibly randomized functions at the decoder inthis multiple-access network of noiseless links remains largely open even for the case of independent sources. Sefidgaran andTchamkerten [29] determined the optimal communication rates for computing deterministic functions when the sources at theencoders are conditionally independent given the side-information at the decoder, i.e., I ( X ; X | Z ) = 0 . Note that Sefidgaranand Tchamkerten [29] in fact studied a more general setting consisting of multiple terminals over a rooted multi-level directedtree, where the multiple-access network of noiseless links is a special case. Atif et al. [41] studied multiple access channelsimulation in the presence of three-way common randomness instead of pairwise shared randomness as above and obtainedan achievable inner bound. Atif et al. [42] obtained a similar inner bound in the quantum information-theoretic setting. Whenthe (possibly randomized) function being computed has a certain algebraic structure, Atif and Pradhan [43] showed that thisstructure can be exploited to obtain better inner bounds using structured codes in the spirit of K¨orner and Marton [15]. Main Contributions.
We derive general inner and outer bounds (Theorems 1 and 2) for the multiple access channel simulationproblem, and obtain the optimal trade-off between communication and shared randomness rates when I ( X ; X | Z ) = 0 (Theorem 3). While the result of Sefidgaran and Tchamkerten [29] for deterministic function computation specialized to themultiple-access network of noiseless links is a special case of our Theorem 3, we remark here that the techniques used therecannot be readily generalized to randomized function computation (see the discussion below Theorem 3, and Remark 3). Aninner bound for the setting with three-way common randomness amongst all the users (instead of pairwise shared randomness) It turns out that shared randomness does not aid in deterministic function computation – see Remark 1. can be inferred from our inner bound in Theorem 1, and is potentially stronger than that of [41], and also that of [42] specializedto the classical setting (see Remark 2). Finally, we also consider a setting where the encoders can share randomness that is notavailable to the decoder (see Figure 2). We show that this strictly improves the communication rates required for coordination(see Example 1) and also obtain general inner and outer bounds (Theorems 4, 5, and 6).The remainder of this paper is organized as follows. We present our system model (with pairwise shared randomness betweeneach encoder and the decoder) in Section II, and give the main results for this model in Section III. In Sections IV and V,we consider the case where the model is augmented with independent pairwise shared randomness between the encoders.In Section IV we present an example where this may lead to a strict improvement of the communication rate region. Innerand outer bounds on the rate region for this setting are presented in Section V. The proofs for our main results are given inSection VI. II. S
YSTEM M ODEL
We study the problem of strong coordination of signals in a three-node multiple-access network. There are two encoders withinputs X n and X n , respectively, and a decoder with side-information Z n , where ( X i , X i , Z i ) , i = 1 , . . . , n , are independentand identically distributed (i.i.d.) with distribution q X X Z , with X , X , and Z taking values in finite alphabets X , X ,and Z , respectively. For j = 1 , , encoder j and the decoder have access to a pairwise shared randomness S j uniformlydistributed on [1 : 2 nR j ] . The random variables S and S are independent of each other and also jointly independent of ( X n , X n , Z n ) . Encoder j ∈ { , } observes X nj and shared randomness S j , and sends a message M j ∈ [1 : 2 nR j ] over anoise-less communication link to the decoder. The decoder observes M , M , in addition to the shared randomness S , S , and side-information Z n . The goal is to output Y n (where Y i , i = 1 , . . . , n, takes values in a finite alphabet Y ) which alongwith the input sources and decoder side-information is approximately distributed according to q ( n ) X X ZY ( x n , x n , z n , y n ) := (cid:81) ni =1 q X X ZY ( x i , x i , z i , y i ) (see Figure 1). Definition 1. A (2 nR , nR , nR , nR , n ) code consists of two randomized encoders p E ( m | s , x n ) and p E ( m | s , x n ) and a randomized decoder p D ( y n | s , s , m , m , z n ) , where s j ∈ [1 : 2 nR j ] , m j ∈ [1 : 2 nR j ] , j = 1 , . The joint distribution of ( S , S , X n , X n , Z n , M , M , Y n ) and the resulting induced joint distribution on ( X n , X n , Z n , Y n ) are respectively given by p ( s , s , x n , x n , z n , m , m , y n ) = 12 n ( R + R ) p ( x n , x n , z n ) (cid:89) j =1 p E j ( m j | s j , x nj ) p D ( y n | s , s , m , m , z n ) , and p ind ( x n , x n , z n , y n ) = (cid:88) s ,s ,m ,m p ( s , s , x n , x n , z n , m , m , y n ) . Recall that the total variation between two p.m.f.’s p X and q X on the same alphabet X is defined as || p X − q X || (cid:44) (cid:88) x ∈X | p X ( x ) − q X ( x ) | . Definition 2.
A rate tuple ( R , R , R , R ) is said to be achievable for a distribution q X X ZY if there exists a sequence of (2 nR , nR , nR , nR , n ) codes such that lim n →∞ || p ind X n ,X n ,Z n ,Y n − q ( n ) X X ZY || = 0 , (1)where q ( n ) X X ZY ( x n , x n , z n , y n ) := n (cid:89) i =1 q X X ZY ( x i , x i , z i , y i ) . Remark 1. If q X X ZY is such that Y is a deterministic function of ( X , X , Z ) , then the pairwise shared randomness andthe private randomness at the encoders and the decoder do not have any effect on the communication rates. In fact, moregenerally, common randomness available to both the encoders and the decoder does not help to improve communication ratesin this case. This follows from standard probabilistic method arguments . To see this, notice that if Y = f ( X , X , Z ) , then (1) reduces to lim n →∞ P ( Y n (cid:54) = f ( X n , X n , Z n )) = 0 . Let W denote the random variable correspondingto the common randomness. A simple application of the law of total probability implies that there exists a realization w ∗ of W such that P ( Y n (cid:54) = f ( X n , X n , Z n ) | W = w ∗ ) ≤ P ( Y n (cid:54) = f ( X n , X n , Z n )) . Therefore, new deterministic encoding and decoding functions can be defined by fixing W = w ∗ and at the same time not increasing the probability of error. Definition 3.
The rate region R MAC-coord is the closure of the set of all achievable rate tuples ( R , R , R , R ) . The communication rate region is given by { ( R , R ) : ∃ R and R s.t. ( R , R , R , R ) ∈ R MAC-coord } . The communication rate region can be thought of as the trade-off between the communication rates R and R undersufficiently large pairwise shared randomness rates R and R . We also consider a set-up where there is an additionalshared randomness between the encoders. A code, an achievable rate tuple and the rate region can be defined analogouslythere (see Section V for details). III. I NNER AND O UTER B OUNDS , AND A R ATE R EGION
The following theorem provides an inner bound to the region R MAC-coord . Theorem 1 (Inner Bound) . Given a p.m.f. q X X ZY , the rate tuple ( R , R , R , R ) is in R MAC-coord if R ≥ I ( U ; X | U , Z, T ) R ≥ I ( U ; X | U , Z, T ) R + R ≥ I ( U , U ; X , X | Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) − I ( U ; U | Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) − I ( U ; U | Z, T ) R + R + R ≥ I ( U ; X , X , Y | Z, T ) + I ( U ; X | U , Z, T ) R + R + R ≥ I ( U ; X , X , Y | Z, T ) + I ( U ; X | U , Z, T ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x ,x , z, t, u , u , y ) = p ( x , x , z ) p ( t ) p ( u | x , t ) p ( u | x , t ) p ( y | u , u , z, t ) such that (cid:88) u ,u p ( x , x , z, u , u , y | t ) = q ( x , x , z, y ) , for all t. Our proof of Theorem 1 is in the spirit of the Berger-Tung inner bound [44], [45] and employs the Output Statistics ofRandom Binning (OSRB) framework developed by Yassaee et al. [46]. A detailed proof is given in Section VI-A.
Remark 2.
The inner bound in Theorem 1 implies an inner bound for the setting studied by Atif et al. [41] (and also theclassical version of the quantum model in Atif et al. [42]), where there is three-way common randomness instead of pairwiseshared randomness as above. Furthermore, this implied inner bound is potentially stronger than the corresponding inner boundof [41, Theorem 2] (with a correction [47]) and also the classical version of [42, Theorem 6]. See Appendix A for the details.Also, we mention here that in the settings of [41] and [42], the available three-way common randomness is simply used toextract independent pairwise shared randomness between each encoder and the decoder (similar to Figure 1). However, wewill show in Section IV that the inner bound in Theorem 1 can be strictly improved if additional independent pairwise sharedrandomness between the encoders is available.We now provide an outer bound to the region R MAC-coord . Theorem 2 (Outer Bound) . Given a p.m.f. q X X ZY , any rate tuple ( R , R , R , R ) in R MAC-coord satisfies R ≥ max { I ( U ; X | Z, T ) , I ( U ; X | U , X , Z, T ) } R ≥ max { I ( U ; X | Z, T ) , I ( U ; X | U , X , Z, T ) } R + R ≥ I ( U , U ; X , X | Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x ,x , z, t, u , u , y ) = p ( x , x , z ) p ( t ) p ( u , u | x , x , t ) p ( y | u , u , z, t ) such that p ( u | x , x , z, t ) = p ( u | x , t ) ,p ( u | x , x , z, t ) = p ( u | x , t ) , (cid:88) u ,u p ( x , x , z, u , u , y | t ) = q ( x , x , z, y ) , for all t. Theorem 2 is, in fact, a direct consequence of our more general result stated as Theorem 5. The details are given inSection VI-B.When the random variables X and X are conditionally independent given Z , we can show that the inner bound ofTheorem 1 is tight by obtaining a stronger outer bound, thus completely characterizing the rate region. Theorem 3 (Rate Region - Conditionally Independent Sources Given the Side Information) . Consider a p.m.f. q X X ZY suchthat the random variables X and X are conditionally independent given Z . Then the rate region R MAC-coord is given by theset of all rate tuples ( R , R , R , R ) such that R ≥ I ( U ; X | Z, T ) R ≥ I ( U ; X | Z, T ) R + R ≥ I ( U ; X , Y | X , Z, T ) R + R ≥ I ( U ; X , Y | X , Z, T ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x , x , z, t, u , u , y ) = p ( z ) p ( x | z ) p ( x | z ) p ( t ) p ( u | x , t ) p ( u | x , t ) p ( y | u , u , z, t ) (2) such that (cid:80) u ,u p ( x , x , z, u , u , y | t ) = q ( x , x , z, y ) , for all t . In Remark 4 (on page 7), we show that the outer bound implicit in Theorem 3 is strictly stronger than that of Theorem 2.The non-trivial part in the converse of Theorem 3 is that we single-letterize the distributed protocol in order to obtain a p.m.f.structure matching that of the inner bound in Theorem 1, particularly leveraging the conditional independence of the sourcesgiven the side-information. In general, obtaining single-letter forms matching the inner bound is known to be notoriouslydifficult for distributed source coding problems [44], [45]. It is interesting to note that for the case of deterministic functioncomputation when the sources are conditionally independent given the side-information, the inner and outer bounds of Sefidgaranand Tchamkerten [29] specialized to the two-user multiple-access network of noiseless links (which can be recovered fromour Theorems 1 and 2 – see Remark 3 below) match, analogous to a result of Gastpar [48]. However, for randomized functioncomputation, the inner and outer bounds in Theorems 1 and 2 do not match for sources conditionally independent givenside-information and we need a strictly stronger outer bound to show Theorem 3. A detailed proof of Theorem 3 is given inSection VI-A (achievability) and Section VI-B (converse).
Remark 3.
As mentioned earlier (see Remark 1), when Y is a deterministic function of ( X , X , Z ) , the pairwise sharedrandomness does not have any effect on the communication rates. Indeed, the rate constraints in Theorems 1, 2, and 3 involvingshared randomness rates become redundant, as we show in Appendices B, C and D respectively. In Appendix C, we also showthat our outer bound in Theorem 2 restricted to deterministic function computation implies the outer bound of Sefidgaranand Tchamkerten [29, Theorem 1] specialized to the two-user multiple-access network of noiseless links (also reported in[49, Corollary 2]). However, it is unclear whether our outer bound strictly subsumes the latter for deterministic functions. Inappendices B and D, we also show that for deterministic function computation, our Theorems 1 and 3, respectively, reduce toTheorems 2 and 3 of Sefidgaran and Tchamkerten [29] specialized to the two-user multiple-access network of noiseless links,also reported in [49, Proposition 1 and Theorem 3].Sefidgaran and Tchamkerten [49] already observed that their inner bound (and hence, our Theorem 1) is not tight, ingeneral. The deterministic function computation problem of K¨orner and Marton [15] illustrates this (see [50, Example 2]).For computing the mod- sum of binary X and X with symmetric input distribution, K¨orner and Marton [15] showed thatstructured codes can strictly outperform standard random coding schemes. Achievable schemes using algebraic-structured codesexploiting the specific structure of the function to be computed were further explored by Krithivasan and Pradhan [51] andAtif and Pradhan [43], where the latter considered the quantum setting.IV. R OLE OF S HARED R ANDOMNESS B ETWEEN THE E NCODERS
In this section, we show that if the encoders share additional independent randomness (see Figure 2), the communicationrates can be strictly improved, even if the additional randomness is not available to the decoder. This is done via an example forwhich we first explicitly compute the communication rate region of Theorem 3 when there is no shared randomness betweenthe encoders, assuming sufficiently large pairwise shared randomness rates. Then we show that a rate pair outside this regionis achievable in the presence of shared randomness between the encoders. This discussion also motivates the next sectionwhere we obtain general inner and outer bounds to the rate coordination region in the presence of shared randomness betweenthe encoders.
Enc Dec M ∈ [1 : 2 nR ] Y n S ∼ Unif [1 : 2 nR ] S ∼ Unif [1 : 2 nR ] X n Enc X n M ∈ [1 : 2 nR ] Z n S ∼ Unif [1 : 2 nR ] Fig. 2. Strong coordination over a multiple-access network with encoder shared randomness. Encoder j ∈ { , } on observing the source X nj and sharedrandomness S , S j sends a message M j over a noiseless link to the decoder, which has side-information Z n . Here ( X i , X i , Z i ) , i = 1 , . . . , n , are i.i.d.with q X X Z . Also ( X n , X n , Z n ) , S , S , and S are mutually independent. The decoder has to output Y n such that ( X i , X i , Z i , Y i ) , i = 1 , . . . , n ,are approximately i.i.d. with q X X ZY . Example 1.
Let X = ( X , X ) be a vector of two independent and uniformly distributed binary random variables. Similarly,let X = ( X , X ) be another vector of two independent and uniformly distributed binary random variables independentof X . Consider simulating a channel q Y | X X with Y = ( X J , X J ) , where J is a random variable uniformly distributedon { , } and independent of ( X , X ) . For simplicity, we let Z = ∅ , i.e. there is no side information at the decoder. Let usassume unlimited rates R and R .When there is no additional shared randomness between the encoders, from Theorem 3, the communication rate region (seeDefinition 3) is given by the set of all rate pairs ( R , R ) such that R ≥ I ( U ; X | T ) , (3) R ≥ I ( U ; X | T ) , (4)for some p.m.f. p ( x ,x , t, u , u , y ) = p ( x ) p ( x ) p ( t ) p ( u | x , t ) p ( u | x , t ) p ( y | u , u , t ) (5)satisfying (cid:88) u ,u p ( x , x , u , u , y | t ) = q ( x , x , y ) , (6)for all t . The following proposition (proved at the end of this section) explicitly characterizes the communication rate regionfor this q X X Y . Proposition 1.
For the joint distribution q X X Y in Example 1, the communication rate region of Theorem 3 under sufficientlylarge shared randomness rates is equal to the region defined by the constraints R ≥ , R ≥ , and R + R ≥ . Now we show that if there exists an additional source of shared randomness between the encoders, then the rate pair ( R , R ) = (1 , is achievable (see Figure 3). In particular, we prove that if this additional shared randomness is of rate atleast , then a rate pair ( R , R ) = (1 , is achievable under sufficiently large shared randomness rates R and R . Tosee this, notice that both the encoders, using a shared randomness of rate , can sample a sequence W , W , · · · , W n i.i.d.distributed on { , } such that p W (1) = p W (2) = 0 . and independent of ( X n , X n ) . Now we invoke Theorem 1 with ( X i , W ) as the input source to the encoder- i , i = 1 , . This implies that a rate pair ( R , R ) is achievable under sufficiently large sharedrandomness rates R and R if R ≥ I ( U ; X , W | U ) ,R ≥ I ( U ; X , W | U ) ,R + R ≥ I ( U , U ; X , X , W ) , for some p.m.f. p ( x , x , w, u , u , y ) = q ( x , x , w ) p ( u | x , w ) p ( u | x , w ) p ( y | u , u ) . (7) R R Fig. 3. The communication rate regions for X k = ( X k , X k ) , k ∈ { , } ( X , X , X , X are mutually independent, uniform binary randomvariables) and Y = ( X J , X J ) , where J is uniform on { , } and independent of ( X , X ) . The region is defined by the constraints R ≥ , R ≥ , and R + R ≥ (shown via solid line) when the shared randomness between each encoder and the decoder are of sufficiently large rates. When an additionalshared randomness is available between the encoders, the region is defined by the constraints R ≥ and R ≥ (shown via dotted line) provided theshared randomness between all the three pairs are of sufficiently large rates (see Section V-A). It is easy to see that U = X W and U = X W satisfy the conditions on the structure of the probability distribution p ( x , x , w, u , u , y ) . This gives I ( X W ; X , W | X W ) = 1 ,I ( X W ; X , W | X W ) = 1 ,I ( X W , X W ; X , X , W ) = 2 , which imply that a rate pair (1 , is achievable, thereby strictly improving over the rate region without encoder sharedrandomness, defined by the constraints R ≥ , R ≥ , and R + R ≥ . This motivates the next section where we analyzethe model with shared randomness between the encoders. In Section V-A, we will show that the rate region defined by R ≥ , R ≥ is indeed optimal for our example with sufficiently large pairwise shared randomness rates (all three pairs). Remark 4.
We prove that the outer bound implicit in Theorem 3 is strictly stronger than that of Theorem 2 for the p.m.f. q X X ZY in Example 1. We first observe that that the communication rate pair (1 , is contained in the outer bound given byTheorem 2 (under unlimited shared randomness rates). To see this, first notice that the choice of U = ( X J , J ) , U = ( X J , J ) ,where J is a random variable uniformly distributed on { , } and independent of ( X , X ) , satisfies the conditions on thestructure of the p.m.f. in the outer bound of Theorem 2. Now we evaluate the bounds on communication rates in Theorem 2for this choice of auxiliary random variables. max { I ( X J , J ; X ) , I ( X J ; X | X J , J, X ) } = 1max { I ( X J , J ; X ) , I ( X J ; X | X J , J, X ) } = 1 I ( X J , X J , J ; X , X ) = 2 . This implies that a rate pair ( R , R ) = (1 , is contained in the outer bound on the communication rate region impliedby Theorem 2 (under sufficiently large shared randomness rates). However, by Proposition 1, this rate pair lies outside thecommunication rate region implied by Theorem 3.We conclude the section with a proof of Proposition 1 below. Proof of Proposition 1.
For the achievability, it suffices to show that the corner points (1 , and (2 , are in the region definedby (3) – (6). By symmetry, it is enough to show that there exists a p.m.f. p ( x , x , u , u , y ) = p ( x , x ) p ( u | x ) p ( u | x ) p ( y | u , u ) such that (cid:80) u ,u p ( x , x , u , u , y ) = q ( x , x , y ) , I ( U ; X ) = 2 , and I ( U ; X ) = 1 . It is easy to see that U = X and U = ( J, X J ) , where J is a random variable uniformly distributed on { , } and independent of ( X , X ) , satisfythe conditions on the structure of the joint probability distribution p ( x , x , u , u , y ) . Now I ( U ; X ) = H ( X ) = 2 and I ( U ; X ) = I ( J, X J ; X ) = I ( X J ; X | J ) = 1 .For the converse, it suffices to show that for any p.m.f. p ( x , x , u , u , y ) = p ( x , x ) p ( u | x ) p ( u | x ) p ( y | u , u ) suchthat (cid:80) u ,u p ( x , x , u , u , y ) = q ( x , x , y ) , we have I ( U ; X ) ≥ , I ( U ; X ) ≥ , and I ( U ; X )+ I ( U ; X ) ≥ . Considerthe p.m.f. in (5) for a fixed value of t , i.e., p ( x , x , u , u , y ) = p ( x ) p ( x ) p ( u | x ) p ( u | x ) p ( y | u , u ) (8a) such that (cid:88) u ,u p ( x , x , u , u , y ) = q ( x , x , y ) . (8b)Note that the independence of X and X along with the long Markov chain U → X → X → U implies the independenceof ( U , X ) and ( U , X ) . The following three cases now arise based on H ( X | U ) and H ( X | U ) .Case : H ( X | U ) = 0 ,Case : H ( X | U ) = 0 ,Case : H ( X | U ) > and H ( X | U ) > .Case ( H ( X | U ) = 0 ):We have I ( X ; U ) = H ( X ) = 2 . Now we prove that I ( U ; X ) ≥ . We show this by contradiction. Suppose that I ( U ; X ) < . Then H ( X | U ) = H ( X ) − I ( U ; X ) = 2 − I ( U ; X ) > and hence there exists a u with P ( U = u ) > such that p X | U = u has a support whose size is larger than 2. Notice that the Markov chain Y → ( X , U ) → X holdsbecause I ( Y ; X | X , U ) ≤ I ( Y, U ; X | X , U )= I ( U ; X | X , U ) + I ( Y ; X | U , U , X ) ≤ I ( U , X ; U , X ) + I ( Y ; X , X | U , U )= 0 , where the last equality follows because ( U , X ) is independent of ( U , X ) and the Markov chain Y → ( U , U ) → ( X , X ) holds.Suppose p X | U = u has the support { (0 , , (1 , , (0 , } . Consider the induced distribution p Y | X =(0 , ,U = u . This iswell-defined because P ( X = (0 , , U = u ) > as X is independent of U and P ( X = (0 , , P ( U = u ) > .Since P ( X = (0 , | X = (0 , , U = u ) = P ( X = (0 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (1 , | X = (0 , , U = u ) = P ( Y = (0 , | X = (0 , , U = u ) = 0 . Since P ( X = (1 , | X = (0 , , U = u ) = P ( X = (1 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (1 , | X = (0 , , U = u ) = P ( Y =(0 , | X = (0 , , U = u ) = 0 . This is a contradiction since p Y | X =(0 , ,U = u has to be a probability distribution.Suppose p X | U = u has the support { (0 , , (1 , , (0 , } . Since P ( X = (0 , | X = (0 , , U = u ) = P ( X =(0 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (1 , | X = (0 , , U = u ) = P ( Y = (0 , | X =(0 , , U = u ) = P ( Y = (1 , | X = (0 , , U = u ) = 0 . Since P ( X = (1 , | X = (0 , , U = u ) = P ( X =(1 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (0 , | X = (0 , , U = u ) = 0 . This is a contradictionsince p Y | X =(0 , ,U = u is a probability distribution.The other supports { (1 , , (0 , , (1 , } and { (0 , , (1 , , (1 , } can be analysed in a similar manner to arrive at acontradiction. Hence, I ( U ; X ) ≥ .Case ( H ( X | U ) = 0 ):By symmetry, the analysis for this case is similar to that of Case .Case ( H ( X | U ) > and H ( X | U ) > ):We prove the following claim in Appendix E. Claim 1.
When H ( X | U ) > and H ( X | U ) > , there exists a k ∈ { , } such that for all u and u with P ( U = u ) > , P ( U = u ) > , we have H ( X k | U = u ) = 0 and H ( X k | U = u ) = 0 . Claim 1 roughly states that, under Case , U and U always reveal the k th components of X and X , respectively, for afixed k ∈ { , } . Let us assume k = 1 without loss of generality, i.e., U i conveys atleast X i losslessly, for i = 1 , . Let A i be the event “ U i conveys both X i and X i losslessly”, i.e., for i = 1 , , A i = (cid:91) u i : H ( X i | U i = u i )= H ( X i | U i = u i )=0 ( U i = u i ) . (9)Let, for i = 1 , , p i := P ( A i ) = (cid:88) u i : H ( X i | U i = u i )= H ( X i | U i = u i )=0 P ( U i = u i ) . For i = 1 , , it follows that A ci is the event “ U i conveys only X i losslessly” from Claim 1 along with our assumption that k = 1 . Now, in view of the independence of ( U , X ) and ( U , X ) , it follows that − p p = 1 − P ( A ∩ A ) = Pr( A c ∪ A c ) .Notice that this event A c ∪ A c must be a subset of the event ( J = 1) , otherwise the correctness condition (8b) is violated.Thus we have − p p ≤ P ( J = 1) = 0 . . (10) Also, we have I ( U ; X ) = H ( X ) − H ( X | U )= 2 − (cid:88) u P ( U = u ) H ( X | U = u ) ( a ) = 2 − (cid:88) u : H ( X | U = u )=0 ,H ( X | U = u ) > P ( U = u ) H ( X | U = u ) − (cid:88) u : H ( X | U = u )=0 ,H ( X | U = u )=0 P ( U = u ) H ( X | U = u ) ≥ − (cid:88) u : H ( X | U = u )=0 ,H ( X | U = u ) > P ( U = u ) H ( X | U = u ) ≥ − (1 − p )= 1 + p , (11)where (a) follows from Claim 1 along with our assumption that k = 1 , since U i reveals atleast X i losslessly for i = 1 , .Similarly, we have I ( U ; X ) ≥ p . From (10), we have p p ≥ . . Now since min p ,p :0 ≤ p ,p ≤ and p p ≥ . p + p > , (12)we have I ( U ; X ) + I ( U ; X ) ≥ p + p ≥ . This proves that the communication rate region is equal to the regiondefined by the constraints R ≥ , R ≥ , and R + R ≥ . (cid:4) V. I
NNER AND O UTER B OUNDS WITH E NCODER S HARED R ANDOMNESS
In this section, we study the setting in Section II augmented with a source of pairwise shared randomness S (uniformlydistributed on [1 : 2 nR ] ) between the two encoders – see Figure 2. The random variables S , S , and S are independent ofeach other and also jointly independent of ( X n , X n , Z n ) . We will also exploit common components [52] between the twosources, i.e. random variables X such that there exist deterministic functions f and f with X = f ( X ) = f ( X ) a.s . (13)Encoder j ∈ { , } observes X nj along with the shared randomness ( S , S j ) , and sends a message M j ∈ [1 : 2 nR j ] over anoise-less communication link to the decoder. The decoder observes ( M , M ) , in addition to the shared randomness ( S , S ) , and side-information Z n . The goal is to output Y n which along with the input sources and decoder side-information isapproximately distributed according to q ( n ) X X ZY ( x n , x n , z n , y n ) := (cid:81) ni =1 q X X ZY ( x i , x i , z i , y i ) . Definition 4. A (2 nR , nR , nR , nR , nR , n ) code consists of two randomized encoders p E ( m | s , s , x n ) and p E ( m | s , s , x n ) and a randomized decoder p D ( y n | s , s , m , m , z n ) , where s ∈ [1 : 2 nR ] and s j ∈ [1 : 2 nR j ] , m j ∈ [1 : 2 nR j ] , j = 1 , . The joint distribution of ( S , S , S , X n , X n , Z n , M , M , Y n ) and the resulting induced joint distribution on ( X n , X n , Z n , Y n ) are respectively given by p ( s , s , s , x n , x n , z n , m , m , y n ) = 12 n ( R + R + R ) p ( x n , x n , z n ) (cid:89) j =1 p E j ( m j | s , s j , x nj ) p D ( y n | s , s , m , m , z n ) , and p ind ( x n , x n , z n , y n ) = (cid:88) s ,s ,s ,m ,m p ( s , s , s , x n , x n , z n , m , m , y n ) . Definition 5.
A rate tuple ( R , R , R , R , R ) is said to be achievable for a distribution q X X ZY if there exists a sequenceof (2 nR , nR , nR , nR , nR , n ) codes such that lim n →∞ || p ind X n ,X n ,Z n ,Y n − q ( n ) X X ZY || = 0 , (14) where q ( n ) X X ZY ( x n , x n , z n , y n ) := n (cid:89) i =1 q X X ZY ( x i , x i , z i , y i ) . Definition 6.
The rate region R enc-SRMAC-coord is the closure of the set of all achievable rate tuples ( R , R , R , R , R ) . The following theorem provides an inner bound to the region R enc-SRMAC-coord . Theorem 4 (Inner Bound with Encoder Shared Randomness) . Given a p.m.f. q X X ZY , the rate tuple ( R , R , R , R , R ) is in R enc-SRMAC-coord if R ≥ H ( U | X , T ) R ≥ I ( U ; X , U | U , Z, T ) R ≥ I ( U ; X , U | U , Z, T ) R + R ≥ I ( U , U ; X , X , U | Z, T ) R + R ≥ I ( U ; X , X , U , Y | Z, T ) − I ( U ; U | Z, T ) R + R ≥ I ( U ; X , X , U , Y | Z, T ) − I ( U ; U | Z, T ) R + R + R ≥ I ( U ; X , X , U , Y | Z, T ) + I ( U ; X , U | U , Z, T ) R + R + R ≥ I ( U ; X , X , U , Y | Z, T ) + I ( U ; X , U | U , Z, T ) R + R + R + R ≥ I ( U , U ; X , X , U , Y | Z, T ) , for some p.m.f. p ( x , x , z, t, u , u , u , y ) = p ( x , x , z ) p ( t ) p ( u | x , t ) p ( u | x , u , t ) p ( u | x , u , t ) p ( y | u , u , z, t ) such that (cid:88) u ,u ,u p ( x , x , u , u , u , y, z | t ) = q ( x , x , z, y ) , for all t. The main idea behind the proof is to make use of the shared randomness between the encoders in order to simulatea common description of X n , viz. U n at both the encoders approximately distributed according to q ( n ) U n | X n ( u n | x n ) := (cid:81) ni =1 p U | X ( u i | x i ) . Then we invoke Theorem 1 with X j replaced by ( X j , U ) for j ∈ { , } . A detailed proof is given inSection VI-A.We now provide an outer bound to the region R enc-SRMAC-coord . Theorem 5 (Outer Bound with Encoder Shared Randomness) . Given a p.m.f. q X X ZY , any rate tuple ( R , R , R , R , R ) in R enc-SRMAC-coord satisfies R ≥ max { I ( U ; X | U , Z, T ) , I ( U ; X | U , U , X , Z, T ) } R ≥ max { I ( U ; X | U , Z, T ) , I ( U ; X | U , U , X , Z, T ) } R + R ≥ I ( U , U ; X , X | U , Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) R + R ≥ I ( U ; X , X , Y | Z, T ) R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R + R + R + R ≥ I ( U , U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x , x , z, t, u , u , u , y ) = p ( x , x , z ) p ( t ) p ( u | t ) p ( u , u | x , x , u , t ) p ( y | u , u , z, t ) (15) such that p ( u | x , x , u , z, t ) = p ( u | x , u , t ) (16) p ( u | x , x , u , z, t ) = p ( u | x , u , t ) (17) (cid:88) u ,u ,u p ( x , x , z, u , u , u , y | t ) = q ( x , x , z, y ) , for all t. (18) A detailed proof is given in Section VI-B. When the random variables X and X are conditionally independent given Z ,we obtain a potentially stronger outer bound. Theorem 6 (Outer Bound - Conditionally Independent Sources Given the Side Information) . Consider a p.m.f. q X X ZY suchthat the random variables X and X are conditionally independent given Z . Then any rate tuple ( R , R , R , R , R ) in R enc-SRMAC-coord satisfies R ≥ I ( U , U ; X | Z, T ) R ≥ I ( U , U ; X | Z, T ) R + R ≥ I ( U , U , U ; X , X | Z, T ) R + R ≥ I ( U ; X , Y | X , Z, T ) R + R ≥ I ( U ; X , Y | X , Z, T ) R + R + R ≥ I ( U , U ; X , Y | X , Z, T ) R + R + R ≥ I ( U , U ; X , Y | X , Z, T ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R + R + R + R ≥ I ( U , U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x , x , z, t, u , u , u , y ) = p ( z ) p ( x | z ) p ( x | z ) p ( t ) p ( u | t ) p ( u | x , u , t ) p ( u | x , u , t ) p ( y | u , u , z, t ) such that (cid:80) u ,u ,u p ( x , x , z, u , u , u , y | t ) = q ( x , x , y ) , for all t . Notice that the improvement is in the structure of the p.m.f. compared to that of Theorem 5. A detailed proof can be foundin Section VI-B.
A. Optimal Region for Example 1 with unlimited shared randomness
Here, we show that in the setting of Example 1, the region R ≥ and R ≥ is indeed the optimal rate region (not justachievable as shown in Section IV) with unlimited pairwise shared randomness (all three pairs). The achievability can also beinferred from Theorem 4 with the choice of U = J , U = X J and U = X J , where J is a random variable uniformlydistributed on { , } and independent of ( X , X ) . To prove the converse, first note that Theorem 6 (with Z = ∅ and unlimitedshared randomness rates) implies that any achievable rate pair ( R , R ) must satisfy R ≥ I ( U , U ; X | T ) R ≥ I ( U , U ; X | T ) R + R ≥ I ( U , U , U ; X , X | T ) , for some p.m.f. p ( x , x , t, u , u , u , y ) = p ( x ) p ( x ) p ( t ) p ( u | t ) p ( u | x , u , t ) p ( u | x , u , t ) p ( y | u , u , t ) such that (cid:80) u ,u ,u p ( x , x , u , u , u , y | t ) = q ( x , x , y ) , for all t .For the converse, it suffices to show that for any p.m.f. p ( x , x , u , u , u , y ) = p ( x ) p ( x ) p ( u ) p ( u | x , u ) p ( u | x , u ) p ( y | u , u ) with (cid:80) u ,u ,u p ( x , x , u , u , u , y ) = q ( x , x , y ) , we have I ( U , U ; X ) ≥ and I ( U , U ; X ) ≥ . Equivalently, bymarginalizing away U and letting ( U , U ) (cid:44) U , it suffices to show that for any p.m.f. p ( x , x , u, y ) = p ( x ) p ( x ) p ( u | x ) p ( y | u, x ) with (cid:80) u p ( x , x , u, y ) = q ( x , x , y ) , we have I ( U ; X ) ≥ (The condition I ( U , U ; X ) ≥ can be shown analogously.).This can be established by proving that for each U = u such that H ( X | U = u ) > , there exists k ( u ) ∈ { , } such that H ( X k | U = u ) = 0 . Indeed this yields I ( U ; X ) = H ( X ) − H ( X | U )= 2 − (cid:88) u P ( U = u ) H ( X | U = u )= 2 − (cid:88) u P ( U = u ) ( H ( X k | U = u ) + H ( X k (cid:48) | U = u, X k )) ≥ − (cid:88) u P ( U = u )(0 + 1)= 1 . (19)We have the following claim. Claim 2.
When H ( X | U ) > , for all u with H ( X | U = u ) > there exists a k in { , } such that H ( X k | U = u ) = 0 .Proof of Claim 2. We prove this by contradiction. Suppose H ( X i | U = u ) > , for i = 1 , . Then the support of p X | U = u has to be a superset of either { (0 , , (1 , } or { (0 , , (1 , } . However, it turns out that, the support cannot be a supersetof { (0 , , (1 , } . To see this, first notice that P ( X = (0 , | U = u, X = (0 , P ( X = (0 , | U = u ) > , wherethe equality follows from the independence of ( U, X ) and X . Similarly, P ( X = (1 , | U = u, X = (0 , > . Nowsince X J − ( U, X ) − X , we have P ( X J = 0 | U = u, X = (0 , P ( X J = 0 | U = u, X = (0 , , X = (1 , ,where the last equality follows from the correctness of the output Y = ( X J , X J ) . Similarly, P ( X J = 1 | U = u, X =(0 , P ( X J = 1 | U = u, X = (0 , , X = (0 , . This is a contradiction since p Y | U = u,X =(0 , has to be aprobability distribution. The only other possibility is that p X | U = u has a support that is a superset of { (0 , , (1 , } . Considerthe induced distribution p Y | X =(0 , ,U = u . This is well-defined because P ( X = (0 , , U = u ) > as X is independentof U and P ( X = (0 , , P ( U = u ) > . Since P ( X = (0 , | X = (0 , , U = u ) = P ( X = (0 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (1 , | X = (0 , , U = u ) = P ( Y = (0 , | X = (0 , , U = u ) = 0 . Since P ( X = (1 , | X = (0 , , U = u ) = P ( X = (1 , | U = u ) > and Y → ( X , U ) → X , we have P ( Y = (1 , | X =(0 , , U = u ) = P ( Y = (0 , | X = (0 , , U = u ) = 0 . This is a contradiction since p Y | X =(0 , ,U = u has to be a validprobability distribution. (cid:4) This proves that the optimal communication rate region with unlimited pairwise shared randomness (all three pairs) is indeeddefined by the constraints R ≥ and R ≥ . VI. P ROOFS
A. Achievability ProofsProof of Theorem 1.
We prove the achievability for |T | = 1 , and the rest of the proof follows using time sharing. The proofemploys the Output Statistics of Random Binning (OSRB) framework developed by Yassaee et al. [46]. In the sequel, we usecapital letters (like P X ) to denote random p.m.f.’s (see, e.g., [5], [46]) and lower-case letters (like p X ) to denote non-randomp.m.f.’s. We use p U A to denote the uniform distribution over the set A . The notation ≈ for pmf approximations is adopted from[46] – for two random pmfs P X and Q X on the same alphabet X , we say that P X (cid:15) ≈ Q X provided E [ || P X − Q X || ] ≤ (cid:15) . Wealso quote the following results that will prove useful in the proof of Theorem 1, where the first one is a restatement of theOSRB result [46, Theorem 1]. Theorem 7. [46, Theorem 1] Given a discrete memoryless source ( A n , B n , · · · , B nL ) ∼ i.i.d. with p A,B , ··· ,B L on A× (cid:81) Lj =1 B i .For j ∈ [1 : L ] , let φ j : B nj → [1 : 2 nR j ] be a random binning in which φ j maps each sequence of B nj uniformly andindependently to the set [1 : 2 nR j ] . Let K j = φ j ( B nj ) , j ∈ { , · · · , L } . If for each S ⊆ [1 : L ] , the following constraint (cid:88) t ∈S R t ≤ H ( B S | A ) (20) holds, then we have lim n →∞ E φ , ··· ,φ L || P A n ,K , ··· ,K L − p A n L (cid:89) j =1 p U K j || = 0 , (21) where E φ , ··· ,φ L denotes the expectation over the random binnings, P A n ,K , ··· ,K L is a random p.m.f., and p U K j is the uniformdistribution over [1 : 2 nR j ] . Lemma 1. [46, Lemma 4]1) If P X n ≈ Q X n , then P X n P Y n | X n ≈ Q X n P Y n | X n . Also if P X n P Y n | X n ≈ Q X n Q Y n | X n , then P X n ≈ Q X n .2) If p X n p Y n | X n ≈ q X n q Y n | X n , then there exists a sequence x n ∈ X n such that p Y n | X n = x n ≈ q Y n | X n = x n .3) If P X n ≈ Q X n and P X n P Y n | X n ≈ P X n Q Y n | X n , then P X n P Y n | X n ≈ Q X n Q Y n | X n . We now follow the standard structure of an achievability proof via OSRB. This involves defining two protocols, one eachbased on random coding and random binning, that induce a joint distribution on the random variables defined during the protocols. Random Binning Scheme:
Let ( U n , U n , X n , X n , Z n , Y n ) be drawn i.i.d. with the joint distribution p ( x , x , z ) p ( u | x ) p ( u | x ) p ( y | u , u , z ) such that p ( x , x , z, y ) = q ( x , x , z, y ) . Now we employ the following random binning: • Generate ( S , M , F ) as three uniform binnings of U n independently, i.e. S = φ ( U n ) ∈ [1 : 2 nR ] , M = φ ( U n ) ∈ [1 : 2 nR ] and F = φ ( U n ) ∈ [1 : 2 n ˜ R ] . • Likewise, generate ( S , M , F ) as three uniform binnings of U n independently, i.e. S = φ ( U n ) ∈ [1 : 2 nR ] , M = φ ( U n ) ∈ [1 : 2 nR ] and F = φ ( U n ) ∈ [1 : 2 n ˜ R ] .The receiver uses a Slepian-Wolf decoder to estimate (ˆ u n , ˆ u n ) from ( s , s , f , f , m , m , z n ) . The corresponding randomp.m.f. induced is (the randomness is due to the binning) P ( x n , x n , z n , y n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n )= p ( x n , x n , z n ) p ( u n | x n ) p ( u n | x n ) p ( y n | u n , u n , z n ) × P ( s , f , m | u n ) P ( s , f , m | u n ) × P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) (22) = p ( x n , x n , z n ) P ( s , f , m , u n | x n ) P ( s , f , m , u n | x n ) × P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) p ( y n | u n , u n , z n )= p ( x n , x n , z n ) P ( s , f | x n ) P ( u n | s , f , x n ) P ( m | u n ) × P ( s , f | x n ) P ( u n | s , f , x n ) P ( m | u n ) × P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) p ( y n | u n , u n , z n )= P ( x n , x n , z n , s , s , f , f ) P ( u n | s , f , x n ) P ( m | u n ) × P ( u n | s , f , x n ) P ( m | u n ) × P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) p ( y n | u n , u n , z n ) . (23)where (22) uses the Markov chains ( U , U ) → ( X , X ) → Z , U → X → X → U , Y → ( U , U , Z ) → ( X , X ) , andthe binning construction. Random Coding Scheme:
We assume that additional shared randomness F j of rate ˜ R j , j ∈ { , } are available between therespective encoders and the decoder in the main problem. Encoder j ∈ { , } , knowing ( s j , f j , x nj ) , generates u nj accordingto the p.m.f. P ( u nj | s j , f j , x nj ) (from the previous protocol) and sends the bin index of u nj corresponding to the binning φ j in the previous protocol over the noiseless link to the decoder. The decoder obtains ( s , s , f , f , m , m , z n ) , and employsthe Slepian-Wolf decoder from the previous protocol, i.e. P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) , to estimate ( u n , u n ) . Thenit constructs y n according to the distribution p Y n | U n ,U n ,Z n ( y n | ˆ u n , ˆ u n , z n ) . The induced random p.m.f. from this protocol isgiven by ˆ P ( x n , x n , z n , y n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n )= p U ( s ) p U ( f ) p U ( s ) p U ( f ) p ( x n , x n , z n ) × P ( u n | s , f , x n ) P ( m | u n ) × P ( u n | s , f , x n ) P ( m | u n ) × P SW (ˆ u n , ˆ u n | s , f , m , s , f , m , z n ) p ( y n | ˆ u n , ˆ u n , z n ) . (24)Next we find constraints that imply that the induced p.m.f.’s from the two protocols are almost identical. Then one canrestrict attention to the source coding side of the problem (related to the random binning protocol) and investigate the desiredproperties like vanishing total variation distance. Analysis of Rate Constraints:
We now derive sufficient conditions for the joint statistics of the random variables from the two protocols to be identical. Since ( s j , f j ) are the bin indices of u nj for j ∈ { , } , if we ensure that R + ˜ R ≤ H ( U | X , X , Z ) = H ( U | X ) , (25) R + ˜ R ≤ H ( U | X , X , Z ) = H ( U | X ) , (26) R + ˜ R + R + ˜ R ≤ H ( U , U | X , X , Z )= H ( U , U | X , X ) (27)then by Theorem 7, we obtain P ( x n , x n , z n , s , s , f , f ) ≈ p U ( s ) p U ( f ) p U ( s ) p U ( f ) p ( x n , x n , z n )= ˆ P ( x n , x n , z n , s , s , f , f ) . (28)This in turn results in P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n ) ≈ ˆ P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n ) . (29)Note that the condition (27) above is redundant because H ( U , U | X , X ) = H ( U | X ) + H ( U | X ) using the Markov chain U → X → X → U .For the Slepian-Wolf decoder to succeed, we require (by Slepian-Wolf theorem [53], see also [46, Lemma 1]) R + ˜ R + R ≥ H ( U | U , Z ) , (30) R + ˜ R + R ≥ H ( U | U , Z ) , (31) R + ˜ R + R + R + ˜ R + R ≥ H ( U , U | Z ) . (32)This ensures that P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n ) ≈ P ( x n , x n , z n , u n , u n , s , f , m , s , f , m ) × { ˆ u n = u n , ˆ u n = u n } . (33)Using (33), (29) and the first and third parts of Lemma 1, we can write the following for the joint probability distributioninvolving y n ˆ P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n , y n )= ˆ P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , ˆ u n , ˆ u n ) × p ( y n | ˆ u n , ˆ u n , z n ) ≈ P ( x n , x n , z n , u n , u n , s , f , m , s , f , m ) × { ˆ u n = u n , ˆ u n = u n } p ( y n | ˆ u n , ˆ u n , z n )= P ( x n , x n , z n , u n , u n , s , f , m , s , f , m ) × { ˆ u n = u n , ˆ u n = u n } p ( y n | u n , u n , z n )= P ( x n , x n , z n , u n , u n , s , f , m , s , f , m , y n ) × { ˆ u n = u n , ˆ u n = u n } . (34)Thus, using the first part of Lemma 1, we can conclude that ˆ P ( x n , x n , z n , y n , f , f ) ≈ P ( x n , x n , z n , y n , f , f ) . (35)We require ( X n , X n , Z n , Y n ) to be independent of the extra shared randomness ( F , F ) to eliminate them without disturbingthe desired i.i.d. distribution. This can be accomplished by imposing the following conditions according to Theorem 7. ˜ R ≤ H ( U | X , X , Y, Z ) , (36) ˜ R ≤ H ( U | X , X , Y, Z ) , (37) ˜ R + ˜ R ≤ H ( U , U | X , X , Y, Z ) . (38)This ensures that P ( x n , x n , z n , y n , f , f ) ≈ p U ( f ) p U ( f ) p ( x n , x n , z n , y n ) , (39)which along with (35) and the triangle inequality, implies that ˆ P ( x n , x n , z n , y n , f , f ) ≈ p U ( f ) p U ( f ) p ( x n , x n , z n , y n ) . (40)Hence there exists a fixed binning with corresponding pmf ˜ p such that if we replace P by ˜ p in (24) and denote the resultingpmf by ˆ p , then ˆ p ( x n , x n , z n , y n , f , f ) ≈ p U ( f ) p U ( f ) p ( x n , x n , y n , z n ) . (41)Now the second part of Lemma 1 allows us to conclude that there exist instances F = f ∗ , F = f ∗ such that ˆ p ( x n , x n , z n , y n | f ∗ , f ∗ ) ≈ p ( x n , x n , z n , y n ) . (42) Now along with the rate constraints imposed in equations (25) – (26), (30) – (32) and (36) – (38), we also need to impose the non-negativity constraints on all the rates. But it turns out that the constraints ˜ R ≥ and ˜ R ≥ are redundant, which can be shownalong the lines of [46, Remark 4]. We prove that if ( ˜ R , ˜ R ) are not necessarily all positive and satisfy (25) – (26), (30) – (32)and (36) – (38) along with ( R , R , R , R ) for some ( U , U ) such that ( U , U ) → ( X , X ) → Z , U → X → X → U and Y → ( U , U , Z ) → ( X , X ) , then there exists ( ¯ U , ¯ U ) with ( ¯ U , ¯ U ) → ( X , X ) → Z , ¯ U → X → X → ¯ U and Y → ( ¯ U , ¯ U , Z ) → ( X , X ) and ¯ R ≥ , ¯ R ≥ such that ( ¯ R , ¯ R ) along with ( R , R , R , R ) satisfy (25) – (26),(30) – (32) and (36) – (38) for ( ¯ U , ¯ U ) instead of ( U , U ) . Suppose ˜ R < and ˜ R < (other configurations can be dealtsimilarly). Let ( W , W ) be random variables such that H ( W ) > | ˜ R | and H ( W ) > | ˜ R | . We also assume that W as wellas W are independent of all the other random variables. Define ¯ R = ˜ R + H ( W ) , ¯ R = ˜ R + H ( W ) and ¯ U = ( U , W ) , ¯ U = ( U , W ) . Clearly, we have ¯ R , ¯ R ≥ and it is easy to see that ( ¯ R , ¯ R ) along with ( R , R , R , R ) satisfy (25)– (26), (30) – (32) and (36) – (38) for ( ¯ U , ¯ U ) using the independence of W and W from all other random variables andthe fact that ( ˜ R , ˜ R ) satisfy (25) – (26), (30) – (32) and (36) – (38) along with ( R , R , R , R ) . Finally on eliminating ( ˜ R , ˜ R ) from equations (25) – (26), (30) – (32) and (36) – (38) by Fourier-Motzkin elimination (FME), we obtain the rateconstraints R ≥ H ( U | U , Z ) − H ( U | X , U , Z ) = I ( U ; X | U , Z ) , (43) R ≥ H ( U | U , Z ) − H ( U | X , U , Z ) = I ( U ; X | U , Z ) , (44) R + R ≥ H ( U , U | Z ) − H ( U , U | X , X , Z )= I ( U , U ; X , X | Z ) , (45) R + R ≥ H ( U | U , Z ) − H ( U | X , X , Y, Z )= I ( U ; X , X , Y | Z ) − I ( U ; U | Z ) , (46) R + R ≥ H ( U | U , Z ) − H ( U | X , X , Y, Z )= I ( U ; X , X , Y | Z ) − I ( U ; U | Z ) , (47) R + R + R ≥ H ( U , U | Z ) − H ( U | X , Z ) − H ( U | X , X , Y, Z )= I ( U ; X , X , Y | Z ) + I ( U ; X | U , Z ) , (48) R + R + R ≥ H ( U , U | Z ) − H ( U | X , Z ) − H ( U | X , X , Y, Z )= I ( U ; X , X , Y | Z ) + I ( U ; X | U , Z ) , (49) R + R + R + R ≥ H ( U , U | Z ) − H ( U , U | X , X , Y, Z )= I ( U , U ; X , X , Y | Z ) . (50)Thus when the rate constraints in (43) – (50) are met, there exists a sequence of (2 nR , nR , nR , nR , n ) codes withencoders and decoders as described in the second protocol with the particular realization of binning along with the fixedinstances f ∗ , f ∗ resulting in vanishing total variation distance. (cid:4) Achievability Proof of Theorem 3.
We argue achievability for |T | = 1 that follows from Theorem 1 by enforcing the constraint p ( x , x , z ) = p ( z ) p ( x | z ) p ( x | z ) . The rest of the proof follows using time sharing. In this case, the joint distribution on ( X , X , Z, U , U ) is such that ( U , X ) → Z → ( U , X ) is a Markov chain. We note the following simplifications to therate constraints in Theorem 1. R ≥ I ( U ; X | U , Z ) = I ( U ; X | Z ) R ≥ I ( U ; X | U , Z ) = I ( U ; X | Z ) R + R ≥ I ( U , U ; X , X | Z )= I ( U ; X | Z ) + I ( U ; X | Z ) R + R ≥ I ( U ; X , X , Y | Z ) − I ( U ; U | Z )= I ( U ; X , X , Y | Z )= I ( U ; X , Y | X , Z ) R + R ≥ I ( U ; X , X , Y | Z ) − I ( U ; U | Z ) = I ( U ; X , X , Y | Z )= I ( U ; X , Y | X , Z ) R + R + R ≥ I ( U ; X ,X ,Y | Z )+ I ( U ; X | U , Z )= I ( U ; X , X , Y | Z ) + I ( U ; X | Z ) R + R + R ≥ I ( U ; X ,X ,Y | Z )+ I ( U ; X | U , Z )= I ( U ; X , X , Y | Z ) + I ( U ; X | Z ) R + R + R + R ≥ I ( U , U ; X , X , Y | Z ) . Noticing that the constraints on R + R , R + R + R and R + R + R are redundant and using time sharing completesthe achievability. (cid:4) Proof of Theorem 4.
We prove the achievability for |T | = 1 , and the rest of the proof follows using time sharing. Firstly, weshow that the shared randomness between the encoders can be harnessed to simulate U n approximately distributed accordingto q ( n ) U n | X n ( u n | x n ) = (cid:81) ni =1 p U | X ( u i | x i ) . The rate of shared randomness needed here will turn out to be R ≥ H ( U | X ) ,i.e. the constraint on R in Theorem 4.We make use of the following set-up from Cuff [5, Corollary VII.5]. Let X n be an i.i.d. sequence with distribution p X and f ( · ) P V | U ,X X n U n V n R Fig. 4. Implications of soft covering [5] p U | X p V | U ,X be a memoryless channel. A deterministic encoder f ( · ) receives both X n and a uniformly distributed randomvariable S ∈ [1 : 2 nR ] . Cuff [5] gave the following sufficient condition on the rate R so that the induced distribution onthe channel output in Figure 4 is i.i.d. in the limit of large n . Lemma 2. [5, Corollary VII.5] Let p V n be the induced distribution on the channel output in Figure 4 and q V n be the i.i.d.output distribution specified by (cid:80) u ,x p X p U | X p V | U ,X . Then if R ≥ I ( X , U ; V ) − H ( X ) , (51) we have lim n →∞ || p V n − q V n || = 0 . (52)For our purposes, we take p V | U ,X to be an identity channel i.e. V = ( U , X ) in Lemma 2. Hence lim n →∞ || p U n ,X n − q U n ,X n || = 0 , (53)provided that R satisfies R ≥ I ( X , U ; X , U ) − H ( X )= H ( U | X ) . (54)Suppose if ( X n , X n , U n ) are generated exactly i.i.d., then we can invoke Theorem 1 with X j replaced by ( X j , U ) for j ∈ { , } and it can be verified that this exactly yields the eight rate constraints involving ( R , R , R , R ) in Theorem4. Here, we show that the same set of rate constraints suffices even if ( U n , X n ) are approximately i.i.d. in the sense of(53). All that remains to be shown is that under the rate constraints in Theorem 4, there exists a sequence of codes with aninduced p.m.f. on ( X n , X n , U n , Z n , Y n ) such that the total variation distance between this p.m.f. and the desired i.i.d. p.m.f. (cid:81) q X ,X ,U ,Z,Y vanishes in the limit of large blocklength n . This can be argued out as follows.If ( X n , X n , U n ) were exactly i.i.d., by invoking Theorem 1 there exists p ( m , m , y n | x n , x n , u n , z n , s , s ):= p ( m | s , x n , u n ) p ( m | s , x n , u n ) p ( y n | m , m , s , s , z n ) such that || p X n ,X n ,U n ,Z n ,Y n − (cid:89) q X X U ZY || ≤ (cid:15). (55)On the other hand, let ˜ p ( x n , x n , u n , z n , s , s ) be the distribution in which ( U n , X n ) are approximately i.i.d. in the sense of(53). Let us define ˜ p ( x n ,x n , u n , z n , s , s , m , m , y n ):=˜ p ( x n , x n , u n , z n , s , s ) × p ( m , m , y n | x n , x n , u n , z n , s , s ) . (56)Now, the total variation distance of interest can be bounded using triangle inequality as || ˜ p X n ,X n ,U n ,Z n ,Y n − (cid:89) q X X U ZY || ≤ || ˜ p X n ,X n ,U n ,Z n ,Y n − p X n ,X n ,U n ,Z n ,Y n || + || p X n ,X n ,U n ,Z n ,Y n − (cid:89) q X X U Y || ≤ || ˜ p X n ,X n ,U n ,Z n ,Y n − p X n ,X n ,U n ,Z n ,Y n || + (cid:15), (57)where (57) follows from (55). Next consider the first term on the RHS of (57). || ˜ p X n ,X n ,U n ,Z n ,Y n − p X n ,X n ,U n ,Z n ,Y n || = (cid:88) x n ,x n ,u n ,z n ,y n | (cid:88) m ,m ,s ,s ˜ p ( x n , x n , u n , z n , y n , m , m , s , s ) − p ( x n , x n , u n , z n , y n , m , m , s , s ) |≤ (cid:88) x n ,x n ,u n ,z n ,y n ,m ,m ,s ,s p ( y n , m , m | x n , x n , u n , z n , s , s )2 n ( R + R ) × | ˜ p ( x n , x n , u n , z n ) − p ( x n , x n , u n , z n ) | = (cid:88) x n ,x n ,u n ,z n | ˜ p ( x n , x n , u n , z n ) − p ( x n , x n , u n , z n ) |≤ (cid:88) x n ,x n ,x n ,u n ,z n | ˜ p ( x n , x n , x n , u n , z n ) − p ( x n , x n , x n , u n , z n ) | = (cid:88) x n ,x n ,x n ,u n ,z n p ( x n , x n , x n , z n ) | ˜ p ( u n | x n ) − p ( u n | x n ) | = (cid:88) x n ,u n | ˜ p ( x n , u n ) − p ( x n , u n ) |≤ (cid:15). (58)From expressions (57) and (58), we conclude that || ˜ p X n ,X , U n ,Z n ,Y n − (cid:89) q X ,X ,U ,Z,Y || ≤ (cid:15). (cid:4) B. Converse Proofs
We will show that Theorem 2 is a direct consequence of Theorem 5, which we prove first.
Proof of Theorem 5.
Consider a code that induces a joint distribution on ( X n , X n , Z n , Y n ) such that (cid:107) p X n ,X n ,Z n ,Y n − q ( n ) X X ZY (cid:107) < (cid:15). (59)For ease of notation, for a vector A n , we write A ∼ i (cid:44) ( A i − , A ni +1 ) . Also let M [0:2] (cid:44) ( M , M , M ) and S [0:2] (cid:44) ( S , S , S ) .We quote the following lemmas that will prove useful in the outer bound. Lemma 3. [54, Lemma 6] Let p X n be such that || p X n − q ( n ) X || ≤ (cid:15) , where q ( n ) X ( x n ) = (cid:81) ni =1 q X ( x i ) , then n (cid:88) i =1 I p ( X i ; X ∼ i ) ≤ ng ( (cid:15) ) , (60) where g ( (cid:15) ) → as (cid:15) → . Lemma 4. [5, Lemma VI.3] Let p X n be such that || p X n − q ( n ) X || ≤ (cid:15) , where q ( n ) X ( x n ) = (cid:81) ni =1 q X ( x i ) , then for any RV T ∈ [1 : n ] independent of X n , I p ( T ; X T ) ≤ g ( (cid:15) ) , (61) where g ( (cid:15) ) → as (cid:15) → . Let us first consider one of the lower bounds on R j for j ∈ { , } . nR j ≥ H ( M j ) ≥ H ( M j | S , S j , Z n ) ≥ I ( M j ; X nj | S , S j , Z n ) ( a ) = I ( M j , S j ; X nj | S , Z n )= n (cid:88) i =1 I ( M j , S j ; X ji | X nj,i +1 , S , Z i , Z ∼ i ) ( b ) = n (cid:88) i =1 I ( M j , S j , X nj,i +1 , Z ∼ i ; X ji | S , Z i ) ≥ n (cid:88) i =1 I ( M j , S j , X nj,i +1 , Z i − ; X ji | S , Z i ) ( c ) ≥ n (cid:88) i =1 I ( U ji ; X ji | U i , Z i ) ( d ) = nI ( U jT ; X jT | U T , Z T , T ) ( e ) = nI ( U j ; X j | U , Z, T ) , (62)where (a) follows since S j is independent of ( S , X nj , Z n ) , (b) follows since ( X ji , Z i ) , i = 1 , . . . , n, are jointly i.i.d. and S is independent of ( X nj , Z n ) , (c) follows by defining U i = S , U i = ( M , S , X n ,i +1 , Z i − ) and U i = ( M , S , X n ,i +1 ) , (d)follows by introducing a uniform time-sharing random variable T ∈ [1 : n ] that is independent of everything else, while (e)follows by defining U := U T , U := U T , U := U T , X := X T , X := X T , Y := Y T and Z := Z T . The other lowerbound on R is obtained as follows. nR ≥ H ( M ) ≥ H ( M | X n , S [0:2] , Z n ) ≥ I ( M ; X n | X n , S [0:2] , Z n ) ( a ) = I ( M [1:2] ; X n | X n , S [0:2] , Z n ) ( b ) = I ( M [1:2] , S [1:2] ; X n | S , X n , Z n )= n (cid:88) i =1 I ( M [1:2] , S [1:2] ; X i | X n ,i +1 , S , X n , Z n )= n (cid:88) i =1 I ( M [1:2] , S [1:2] , X n ,i +1 , X , ∼ i , Z ∼ i ; X i | S , X i , Z i ) ≥ n (cid:88) i =1 I ( M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i − ; X i | S , X i , Z i )= n (cid:88) i =1 I ( U i , U i ; X i | U i , X i , Z i ) ( c ) = n (cid:88) i =1 I ( U i ; X i | U i , U i , X i , Z i )= nI ( U T ; X T | U T , U T , X T , Z T , T )= nI ( U ; X | U , U , X , Z, T ) , (63) where (a) follows from the Markov chain M → ( X n , S , S ) → ( M , S , X n , Z n ) , (b) follows since ( S [1:2] is independentof ( S , X n , X n , Z n ) , and (c) follows from the Markov chain U i → ( U i , X i ) → ( X i , Z i ) . Similarly, we obtain nR ≥ nI ( U ; X | U , U , X , Z, T ) . (64)We next derive the lower bound on ( R + R ) . n ( R + R ) ≥ H ( M [1:2] ) ≥ H ( M [1:2] | S [0:2] , Z n ) ≥ I ( M [1:2] ; X n , X n | S [0:2] , Z n ) ( a ) = I ( M [1:2] , S [1:2] ; X n , X n | S , Z n )= n (cid:88) i =1 I ( M [1:2] , S [1:2] ; X i , X i | X n ,i +1 , X n ,i +1 , S , Z n ) ( b ) = n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Z ∼ i ; X i ,X i | S , Z i ) ≥ n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Z i − ; X i ,X i | S , Z i )= n (cid:88) i =1 I ( U i , U i ; X i , X i | U i , Z i )= nI ( U T , U T ; X T , X T | U T , Z T , T )= nI ( U , U ; X , X | U , Z, T ) , (65)where (a) follows since S [1:2] is independent of ( S , X n , X n , Z n ) while (b) follows since ( X i , X i , Z i ) , i = 1 , . . . , n, arejointly i.i.d. and S is independent of ( X n , X n , Z n ) . We next derive the lower bound on ( R j + R j ) for j ∈ { , } . n ( R j + R j ) ≥ H ( M j , S j ) ≥ H ( M j , S j | Z n ) ≥ I ( M j , S j ; X n , X n , Y n | Z n )= n (cid:88) i =1 I ( M j , S j ; X i , X i , Y i | X n ,i +1 , X n ,i +1 , Y ni +1 , Z n )= n (cid:88) i =1 I ( M j , S j , X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) − n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ( a ) ≥ n (cid:88) i =1 I ( M j , S j , X nj,i +1 , Z i − ; X i , X i , Y i | Z i ) − ng ( (cid:15) ) ≥ n (cid:88) i =1 I ( U ji ; X i , X i , Y i | Z i ) − ng ( (cid:15) )= nI ( U jT ; X T , X T , Y T | Z T , T ) − ng ( (cid:15) )= nI ( U j ; X , X , Y | Z, T ) − ng ( (cid:15) ) , (66)where (a) follows since n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ≤ n (cid:88) i =1 I ( X ∼ i , X ∼ i , Y ∼ i , Z ∼ i ; X i , X i , Y i , Z i ) ≤ ng ( (cid:15) ) (67)by (59) and Lemma 3. We next derive the lower bound on ( R + R j + R j ) for j ∈ { , } . n ( R + R j + R j ) ≥ H ( M j , S , S j ) ≥ H ( M j , S , S j | Z n ) ≥ I ( M j , S , S j ; X n , X n , Y n | Z n )= n (cid:88) i =1 I ( M j , S , S j ; X i , X i , Y i | X n ,i +1 , X n ,i +1 , Y ni +1 , Z n )= n (cid:88) i =1 I ( M j ,S ,S j ,X n ,i +1 ,X n ,i +1 ,Y ni +1 ,Z ∼ i ; X i ,X i ,Y i | Z i ) − n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ( a ) ≥ n (cid:88) i =1 I ( M j , S , S j , X nj,i +1 , Z i − ; X i , X i , Y i | Z i ) − ng ( (cid:15) ) ≥ n (cid:88) i =1 I ( U i , U ji ; X i , X i , Y i | Z i ) − ng ( (cid:15) )= nI ( U T , U jT ; X T , X T , Y T | Z T , T ) − ng ( (cid:15) )= nI ( U , U j ; X , X , Y | Z, T ) − ng ( (cid:15) ) , (68)where (a) follows from (67). We next derive the lower bound on ( R + R + R + R ) . n ( R + R + R + R ) ≥ H ( M [1:2] , S [1:2] ) ≥ H ( M [1:2] , S [1:2] | Z n ) ≥ I ( M [1:2] , S [1:2] ; X n , X n , Y n | Z n )= n (cid:88) i =1 I ( M [1:2] , S [1:2] ; X i , X i , Y i | X n ,i +1 , X n ,i +1 , Y ni +1 , Z n )= n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Y ni +1 ,Z ∼ i ; X i ,X i ,Y i | Z i ) − n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ( a ) ≥ n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Z i − ; X i ,X i ,Y i | Z i ) − ng ( (cid:15) )= n (cid:88) i =1 I ( U i , U i ; X i , X i , Y i | Z i ) − ng ( (cid:15) )= nI ( U T , U T ; X T , X T , Y T | Z T , T ) − ng ( (cid:15) )= nI ( U , U ; X , X , Y | Z, T ) − ng ( (cid:15) ) , (69)where (a) follows from (67). We finally derive the lower bound on ( R + R + R + R + R ) . n ( R + R + R + R + R ) ≥ H ( M [1:2] , S [0:2] ) ≥ H ( M [1:2] , S [0:2] | Z n ) ≥ I ( M [1:2] , S [0:2] ; X n , X n , Y n | Z n )= n (cid:88) i =1 I ( M [1:2] , S [0:2] ; X i , X i , Y i | X n ,i +1 , X n ,i +1 , Y ni +1 , Z n )= n (cid:88) i =1 I ( M [1:2] ,S [0:2] ,X n ,i +1 ,X n ,i +1 ,Y ni +1 ,Z ∼ i ; X i ,X i ,Y i | Z i ) − n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ( a ) ≥ n (cid:88) i =1 I ( M [1:2] ,S [0:2] ,X n ,i +1 ,X n ,i +1 ,Z i − ; X i ,X i ,Y i | Z i ) − ng ( (cid:15) )= n (cid:88) i =1 I ( U i , U i , U i ; X i , X i , Y i | Z i ) − ng ( (cid:15) )= nI ( U T , U T , U T ; X T , X T , Y T | Z T , T ) − ng ( (cid:15) )= nI ( U , U , U ; X , X , Y | Z, T ) − ng ( (cid:15) ) , (70)where (a) follows from (67).We now prove the Markov chains U i → ( X i , U i ) → ( X i , Z i ) , U i → ( X i , U i ) → ( X i , Z i ) , ( U i , U i ) → ( X i , X i , U i ) → Z i and Y i → ( U i , U i , Z i ) → ( X i , X i , U i ) . Note that this implies that the joint p.m.f. satisfies (15)-(17). Recall the auxiliaryrandom variable identifications U i = S ,U i = ( M , S , X n ,i +1 , Z i − ) ,U i = ( M , S , X n ,i +1 ) . Let us first show that ( U i , U i ) → ( X i , X i , U i ) → Z i . I ( U i , U i ; Z i | X i , X i , U i )= I ( M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i − ; Z i | X i , X i , S ) ≤ I ( M [1:2] , S [1:2] , X ∼ i , X ∼ i , Z i − ; Z i | X i , X i , S )= I ( S [1:2] , X ∼ i , X ∼ i , Z i − ; Z i | X i , X i , S )+ I ( M [1:2] ; Z i | X n , X n , S [0:2] , Z i − ) ( a ) = 0 + 0 = 0 , (71)where in (a), the first term is zero since S [0:2] is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointlyi.i.d., while the second term is zero because of the Markov chain M [1:2] → ( X n , X n , S [0:2] ) → Z n .We now show that U i → ( X i , U i ) → ( X i , Z i ) holds. I ( U i ; X i , Z i | X i , U i )= I ( M , S , X n ,i +1 , Z i − ; X i , Z i | X i , S ) ≤ I ( M , S , X ∼ i , Z i − ; X i , Z i | X i , S )= I ( S , X ∼ i , Z i − ; X i , Z i | X i , S )+ I ( M ; X i , Z i | X n , S , Z i − , S ) ( a ) = 0 + 0 = 0 , (72)where in (a), the first term is zero since ( S , S ) is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointlyi.i.d., while the second term is zero because of the Markov chain M → ( X n , S , S ) → ( X n , Z n ) . In a similar fashion, wecan show that U i → ( X i , U i ) → ( X i , Z i ) holds as well. Finally, let us show that Y i → ( U i , U i , Z i ) → ( X i , X i , U i ) isa Markov chain as well. I ( Y i ; X i , X i , U i | U i , U i , Z i )= I ( Y i ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i − , Z i ) ≤ I ( Y i , Z ni +1 ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i )= I ( Z ni +1 ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i )+ I ( Y i ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z n ) ( a ) = I ( Z ni +1 ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i ) ≤ I ( Z ni +1 ; X i , X i , S | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i ) ≤ I ( Z ni +1 ; X i , X i , M [1:2] , S | S [1:2] , X n ,i +1 , X n ,i +1 , Z i )= I ( Z ni +1 ; X i , X i , S | S [1:2] , X n ,i +1 , X n ,i +1 , Z i )+ I ( Z ni +1 ; M [1:2] | S [0:2] , X n , X n , Z i ) ( b ) = 0 + 0 = 0 , where (a) follows from the Markov chain Y n → ( M [1:2] , S [1:2] , Z n ) → ( S , X n , X n ) , and in (b), the first term is zero since S [0:2] is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d., while the second term is zero becauseof the Markov chain M [1:2] → ( X n , X n , S [0:2] ) → Z n . Note that, for all t ∈ [1 : n ] , (cid:107) p X t ,X t ,Z t ,Y t | T = t − q X ,X ,Z,Y | T = t (cid:107) a ) ≤ (cid:107) p X n ,X n ,Z n ,Y n − q ( n ) X ,X ,Z,Y (cid:107) b ) < (cid:15), where ( a ) follows from [5, Lemma V.1] and the fact that T is independent of all other random variables, while ( b ) followsfrom (73). Therefore, (cid:107) p X ,X ,Z,Y | T = t − q X ,X ,Z,Y | T = t (cid:107) = (cid:107) p X T ,X T ,Z T ,Y T | T = t − q X ,X ,Z,Y | T = t (cid:107) = (cid:107) p X t ,X t ,Z t ,Y t | T = t − q X ,X ,Z,Y | T = t (cid:107) ≤ (cid:15). Now using the continuity of total variation distance and mutual information in the probability simplex along the same lines as[8, Lemma 6], the converse for Theorem 3 follows. (cid:4)
Proof of Theorem 2.
Note that by definitions 2 and 5, if ( R , R , R , R ) ∈ R MAC-coord , then we have ( R , R , , R , R ) ∈ R enc-SRMAC-coord . In other words, an outer bound for the case without shared randomness between the encoders can be obtainedby invoking Theorem 5 with R = 0 . The proof of the implication would be complete by proving that the resulting outerbound is exactly identical to Theorem 2.With R = 0 , Theorem 5 reduces to the set of ( R , R , R , R ) such that R ≥ max { I ( U ; X | U , Z, T ) ,I ( U ; X | U , U , X , Z, T ) } R ≥ max { I ( U ; X | U , Z, T ) ,I ( U ; X | U , U , X , Z, T ) } R + R ≥ I ( U , U ; X , X | U , Z, T ) R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R ≥ I ( U , U ; X , X , Y | Z, T ) R + R + R + R ≥ I ( U , U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x , x , z, t, u , u , u , y ) = p ( x , x , z ) p ( t ) p ( u | t ) p ( u , u | x , x , u , t ) p ( y | u , u , z, t ) such that p ( u | x , x , u , z, t ) = p ( u | x , u , t ) p ( u | x , x , u , z, t ) = p ( u | x , u , t ) (cid:88) u ,u ,u p ( x , x , z, u , u , u , y | t ) = q ( x , x , z, y ) , for all t. Let us define U (cid:48) j (cid:44) ( U , U j ) for j = 1 , . Using the Markov chains U → T → ( X , X , Z ) , Y → ( U , U , Z, T ) → ( X , X , U ) , ( U , U ) → ( X , X , U , T ) → Z , U → ( X , U , T ) → ( X , Z ) , and U → ( X , U , T ) → ( X , Z ) , theregion can be simplified as the set of ( R , R , R , R ) such that R ≥ max { I ( U (cid:48) ; X | Z, T ) ,I ( U (cid:48) ; X | U (cid:48) , X , Z, T ) } R ≥ max { I ( U (cid:48) ; X | Z, T ) ,I ( U (cid:48) ; X | U (cid:48) , X , Z, T ) } R + R ≥ I ( U (cid:48) , U (cid:48) ; X , X | Z, T ) R + R ≥ I ( U (cid:48) ; X , X , Y | Z, T ) R + R ≥ I ( U (cid:48) ; X , X , Y | Z, T ) R + R + R + R ≥ I ( U (cid:48) , U (cid:48) ; X , X , Y | Z, T ) , for some p.m.f. p ( x ,x , z, t, u (cid:48) , u (cid:48) , y ) = p ( x , x , z ) p ( t ) p ( u (cid:48) , u (cid:48) | x , x , t ) p ( y | u (cid:48) , u (cid:48) , z, t ) such that p ( u (cid:48) | x , x , z, t ) = p ( u (cid:48) | x , t ) p ( u (cid:48) | x , x , z, t ) = p ( u (cid:48) | x , t ) (cid:88) u (cid:48) ,u (cid:48) p ( x , x , z, u (cid:48) , u (cid:48) , y | t ) = q ( x , x , z, y ) , for all t. (cid:4) Converse Proof of Theorem 3.
Consider a code that induces a joint distribution on ( X n , X n , Z n , Y n ) such that (cid:107) p X n ,X n ,Z n ,Y n − q ( n ) X X ZY (cid:107) < (cid:15). (73)Let us first prove the lower bounds on R j for j ∈ { , } . nR j ≥ H ( M j ) ≥ H ( M j | S j , Z n ) ≥ I ( M j ; X nj | S j , Z n ) ( a ) = I ( M j , S j ; X nj | Z n )= n (cid:88) i =1 I ( M j , S j ; X ji | X nj,i +1 , Z n ) ( b ) = n (cid:88) i =1 I ( M j , S j , Z ∼ i , X nj,i +1 ; X ji | Z i ) ≥ n (cid:88) i =1 I ( M j , S j , X nj,i +1 , Z i − ; X ji | Z i ) ( c ) ≥ n (cid:88) i =1 I ( U ji ; X ji | Z i ) ( d ) = nI ( U jT ; X jT | Z T , T ) ( e ) = nI ( U j ; X j | Z, T ) , (74)where (a) follows since S j is independent of ( X nj , Z n ) , (b) follows since ( X ji , Z i ) , i = 1 , . . . , n, are jointly i.i.d., (c) followsby defining U i = ( M , S , X n ,i +1 , Z i − ) and U i = ( M , S , X n ,i +1 ) , (d) follows by introducing a uniform time-sharingrandom variable T ∈ [1 : n ] that is independent of everything else, while (e) follows by defining U := U T , U := U T , X := X T , X := X T , Y := Y T and Z := Z T .We next derive the lower bound on ( R + R ) . n ( R + R ) ≥ H ( M , S ) ≥ H ( M , S | X n , Z n ) ≥ I ( M , S ; X n , Y n | X n , Z n )= n (cid:88) i =1 I ( M , S ; X i , Y i | X n ,i +1 , Y ni +1 , X n , Z n )= n (cid:88) i =1 I ( M , S , X n ,i +1 , Y ni +1 , X ∼ i , Z ∼ i ; X i , Y i | X i , Z i ) − n (cid:88) i =1 I ( X n ,i +1 , Y ni +1 , X ∼ i , Z ∼ i ; X i , Y i | X i , Z i ) ( a ) ≥ n (cid:88) i =1 I ( M , S , X n ,i +1 , Z i − ; X i , Y i | X i , Z i ) − ng ( (cid:15) )= n (cid:88) i =1 I ( U i ; X i , Y i | X i , Z i ) − ng ( (cid:15) )= nI ( U T ; X T , Y T | X T , Z T , T ) − ng ( (cid:15) )= nI ( U ; X , Y | X , Z, T ) − ng ( (cid:15) ) , (75)where (a) follows since n (cid:88) i =1 I ( X n ,i +1 , Y ni +1 , X ∼ i , Z ∼ i ; X i , Y i | X i , Z i ) ≤ n (cid:88) i =1 I ( X ∼ i , X ∼ i , Y ∼ i , Z ∼ i ; X i , X i , Y i , Z i ) ≤ ng ( (cid:15) ) by (73) and Lemma 3. The bound n ( R + R ) ≥ I ( U ; X , Y | X , Z, T ) follows in a similar manner.For the lower bound on ( R + R + R + R ) , we proceed as follows. n ( R + R + R + R ) ≥ H ( M [1:2] , S [1:2] ) ≥ H ( M [1:2] , S [1:2] | Z n ) ≥ I ( M [1:2] , S [1:2] ; X n , X n , Y n | Z n )= n (cid:88) i =1 I ( M [1:2] , S [1:2] ; X i , X i , Y i | X n ,i +1 , X n ,i +1 , Y ni +1 , Z n )= n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Y ni +1 ,Z ∼ i ; X i ,X i ,Y i | Z i ) − n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ( a ) ≥ n (cid:88) i =1 I ( M [1:2] ,S [1:2] ,X n ,i +1 ,X n ,i +1 ,Z i − ; X i ,X i ,Y i | Z i ) − ng ( (cid:15) )= n (cid:88) i =1 I ( U i , U i ; X i , X i , Y i | Z i ) − ng ( (cid:15) )= nI ( U T , U T ; X T , X T , Y T | Z T , T ) − ng ( (cid:15) )= nI ( U , U ; X , X , Y | Z, T ) − ng ( (cid:15) ) , (76)where (a) follows since n (cid:88) i =1 I ( X n ,i +1 , X n ,i +1 , Y ni +1 , Z ∼ i ; X i , X i , Y i | Z i ) ≤ n (cid:88) i =1 I ( X ∼ i , X ∼ i , Y ∼ i , Z ∼ i ; X i , X i , Y i , Z i ) ≤ ng ( (cid:15) ) by (73) and Lemma 3.We now prove the Markov chains U i → X i → ( X i , Z i ) , U i → X i → ( U i , X i , Z i ) , ( U i , U i ) − ( X i , X i ) − Z i , and Y i → ( U i , U i , Z i ) → ( X i , X i ) . Note that this implies that the joint p.m.f. satisfies (2). Recall the auxiliary random variableidentifications U i = ( M , S , X n ,i +1 , Z i − ) , (77) U i = ( M , S , X n ,i +1 ) . (78)Let us first show that ( U i , U i ) → ( X i , X i ) → Z i . I ( U i , U i ; Z i | X i , X i ) = I ( M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i − ; Z i | X i , X i ) ≤ I ( M [1:2] , S [1:2] , X ∼ i , X ∼ i , Z i − ; Z i | X i , X i )= I ( S [1:2] , X ∼ i , X ∼ i , Z i − ; Z i | X i , X i )+ I ( M [1:2] ; Z i | X n , X n , S [1:2] , Z i − ) ( a ) = 0 + 0 = 0 , (79)where in (a), the first term is zero since S [1:2] is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointlyi.i.d., while the second term is zero because of the Markov chain M [1:2] → ( X n , X n , S [1:2] ) → Z n . We now show that U i → X i → ( X i , Z i ) is a Markov chain. I ( U i ; X i , Z i | X i )= I ( M , S , X n ,i +1 , Z i − ; X i , Z i | X i ) ≤ I ( M , S , X ∼ i , Z i − ; X i , Z i | X i )= I ( S , X ∼ i , Z i − ; X i , Z i | X i )+ I ( M ; X i , Z i | X n , S , Z i − ) ( a ) = 0 + 0 = 0 , (80)where in (a), the first term is zero since S is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointlyi.i.d., while the second term is zero because of the Markov chain M → ( X n , S ) → ( X n , Z n ) . Next, we show that Y i → ( U i , U i , Z i ) → ( X i , X i ) is a Markov chain. I ( Y i ; X i , X i | U i , U i , Z i )= I ( Y i ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i − , Z i ) ≤ I ( Y i , Z ni +1 ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i )= I ( Z ni +1 ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i )+ I ( Y i ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z n ) ( a ) = I ( Z ni +1 ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i ) ≤ I ( Z ni +1 ; X i , X i | M [1:2] , S [1:2] , X n ,i +1 , X n ,i +1 , Z i ) ≤ I ( Z ni +1 ; X i , X i , M [1:2] | S [1:2] , X n ,i +1 , X n ,i +1 , Z i )= I ( Z ni +1 ; X i , X i | S [1:2] , X n ,i +1 , X n ,i +1 , Z i )+ I ( Z ni +1 ; M [1:2] | S [1:2] , X n , X n , Z i ) ( a ) = 0 + 0 = 0 , where (a) follows from the Markov chain Y n → ( M [1:2] , S [1:2] , Z n ) → ( X n , X n ) and in (b), the first term is zero since S [1:2] is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d., while the second term is zero because ofthe Markov chain M [1:2] → ( X n , X n , S [1:2] ) → Z n .It remains to prove the Markov chain U i → X i → ( U i , X i , Z i ) . Consider the following chain of inequalities. I ( U i ; U i , X i , Z i | X i )= I ( M , S , X n ,i +1 ; M , S , X n ,i , Z i | X i )= I ( S , X n ,i +1 ; M , S , X n ,i , Z i | X i )+ I ( M ; M , S , X n ,i , Z i | S , X n ,i )= I ( S , X n ,i +1 ; S , X n ,i , Z i | X i )+ I ( S , X n ,i +1 ; M | S , X n ,i , Z i , X i )+ I ( M ; S , X n ,i , Z i | S , X n ,i )+ I ( M ; M | S , S , X n ,i , X n ,i , Z i ) ( a ) ≤ I ( S , X n ,i +1 ; M , X i − , Z i − | S , X n ,i , Z i , X i )+ I ( M , X i − ; S , X n ,i , Z i | S , X n ,i )+ I ( M , X i − ; M | S , S , X n ,i , X n ,i , Z i )= I ( S , X n ,i +1 ; X i − , Z i − | S , X n ,i , Z i , X i ) + I ( S , X n ,i +1 ; M | S , X n , Z n , X i )+ I ( X i − ; S , X n ,i , Z i | S , X n ,i )+ I ( M ; S , X n ,i , Z i | S , X n )+ I ( X i − ; M | S , S , X n ,i , X n ,i , Z i )+ I ( M ; M | S , S , X n ,i , X n , Z i ) ( b ) = 0+ 0+ 0+ 0+ I ( X i − ; M | S , S , X n ,i , X n ,i , Z i )+ 0 ≤ I ( X i − ; M , X i − | S , S , X n ,i , X n ,i , Z i )= I ( X i − ; X i − | S , S , X n ,i , X n ,i , Z i )+ I ( X i − ; M | S , S , X n , X n ,i , Z i ) ( c ) = I ( X i − ; X i − | Z i − )+ I ( X i − ; M | S , S , X n , X n ,i , Z i ) ( d ) = 0 + 0 = 0 , (81)where the fact that the first term in (a) and the first and third terms in (b) are zeros, as well as (c), follow since S [1:2] isindependent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d. The second, fourth and sixth terms in (b) andthe second term in (d) are zeros because of the Markov chains M → ( X n , S ) → ( S , X n , Z n ) and M → ( X n , S ) → ( S , M , X n , Z n ) . The first term in (d) is zero because ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d. with q X X ,Z and I ( X ; X | Z ) = 0 . Notice that we also have (cid:107) p X ,X ,Z,Y | T = t − q X ,X ,Z,Y | T = t (cid:107) ≤ (cid:15) along the same lines as in the proof ofTheorem 5. Using the continuity of total variation distance and mutual information in the probability simplex along the samelines as [8, Lemma 6], the outer bound in Theorem 3 follows. (cid:4) Proof of Theorem 6.
With the same choice of auxiliary random variables as in the proof of Theorem 5, we will show thatthe Markov chain U → ( X , U , T ) → ( U , X , Z ) holds when X → Z → X . Then in addition to the Markov chains U → T → ( X , X , Z ) , U → ( X , U , T ) → ( X , Z ) , ( U , U ) → ( X , X , U , T ) → Z and Y → ( U , U , Z, T ) → ( X , X , U ) , we note the following simplifications to the rate constraints in Theorem 5: R ≥ max { I ( U ; X | U , Z, T ) , I ( U ; X | U , U , X , Z, T ) } = max { I ( U ; X | U , Z, T ) , I ( U ; X | U , Z, T ) } = I ( U , U ; X | Z, T ) ,R ≥ max { I ( U ; X | U , Z, T ) , I ( U ; X | U , U , X , Z, T ) } = max { I ( U ; X | U , Z, T ) , I ( U ; X | U , Z, T ) } = I ( U , U ; X | Z, T ) ,R + R ≥ I ( U , U ; X , X | U , Z, T )= I ( U , U , U ; X , X | Z, T ) ,R + R ≥ I ( U ; X , X , Y | Z, T )= I ( U ; X , Y | X , Z, T ) ,R + R ≥ I ( U ; X , X , Y | Z, T )= I ( U ; X , Y | X , Z, T ) ,R + R + R ≥ I ( U , U ; X , X , Y | Z, T )= I ( U , U ; X , Y | X , Z, T ) R + R + R ≥ I ( U , U ; X , X , Y | Z, T )= I ( U , U ; X , Y | X , Z, T ) ,R + R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) ,R + R + R + R + R ≥ I ( U , U , U ; X , X , Y | Z, T ) . It remains to prove the Markov chain U i → ( X i , U i ) → ( U i , X i , Z i ) . Recall the choice of auxiliary random variables. U i = S ,U i = ( M , S , X n ,i +1 , Z i − ) ,U i = ( M , S , X n ,i +1 ) . Consider the following chain of inequalities. I ( U i ; U i , X i , Z i | X i , U i )= I ( M , S , X n ,i +1 ; M , S , X n ,i , Z i | X i , S )= I ( S , X n ,i +1 ; M , S , X n ,i , Z i | X i , S )+ I ( M ; M , S , X n ,i , Z i | S , S , X n ,i )= I ( S , X n ,i +1 ; S , X n ,i , Z i | X i , S )+ I ( S , X n ,i +1 ; M | S , X n ,i , Z i , X i , S )+ I ( M ; S , X n ,i , Z i | S , S , X n ,i )+ I ( M ; M | S , S , S , X n ,i , X n ,i , Z i ) ( a ) ≤ I ( S , X n ,i +1 ; M , X i − , Z i − | S , S , X n ,i , Z i , X i )+ I ( M , X i − ; S , X n ,i , Z i | S , S , X n ,i )+ I ( M , X i − ; M | S , S , S , X n ,i , X n ,i , Z i )= I ( S , X n ,i +1 ; X i − , Z i − | S , S , X n ,i , Z i , X i )+ I ( S , X n ,i +1 ; M | S , S , X n , Z n , X i )+ I ( X i − ; S , X n ,i , Z i | S , S , X n ,i )+ I ( M ; S , X n ,i , Z i | S , S , X n )+ I ( X i − ; M | S , S , S , X n ,i , X n ,i , Z i )+ I ( M ; M | S , S , S , X n ,i , X n , Z i ) ( b ) = 0+ 0+ 0+ 0+ I ( X i − ; M | S , S , S , X n ,i , X n ,i , Z i )+0 ≤ I ( X i − ; M , X i − | S , S , S , X n ,i , X n ,i , Z i )= I ( X i − ; X i − | S , S , S , X n ,i , X n ,i , Z i )+ I ( X i − ; M | S , S , S , X n , X n ,i , Z i ) ( c ) = I ( X i − ; X i − | Z i − )+ I ( X i − ; M | S , S , S , X n , X n ,i , Z i ) ( d ) = 0 + 0 = 0 , (82)where the first term in (a), the first and third terms in (b) are zeros and (c) follows since S [0:2] is independent of ( X n , X n , Z n ) and ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d. The second, fourth and sixth terms in (b) and the second term in (d) arezeros because of the Markov chains M → ( X n , S , S ) → ( S , X n , Z n ) and M → ( X n , S , S ) → ( S , M , X n , Z n ) . Thefirst term in (d) is zero because ( X i , X i , Z i ) , i = 1 , . . . , n, are jointly i.i.d. with q X X ,Z and I ( X ; X | Z ) = 0 . (cid:4) A PPENDIX AE XPLANATION FOR R EMARK R = R + R in the region of Theorem 1 and eliminate the rates R , R to get an inner bound for the problem with three-way common randomness. Using Fourier-Motzkin elimination (FME), weconclude that rate tuples ( R , R , R ) satisfying R ≥ I ( U ; X | U , Z, T ) ,R ≥ I ( U ; X | U , Z, T ) , R + R ≥ I ( U , U ; X , X | Z, T ) ,R + R ≥ I ( U ; X , X , Y | Z, T ) − I ( U ; U | Z, T ) ,R + R ≥ I ( U ; X , X , Y | Z, T ) − I ( U ; U | Z, T ) ,R + R + R ≥ I ( U , U ; X , X , Y | Z, T ) , for some p.m.f. p ( x ,x , z, t, u , u , y ) = p ( x , x , z ) p ( t ) p ( u | x , t ) p ( u | x , t ) p ( y | u , u , z, t ) such that (cid:88) u ,u p ( x , x , z, u , u , y | t ) = q ( x , x , z, y ) , for all t, are achievable. The inner bound of Atif et al. [41, Theorem 2] differs from the above region only with respect to the rateconstraint on R + R after correcting for a possible error (see below) in the former (note that Z = ∅ in their setting). Inparticular, the corrected region of Atif et al. [41, Theorem 2] must be the set of all rate tuples ( R , R , R ) satisfying R ≥ I ( U ; X | U , T ) ,R ≥ I ( U ; X | U , T ) ,R + R ≥ I ( U , U ; X , X | T ) ,R + R ≥ I ( U ; X , X , Y, U | T ) − I ( U ; U | T ) ,R + R ≥ I ( U ; X , X , Y | T ) − I ( U ; U | T ) ,R + R + R ≥ I ( U , U ; X , X , Y | T ) , for some p.m.f. p ( x ,x , t, u , u , y ) = p ( x , x ) p ( t ) p ( u | x , t ) p ( u | x , t ) p ( y | u , u , t ) such that (cid:88) u ,u p ( x , x , u , u , y | t ) = q ( x , x , y ) , for all t. Comparing the corresponding constraints on R + R clearly implies that our inner bound above is potentially stronger thanthe inner bound of [41, Theorem 2] (and the classical version of [42, Theorem 6]). Corrections to Atif et al. [41, Theorem 2] based on [47]:
The bounds on R + R and R + R do not appear in [41,Theorem 2]. We remark here that the proof of inner bound [55] gives the two additional constraints R + R ≥ I ( U ; X , X , Y, U | T ) − I ( U ; U | T ) , (83) R + R ≥ I ( U ; X , X , Y | T ) − I ( U ; U | T ) . (84)However, these are missing in the statement of [41, Theorem 2] due to an error in the last step of the proof, i.e, in FME,where the non-negativity constraints on rates R and R were not taken care of. This has been corrected in a quantumgeneralization of the same setting by Atif et al. [42, Theorem 6].Also, the structure of the p.m.f. with respect to the time sharing random variable in [41, Theorem 2] might only guaranteea weaker form of coordination, namely, empirical coordination [1]. In particular, it turns out that the condition (cid:88) u ,u ,t p ( x , x , u , u , y, t ) = q ( x , x , y ) (85)of [41, Theorem 2] might not suffice for a vanishing total variation distance between the required distributions. Instead, astronger condition (cid:88) u ,u p ( x , x , u , u , y | t ) = q ( x , x , y ) , for all t, (86)ensures vanishing total variation distance and hence strong coordination. Note that (85) may not imply (86) for all (or evenany) t . So, the corrected statement of [41, Theorem 2] should contain two additional rate constraints (83) and (84), and thep.m.f. structure as in (86) instead of (85). A PPENDIX BS PECIALIZATION OF T HEOREM TO D ETERMINISTIC F UNCTIONS
Let |T | = 1 for simplicity. When Y is a deterministic function of ( X , X , Z ) , we note the following simplifications to therate constraints of Theorem 1. R + R ≥ I ( U ; X , X , Y | Z ) − I ( U ; U | Z ) ( a ) = I ( U ; X , X | Z ) − I ( U ; U | Z )= I ( U ; X , X | U , Z ) − I ( U ; U | X , X , Z ) ( b ) = I ( U ; X | U , Z ) , (87)where (a) follows since Y is determined by ( X , X , Z ) while (b) follows from the Markov chains U → X → ( U , X , Z ) and U → X → ( U , X , Z ) . Similarly, it follows that R + R ≥ I ( U ; X | U , Z ) . (88)Furthermore, we note that R + R + R ≥ I ( U ; X , X , Y | Z ) + I ( U ; X | U , Z ) ( a ) = I ( U ; X , X | Z ) + I ( U ; X | U , Z ) ( b ) = I ( U ; X , X | Z ) + I ( U ; X , X | U , Z )= I ( U , U ; X , X | Z ) , (89)where (a) follows since Y is determined by ( X , X , Z ) while (b) follows from the Markov chain U → X → ( U , X , Z ) .Similarly, it follows that R + R + R ≥ I ( U , U ; X , X | Z ) . (90)Finally, we note that R + R + R + R ≥ I ( U , U ; X , X , Y | Z )= I ( U , U ; X , X | Z ) . (91)Thus when Y is a deterministic function of ( X , X , Z ) , the rate constraints involving the shared randomness rates R and R become redundant in Theorem 1. Hence, the region simplifies to the set of rate pairs ( R , R ) satisfying R ≥ I ( U ; X | U , Z ) ,R ≥ I ( U ; X | U , Z ) ,R + R ≥ I ( U , U ; X , X | Z ) , for some p.m.f. p ( x , x , z, u , u ) = p ( x , x , z ) p ( u | x ) p ( u | x ) such that H ( Y | U , U , Z ) = 0 . This is precisely the inner bound of [29, Theorem 2] specialized to the multiple-access networkwhere the condition H ( Y | U , U , Z ) = 0 , in conjunction with the structure of the p.m.f., is expressed in an alternate forminvolving graph entropy using [50, Lemma 3]. A PPENDIX CS PECIALIZATION OF T HEOREM TO D ETERMINISTIC F UNCTIONS
Let |T | = 1 again for simplicity. When Y is a deterministic function of ( X , X , Z ) , say, Y = f ( X , X , Z ) , it followsthat the Markov chains U → X → ( X , Z ) and U → X → ( X , Z ) render the rate bounds on R + R , R + R and R + R + R + R in Theorem 2 redundant. The characterization reduces to the set of ( R , R ) satisfying R ≥ max { I ( U ; X | Z ) , I ( U ; X | U , X , Z ) } , (92) R ≥ max { I ( U ; X | Z ) , I ( U ; X | U , X , Z ) } , (93) R + R ≥ I ( U , U ; X , X | Z ) , (94)for some p.m.f. p ( x , x , z, u , u ) = p ( x , x , z ) p ( u , u | x , x ) such that p ( u | x , x , z ) = p ( u | x ) ,p ( u | x , x , z ) = p ( u | x ) ,H ( Y | U , U , Z ) = 0 . We now prove that this region recovers the outer bound of [29, Theorem 1] specialized to the multiple-access network. In thefollowing, we write X ∈ V for two random variables X ∈ X and V taking values over subsets of X provided P ( X ∈ V ) = 1 .Recall the definition of conditional graph entropy. Definition 7 (Conditional Graph Entropy [20]) . Conditional graph entropy is defined as H G X | Y (cid:44) min V → X → YX ∈ V ∈ Γ ∗ ( G X | Y ) I ( V ; X | Y ) , (95) where Γ ∗ ( G X | Y ) denotes the set of maximal independent sets of the conditional characteristic graph G X | Y [56], [57]. To obtain the graph entropy based characterization of [29, Theorem 1], we proceed as in [50, Proof of Corollary 1]. If ( R , R ) is an achievable rate pair for ( X , X , Z ) , then ( R , is achievable for ( X , ∅ , ( X , Z )) i.e. for the setting where X is revealed to the receiver. The second inequality of (92) for ( X , ∅ , ( X , Z )) becomes R ≥ I ( U ; X | U , ∅ , ( X , Z ))= I ( U ; X | U , X , Z ) , for some U and U that satisfy H ( f ( X , X , Z ) | U , U , X , Z ) = 0 , (96) U → X → ( X , Z ) ,U → ∅ → ( X , X , Z ) , (97) ( U , U ) → X → ( X , Z ) . (98)Thus we obtain R ≥ I ( U ; X | U , X , Z ) ( a ) = I ( U , U ; X | X , Z ) , (99)where (a) follows from (97). Furthermore, from (96), (98), and [50, Corollary 2], it follows that X ∈ ( U , U ) ∈ Γ( G X | X ,Z ) . This along with (99) and Definition 7 imply R ≥ H G X | X ,Z ( X | X , Z ) . Similarly it can be shown that R ≥ H G X | X ,Z ( X | X , Z ) . For the sum rate bound, we recall that H ( f ( X , X , Z ) | U , U , Z ) = 0 . This along with the Markov chain ( U , U ) → ( X , X ) → Z and [50, Corollary 2] gives ( X , X ) ∈ ( U , U ) ∈ Γ( G X ,X | Z ) . This along with (94) and Definition 7 imply R + R ≥ H G X ,X | Z ( X , X | Z ) . A PPENDIX DS PECIALIZATION OF T HEOREM TO D ETERMINISTIC F UNCTIONS
Let |T | = 1 again for simplicity. The joint distribution in (2) is such that ( U , X ) → Z → ( U , X ) is a Markov chain. Thisleads to the following simplifications to the rate constraints involving shared randomness when Y is a deterministic functionof ( X , X , Z ) , say, Y = f ( X , X , Z ) . I ( U ; X , Y | X , Z ) = I ( U ; X , X , Y | Z )= I ( U ; X , X | Z )= I ( U ; X | Z ) . Similarly, I ( U ; X , Y | X , Z ) = I ( U ; X | Z ) . Furthermore, I ( U , U ; X , X , Y | Z ) = I ( U , U ; X , X | Z )= I ( U ; X | Z ) + I ( U ; X | Z ) . These simplifications render the bounds on R + R , R + R and R + R + R + R in Theorem 3 redundant. Hence,the region simplifies to the set of rate pairs ( R , R ) satisfying R ≥ I ( U ; X | Z ) ,R ≥ I ( U ; X | Z ) , for some p.m.f. p ( x ,x , z, u , u , y ) = p ( z ) p ( x | z ) p ( x | z ) p ( u | x ) p ( u | x ) p ( y | u , u , z ) such that H ( Y | U , U , Z ) = 0 . This is the rate region in [29, Theorem 3] specialized to the multiple-access network where itis expressed in an alternate form involving graph entropies, which follows by using [50, Proof of Theorem 3].A PPENDIX EP ROOF OF C LAIM u , u for which H ( X | U = u ) > and H ( X | U = u ) > . We prove theclaim in three steps.Step : We prove that under Case , for any u with P ( U = u ) > , there exists a k ∈ { , } such that H ( X k | U = u ) = 0 .Step : Then for any u with P ( U = u ) > , for the k from Step , we show that H ( X k | U = u ) = 0 .Step : Finally, for any u (cid:48) with u (cid:48) (cid:54) = u and P ( U = u (cid:48) ) > , for the k from Step , we prove that H ( X k | U = u (cid:48) ) = 0 .Step : We prove this by contradiction. Suppose H ( X i | U = u ) > , for i = 1 , . This implies that the support of p X | U = u cannot be of the form { b , b } , { ( b , b ) , ( b , − b ) } , { ( b , b ) , (1 − b , b ) } , for b , b ∈ { , } . The remainingpossibility is that the support has to be a superset of either { (0 , , (1 , } or { (0 , , (1 , } . In the sequel, we use theindependence of ( U , X ) and ( U , X ) , and the Markov chain Y → ( U , U ) → ( X , X ) repeatedly. Consider a u suchthat P ( U = u ) > . It turns out that the support cannot be a superset of { (0 , , (1 , } . To see this, first notice that P ( X = (0 , | U = u , U = u ) , P ( X = (1 , | U = u , U = u ) > . Now since X J − ( U , U ) − X , we have P ( X J = 0 | U = u , U = u ) = P ( X J = 0 | U = u , U = u , X = (1 , , where the last equality follows from thecorrectness of output Y = ( X J , X J ) . Similarly, P ( X J = 1 | U = u , U = u ) = P ( X J = 1 | U = u , U = u , X =(0 , . This is a contradiction since p X J | U = u ,U = u has to be a probability distribution. The only other possibility isthat the support of p X | U = u is a superset of { (0 , , (1 , } . Since H ( X | U ) > , there exists a u with P ( U = u ) > such that H ( X | U = u ) > . So, either exactly one or both of H ( X | U = u ) and H ( X | U = u ) will be strictlypositive. Suppose exactly one of them is strictly positive. Without loss of generality, suppose H ( X | U = u ) > , i.e., P ( X = 0 | U = u ) , P ( X = 1 | U = u ) > and H ( X | U = u ) = 0 . Also, assume that P ( X = 1 | U = u ) = 1 .Consider the probability distribution p Y | U = u ,U = u . This is well defined because P ( U = u , U = u ) > as U isindependent of U and P ( U = u ) , P ( U = u ) > . From the above we have the following. P ( X = (0 , , X = (0 , | U = u , U = u ) > which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 . P ( X = (1 , , X = (0 , | U = u , U = u ) > which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 .This is a contradiction since p Y | U = u ,U = u has to be a probability distribution. Now suppose both H ( X | U = u ) and H ( X | U = u ) are strictly positive. Since Y := X J ∈ { X , X } , the only possibility is that p X | U = u has a supportthat is a superset of { (0 , , (1 , } . Then we have the following. P ( X = (0 , , X = (0 , | U = u , U = u ) > which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 . P ( X = (0 , , X = (1 , | U = u , U = u ) > which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 .This is a contradiction since p Y | U = u ,U = u has to be a probability distribution. Thus, we have, under Case , if H ( X | U = u ) > , then there exists a k ∈ { , } such that H ( X k | U = u ) = 0 .Step : Note that there exists u with P ( U = u ) > such that H ( X | U = u ) > since H ( X | U ) > . So, bythe discussion in Step , there exists a k such that H ( X k | U = u ) = 0 . Note that H ( X k (cid:48) | U = u ) > , where k (cid:48) = 3 − k , since H ( X | U = u ) > . Similarly, there exists u with P ( U = u ) > such that H ( X | U = u ) > .Now we show that H ( X k | U = u ) = 0 . We prove this by contradiction. Suppose H ( X k | U = u ) > . Then in viewof the above discussion, we have H ( X k (cid:48) | U = u ) = 0 . Without loss of generality, let k = 1 , i.e., k (cid:48) = 2 . Note that p X ,X | U = u ,U = u = p X | U = u · p X | U = u has full support since H ( X | U = u ) , H ( X | U = u ) > . Also, assumethat P ( X = 0 | U = u ) = 1 and P ( X = 0 | U = u ) = 1 (other choices can be dealt similarly). Consider the probabilitydistribution p Y | U = u ,U = u . Then we have the following. P ( X = (0 , , X = (1 , | U = u , U = u ) = P ( X = 1 , X = 1 | U = u , U = u ) > , which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 since we have the Markov chain Y − ( U , U ) − ( X , X ) . P ( X = (0 , , X = (0 , | U = u , U = u ) = P ( X = 0 , X = 0 | U = u , U = u ) > , which implies that P ( Y = (0 , | U = u , U = u ) = 0 and P ( Y = (1 , | U = u , U = u ) = 0 since we have the Markov chain Y − ( U , U ) − ( X , X ) .This is a contradiction since p Y | U = u ,U = u has to be a probability distribution.Step : Now suppose that there exists a u (cid:48) with u (cid:48) (cid:54) = u and P ( U = u (cid:48) ) > such that H ( X | U = u (cid:48) ) > . Sincethere is a u such that H ( X k | U = u ) = 0 , by the same argument as above (reversing the roles of and ), we have H ( X k | U = u (cid:48) ) = 0 . This completes the proof of Claim 1.R EFERENCES[1] P. Cuff, H. Permuter, and T. Cover, “Coordination capacity,”
IEEE Transactions on Information Theory , vol. 56, no. 9, pp. 4181–4206, 2010.[2] C. Bennett, P. Shor, J. Smolin, and A. Thapliyal, “Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem,”
IEEETransactions on Information Theory , vol. 48, no. 10, pp. 2637–2655, 2002.[3] P. Harsha, R. Jain, D. McAllester, and J. Radhakrishnan, “The communication complexity of correlation,”
IEEE Transactions on Information Theory ,vol. 56, no. 1, pp. 438–449, 2010.[4] A. Winter, “Compression of sources of probability distributions and density operators,” arXiv preprint quant-ph/0208131 , 2002.[5] P. Cuff, “Distributed channel synthesis,”
IEEE Transactions on Information Theory , vol. 59, no. 11, pp. 7071–7096, 2013.[6] C. Bennett, I. Devetak, A. Harrow, P. Shor, and A. Winter, “The quantum reverse Shannon theorem and resource tradeoffs for simulating quantumchannels,”
IEEE Transactions on Information Theory , vol. 60, no. 5, pp. 2926–2959, 2014.[7] M. Wilde, P. Hayden, F. Buscemi, and M.-H. Hsieh, “The information-theoretic costs of simulating quantum measurements,”
Journal of Physics A:Mathematical and Theoretical , vol. 45, no. 45, p. 453001, 2012.[8] M. Yassaee, A. Gohari, and M. Aref, “Channel simulation via interactive communications,”
IEEE Transactions on Information Theory , vol. 61, no. 6,pp. 2964–2982, 2015.[9] G. Cervia, L. Luzzi, M. Bloch, and M. Le Treust, “Polar coding for empirical coordination of signals and actions over noisy channels,” in
IEEEInformation Theory Workshop , 2016, pp. 81–85.[10] M. Le Treust, “Joint empirical coordination of source and channel,”
IEEE Transactions on Information Theory , vol. 63, no. 8, pp. 5087–5114, 2017.[11] R. Chou, M. Bloch, and J. Kliewer, “Empirical and strong coordination via soft covering with polar codes,”
IEEE Transactions on Information Theory ,vol. 64, no. 7, pp. 5087–5100, 2018.[12] M. Mylonakis, P. Stavrou, and M. Skoglund, “Empirical coordination with multiple descriptions,” in
Allerton Conference , 2019, pp. 1074–1081.[13] E. Kushilevitz and N. Nisan,
Communication Complexity . Cambridge University Press, 1996.[14] M. Braverman and A. Rao, “Information equals amortized communication,”
IEEE Transactions on Information Theory , vol. 60, no. 10, pp. 6058–6069,2014.[15] J. K¨orner and K. Marton, “How to encode the modulo-two sum of binary sources (corresp.),”
IEEE Transactions on Information Theory , vol. 25, no. 2,pp. 219–221, 1979.[16] H. Yamamoto, “Wyner-Ziv theory for a general function of the correlated sources (corresp.),”
IEEE Transactions on Information Theory , vol. 28, no. 5,pp. 803–807, 1982.[17] A. Kaspi, “Two-way source coding with a fidelity criterion,”
IEEE Transactions on Information Theory , vol. 31, no. 6, pp. 735–740, 1985.[18] T. Han and K. Kobayashi, “A dichotomy of functions f ( x, y ) of correlated sources ( x, y ) from the viewpoint of achievable rate region,” IEEE Transactionson Information Theory , vol. 33, no. 1, pp. 69–76, 1987.[19] N. Alon and A. Orlitsky, “Source coding and graph entropies,”
IEEE Transactions on Information Theory , vol. 42, no. 5, pp. 1329–1339, 1996.[20] A. Orlitsky and J. Roche, “Coding for computing,”
IEEE Transactions on Information Theory , vol. 47, no. 3, pp. 903–917, 2001.[21] B. Nazer and M. Gastpar, “Computation over multiple-access channels,”
IEEE Transactions on Information Theory , vol. 53, no. 10, pp. 3498–3516,2007.[22] B. Nazer and M. Gastpar, “Lattice coding increases multicast rates for gaussian multiple-access networks,” in
Allerton Conference , 2007, pp. 1089–1096.[23] M. Gastpar, “Uncoded transmission is exactly optimal for a simple Gaussian sensor network,”
IEEE Transactions on Information Theory , vol. 54, no. 11,pp. 5247–5251, 2008.[24] N. Ma and P. Ishwar, “Some results on distributed source coding for interactive function computation,”
IEEE Transactions on Information Theory ,vol. 57, no. 9, pp. 6180–6195, 2011.[25] R. Appuswamy, M. Franceschetti, N. Karamchandani, and K. Zeger, “Network coding for computing: Cut-set bounds,”
IEEE Transactions on InformationTheory , vol. 57, no. 2, pp. 1015–1030, 2011.[26] H. Kowshik and P. Kumar, “Optimal function computation in directed and undirected graphs,”
IEEE Transactions on Information Theory , vol. 58, no. 6,pp. 3407–3418, 2012. [27] S. Kuzuoka and S. Watanabe, “A dichotomy of functions in distributed coding: An information spectral approach,” IEEE Transactions on InformationTheory , vol. 61, no. 9, pp. 5028–5041, 2015.[28] M. Sefidgaran, A. Gohari, and M. Aref, “On K¨orner-Marton’s sum modulo two problem,” in
Iran Workshop on Communication and Information Theory ,2015, pp. 1–6.[29] M. Sefidgaran and A. Tchamkerten, “Distributed function computation over a rooted directed tree,”
IEEE Transactions on Information Theory , vol. 62,no. 12, pp. 7135–7152, 2016.[30] S. Watanabe, “A classification of functions in multiterminal distributed computing,”
IEEE Transactions on Information Theory , vol. 66, no. 10, pp.6169–6183, 2020.[31] S. Satpathy and P. Cuff, “Secure cascade channel synthesis,”
IEEE Transactions on Information Theory , vol. 62, no. 11, pp. 6081–6094, 2016.[32] B. Vellambi, J. Kliewer, and M. Bloch, “Strong coordination over multi-hop line networks using channel resolvability codebooks,”
IEEE Transactionson Information Theory , vol. 64, no. 2, pp. 1132–1162, 2017.[33] A. Bereyhi, M. Bahrami, M. Mirmohseni, and M. Aref, “Empirical coordination in a triangular multiterminal network,” in
IEEE International Symposiumon Information Theory , 2013, pp. 2149–2153.[34] M. Mylonakis, P. Stavrou, and M. Skoglund, “Empirical coordination subject to a fidelity criterion,” in
IEEE Information Theory Workshop , 2019, pp.1–5.[35] C. Shannon, “Two-way communication channels,” in
In Proc. 4TH Berkely Symp. Math, Statist. Probab , 1961, pp. 611–644.[36] R. Ahlswede, “Multi-way communication channels,” in
Second International Symposium on Information Theory: Tsahkadsor, Armenia, USSR, Sept. 2-8,1971 , 1973.[37] H. Liao, “Multiple access channels,” Ph.D. dissertation, University of Hawaii, Honolulu, HI, 1972.[38] F. Willems, “The discrete memoryless multiple access channel with partially cooperating encoders (corresp.),”
IEEE Transactions on Information Theory ,vol. 29, no. 3, pp. 441–445, 1983.[39] S. Bross, A. Lapidoth, and M. Wigger, “Dirty-paper coding for the Gaussian multiaccess channel with conferencing,”
IEEE Transactions on InformationTheory , vol. 58, no. 9, pp. 5640–5668, 2012.[40] P. Noorzad, M. Effros, and M. Langberg, “The unbounded benefit of encoder cooperation for the k -user MAC,” IEEE Transactions on InformationTheory , vol. 64, no. 5, pp. 3655–3678, 2017.[41] T. Atif, A. Padakandla, and S. Pradhan, “Source coding for synthesizing correlated randomness,” in
IEEE International Symposium on InformationTheory , 2020, pp. 1576–1581.[42] T. Atif, M. Heidari, and S. Pradhan, “Faithful simulation of distributed quantum measurements with applications in distributed rate-distortion theory,” arXiv preprint arXiv:1901.06188 , 2020.[43] T. Atif and S. Pradhan, “Distributed quantum faithful simulation and function computation using algebraic structured measurements,” arXiv preprintarXiv:2101.02360 , 2021.[44] S.-Y. Tung, “Multiterminal source coding,” Ph.D. dissertation, Cornell University, Ithaca, NY, 1978.[45] T. Berger, “Multiterminal source coding,”
The information theory approach to communications , vol. 229, pp. 171–231, 1977.[46] M. Yassaee, M. Aref, and A. Gohari, “Achievability proof via output statistics of random binning,”
IEEE Transactions on Information Theory , vol. 60,no. 11, pp. 6760–6786, 2014.[47] S. Pradhan, Private Communication, January 2021.[48] M. Gastpar, “The Wyner-Ziv problem with multiple sources,”
IEEE Transactions on Information Theory , vol. 50, no. 11, pp. 2762–2768, 2004.[49] M. Sefidgaran and A. Tchamkerten, “Computing a function of correlated sources: A rate region,” in
IEEE International Symposium on InformationTheory , 2011, pp. 1856–1860.[50] M. Sefidgaran and A. Tchamkerten, “On computing a function of correlated sources,” arXiv preprint arXiv:1107.5806 , 2011.[51] D. Krithivasan and S. Pradhan, “Distributed source coding using Abelian group codes: A new achievable rate-distortion region,”
IEEE Transactions onInformation Theory , vol. 57, no. 3, pp. 1495–1519, 2011.[52] P. G´acs and J. K¨orner, “Common information is far less than mutual information,”
Problems of Control and Information Theory , vol. 2, no. 2, pp.149–162, 1973.[53] D. Slepian and J. Wolf, “Noiseless coding of correlated information sources,”
IEEE Transactions on information Theory , vol. 19, no. 4, pp. 471–480,1973.[54] G. Cervia, L. Luzzi, M. Le Treust, and M. Bloch, “Strong coordination of signals and actions over noisy channels with two-sided state information,”
IEEE Transactions on Information Theory , 2020.[55] T. Atif, A. Padakandla, and S. Pradhan, “Source coding for synthesizing correlated randomness,” arXiv preprint arXiv:2004.03651v1 , 2020.[56] J. K¨orner, “Coding of an information source having ambiguous alphabet and the entropy of graphs,” in ,1973, pp. 411–425.[57] H. Witsenhausen, “The zero-error side information problem and chromatic numbers (corresp.),”