Source-Channel Matching for Sources with Memory
Christos Kourtellaris, Charalambos D. Charalambous, Photios A. Stavrou
aa r X i v : . [ c s . I T ] M a r Source-Channel Matching for Sources with Memory
Christos Kourtellaris, Photios A. Stavrou, Charalambos D. Charalambous
Dep. of Electrical & Computer Engineering, University of Cyprus, Nicosia, CyprusEmail: { kourtellaris.christos, stavrou.fotios, chadcha } @ucy.ac.cy Abstract —To be considered for an IEEE Jack Keil Wolf ISITStudent Paper Award. In this paper we analyze the probabilisticmatching of sources with memory to channels with memory sothat symbol-by-symbol code with memory without anticipationare optimal, with respect to an average distortion and excessdistortion probability. We show achievability of such a symbol-by-symbol code with memory without anticipation, and we showmatching for the Binary Symmetric Markov source (BSMS(p))over a first-order symmetric channel with a cost constraint.
I. I
NTRODUCTION
In this paper we address the problem of Joint Source-Channel Coding JSCC based on symbol-by-symbol codetransmission with memory without anticipation. Thus, at eachinstant of time i , we impose real-time transmission constrainson the encoder and decoder to process samples independently,with memory on past symbols, and without anticipation withrespect to symbols occurring future times j > i . The aim is tomatch probabilistically the source to a channel, and evaluateits performance with respect to excess distortion probability.For memoryless sources and channels, necessary and suffi-cient conditions for symbol-by-symbol transmission are givenin [1] (see also [2]). However, extending these results tosources with memory is not a trivial task for the following tworeasons. i) The optimal reproduction distribution of classicalRate Distortion Function (RDF), used during the realizationprocedure, to match the source to a channel is, in generalnoncausal (anticipative on future symbols); ii) the solution tothe RDF is often unknown.In this paper we consider a nonanticipative informationRDF which is realizable in the above sense, and we proceedto obtain the expression of the optimal causal reproductiondistribution. 1) We prove under certain conditions involvingthe nonanticipative information RDF, and the capacity ofcertain channels with memory and feedback, that symbol-by-symbol code with memory without anticipation is achievable.2) we consider a BSMS(p) and we show that matching ispossible over a symmetric channel with memory and costconstraint, 3) we evaluate the excess distortion probability andwe show that convergence to zero, as the number of channeluses increases, establishing achievability.II. S YMBOL - BY -S YMBOL CODES WITH M EMORY W ITHOUT A NTICIPATION
Let N △ = { , , . . . } , N n △ = { , , . . . , n } . The spaces X , A , B , Y denote the source output, channel input, channeloutput, and decoder output alphabets, respectively, which EncoderSource Channel DecoderOptimalReproduction Distribution , ,...
Y Y , ,...
B B , ,...
X X , ,...
A A | ii X X P | , i ii Y Y B P | , i ii Y Y X P | , , i i ii A A X B P | , , i i ii B B A X P ( , ) ( 1) ! " , n i n i nii T x T y n d
Fig. 1. Communication scheme with feedback. are assumed to be complete separable metric spaces (Polishspaces) to avoid excluding continuous alphabets. We definetheir product spaces by X ,n △ = × ni =0 X , A ,n △ = × ni =0 A , B ,n △ = × ni =0 B , Y ,n △ = × ni =0 Y , and associate them with theirmeasurable spaces. Let x n △ = { x , x , . . . , x n } ∈ X ,n denotethe source sequence of length n + 1 , and similarly for the restof the blocks. Next, we introduce the various distributions.. Definition II.1. (Source) The source is a sequence of condi-tional distributions defined by P X n ( dx n ) △ = ⊗ ni =0 P X i | X i − ( dx i | x i − ) . Definition II.2. (Encoder) The encoder is a sequence ofconditional distributions defined by −→ P A n | B n − ,X n ( da n | b n − , x n ) △ = ⊗ ni =0 P A i | A i − ,B i − ,X i ( da i | a i − , b i − , x i ) . Thus, the encoder is nonanticipative in the sense that ateach time i ∈ N n , P A i | A i − ,B i − ,X i ( da i | a i − , b i − , x i ) is ameasurable function of past and present symbols x i ∈ X ,i and past symbols a i − ∈ A ,i − , b i − ∈ B ,i − . Definition II.3. (Channel) The channel is a sequence ofconditional distributions defined by −→ P B n | A n ,X n ( db n | a n , x n ) △ = ⊗ ni =0 P B i | B i − ,A i ,X i ( db i | b i − , a i , x i ) . Thus the channel has memory, feedback and it is nonantic-ipative with respect to the source sequence.
Definition II.4. (Decoder) The decoder is a sequence ofconditional distributions defined by −→ P Y n | B n ( dy n | b n ) △ = ⊗ ni =0 P Y i | Y i − ,B i ( dy i | y i − , b i ) . Definitions II.1-II.4 are general, since they allow memoryand feedback without anticipation, hence we call the source-channel code symbol-by-symbol code with memory withoutnticipation. Given the source, encoder, channel, decoder, wecan define uniquely the joint measure by P X n ,A n ,B n ,Y n ( dx n , da n , db n , dy n )= ⊗ ni =0 P Y i | Y i − ,B i ( dy i | y i − , b i ) ⊗ P B i | B i − ,A i ,X i ( db i | b i − , a i , x i ) ⊗ P A i | A i − ,B i − ,X i ( da i | a i − , b i − , x i ) ⊗ P X i | X i − ( dx i | x i − ) . (1)The previous equation implies the Markov Chains (MCs): ( A i − , B i − , Y i − ) ↔ X i − ↔ X i , ∀ i ∈ N n (2) Y i − ↔ ( A i − , B i − , X i ) ↔ A i , ∀ i ∈ N n (3) Y i − ↔ ( A i , B i − , X i ) ↔ B i , ∀ i ∈ N n (4) ( A i , X i ) ↔ ( B i , Y i − ) ↔ Y i , ∀ i ∈ N n . (5)The distortion between the source and its reproduction isa measurable function d ,n : X ,n × Y ,n [0 , ∞ ) , and thecost of transmitting symbols over the channel is a measurablefunction c ,n : A ,n × Y ,n − [0 , ∞ ) defined by d ,n ( x n , y n ) △ = n X i =0 ρ ,i ( T i x n , T i y n ) c ,n ( a n , b n − ) △ = n X i =0 γ ,i ( a i , b i − ) , where ( T i x n , T i y n ) are the shift operations on ( x n , y n ) ,respectively. For a single letter distortion function we take ρ ,i ( T i x n , T i y n ) = ρ ( x i , y i ) . Next, we state the definition ofa symbol-by-symbol code (with memory without anticipation). Definition II.5. (Symbol-by-Symbol Code)An (n,d, ǫ ,P) symbol-by-symbol code for( X ,n , A ,n , B ,n , Y ,n , P X n , −→ P B n | A n ,X n , d ,n , c ,n )is a code { P A i | A i − ,B i − ,X i ( ·|· ) : ∀ i ∈ N n } , { P Y i | Y i − ,B i ( ·|· ) : ∀ i ∈ N n } with excess distortion probability P n d ,n ( x n , y n ) > ( n + 1) d o ≤ ǫ, ǫ ∈ (0 , , d ≥ , andtransmission cost n +1 E n c ,n ( A n , B n − ) o ≤ P, P ≥ . Definition II.6. (Minimum Excess Distortion) The minimumexcess distortion achievable by a symbol-by-symbol code ( n, d, ǫ, P ) is defined by D o ( n, ǫ, P ) △ = inf n d : ∃ ( n, d, ǫ, P ) symbol-by- symbol code o Our definition of symbol-by-symbol code is randomized,hence it embeds deterministic codes as a special case.III. N
ONANTICIPATIVE
RDFThe necessary conditions for transmitting a symbol-by-symbol code (they also hold for memoryless sources andchannels) is the following.1) Computation of the RDF and that of the optimal repro-duction distribution so that probabilistic matching of thesource and channel is feasible; 2) Realization of the optimal reproduction distribution oflossy compression with fidelity by an encoder-channel-decoder scheme, processing information causally.Therefore, to facilitate the matching we introduce the RDF.Given a source distribution P X n ( · ) and a reproduction distri-bution P Y n | X n ( ·| x n ) the average fidelity set is Q ,n ( D ) △ = n P Y n | X n :1 n + 1 Z d ,n ( x n , y n )( P Y n | X n ⊗ P X n )( dx n , dy n ) ≤ D o . It is known that for stationary ergodic sources, the OPTA isgiven by the RDF [3] R ( D ) = lim n →∞ R ,n ( D ) , R ,n ( D ) =inf P Y n | Xn ∈Q ,n ( D ) 1 n +1 I ( X n ; Y n ) , provided the infimum isachievable. However, R ( D ) is only known for IID and Gaus-sian sources, and in generally fails to satisfy 1), 2).Now, we introduce the nonanticipative information RDFwhich by construction is realizable. Given a source P X n ( dx n ) and a causal conditional distribution defined by −→ P Y n | X n ( dy n | x n ) △ = ⊗ ni =0 P Y i | Y i − ,X i ( dy i | y i − , x i ) (6)we introduce the information measure I P Xn ( X n → Y n ) △ = D ( −→ P Y n | X n ⊗ P X n || P Y n × P X n ) ≡ I X n → Y n ( P X n , −→ P Y n | X n ) . Consider the fidelity set defined by −→Q ,n ( D ) △ = n −→ P Y n | X n : 1 n + 1 Z X ,n ×Y ,n d ,n ( x n , y n ) −→ P Y n | X n ( dy n | x n ) ⊗ P X n ( dx n ) ≤ D o . (7) Definition III.1. (Nonanticipative Information RDF) Given −→Q ,n ( D ) , the nonanticipative information RDF is defined by R na ,n ( D ) △ = inf −→ P Y n | Xn ∈−→Q ,n ( D ) n + 1 I X n → Y n ( P X n , −→ P Y n | X n ) (8) and its rate by R na ( D ) = lim n →∞ R na ,n ( D ) provided infimumand the limit exist. Clearly, if the minimum of R na ,n ( D ) exists the optimalreproduction distribution is nonanticipative, and hence real-izable.It can be shown that R na ,n ( D ) is equal to the nonanticipatory ǫ − entropy introduced by Gorbunov and Pinsker in [4], via R ε ,n ( D ) = inf P Y n | Xn ∈Q ,n ( D ) X ni +1 ↔ X i ↔ Y i , i =0 , ,...,n − n + 1 I ( X n ; Y n ) (9)The MC in (9) implies that the reproduction distributionwhich minimizes (9) can be realized via an encoder-channel-decoder, using nonanticipative operations (causal).Under the conditions in [4], or assuming the solution of R na ,n ( D ) is stationary, which implies −→ P Y n | X n ( dy n | x n ) isa stationary conditional distribution, we have the followingtheorem [5]. heorem III.2. Suppose there exist an interior point of thefidelity set, and the optimal reproduction is stationary. Thenthe infimum over −→Q ,n ( D ) in (8) is attained by −→ P ∗ Y n | X n ( dy n | x n ) = ⊗ ni =0 e sρ ( T i x n ,T i y n ) P ∗ Y i | Y i − ( dy i | y i − ) R Y i e sρ ( T i x n ,T i y n ) P ∗ Y i | Y i − ( dy i | y i − ) (10) where s ≤ is the Lagrange multiplier associated with theconstraint which is satisfied with equality, and R na ,n ( D ) = sD − n + 1 n X i =0 Z X ,i ×Y ,i − log (cid:16) Z Y i e sρ ( T i x n ,T i y n ) P ∗ Y i | Y i − ( dy i | y i − ) (cid:17) ⊗ P X i | X i − ( dx i | x i − ) ⊗ P ∗ X i − ,Y i − ( dx i − , dy i − ) (11) where P ∗ X i − ,Y i − ( · , · ) = −→ P ∗ Y i − | X i − ( ·|· ) ⊗ P X i − ( · ) .Proof: The derivation is given in [6].Clearly, (10) is nonanticipative, and as we show in the nextsection, easy to compute, even for sources with memory.IV. C
ODING T HEOREM
In this section we show achievability of symbol-by-symbolcode. First, we define the probabilistic realization of optimalreproduction distribution.
Definition IV.1. (Realization) Given a source { P X i | X i − ( dx i | x i − ) : ∀ i ∈ N n } , a general channel { P B i | B i − ,A i ,X i ( db i | b i − , a i , x i ) : ∀ i ∈ N n } is a realization of the optimalreproduction distribution { P ∗ Y i | Y i − ,X i ( dy i | y i − , x i ) : ∀ i ∈ N n } of theorem III.2, if there exists a pre-channel encoder { P A i | A i − ,B i − ,X i ( da i | a i − , b i − , x i ) : ∀ i ∈ N n } and a post-channel decoder { P Y i | Y i − ,B i ( dy i | y i − , b i ) : ∀ i ∈ N n } suchthat −→ P ∗ Y n | X n ( dy n | x n ) = ⊗ ni =0 P ∗ Y i | Y i − ,X i ( dy i | y i − , x i )= ⊗ ni =0 P Y i | Y i − ,X i ( dy i | y i − , x i ) (12) where the joint distribution from which (12) is obtained isgiven precisely by (1). Moreover we say that R na ,n ( D ) isrealizable if in addition the realization operates with averagedistortion D and I P Xn ( P X n , −→ P Y n | X n ) = R na ,n ( D ) If the optimal reproduction distribution is realizable (seeDefinition IV.1), then the data processing inequality holds: I X n → Y n ( P X n , −→ P Y n | X n ) ≤ I ( X n → B n ) , ∀ n ∈ N . (13)If R na ,n ( D ) is realizable according to Definition IV.1, then thesource is not necessarily matched to the channel. Next, weprove (under certain conditions) achievability.Consider the following average cost set defined by P ,n ( P ) △ = n ( X n , A n ) : 1 n + 1 E { c ,n ( A n , B n − ) } ≤ P o . Since we consider the general scenario that (2)-(5) hold, thenwe define the information channel capacity as follows [7]. C ,n ( P ) △ = sup ( X n ,A n ) ∈P ,n ( P ) n + 1 I ( X n → B n ) and its rate (provided sup is finite and the limit exists) by C ( P ) = lim n →∞ C ,n ( P ) .Next, we prove achievability of a symbol-by-symbol code. Theorem IV.2. (Achievability of Symbol-by-Symbol Code).Suppose the following conditions hold. (1) R na ,n ( D ) has a solution, and the optimal reproductiondistribution is stationary of the form { P Y i | Y i − ,X i : ∀ i =0 , , . . . , n } ; (2) C ,n ( P ) has a solution, the maximizing processes arestationary, and the encoder is of the form { P A i | A i − ,X i : ∀ i =0 , , . . . , n } ; (3) The optimal reproduction distribution −→ P Y n | X n ( dy n | x n ) given by Theorem III.2 is realizable,and R na ,n ( D ) is also realizable. (4) For a given D there exists a P such that R na ( D ) = C ( P ) .If P n n X i =0 ρ ,i ( T i X n , T i Y n ) > ( n + 1) d o ≤ ǫ (14) where P is taken with respect to P Y n ,X n ( dy n , dx n ) = −→ P ∗ Y n | X n ( dy n | x n ) ⊗ P X n ( dx n ) induced by matching, thenthere exists an ( n, d, ǫ, P ) symbol-by-symbol code with mem-ory without anticipation.Proof: The derivation is similar to [1]. If conditions(1), (3) hold then the optimal reproduction distribution isrealizable, and this realization achieves R na ,n ( D ) . By (4) thesource is matched to the channel so that the excess distortionprobability of a symbol-by-symbol code with memory withoutanticipation satisfies (18). A. Existence of Symbol-by-Symbol Codes
Next, we give sufficient conditions so that the conditionsof Theorem IV.2, (1), (2) hold, i.e., establishing existenceof a symbol-by symbol encoder { P A i | A i − ,B i − ,X i : i =0 , , . . . , n } . Suppose the following conditions hold. (A1) ρ ,i ( T i x n , T i y n ) = ρ ,i ( x i , T i y n ) , ∀ i ∈ N n ; (A2) P X i | X i − ( x i | x i − ) = P X i | X i − ( x i | x i − ) , ∀ i ∈ N n ; (A3) P B i | B i − ,A i ,X i ( db i | b i − , a i , x i )= P B i | B i − ,A i ,X i ( db i | b i − , a i , x i ) , ∀ i ∈ N n .If (A1) holds, then by Theorem III.2 the optimal stationaryreproduction distribution is P ∗ Y i | Y i − ,X i = P ∗ Y i | Y i − ,X i , ∀ i ∈ N n , and hence the form of the optimal reproduction distribu-tion in Theorem IV.2, (1) holds. Moreover, if (A2), (A3) hold,then maximizing directed information I ( X n → B n ) overnon-Markov encoders { P A i | A i − ,B i − ,X i : i = 0 , , . . . , n } is equivalent to maximizing it over encoders { P A i | B i − ,X i : i = 0 , , . . . , n } , and similarly, maximizing I ( X n → B n ) over non-Markov deterministic encoders { e i ( x i , a i − , b i − ) : i = 1 , . . . , n } is equivalent to the maximization with respect toncoders { g i ( x i , b i − ) : i = 1 , . . . , n } . This result appeared in[8] and is calculated using dynamic programming. Hence, theform of the encoder in Theorem IV.2, (2) holds. Thus, based onthese two conditions the encoder is symbol-by-symbol Markovwith respect to the source, and nothing can be gained byconsidering an encoder that depends on the entire past of thesource causally.V. S YMBOL - BY -S YMBOL
JSCC
OF A B INARY S YMMETRIC M ARKOV SOURCE VIA A B INARY S TATE S YMMETRIC C HANNEL
In this section we provide a noisy coding theorem for aBinary Symmetric Markov Source with crossover probability p , BSM S ( p ) . This is achieved by symbol-by-symbol jointsource channel matching of the current source via a BinaryState Symmetric Channel BSSC ( α , β ) with an average costconstraint. First, we give the expression of the nonanticipativereproduction distribution which achieves the infimum in (8).Next, we give the capacity expression of the BSSC ( α , β ) and the optimal input distributions without feedback thatachieve it. For this channel feedback does not increase thecapacity. Then, by merging these results we show achievabilityof symbol-by-symbol code such that, R na ( D ) = C ( κ ) . A. Results on
BSM S ( p ) and BSSC ( α , β ) Consider a Binary Symmetric Markov Source.
BSM S ( p ) , P X i | X i − (0 |
0) = P X i | X i − (1 |
1) = 1 − p and P X i | X i − (1 |
0) = P X i | X i − (0 |
1) = p and i ∈ N n and Hamming distortioncriterion ρ ( x, y ) = 0 if x = y and ρ ( x, y ) = 1 if x = y . Theorem V.1.
For a BSMS(p) and single letter Hammingdistortion criterion R na ( D ) is given by R na ( D ) = (cid:26) H ( p ) − mH ( α ) − (1 − m ) H ( β ) if D ≤ otherwise m = 1 − p − D + 2 pD , α = (1 − p )(1 − D )1 − p − D +2 pD , β = p (1 − D ) p + D − pD .Proof: We describe the main steps. The steady statedistribution of the source is P ( X i = 0) = P ( X i = 1) = 0 . and the reproduction distribution is P ∗ Y i | X i ,Y i − = P ∗ Y i | X i ,Y i − = e sρ ( x i ,y i ) P ( y i | y i − ) P y i e sρ ( x i ,y i ) P ( y i | y i − ) and we can show that P ∗ Y i | X i ,Y i − = P ∗ Y i | X i ,Y i − and that P ∗ Y i | X i ,Y i − ( y i | x i , y i − ) = , , , , α β − β − α − α − β β α Using the stationary distributions P ∗ Y i | X i ,Y i − and P X i | X i − ,we obtain R na ( D ) .To perform the matching on the source to the channel we usethe Binary State Symmetric Channel BSSC ( α , β ) definedby P B i | A i ,B i − ( b i | a i , b i − )= (cid:20) , , , , α β − β − α − α − β β α (cid:21) . (15) The form of the channel 15 is motivated by the form of the P ∗ Y i | X i ,Y i − (as in the IID Bernoulli source is matched via abinary symmetric channel). The state of the channel is definedas the modulo2 addition of the current input and previousoutput symbol, s i = a i ⊕ b i − . Then we may transform thechannel to its equivalent form defined by P B i | A i ,S i ( b i | a i , s i ) .This channel is called binary state symmetric channel, sincegiven the state the channel it is binary symmetric. We intro-duce a cost constraint on the channel that has the followingphysical interpretation. Assume α > β ≥ . . Then thecapacity of the state zero channel (1 − H ( α )) , is greater thanthe capacity of the state one channel (1 − H ( β )) . With“abuse”of terminology, we interpret the ( BSC (1 − α )) as the “goodchannel” and the ( BSC (1 − β )) as the bad channel. It isfurther reasonable to assume that the we pay a larger fee touse the “good channel” and a smaller fee to use the “badchannel”. We quantify this policy by assigning a binary payoff to each of the channels. Hence, we assign a cost equalto for the good channel, and a cost equal to for the badchannel, defined by c ( a i , b i − ) △ = (cid:26) if a i = b i − , or s i = 00 if a i == b i − , or s i = 1 hence the average cost constraint is E { c ( a i , b i − ) } = P A i ,B i − (0 ,
0) + P A i ,B i − (1 ,
1) = P S i (0) . Note that c ( a i , b i − ) is not required to be binary and can beeasily upgraded to more complex forms. We know that for the BSSC ( α , β ) [9] feedback does not increase the capacity.The definition of the constrained capacity without feedback isdefined by C fb ( k ) = lim n →∞ max P Xn : P ni =0 1 n +1 E { P ni =0 c ,i ( x i ,y i − ) } = κ n + 1 I ( X n → Y n ) (16) Proposition V.2.
The capacity of the
BSSC ( α , β ) , withor without feedback, subject to the average cost constrain E { c ( a i , b i − ) } = k , where κ = constant , given by C ( κ ) = H ( α κ +(1 − β )(1 − κ )) − κH ( α ) − (1 − κ ) H ( β ) (17) The optimal input distribution without feedback is given by P ∗ A i | A i − ( a i | a i − ) = − κ − γ − γ κ − γ − γκ − γ − γ − κ − γ − γ , where γ = α κ + β (1 − κ ) .Proof: see [10]. B. Symbol-By-Symbol Joint Source Channel Matching
Recall that symbol-by-symbol joint source channel match-ing is achievable if R na ( D ) = C ( κ ) and if there exists anencoder decoder scheme for d ≥ D , such that P n n X i =0 ρ ,i ( T i X n , T i Y n ) > ( n + 1) d o ≤ ǫ (18)
500 1000 1500 200000.050.10.150.20.250.30.350.40.450.5 p=0.39, D=0.18n D Fig. 2. The distortion between the source and reproduction symbols fora random realization of the source, as a function of n using the optimalreproduction distribution as the channel and uncoded transmission. By setting κ = m , α = α , β = β , then − κ − γ − γ = p , C ( κ ) = H ( β (1 - κ ) + (1 - α ) κ ) - κH ( α ) - (1 - κ ) H ( β )= H ( β (1 - m ) + (1 - α ) m ) - mH ( α ) - (1 - m ) H ( β )= H ( p ) - mH ( α ) - (1 - m ) H ( β ) = R na ( D ) Moreover, the optimal input distribution is given by P ∗ A i | A i − ( a i | a i − ) = (cid:20) p − p − p p (cid:21) , (19)Since the optimal input distribution is identical to theprobability distribution of the source, then no encoder isrequired. Next, we check whether the average distortion issatisfied in the absence of a decoder. The average distortionbetween the source symbols and the reproduction symbols, ∆ ,is equal to ∆ = E [ d ( X i , Y i )]= E [ d ( A i , B i )]= X A i ,B i ,B i − d ( A i , B i ) P B i | A i ,B i − ( b i | a i , b i − ) P A i | B i − ( a i | b i − ) P B i − ( b i − )= (1 − β )(1 − m ) + (1 − α ) m = D Thus, we established source channel matching of a
BSM S ( p ) with Hamming fidelity constraint over a BSSC ( α , β ) subject to cost constraint, in the spirit of [1].A realization of the described scheme is illustrated in Fig. 2,where it is shown that as the number of channel uses n is increased, the single letter distortion between the sourcesymbol sequence and the reproduction sequence converges tothe average distortion D .Next, we bound the excess distortion probability of TheoremIV.2, by applying an extension of Hoeffding’s inequality forMCs [11], to the Markov process { Z i △ = ( Y i , X i ) : ∀ i ∈ N } (this is easily shown to hold). Set ρ ( x, y ) = x ⊕ y and let S n △ = P ni =0 ρ ( X i , Y i ) . Let d △ = δ + E [ S n ] n +1 , δ > . By Hoeffding’s P e p=0.39 D=0.18D=0.12D=0.25 Fig. 3. Excess Probability of Distortion for δ = 0 . . inequality [11], the excess distortion probability is boundedby P n S n > ( n + 1) d o ≤ exp (cid:16) − λ (( n + 1) δ − k f k m/λ ) n + 1) k f k m (cid:17) where k f k △ = sup { y i : i = 0 , , . . . } = 1 , m = 1 , λ =min { p, − p } min { α, β, − α, − β } , for n > k f k m/ ( λδ ) .This bound is illustrated in Fig. 3. Although, this bound is nottight and holds for n large enough, it shows the achievability ofMarkov sources via uncoded transmission. It might be possibleto compute the excess distortion probability in closed form toget tighter bounds. VI. C ONCLUSIONS
This paper discusses General Source-Channel Matching forsymbol-by-symbol. Using the nonanticipative RDF it is showsachievability of a symbol-by-symbol code with respect toaverage and excess distortion probability. Then it considers the
BSM S ( p ) , it computes the nonanticipative RDF with respectto Hamming distortion, and shows that is is matched, uncoded,over a BSSC ( α , β ) subject to cost constraint but withoutfeedback. R EFERENCES[1] M. Gastpar, B. Rimoldi, and M. Vetterli, “To code, or not to code:lossy source-channel communication revisited,”
IEEE Transactions onInformation Theory, , vol. 49, no. 5, pp. 1147–1158, May 2003.[2] V. Kostina and S. Verdu, “Fixed-length lossy compression in thefinite blocklength regime: Discrete memoryless sources,” in , 2011, pp. 41–45.[3] T. Berger,
Rate Distortion Theory: A Mathematical Basis for DataCompression . Englewood Cliffs, NJ: Prentice-Hall, 1971.[4] A. K. Gorbunov and M. S. Pinsker, “Nonanticipatory and prognosticepsilon entropies and message generation rates,”
Probl. Peredachi Inf. ,vol. 9, no. 3, pp. 12–21, 1973, (English version).[5] P. Stavrou and C. Charalambous, “Variational equalities of directed in-formation and applications,” in
Information Theory Proceedings (ISIT),2013 IEEE International Symposium on , 2013, pp. 2577–2581.[6] C. D. Charalambous, P. A. Stavrou, and N. U. Ahmed, “Nonanticipativerate distortion function and relations to filtering theory,” accepted toIEEE Transactions on Automatic Control , 2013. [Online]. Available:http://arxiv.org/abs/1210.1266v2[7] T. M. Cover and S. Pombra, “Gaussian feedback capacity,”
IEEETransactions on Information Theory , vol. 35, no. 1, pp. 37–43, 1989.[8] C. Charalambous, C. Kourtellaris, and C. Hadjicostis, “Optimal encoderand control strategies in stochastic control subject to rate constraints forchannels with memory and feedback,” in
CDC-ECC, 2011 50th IEEEConference on , dec. 2011, pp. 4522 –4527.9] H. Asnani, H. Permuter, and T. Weissman, “Capacity of a post channelwith and without feedback,” in
Information Theory Proceedings (ISIT),2013 IEEE International Symposium on , 2013, pp. 2538–2542.[10] C. Kourtellaris and C. D. Charalambous, “Capacity of the binary statesymmetric channel with cost constraint,” arxiv.org , 2014. [Online].Available: arxiv.org[11] P. Glyn, W. and D. Ormoneit, “Hoeffding’s inequality for uniformergosic markov chains,”