Joint Source-Channel Coding for Semantics-Aware Grant-Free Radio Access in IoT Fog Networks
Johannes Dommel, Zoran Utkovski, Osvaldo Simeone, Slawomir Stanczak
11 Joint Source-Channel Coding for Semantics-AwareGrant-Free Radio Access in IoT Fog Networks
Johannes Dommel, Zoran Utkovski, Osvaldo Simeone and Sławomir Sta´nczak
Abstract —A fog-radio access network (F-RAN) architecture isstudied for an Internet-of-Things (IoT) system in which wirelesssensors monitor a number of multi-valued events and transmitin the uplink using grant-free random access to multiple edgenodes (ENs). Each EN is connected to a central processor (CP)via a finite-capacity fronthaul link. In contrast to conventionalinformation-agnostic protocols based on separate source-channel(SSC) coding, where each device uses a separate codebook, thispaper considers an information-centric approach based on jointsource-channel (JSC) coding via a non-orthogonal generalizationof type-based multiple access (TBMA). By leveraging the seman-tics of the observed signals, all sensors measuring the same eventshare the same codebook (with non-orthogonal codewords), andall such sensors making the same local estimate of the eventtransmit the same codeword. The F-RAN architecture directlydetects the events’ values without first performing individualdecoding for each device. Cloud and edge detection schemesbased on Bayesian message passing are designed and trade-offsbetween cloud and edge processing are assessed.
Index Terms —approximate message passing, fog-radio accessnetwork, random access, type-based multiple access, semanticcommunications.
I. I
NTRODUCTION D UE to the growing interest in Internet-of-Things (IoT)applications, there has been an intense research efforton massive machine-type communications (mMTC) for 5Gnetworks and beyond [1]–[3]. In these networks, standardmedium access control protocols that recover the individualmessages of participating devices require spectral resourcesthat scale at least linearly with the number of active users[4]–[7]. This paper proposes the integration of two distinctmechanisms that aim at reducing the communication overhead,namely ( i ) the use of cloud and edge processing in fog-radioaccess networks (F-RANs) [8]; and ( ii ) the application ofsemantics-aware medium access protocols that are designedto recover the aggregated information of interest rather thanthe individual messages (see, e.g., [9], [10]). To elaborate, weconsider a multi-cell F-RAN architecture [8], as illustratedin Fig. 1, where IoT devices are connected to edge nodes(ENs) in a cell-free fashion. Each EN is connected via afinite-capacity fronthaul link to a central processor (CP). Inthe system under study, multiple IoT sensor devices measurecorrelated events and transmit messages in a grant-free fashionvia wireless channels to the ENs. The events may be inactive, J. Dommel (e-mail: [email protected]), Z. Utkovski andS. Sta´nczak are with the Department of Wireless Communications andNetworks, Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany. S. Sta´nczakis with the Department of Telecommunication Systems, Technical Universityof Berlin, Berlin, Germany. O. Simeone is with King’s Communications,Learning & Information Processing (KCLIP) lab, CTR, Dept. of Engineering,King’s College London.
CPEN EN c EN LB B c B L K l = { k : M k l } h Li ∈K l h i ∈K l h ci ∈K l ξ = 0 ξ M = 0 ξ l > P ξ ( ξ , . . . , ξ M ) K M K Transmitting NodeIdle Node Estimator ˆ ξ = [ˆ ξ . . . ˆ ξ M ] Fig. 1. A wireless fog-radio access network (F-RAN) for event driven randomaccess: a set of sensors K monitors jointly M independent events, with eachevent m being either inactive ( ξ m = 0 ) or active with an associated scalarstate value ξ m ∈ { , ..., R } . Each sensor k ∈ K m ⊂ K measuring an activeevent m ∈ M k transmits a message to L edge nodes (ENs) over a wirelessfading channel. The ENs are connected via capacity-limited fronthaul linksof capacity B , . . . , B L to a central processor (CP) for joint decoding andestimation of the events values ˆ ξ = [ˆ ξ . . . ˆ ξ M ] . and functions of multiple IoT sensors’ measurements, ratherthan individual measurements, are of interest to the receiver.In information-theoretic terms, the problem is thus not oneof channel coding in a multiple-access channel (MAC) forreliable communication of individual messages, but rather thatof joint source-channel (JSC) coding for effective inference ofcorrelated quantities of interest (QoIs). Related work:
A notable instance of information-centric MACprotocols is type-based multiple access (TBMA) [11]. WithTBMA, each measurement value for a given QoI is assignedan orthogonal codeword, and the receiver infers the desiredQoI from a histogram at the outputs of a filter-bank matched tothe codewords [11]–[13]. A potentially more efficient solutionbased on a non-orthogonal generalization of TBMA has beenproposed in [14]. Accordingly, all sensors measuring the sameevent share the same codebook with non-orthogonal code-words, and the base station directly detects the events’ valuesusing a Bayesian message passing technique. Recently, TBMAhas been extended to multi-cell F-RANs for IoT applicationsunder centralized or decentralized decoding in [15].
Contribution:
In this paper, we study the integration of clouddetection in F-RAN with grant-free transmission based onthe semantics-aware non-orthogonal TBMA protocol [14]. Inthe proposed approach, detection is performed in a central-ized fashion in the cloud based on either detect-and-forward(DtF) [16] or quantize-and-forward (QF) utilizing capacity a r X i v : . [ c s . I T ] J a n limited fronthaul links. We design DtF and QF schemes basedon Bayesian message passing by leveraging the hybrid general-ized approximate message passing (H-GAMP) algorithm [17].Finally, we numerically evaluate the relative performance ofthe proposed DtF and QF schemes under capacity-constrainedfronthaul.II. E VENT -B ASED R ANDOM A CCESS FOR F OG -I O T:S
YSTEM M ODEL AND C ODING S CHEME
Scenario:
We consider an F-RAN IoT architecture consistingof a set L of L ENs, each connected to a CP unit via acapacity constrained fronthaul link, as illustrated in Fig. 1.In this scenario, a set K of K devices jointly monitor a set M of M multi-valued events. Each event m is characterizedby an independent scalar random variable ξ m ∈ { , , . . . , R } ,with P ξ ( ξ m = 0) = 1 − ρ for some ≤ ρ ≤ representingthe probability that event m is inactive. When the event isactive, the event variable ξ m takes one of the values in theset { , . . . , R } , so that parameter R (or, more properly, itslogarithm) measures the amount of information attached tothe occurrence of an event. Each device k can simultaneouslymonitor a subset of events M k ⊆ { , ..., M } . Therefore,the devices can be partitioned into M , generally overlapping,groups K m = { k ∈ { , ..., K } : M k (cid:51) m } . Coding scheme:
Each device k performs a local (real-valued) measurement u k , which is, in general, correlatedwith all the variables ξ m for m ∈ M k . For each event m ∈ M k , the local measurement u k is mapped to avalue φ m ( u k ) ∈ { , , . . . , R } , which is the local estimate of event m . For transmission, each local estimate φ m ( u k ) is mapped by device k into a codeword s mφ m ( u k ) ∈ C N × ,subject to a power constraint (cid:107) s mφ m ( u k ) (cid:107) ≤ . The code-words for each event m are selected from a shared code-book S m = [ s m . . . s mR ] ∈ C N × ( R +1) of R + 1 generally non-orthogonal codewords (columns). For future reference, wedefine S = [ S . . . S M ] ∈ C N × M ( R +1) to be a matrix thatcollects all codebooks. Channel model:
We assume time synchronization and trans-mission over a block-fading channel model with coherencetime–frequency span no smaller than that occupied by thecodewords’ duration. The signal received at EN c can bewritten as y c = (cid:88) k ∈K h ck (cid:88) m ∈M k s mφ m ( u k ) + v c , (1)where h ck denote the fading coefficient for the link betweendevice k and EN c , which is assumed to be identical andindependent distributed (i.i.d.) ∼ CN (0 , σ h ) , and v c ∈ C N × the additive noise vector with elements i.i.d. ∼ CN (0 , σ v ) .To obtain a matrix notation, we define for each device k thebinary measurement vector c k = [( c k ) T . . . ( c Mk ) T ] T ∈ { , } M ( R +1) × , (2)with c mk = (cid:40) e φ m ( u k ) if m ∈ M k e otherwise , (3) where e r is an R + 1 -dimensional binary vector with a singlenon-zero-entry at the ( r + 1) -th position. With this definition, the received signal (1) at EN c can bedescribed in matrix-notation as Sx c + v c with x c = (cid:0) h c ⊗ I M ( R +1) (cid:1) T c , (4)where h c = [ h c . . . h cK ] T is the vector of channel coefficients; ⊗ the Kronnecker product; I M ( R +1) the identity matrix ofsize M ( R + 1) and c = [ c T . . . c TK ] T the stacked vector ofmeasurements. Note, that x c is a (sparse) Bernoulli-Gaussianvector, where each non-zero element constitutes the superpo-sition of complex-normal fading coefficients. Fronhthaul constraint:
We assume a packetized fronthaultransmission, e.g., via Ethernet, by considering a limitedoverall number of bits B c that each EN c can communicateerror-free to the CP per fronthaul use . Error probability:
The CP aims at estimating the state ofeach event ˆ ξ = [ ˆ ξ . . . ˆ ξ M ] , where the average (per event) errorprobability is defined as P e . = 1 M (cid:88) m ∈M Pr (cid:110) ξ m (cid:54) = ˆ ξ m (cid:111) . (5)We note that the outlined MAC protocol can be considered asa generalization of TBMA [11], given that the latter assumesa single event, i.e. M . = 1 , and the use of R + 1 orthogonal codewords of length N ≥ R + 1 .III. F-RAN P ROCESSING WITH L IMITED F RONTHAUL C APACITY
In this section, we introduce a Bayesian decoder basedon generalized approximate message passing (GAMP). Theproposed approach extends the decoder introduced in [14]from single EN-detection to the F-RAN architecture discussedin Section II. We derive a graphical model and developtwo fronthaul processing schemes: ( i ) DtF, whereby eachEN produces local estimates and forwards quantized soft-information to the CP; and ( ii ) QF, whereby each EN directlyforwards a quantized version of the received signal to the CP. Graphical Model:
The relation between the involved random variables, i.e. the (input) x = [( x ) T . . . ( x L ) T ] T , the (output)observations y = [( y ) T . . . ( y L ) T ] T and the (hidden) vari-ables ξ can be described at the CP via a graphical model,where the input x depends on ξ ∼ P ξ via the mapping (2)-(3),which we denote as p x c | ξ . The output y is generated subjectto the conditional probability distribution function (pdf) p y | z capturing the effect of the additive white Gaussian noise,where z = [( z ) T . . . ( z L ) T ] T is the output of a (dense) linearmixing A x with A = I L ⊗ S . According to our transmissionscheme, the i -th element of z c is defined as z ci = ( a ci ) T x c with a ci being the i -th row of A c = S , which is the c -th block of A . In the following, we adopt the graphical model for QF andDtF considering a limited fronthaul capacity. c k can be interpreted as a one-hot encoding of all estimates at device k . A. Quantize-and-Forward
With QF, each EN c ∈ L forwards a quantized versionof the received symbols ˜ y c = Q c ( y c ) via the fronthaul link,and the CP uses ˜ y = [( ˜ y ) T . . . ( ˜ y L ) T ] T to carry out jointdecoding. The impact of the fronthaul quantization can bemodeled as Gaussian test channel [18], such that (1) receivedvia the c -th fronthaul link from EN c at the CP can be writtenas ˜ y c = y c + q c where q c ∈ C N × represents the quantizationnoise vector with elements being i.i.d. CN (0 , σ q c ) [19]. Fol-lowing rate-distortion arguments [20], σ q c is upper boundedas P Cc − , with P being the signal power and C c = B c /N thefronthaul rate in bit per complex sample . ξ P ξ ( ξ ) ... ξ M P ξ ( ξ M ) p x | ξ ( x | ξ ) x x p x | ξ ( x | ξ ) x p x | ξ ( x | ξ ) ... x L p x L | ξ ( x L | ξ ) ˜ y p ˜ y | z (˜ y | z ) z ˜ y p ˜ y | z (˜ y | z ) z ˜ y p ˜ y | z (˜ y | z ) z ...˜ y L p ˜ y | z (˜ y L | z L ) z L A Fig. 2. Factor graph representation of the fog-radio access network (F-RAN)system under study with quantize-and-forward (QF).
By the factor graph representation, see Fig. 2, the joint pdf p ξ , x , ˜ y of the triple ( ξ , x , ˜ y ) factorizes as M (cid:89) m =1 P ξ ( ξ m ) LM ( R +1) (cid:89) j =1 p x | ξ ( x j | ξ m ) LN (cid:89) i =1 p ˜ y | z (˜ y i | z i ) , (6)where the conditional pdf p ˜ y | z captures the effect of thereceiver- and quantization noise.Given the factorization (6), the detector at the CP aims atcomputing the posterior distribution p ξ | ˜ y ( ξ | ˜ y ) of the events’state vector ξ given the quantized observation vector ˜ y . WithQF, the relation between the structured sparsity introducedby the hidden variables ξ on the input variables x , can beexploited using the H-GAMP algorithm [17], which providesan efficient solution with desirable empirical performance forgroup sparsity problems with overlapping groups. H-GAMPoperates by iteratively exchanging soft information betweentwo modules: the first carries out standard GAMP by treatingthe entries of the vector x as independent, while the secondrefines the output of the first by leveraging the correlationstructure of the entries of vector x . With the posterior dis-tribution, the CP calculates for each event the log-likelihoodratios (LLRs) l m,r = ln (cid:18) p ξ | ˜ y ( ξ m = r | ˜ y ) p ξ | ˜ y ( ξ m = 0 | ˜ y ) (cid:19) , (7) associated with the belief that event m is active with value r ∈ [ R ] . The estimator then selects the decisions for ˆ ξ m as ˆ ξ m = if l m,r < l thm,r , ∀ r ∈ [ R ]arg max r ∈ [ R ] l m,r otherwise . (8)The thresholds l thm,r can be selected, e.g., to minimize theBayesian risk [21] that included individual costs for false-alarm (false positive) and missed-detection (false negative). B. Detect-and-Forward
With DtF, each EN c ∈ L performs local detection andforwards quantized soft-information to the CP. The CP thenfuses the local decisions to obtain the final estimates for eachevent. For detection at each EN c , the joint pdf p ξ , x c , y c ( · , · , · ) of the triple ( ξ , x c , y c ) factorizes as M (cid:89) m =1 P ξ ( ξ m ) M ( R +1) (cid:89) j =1 p x c | ξ ( x cj | ξ m ) N (cid:89) i =1 p y c | z ( y ci | z ci ) . (9)Using (9), the local detector at EN c aims at computing theposterior distribution p ξ | y c ( ξ | y c ) of the events’ state vector ξ given the local observation y c . This can be done by applyingthe standard GAMP operating on A c = S .Given the posterior distribution, each EN c ∈ L computes thelocal LLRs for all r ∈ [ R ] and m ∈ M as l cm,r = ln (cid:18) p ξ | y c ( ξ m = r | y c ) p ξ | y c ( ξ m = 0 | y c ) (cid:19) , (10)associated with the (local) belief at EN c , that the eventvariable ξ m is active with value r . For transmission over thecapacity-constraint fronthaul, each EN c applies a quantizationfunction ˜ x = U c ( x ) , to the LLRs (10) according to thefronthaul bit-budget of B c /M bit per event for all m ∈ M .At the CP, the beliefs are reconstructed and merged to obtain ˜ l m,r = (cid:88) c ∈L ˜ l cm,r , r ∈ [ R ] , (11)which is associated with the (global) belief that the eventvariable ξ m is active with value r ∈ [ R ] . Finally, the CPestimates the event activity variable by comparing (11) againstthresholds, c.f. (8). We note that in the case of DtF, an optimalcompression performance can be achieved by using an entropyquantizer operating on the M R -dimensional vector of LLRs.IV. N
UMERICAL R ESULTS
We assume a dense network with K = 80 devices observingin total M = 8 events with R = 4 values in an F-RANdeployment with L = 4 ENs and fronthaul bit budget B c = B ,for all fronthaul links between the ENs c ∈ L and theCP. Each event has an activation probability ρ = 0 . andthe devices are configured such that each individual deviceobserves only one of the events and that the total numberof devices is partitioned into M non-overlapping sets {K m } ,each of cardinality . The variance of the channel coefficients,which we recall are unknown to the transmitter devices andthe receiver, is set to σ h = 1 . To increase the energy efficiency, each device k is configured for transmission, only if the locallyobserved event m ∈ M k is active , i.e., if φ m ( u k ) > . Thesignatures of the shared codebook S of length N are generatedrandomly with entries being i.i.d. ∼ CN (0 , /N ) . We note thatthe convergence of approximate message passing (AMP) forthis codebook has been studied rigorously in the asymptoticlimit [17]. The average signal-to-noise ratio (SNR) is defined per user as SNR . = 1 /σ v . For DtF, only the LLRs associatedwith the non-zero entries of the estimated event activity patternare quantized and forwarded to the CP. The threshold forDtF and QF is chosen to minimize the error probability P e ,according to (5).The impact of fronthaul quantization on the error rate P e isplotted in Fig. 3 as a function of the SNR for both fronthaulprocessing schemes under different fronthaul bit budgets. DtFis seen to outperform QF in the regime of high SNR, withcrossing point occurring at lower SNR levels for a smallfronthaul budget. This is because, with a sufficiently large SNRand small enough fronthaul capacity, the potential advantagesof centralized detection at the CP are offset by the fronthaulquantization noise, and local detection is preferable. − − − − − P e DtF: B =
32 bitDtF: B =
64 bitQF: B =
32 bitQF: B =
64 bitQF: B → ∞ Fig. 3. Error rate versus SNR for F-RAN deployment with L = 4 ENs,signature length N = 16 , and limited fronthaul capacity B . The comparison depends also on the length N of the signature.To elaborate on this point, Fig. 4 plots the SNR required tomeet a predefined reliability target P e ≤ − as a functionof the fronthaul bit budget B and the signature length N . Asdiscussed, QF is preferable only at sufficiently large fronthaulcapacity levels, and the required fronthaul capacity increaseswith the signature length N . In fact, for QF, in the presence ofstringent fronthaul constraints, it is beneficial to trade signaturelength for quantization precision.Finally, in Fig. 5 we analyze the trade-off between falsepositive rate, P F P . = M (cid:80) Mm =1 Pr { ˆ ξ m (cid:54) = 0 | ξ m = 0 } , andfalse negative rate, P F N . = M (cid:80) Mm =1 Pr { ˆ ξ m = 0 | ξ m (cid:54) = 0 } ,obtained by varying the decision threshold. In line with thediscussion so far, QF is seen to provide significant advantageswhen the fronthaul capacity B is sufficiently large. In contrast,the fronthaul requirement of DtF are more modest, but theperformance of DtF is constrained by the limitations of localdetection.
32 48 64 80 B [ bit ] S N R [ dB ] DtF: N = 16QF: N = 16DtF: N = 32QF: N = 32QF: N = B → ∞ QF: N = B → ∞ Fig. 4. Required SNR to achieve a target reliability P e ≤ − as a functionof the fronthaul link budget B and signature length N . − − − False Negative Rate ( P FN ) − − − F a l s e P o s i t i v e R a t e ( P F P ) QFDtF B =
48 bit B =
800 bit
Fig. 5. Trade-off between false positive and false negative rates for differentdecision thresholds for N = 32 and SNR = 0 dB. V. C
ONCLUSIONS
This paper has introduced a semantics-aware protocol forevent-driven grant-free access in IoT F-RANs with fronthaulcapacity constraints. The protocol adopts a joint source-channel coding scheme, based on a non-orthogonal generaliza-tion of TBMA, that directly detects the quantities of interestand yields spectral requirements that scale with the number ofevents to be monitored rather than with the number of devices.The power and spectral requirements are further improvedthrough integration with cloud- and edge detection based onBayesian message passing. We have evaluated numerically therelative performances of edge-cloud processing based on DtFand QF, and assessed the trade-offs between the codewordlength, the fronthaul capacity and the required SNR, given apredefined reliability target. A general observation is that DtFoutperforms QF in the presence of stringent fronthaul con-straints, with the effect being more pronounced for higher SNRvalues. In this operational regime, the potential advantagesof centralized detection at the CP are offset by the fronthaulquantization noise, and local detection is preferable. Further,in line with [14] we conclude that the proposed schemeoffers a significantly higher spectral efficiency as comparedto conventional TBMA by exploiting ( i ) sparse activation and( ii ) structural dependencies between the variables for L > . R EFERENCES[1] N. H. Mahmood, H. Alves, O. A. López, M. Shehab, D. P. M. Osorio,and M. Latva-Aho, “Six Key Features of Machine Type Communicationin 6G,” in , 2020, pp. 1–5.[2] K. David and H. Berndt, “6G Vision and Requirements: Is There AnyNeed for Beyond 5G?,”
IEEE Veh. Technol. Mag. , vol. 13, no. 3, pp.72–80, July 2018.[3] X. Chen, D. W. K. Ng, W. Yu, E. G. Larsson, N. Al-Dhahir, andR. Schober, “Massive Access for 5G and Beyond,” arXiv:2002.03491 ,2020.[4] N. Abramson, “The ALOHA System: Another Alternative for ComputerCommunications,” in
ACM Fall Joint Computer Conference (AFIPS) ,Nov. 1970, p. 281–285.[5] G. Liva, “Graph-Based Analysis and Optimization of ContentionResolution Diversity Slotted ALOHA,”
IEEE Trans. Commun. , vol. 59,no. 2, pp. 477–487, Feb. 2011.[6] E. Paolini, C. Stefanovic, G. Liva, and P. Popovski, “Coded Random Ac-cess: Applying Codes on Graphs to Design Random Access Protocols,”
IEEE Commun. Mag. , vol. 53, no. 6, pp. 144–150, June 2015.[7] X. Chen, T. Chen, and D. Guo, “Capacity of Gaussian Many-AccessChannels,”
IEEE Trans. Inf. Theory , vol. 63, no. 6, pp. 3516–3539, Feb.2017.[8] S. Park, O. Simeone, and S. Shamai Shitz, “Joint Optimization of Cloudand Edge Processing for Fog Radio Access Networks,”
IEEE Trans.Wireless Commun. , vol. 15, no. 11, pp. 7621–7632, 2016.[9] P. Popovski, O. Simeone, F. Boccardi, D. Gündüz, and O. Sahin,“Semantic-Effectiveness Filtering and Control for Post-5G WirelessConnectivity,”
Journal of the Indian Institute of Science , vol. 100, no.2, pp. 435–443, 2020.[10] K. Marios and N. Pappas, “Semantics-Empowered Communication forNetworked Intelligent Systems,” arXiv:2007.11579 , 2020.[11] G. Mergen and L. Tong, “Type Based Estimation Over MultiaccessChannels,”
IEEE Trans. Signal Process. , vol. 54, no. 2, pp. 613–626,Feb. 2006. [12] K. Liu and A. M. Sayeed, “Asymptotically Optimal DecentralizedType-based Detection in Wireless Sensor Networks,” in , May 2004, pp. iii–873.[13] A. Anandkumar and L. Tong, “Type-Based Random Access forDistributed Detection Over Multiaccess Fading Channels,”
IEEE Trans.Signal Process. , vol. 55, no. 10, pp. 5032–5043, Oct. 2007.[14] J. Dommel, Z. Utkovski, S. Sta´nczak, and O. Simeone, “Joint Source-Channel Coding and Bayesian Message Passing Detection for Grant-FreeRadio Access in IoT,” in
IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP) , May 2020, pp. 8574–8578.[15] R. Kassab, O. Simeone, and P. Popovski, “Information-Centric Grant-Free Access for IoT Fog Networks: Edge vs Cloud Detection andLearning,”
IEEE Trans. Wireless Commun. , vol. 19, no. 10, pp. 6347–6361, Oct. 2020.[16] Z. Utkovski, O. Simeone, T. Dimitrova, and P. Popovski, “RandomAccess in C-RAN for User Activity Detection with Limited-CapacityFronthaul,”
IEEE Signal Process. Lett. , vol. 24, no. 1, pp. 17–21, Jan.2017.[17] S. Rangan, A. K. Fletcher, V. K. Goyal, E. Byrne, and P. Schniter,“Hybrid Approximate Message Passing,”
IEEE Trans. Signal Process. ,vol. 65, no. 17, pp. 4577–4592, Sept. 2017.[18] A. E. Gamal and Y.-H. Kim,
Network Information Theory , CambridgeUniversity Press, USA, 2012.[19] S. Park, O. Simeone, O. Sahin, and S. Shamai Shitz, “FronthaulCompression for Cloud Radio Access Networks: Signal ProcessingAdvances Inspired by Network Information Theory,”
IEEE SignalProcess. Mag. , vol. 31, no. 6, pp. 69–79, Nov. 2014.[20] T. M. Cover and J. A. Thomas,
Elements of Information Theory(Wiley Series in Telecommunications and Signal Processing) , Wiley-Interscience, USA, 2006.[21] S. M. Kay,