Mutual Information Approximation
aa r X i v : . [ c s . I T ] J un Mutual Information Approximation
Chongjun Ouyang,
Student Member, IEEE , Sheng Wu, and Hongwen Yang,
Member, IEEE
Abstract
To provide an efficient approach to characterize the input-output mutual information (MI) under additive whiteGaussian noise (AWGN) channel, this short report fits the curves of exact MI under multilevel quadrature amplitudemodulation ( M -QAM) signal inputs via multi-exponential decay curve fitting (M-EDCF). Even though the definitionexpression for instanious MI versus Signal to Noise Ratio (SNR) is complex and the containing integral is intractable,our new developed fitting formula holds a neat and compact form, which possesses high precision as well as lowcomplexity. Generally speaking, this approximation formula of MI can promote the research of performance analysisin practical communication system under discrete inputs. Index Terms
Mutual information, M -QAM, M-EDCF I. I
NTRODUCTION
Mutual information (MI) represents the limit achievable transmission rate in communication system. In 1948,Shannon proposed the famous Shannon formula log (1 + γ ) to evaluate the transmission limitation under Gaussianinputs. Since then, researches on analysis of mutual information or channel capacity have become a hot topic inthe field of telecommunication. Later, the concepts of secrecy capacity [1] and efficient capacity [2] were proposedin and , which further broaden the path of performance analysis and correlated optimization. Besides the mutualinformation analysis in additive white Gaussian noise (AWGN) channel, many theoretical results about the ergodicmutual information in wireless channel were also widely reported in the literature [3]–[5]. Nevertheless, it shouldbe noticed that the inputs follow non-Gaussian distribution due to the limitation of digital modulation system. Morespecifically, the practical implementation of the communication system is based on finite alphabet inputs, and allthe symbols are drawn form discrete constellation. Under this situation, it is impossible to adopt Shannon formulato employ the performance analysis.In 2007, Yang et al. in [6] formulated a series expression for the ergodic mutual information under BPSKmodulation mode over Nakagami- m fading. Although the contribution in [6] is seemingly fascinating, its constraintby limiting m as an integer indeed makes it lack generality. Besides, the proposed formula is hard computable.More serious, existing literature failed to offer an analytical framework to analyze the mutual information underdiscrete inputs. In 2007, the authors in [7] and [8] developed some approximation formulas for the instantaneous Supported by BUPT Excellent Ph.D. Students FoundationC. Ouyang, S. Wu, and H. Yang are with the School of Information and Communication Engineering, Beijing University of Posts andTelecommunications, Beijing 100876, China. (E-mail: { DragonAim, thuraya, yanghong } @bupt.edu.cn) mutual information under AWGN channels. Although these formulas hold compact forms, the approximation errorwas relatively high. Motivated by these work, we will also develop some approximation expression for the mutualinformation under discrete inputs.The remaining parts of this manuscript is structured as follows: Section II presents the approximated expressionfor the mutual information. In Section III, simulation results are provided. Section IV offer some further discussionsof our work. Finally, Section V concludes the paper.II. A PPROXIMATION OF M UTUAL I NFORMATION
Consider a single-input single-output AWGN channel under finite alphabet inputs, the input-output mutualinformation can be expressed [9] I ( X ; Y ) = m − X k =0 p ( k ) Z + ∞−∞ p ( y | x k ) log p ( y | x k ) m − P i =0 p ( i ) p ( y | x k ) d y. (1)Here, we assume the input signal utilizes the M -QAM modulation and x k ∈ X = { x , x , · · · , x M } . Equ. (1) canbe also rewritten as I ( X ; Y ) = m − X k =0 p ( k ) E log p ( y | x k ) m − P i =0 p ( i ) p ( y | x k ) = E m − X k =0 p ( k ) log p ( y | x k ) m − P i =0 p ( i ) p ( y | x k ) . (2)On the basis of Equ. (2), the mutual information can be obtained by Monte-Carlo simulation, which is really time-consuming. On the other hand, the integral in Equ. (1) is intractable. Due to these challenges, we will not continueto try to simplify the integrals with any obscure mathematical theories but to approximate the mutual informationdirectly by curve fitting. From this perspective, we may figure out some elegant approximate expressions of themutual information, with both accurate precision and compact form.In the following part, some classical M -QAM modulation modes will be considered, including 4-QAM, 16-QAM, 64-QAM and 256-QAM. There are many effective tools for curve fitting, such as Matlab, OriginPro, SAS,and SPSS. In this manuscript, a classical software named 1stQpt is utilized, which fits the curves on the basis ofLevenberg-Marquardt(LM) and Universal Global Optimization(UGO) algorithms. Actually, for different modulationmode, they can all be approximated as the following form I M − QAM ( γ ) = (log M ) − N X k =1 a i exp ( − b i γ ) ! , (3)and γ denotes the SNR. As can be seen form this equation, our approximation is based on multi-exponential decaycurve fitting (M-EDCF). After the experiments, the coefficient for different modulation mode are summarized inTable 1. Notice that the RMSE in this table means Root Mean Square Error (RMSE), i.e. the standard deviation ofthe residuals (prediction errors). TABLE IF
ITTING C OEFFICIENTS a a a a b b b b RMSE0.228768 0.229083 0.118223 0.423927 0.183242 0.038011 0.994472 0.006911 0.0005920210.329121 0.197647 0.473233 0.074934 0.662915 0.007544 0.00244403764-QAM a a a a b b b b RMSE0.198324 0.512831 0.209086 0.079759 0.408618 0.027517 0.120616 1.467118 0.0002383690.545415 0.15446 0.300125 0.028609 1.016175 0.191244 0.0008157620.360522 0.639478 0.475473 0.033426 0.00618132216-QAM a a a a b b b b RMSE0.658747 0.117219 0.224034 0.115521 1.467927 0.482023 0.0003180740.277888 0.722112 0.898478 0.123144 0.0016294194-QAM a a a a b b b b RMSE0.143281 0.856719 1.557531 0.57239 0.000364721 0.6507 0.0034287
III. N
UMERICAL R ESULTS
In this section, we use simulation results to verify the precision of the proposed approximate expressions. Ascan be seen from these graphs, our proposed approximation holds high precision, which can be used to evaluatethe mutual information. It should be noted that we only use the approximation with the smallest RMSE to do theestimation. IV. F
URTHER D ISCUSSION
Let us move back to the proposed approximation formula which reads: I M − QAM ( γ ) = (log M ) − N X k =1 a i exp ( − b i γ ) ! . (4)Indeed, the form of this formula is considerably neat and simple. Besides, as can be seen from the simulationresults, it provides a robust approximation to the exact mutual information. Due to its high precision and neat form,this expression can be utilized to analyze the performance of all kinds of wireless channel. For example, the ergodicinformation under any wireless channel with PDF f ( γ ) can be written as: Z ∞ I M − QAM ( γ ) f ( γ ) d γ = (log M ) − N X k =1 a i Z ∞ exp ( − b i γ ) f ( γ ) d γ ! . (5)As it shows, the core point of this calculation is to calculate the moment generating function of f ( γ ) . In the futurework, we will try to use this formula to analyze the ergodic mutual information, the secrecy capacity and theefficient capacity under finite alphabet inputs. −30 −20 −10 0 10 2000.20.40.60.811.21.41.61.82 γ [dB] − Q A M [ b i t s / sy m bo l ] SimulationApproximation (a) 4-QAM −20 −10 0 10 20 3000.511.522.533.54 γ [dB] − Q A M [ b i t s / sy m bo l ] SimulationApproximation (b) 16-QAM −20 −10 0 10 20 300123456 γ [dB] − Q A M [ b i t s / sy m bo l ] SimulationApproximation (c) 64-QAM −30 −20 −10 0 10 20 30 40012345678 γ [dB] − Q A M [ b i t s / sy m bo l ] SimulationApproximation (d) 256-QAMFig. 1. Simulated and approximated ergodic mutual information for different modulation modes.
V. C
ONCLUSION
This paper provides a unified and general framework to analyze the mutual information under finite alphabetinputs. Actually, it has transformed an intractable problem into a relatively solvable one, i.e., the mutual informationanalysis under discrete inputs. Even though these derivations are derived on the basis of numerical fitting, it reallyoffers a fast, efficient and precise method to solve these problem.R
EFERENCES[1] F. Oggier and B. Hassibi, “The secrecy capacity of the MIMO wiretap channel,”
IEEE Trans. Inf. Theory , vol. 57, no. 8, pp. 4961–4972,2011.[2] D. Wu and R. Negi, “Effective capacity: a wireless link model for support of quality of service,”
IEEE Trans. Wireless Commun. , vol. 2,no. 4, pp. 630–643, 2003.[3] S. Khatalin and J. P. Fonseka, “On the channel capacity in Rician and Hoyt fading environments with MRC diversity,”
IEEE Trans. Veh.Technol. , vol. 55, no. 1, pp. 137–141, 2006.[4] D. B. Da Costa, and M. D. Yacoub, “Average channel capacity for generalized fading scenarios,”
IEEE Commun. Lett. , vol. 11, no. 12,pp. 949–951, 2007. [5] J. Zhang, L. Dai, W. H. Gerstacker, and Z. Wang, “Effective capacity of communication systems over κ – µ shadowed fading channels,” Electron. Lett. , vol. 51, no. 19, pp. 1540–1542, 2015.[6] P. Yang, Y. Wu, and H. Yang, “Capacity of Nakagami- m fading channel with BPSK/QPSK modulations,” IEEE Commun. Lett. , vol. 21,no. 3, pp. 564–567, 2017.[7] W. Li, H. Yang, and D. Yang, “Simple approximation formula for the symmetric capacity,”
IEEE 2007 Third International Conference onWireless and Mobile Communications (ICWMC’07) , 2007.[8] P. Yang, L.-Q. Jin, and H.-W. Yang, “An improved approximation formula for the symmetric capacity of 2-dimentional constellationsunder AWGN channel,”
Computer Science, Technology and Application: Proceedings of the 2016 International Conference on ComputerScience, Technology and Application (CSTA2016) , 2017.[9] S. G. Wilson,