Gauge freedom of entropies on q -Gaussian measures
aa r X i v : . [ m a t h . QA ] F e b Gauge freedom of entropieson q -Gaussian measures Hiroshi Matsuzoe ⋆ ⋆⋆ and Asuka Takatsu ∗ ⋆ ⋆ ⋆ , Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan [email protected] Department of Mathematical Sciences, Tokyo Metropolitan University,Tokyo, Japan [email protected] RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan
Abstract. A q -Gaussian measure is a generalization of a Gaussian mea-sure. This generalization is obtained by replacing the exponential func-tion with the power function of exponent 1 / (1 − q ) ( q = 1). The limitcase q = 1 recovers a Gaussian measure. For 1 ≤ q <
3, the set of all q -Gaussian densities over the real line satisfies a certain regularity con-dition to define information geometric structures such as an entropy anda relative entropy via escort expectations. The ordinary expectation ofa random variable is the integral of the random variable with respectto its law. Escort expectations admit us to replace the law to any othermeasures. A choice of escort expectations on the set of all q -Gaussiandensities determines an entropy and a relative entropy. One of most im-portant escort expectations on the set of all q -Gaussian densities is the q -escort expectation since this escort expectation determines the Tsallisentropy and the Tsallis relative entropy.The phenomenon gauge freedom of entropies is that different escort ex-pectations determine the same entropy, but different relative entropies.In this note, we first introduce a refinement of the q -logarithmic function.Then we demonstrate the phenomenon on an open set of all q -Gaussiandensities over the real line by using the refined q -logarithmic functions.We write down the corresponding Riemannian metric. Keywords:
Information geometry · gauge freedom of entropies · refined q -logarithmic function · q -Gaussian measure q -Logarithmic functions and their refinements For q ∈ R , we set χ q : (0 , ∞ ) → (0 , ∞ ) by χ q ( s ) := s q . ⋆ The both authors were supported in part by JSPS Grant-in-Aid for Scientific Re-search (KAKENHI) 16KT0132. ⋆⋆ HM was supported in part by KAKENHI 19K03489. ⋆ ⋆ ⋆
AT was supported in part by KAKENHI 19K03494, 19H01786. H. Matsuzoe and A. Takatsu
We define a strictly increasing function ln q : (0 , ∞ ) → R byln q ( t ) := Z t χ q ( s ) ds and we denote by exp q the inverse function of ln q : (0 , ∞ ) → ln q (0 , ∞ ). Thefunctions ln q and exp q are called the q -logarithmic function and the q -exponentialfunction , respectively. We observe that ddt ln q ( t ) = 1 χ q ( t ) = t − q for t ∈ (0 , ∞ ) ,ddτ exp q ( τ ) = χ q (exp q ( τ )) = exp q ( τ ) q for τ ∈ ln q (0 , ∞ ) . It holds for q ∈ R that χ q (1) = 1 and ln q (1) = 0. Remark 1. (1) For q = 1, we have thatln ( t ) = log( t ) for t ∈ (0 , ∞ ) , ln (0 , ∞ ) = R , exp ( τ ) = exp( τ ) for τ ∈ R . (2) For q = 1, we have thatln q ( t ) = t − q − − q for t ∈ (0 , ∞ ) , ln q (0 , ∞ ) = (cid:18) −∞ , q − (cid:19) if q > , (cid:18) − − q , ∞ (cid:19) if q < , exp q ( τ ) = { − q ) τ } − q for τ ∈ ln q (0 , ∞ ) . Taking account into the negativity of ln q in (0 , q -logarithmic function and the q -exponential function. For q ∈ R and a ∈ R \ { } , define two functions χ q,a : (0 , → (0 , ∞ ) and ln q,a : (0 , → R respectively by χ q,a ( s ) := χ q ( s ) · ( − ln q ( s )) − a , ln q,a ( t ) := − a (cid:0) − ln q ( t ) (cid:1) a . It turns out that dds χ q,a ( s ) = χ ′ q ( s )( − ln q ( s )) − a − (1 − a )( − ln q ( s )) − a for s ∈ (0 , ddt ln q,a ( t ) = 1 χ q,a ( t ) > t ∈ (0 , auge freedom of entropies on q -Gaussian measures 3 Hence the function ln q,a : (0 , → R is strictly increasing. We denote by exp q,a the inverse function of ln q,a : (0 , → ln q,a (0 , q,a ( τ ) = exp q (cid:16) − ( − aτ ) a (cid:17) for τ ∈ ln q,a (0 , q,a and exp q,a are called the a -refined q -logarithmic function andthe a -refined q -exponential function , respectively.On one hand, it holds for q ≥ q,a (0 ,
1) = ( ( −∞ ,
0) if a > , (0 , ∞ ) if a < . On the other hand, it holds for q < q,a (0 ,
1) = (cid:18) − a (1 − q ) − a , (cid:19) if a > , (cid:18) − a (1 − q ) − a , ∞ (cid:19) if a < . Remark 2. (1) The refinement of the ordinary logarithmic function, that is thecase q = 1, was introduced by Ishige, Salani and the second named au-thor [3], where they studied the preservation of concavity by the heat flowin Euclidean space.(2) For a positive function χ : (0 , ∞ ) → (0 , ∞ ) and a ∈ R \{ } , the χ -logarithmicfunction ln χ : (0 , ∞ ) → R and its refinement ln χ,a : (0 , → R are respec-tively defined in the same way as χ q . In this section, we give a condition for ln q,a to be concave and compute the higherorder derivatives of exp q,a , which will be used to define information geometricstructures.For q ∈ R and a ∈ R \ { } , define t q,a := q > q = 0 with a − > , q ≤ a − ≤ , q (cid:16) − aq (cid:17) otherwise ,T q,a := q > − a ≥ qq − , q ≤ , q (cid:16) max n , − aq o(cid:17) otherwise , and set I q,a := ( t q,a , T q,a ). Note that I q,a is nonempty if and only if one of thefollowing three conditions holds: H. Matsuzoe and A. Takatsu • q > − a < qq − • < q ≤ • q ≤ a − > Proposition 1.
Fix q ∈ R and a ∈ R \ { } . For an interval I ⊂ (0 , , the strictconcavity of ln q,a in I is equivalent to the strict convexity of exp q,a in ln q,a ( I ) .Moreover, if I q,a = ∅ , then ln q,a is strictly concave in I q,a .Proof. Due to Equation (1.1), ln q,a is strictly increasing in (0 ,
1) and so is exp q,a in ln q,a (0 , I ⊂ (0 , t i ∈ I, τ i ∈ ln q,a ( I ) ( i = 0 ,
1) with τ i = ln q,a ( t i ) or equivalently t i = exp q,a ( τ i )and λ ∈ (0 , q,a that(1 − λ ) t + λt ∈ I, (1 − λ ) τ + λτ ∈ ln q,a ( I ) . We observe from the monotonicity of ln q,a and exp q,a thatln q,a (cid:0) (1 − λ ) t + λt (cid:1) > (1 − λ ) ln q,a ( t ) + λ ln q,a ( t ) ⇔ ln q,a (cid:0) (1 − λ ) t + λt (cid:1) > (1 − λ ) τ + λτ ⇔ exp q,a (cid:0) ln q,a (cid:0) (1 − λ ) t + λt (cid:1)(cid:1) > exp q,a ((1 − λ ) τ + λτ ) ⇔ (1 − λ ) t + λt > exp q,a ((1 − λ ) τ + λτ ) ⇔ (1 − λ ) exp q,a ( τ ) + λ exp q,a ( τ ) > exp q,a ((1 − λ ) τ + λτ ) , where we used the fact that exp q,a is the inverse function of ln q,a . This provesthe first claim.Assume I q,a = ∅ . A direct calculation provides that d dt ln q,a ( t ) = ddt χ q,a ( t ) = − χ q,a ( t ) ddt χ q,a ( t )= − ( − ln q ( t )) − a χ q,a ( t ) (cid:8) χ ′ q ( t ) ( − ln q ( t )) − (1 − a ) (cid:9) = ( − ln q ( t )) − a χ q,a ( t ) (cid:8) qt q − ln q ( t ) + (1 − a ) (cid:9) . Notice that ( − ln q ( t )) − a /χ q,a ( t ) is positive in t ∈ I q,a . In the case q = 0, thecondition I ,a = ∅ leads to a − >
0, consequently d dt ln ,a ( t ) = ( − ln ( t )) − a χ ,a ( t ) (1 − a ) < . Since the function given by t q − ln q ( t ) = − ln q (cid:18) t (cid:19) = log( t ) q = 1 , − t q − − q q = 1 auge freedom of entropies on q -Gaussian measures 5 is strictly increasing in t ∈ (0 , q > t ∈ I q,a that d dt ln q,a ( t ) = ( − ln q ( t )) − a χ q,a ( t ) (cid:8) qt q − ln q ( t ) + (1 − a ) (cid:9) < ( − ln q ( t )) − a χ q,a ( t ) (cid:26) − q ln q (cid:18) T q,a (cid:19) + (1 − a ) (cid:27) = ( − ln q ( t )) − a χ q,a ( t ) (cid:26) − q · max (cid:26) , − aq (cid:27) + (1 − a ) (cid:27) = ( − ln q ( t )) − a χ q,a ( t ) { min { , a − } + (1 − a ) }≤ . On the other hand, we see that d dt ln q,a ( t ) = ( − ln q ( t )) − a χ q,a ( t ) (cid:8) qt q − ln q ( t ) + (1 − a ) (cid:9) < ( − ln q ( t )) − a χ q,a ( t ) (cid:26) − q ln q (cid:18) t q,a (cid:19) + (1 − a ) (cid:27) = ( − ln q ( t )) − a χ q,a ( t ) (cid:26) − q · − aq + (1 − a ) (cid:27) = 0for q < t ∈ I q,a . This completes the proof of the second claim. ⊓⊔ Lemma 1.
For q ∈ R and a ∈ R \ { } , there exists { b nj = b nj ( q, a ) } n ∈ N , ≤ j ≤ n − such that d n dτ n exp q,a ( τ ) = exp q,a ( τ ) ( n − q − q ( − aτ ) n (1 − a ) a n − X j =0 b nj ( q, a ) · ( − aτ ) − ja for τ ∈ ln q,a (0 , . Moreover, { b nj } n ∈ N , ≤ j ≤ n − satisfies b = 1 ,b n +1 j = { na ( q −
1) + 1 } b n if j = 0 , { ( na + j )( q −
1) + 1 } b nj − { n (1 − a ) − ( j − } b nj − if ≤ j ≤ n − , ( na − b nn − if j = n. Proof.
We observe that ddτ exp q,a ( τ ) = χ q,a (cid:0) exp q,a ( τ ) (cid:1) = χ q (cid:0) exp q,a ( τ ) (cid:1) · (cid:8) − ln q (cid:0) exp q,a ( τ ) (cid:1)(cid:9) − a = exp q,a ( τ ) q · ( − aτ ) − aa , H. Matsuzoe and A. Takatsu where we used Equation (1.2). Thus the lemma holds for n = 1.If the lemma holds for n , then we compute that d n +1 dτ n +1 exp q,a ( τ )= ddτ exp q,a ( τ ) ( n − q − q ( − aτ ) n (1 − a ) a n − X j =0 b nj · ( − aτ ) − ja = (cid:18) ddτ exp q,a ( τ ) ( n − q − q (cid:19) × ( − aτ ) n (1 − a ) a n − X j =0 b nj · ( − aτ ) − ja + exp q,a ( τ ) ( n − q − q × ddτ ( − aτ ) n (1 − a ) a n − X j =0 b nj · ( − aτ ) − ja = { ( n − q −
1) + q } exp q,a ( τ ) ( n − q − q − · exp q,a ( τ ) q ( − aτ ) − aa × ( − aτ ) n (1 − a ) a n − X j =0 b nj · ( − aτ ) − ja + exp q,a ( τ ) ( n − q − q × − a n − X j =0 n (1 − a ) − ja b nj · ( − aτ ) n (1 − a ) − ja − = exp q,a ( τ ) n ( q − q ( − aτ ) ( n +1)(1 − a ) a × { ( n − q −
1) + q } n − X j =0 b nj ( − aτ ) − ja − exp q,a ( τ ) − q n − X j =0 { n (1 − a ) − j } b nj ( − aτ ) − j +1 a . We deduce from exp q,a ( τ ) − q = 1 − (1 − q )( − aτ ) a thatexp q,a ( τ ) − q n − X j =0 { n (1 − a ) − j } b nj · ( − aτ ) − j +1 a = n − X j =0 { n (1 − a ) − j } b nj · ( − aτ ) − j +1 a − (1 − q ) n − X j =0 { n (1 − a ) − j } b nj · ( − aτ ) − ja . This completes the proof of the lemma. ⊓⊔ Remark 3.
For q ∈ R and a ∈ R \ { } , we have that b = 1 , b = a ( q −
1) + 1 , b = { a ( q −
1) + 1 }{ a ( q −
1) + 1 } ,b = a − , b = ( a − { (4 a + 1)( q −
1) + 3 } ,b = ( a − a − . auge freedom of entropies on q -Gaussian measures 7 Corollary 1.
For a ∈ R \ { } and n ∈ N , then b n (1 , a ) = 1 .Proof. It follows from Lemma 1 that b n +10 (1 , a ) = { na (1 −
1) + 1 } b n (1 , a ) = b n (1 , a ) = · · · = b (1 , a ) = 1 . ⊓⊔ Corollary 2.
Let q ∈ R and n ∈ N . For ≤ j < n , then b nj ( q,
1) = 0 .Proof.
This holds for 1 = j < n = 2 by Remark 3. For n ≥
2, if b nj ( q,
1) = 0 holdsfor 1 ≤ j ≤ n −
1, then Lemma 1 implies that b n +1 n ( q,
1) = ( na − b nn − ( q,
1) = 0.For 2 ≤ j ≤ n −
1, we have that b n +1 j ( q,
1) = { ( n + j )( q −
1) + 1 } b nj ( q,
1) + ( j − b nj − ( q,
1) = 0by the assumption b nk ( q,
1) = 0 for 1 ≤ k ≤ n −
1. For j = 1, we have that b n +11 ( q,
1) = { ( n + 1)( q −
1) + 1 } b n ( q,
1) + (1 − b n ( q,
1) = 0 . ⊓⊔ The ordinary expectation of a random variable is the integral of the randomvariable with respect to its law. An introduction to escort expectations admitsus to replace the law to any other measures. The escort expectation with respectto a probability measure was first introduced by Naudts [5].
Definition 1.
For a measure ν on a measurable space Ω , the escort expectation of a function f ∈ L ( ν ) with respect to ν is defined by E ν [ f ] := Z Ω f ( ω ) dν ( ω ) . In this section, we fix a manifold S consisting of positive probability densitieson a measure space ( Ω, ν ). We assume that S is homeomorphic to an open set Ξ in R d and we denote each element in S by p ( · ; ξ ) for ξ ∈ Ξ . Namely, S = (cid:26) p ( · ; ξ ) : Ω → (0 , ∞ ) (cid:12)(cid:12)(cid:12) Z Ω p ( ω ; ξ ) dν ( ω ) = 1 , ξ ∈ Ξ (cid:27) . We moreover require that S satisfies a certain regularity condition to defineinformation geometric structures via escort expectations. For the regularity con-dition, we refer to [1, Chapter 2]. Remark 4.
One of manifolds consisting of probability densities on a measurespace satisfying the regular condition is a q -exponential family, which is a gen-eralization to the space of q -Gaussian densities over R for 1 ≤ q < c ∈ (0 , ∞ ] such that c > sup { p ( ω ) | p ∈ S , ω ∈ Ω } if the above supremum is finite, otherwise c := ∞ . H. Matsuzoe and A. Takatsu
Definition 2.
Let ℓ : (0 , c ) → R be a differentiable function such that ℓ ′ > in (0 , c ) . For p ∈ S , we define a measure ν ℓ ; p on Ω as the absolutely continuousmeasure with respect to ν with Radon–Nikodym derivative dν ℓ ; p dν ( ω ) = 1 ℓ ′ ( p ( ω )) . Note that ℓ is often assumed to be concave such as the logarithmic function. Inthe case ℓ = log, we have that dν ℓ ; p dν = p. Definition 3.
Fix a differentiable function ℓ : (0 , c ) → R such that ℓ ′ > in (0 , c ) and assume that ℓ ( r ) = ℓ ◦ r ∈ L ( ν ℓ ; p ) for p, r ∈ S . (2.1)(1) For p, r ∈ S , the ℓ -cross entropy of p with respect to r is defined by d ℓ ( p, r ) := − E ν ℓ ; p [ ℓ ( r )] . (2) The ℓ -entropy of p ∈ S is defined by Ent ℓ ( p ) := d ℓ ( p, p ) . (3) For p, r ∈ S , the ℓ -relative entropy of p with respect to r is defined by D ( ℓ ) ( p, r ) := − d ℓ ( p, p ) + d ℓ ( p, r ) . Remark 5.
In general, the ℓ -entropy does not satisfy nonextensive Shannon–Khinchin axioms [7]. However, if S is a manifold of all Gaussian densities overEuclidean space and ℓ = log, then the ℓ -entropy coincides with the Boltzmann–Shannon entropy.A choice of differentiable functions ℓ : (0 , c ) → R such that ℓ ′ > , c )determines an entropy and a relative entropy on S . However, the converse isnot true. This phenomenon is related to gauge freedom , which was proposed byZhang and Naudts [8] (see also [6]).In the next section, we demonstrate gauge freedom of entropies on an openset of q -Gaussian densities over R for 1 ≤ q <
3. To be precise, we show that dif-ferent escort expectations determine the same entropy up to scalar multiple, butdifferent relative entropies, where the entropy satisfies nonextensive Shannon–Khinchin axioms. q -Gaussian measures To define q -Gaussian measures, we extend exp q to the whole of R byRexp q ( τ ) := max { , − q ) τ } − q for τ ∈ R , where by convention 0 c := ∞ for c <
0. We recall the following improper integral. auge freedom of entropies on q -Gaussian measures 9 Lemma 2.
For q ∈ R and ( µ, λ ) ∈ R × (0 , ∞ ) , the improper integral of thefunction x Rexp q ( − λ ( x − µ ) ) on R converges if and only if q < . For q < , p − q Z R Rexp q ( − x ) dx = Z q := r − qq − B (cid:18) − q q − , (cid:19) if q > , √ π if q = 1 , r − q − q B (cid:18) − q − q , (cid:19) if q < , where B ( · , · ) stands for the beta function.Proof. By the change of variables, it is enough to show the case ( µ, λ ) = (0 , q = 1, which is well-known.Assume q = 1. There exist c, C, R > q such that cx − q ≤ Rexp q ( − x ) = (cid:8) − (1 − q ) x (cid:9) − q < Cx − q for x > R . Since the improper integral of the function x x − q on [1 , ∞ ) converges if and only if 2 / (1 − q ) < −
1, that is q <
3, so does theimproper integral of the function x Rexp q ( − x ) on R .For 1 < q <
3, we observe that Z R Rexp q ( − x ) dx = 2 Z ∞ (cid:8) − (1 − q ) x (cid:9) − q dx = 1 √ q − Z ∞ (1 + r ) − q r − dr = 1 √ q − B (cid:18) − q q − , (cid:19) , where we used that B ( s − t, t ) = Z ∞ r s − (1 + r ) t dr for s > t > . In the case q <
1, the support of the function x Rexp q ( − x ) on R is (cid:20) − √ − q , √ − q (cid:21) implying that Z R Rexp q ( − x ) dx = 2 Z √ (1 − q ) (cid:2) − (1 − q ) x (cid:3) − q dx = 1 √ − q Z [1 − r ] − q r − dx = 1 √ − q B (cid:18) − q − q , (cid:19) . ⊓⊔ Definition 4.
For q < and ξ = ( µ, σ ) ∈ R × (0 , ∞ ) , the q -Gaussian measure with location parameter µ and scale parameter σ on R is an absolutely continuousprobability measure with respect to the one-dimensional Lebesgue measure withRadon–Nikodym derivative p q ( x ; ξ ) = p q ( x ; µ, σ ) := 1 Z q σ Rexp q − − q (cid:18) x − µσ (cid:19) ! . We call p q ( x ; ξ ) = p q ( x ; µ, σ ) the q -Gaussian density with location parameter µ and scale parameter σ . A q -Gaussian density corresponds to a normal ( Gaussian ) distribution for q = 1, and a Student t -distribution for 1 < q <
3. In the both cases, the supportof each q -Gaussian measure is the whole of R and p q ( x ; ξ ) = p q ( x ; µ, σ ) = 1 Z q σ exp q − − q (cid:18) x − µσ (cid:19) ! . The set of all q -Gaussian densities satisfies the regularity condition to defineinformation geometric structures. For example, see [4]. (2.1)In order to give a rigorous treatment of an escort expectation associated to the a -refined q -logarithmic function, we only deal with the case 1 ≤ q <
3. Set Σ q := (cid:26) σ > (cid:12)(cid:12)(cid:12) Z q σ < (cid:27) , S q := { p q ( · ; ξ ) | ξ ∈ R × Σ q } . It holds for σ ∈ Σ q , p ∈ S q and x ∈ R thatln q (cid:18) Z q σ (cid:19) < ln q (1) = 0 , ln q ( p ( x )) ∈ ( −∞ , . Definition 5.
For ≤ q < and ξ ∈ R × Σ q , define ℓ q ( · ; ξ ) : R → ( −∞ , by ℓ q ( x ; ξ ) := ln q ( p q ( x ; ξ )) , which is called the q -likelihood function of p q ( · ; ξ ) . auge freedom of entropies on q -Gaussian measures 11 For ≤ q < , a ∈ R \ { } and ξ ∈ R × Σ q , we define a measure ν q,a ; ξ on R asthe absolutely continuous measure with respect to the one-dimensional Lebesguemeasure with Radon–Nikodym derivative dν q,a ; ξ dx ( x ) = 1ln ′ q,a ( p q ( x ; ξ )) . Since the inverse function of ln q,a is exp q,a , Lemma 1 in the case n = 1 leads to dν q,a ; ξ dx ( x ) = exp ′ q,a (ln q,a ( p q ( x ; ξ ))) = ( − ℓ q ( x ; ξ )) − a χ q ( p q ( x ; ξ )) . A direct computation leads to the relation that ℓ q ( x ; ξ ) = ln q (cid:18) Z q σ (cid:19) − Z q σ ) − q (3 − q ) (cid:18) x − µσ (cid:19) , ln q,a ( p q ( x ; ξ )) = − a ( − ℓ q ( x ; ξ )) a . (3.1) Lemma 3.
Let ≤ q < , a ∈ R \ { } and ξ ∈ R × Σ q . Then for λ, γ ∈ R with λ > , ( λ + x ) γ ∈ L ( ν q,a ; ξ ) if and only ifeither q = 1 or q > with γ <
12 + 1 q − a − . Proof.
Since the decay rate of ν ,a ; ξ is o (exp( − x ε )) as x → ∞ for ε <
2, thelemma holds for q = 1.Assume q >
1. By the change of variables, it is enough to show the case ξ = (0 , /Z q ). Here we have that Z q σ = 2. There exist c, C, R > q such that cx − a )+ q − q +2 γ < ( − ℓ q ( x ; ξ )) − a · χ q ( p q ( x ; ξ )) · ( λ + x ) γ = ( − ln q (cid:18) (cid:19) + 12 − q (3 − q ) Z q x ) − a · q q − − q Z q x ! q − q · ( λ + x ) γ < Cx − a )+ q − q +2 γ for x > R . This means that ( c + x ) γ ∈ L ( ν q,a ; ξ ) if and only if2(1 − a ) + 2 q − q + 2 γ < − ⇔ γ <
12 + 1 q − a − . ⊓⊔ Lemma 3 in the case γ = 0 provides the condition for ( q, a ) such that ν q,a ; ξ has a finite mass. Corollary 3.
Let ≤ q < , a ∈ R \ { } and ξ ∈ R × Σ q . Then ∈ L ( ν q,a ; ξ ) if and only if either q = 1 or q > with − q − < a. Note that 12 − q − < < q < . Corollary 4.
Let ≤ q < and a ∈ R \ { } . Then ln q,a ( r ) ∈ L ( ν q,a ; ξ ) for ξ ∈ R × Σ q and r ∈ S q .Proof. The corollary trivially holds for q = 1.Assume q >
1. We observe from (3.1) thatln q,a ( p ( x ; µ, σ )) = − a ( − ln q (cid:18) Z q σ (cid:19) + 1( Z q σ ) − q (3 − q ) (cid:18) x − µσ (cid:19) ) a for ( µ, σ ) ∈ R × Σ q . This with Lemma 3 yields thatln q,a ( r ) ∈ L ( ν q,a ; ξ ) ⇔ a <
12 + 1 q − a − , (3.2)which holds for q < ⊓⊔ Following Definition 3, we define an entropy and a relative entropy on S q .Recall the escort expectation of a function f ∈ L ( ν q,a ; ξ ) with respect to ν q,a ; ξ is defined by E ν q,a ; ξ [ f ] = Z R f ( x ) dν q,a ; ξ ( x ) = Z R f ( x ) exp ′ q,a (cid:0) ln q,a ( p q ( x ; ξ )) (cid:1) dx. Definition 6.
Let ≤ q < and a ∈ R \ { } . Take ξ ∈ R × Σ q and set p = p q ( · ; ξ ) ∈ S q . (1) The ( q, a )-cross entropy of p with respect to r ∈ S q is defined by d q,a ( p, r ) := − E ν q,a ; ξ [ln q,a ( r )] . (2) The ( q, a )-entropy of p is defined by Ent q,a ( p ) := d q,a ( p, p ) . (3) The ( q, a )-relative entropy of p with respect to r ∈ S q is defined by D ( q,a ) ( p, r ) := − d q,a ( p, p ) + d q,a ( p, r ) . Remark 6.
The domain of the ( q, q -Gaussian densities. The ( q, Boltzmann–Shannonentropy if q = 1, and the Tsallis entropy otherwise.
Theorem 1 (gauge freedom of entropies).
Let ≤ q < and a ∈ R \ { } .Then Ent q, = a Ent q,a , D ( q, = λD ( q,a ) for a = 1 and λ ∈ R . auge freedom of entropies on q -Gaussian measures 13 Proof.
By the definition, we have that d q,a ( p q ( · ; ξ ) , p q ( · ; ξ )) = 1 a Z R ( − ℓ q ( x ; ξ )) a ν q,a ; ξ ( x )= 1 a Z R ( − ℓ q ( x ; ξ )) a ( − ℓ q ( x ; ξ )) − a χ q ( p q ( x ; ξ )) dx for ξ , ξ ∈ R × Σ q , which implies thatEnt q, ( p ) = a Ent q,a ( p ) = − Z R ln q ( p ( x )) p ( x ) q dx = − Z R p ( x ) − p ( x ) q − q dx q > , − Z R p ( x ) log( p ( x )) dx q = 1for p ∈ S q .Recall that Σ q = { σ > | σ > /Z q } . Since we observe thatlim σ →∞ ( − ℓ q ( x ; 0 , σ )) a − ln q (cid:16) Z q σ (cid:17) = ∞ if a > , a = 1 , a < , a = 0 , we apply the dominated convergence theorem a ≤ a > λD ( q,a ) ( p, p q ( · ; 0; σ )) − D ( q, ( p, p q ( · ; 0; σ )) − ln q (cid:16) Z q σ (cid:17) = − λd q,a ( p, p ) − d q, ( p, p ) − ln q (cid:16) Z q σ (cid:17) + λd q,a ( p, p q ( · ; 0; σ )) − d ( q, ( p, p q ( · ; 0; σ )) − ln q (cid:16) Z q σ (cid:17) σ →∞ −−−−→ λ · ∞ − c if a > , ( λ − c if a = 1 , − c if a < , a = 0for p ∈ S q and λ ∈ R , where we put 0 · ∞ := 0 and c := Z R χ q ( p ( x )) dx. This constant c is obviously positive, and c is finite due to Lemma 5 in the nextsection. This ensures that D ( q,a ) = λD ( q, for a = 1 and λ ∈ R . ⊓⊔ The proof of Theorem 1 immediately gives the following corollary.
Corollary 5.
Let ≤ q < and a ∈ R \ { } . Then d q, = λd q,a for a = 1 and λ ∈ R . Throughout of this section, we fix 1 ≤ q < a ∈ R \ { } such that I q,a = ∅ ,namely either q = 1 or q > − a < qq − . In this case, t q,a = 0. Set Σ q,a := (cid:26) σ ∈ Σ q (cid:12)(cid:12) Z q σ < T q,a (cid:27) , S q,a := { p q ( · ; ξ ) ∈ S q | ξ ∈ R × Σ q,a } . The manifold S q,a admits information geometric structures. q, a )-relative entropy The ( q, a )-relative entropy is nondegenerate on S q,a × S q,a . Lemma 4.
For p, r ∈ S q,a , D ( q,a ) ( p, r ) > .Proof. Proposition 1 yields that exp ′′ q,a (ln q,a ( p ( x ))) > x ∈ R for p ∈ S q,a .The strict convexity of exp q,a leads to the inequality that r ( x ) = exp q,a (ln q,a ( r ( x ))) > exp q,a (ln q,a ( p ( x ))) + { ln q,a ( r ( x )) − ln q,a ( p ( x )) } exp ′ q,a (ln q,a ( p ( x )))= p ( x ) + ln q,a ( r ( x )) exp ′ q,a (ln q,a ( p ( x ))) − ln q,a ( p ( x )) exp ′ q,a (ln q,a ( p ( x )))for x ∈ R and p, r ∈ S q,a . Integrating this inequality on R gives1 > − d q,a ( p, r ) + d q,a ( p, p ) = 1 − D ( q,a ) ( p, r ) . ⊓⊔ Let us define a function ρ ( q,a ) on ( x, ξ , ξ ) ∈ R × ( R × Σ q,a ) by ρ ( q,a ) ( x ; ξ , ξ ) := { ln q,a ( p q ( x ; ξ )) − ln q,a ( p q ( x ; ξ )) } exp ′ q,a (ln q,a ( p q ( x ; ξ ))) , which is the integrand of D ( q,a ) ( p q ( · ; ξ ) , p q ( · ; ξ )).Given ξ i = ( µ i , σ i ) ∈ R × Σ q,a , it turns out that ∂∂s ∂∂s ρ ( q,a ) ( x ; ξ , ξ ) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) = − ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂s exp ′ q,a (ln q,a ( p q ( x ; ξ )) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) = − ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂s ln q,a ( p q ( x ; ξ ) · exp ′′ q,a (ln q,a ( p q ( x ; ξ )) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) = − ∂∂s (cid:26) − a ( − ℓ q ( x ; ξ )) a (cid:27) · ∂∂s (cid:26) − a ( − ℓ q ( x ; ξ )) a (cid:27) (cid:12)(cid:12)(cid:12)(cid:12) ( ξ,ξ ) × p q ( x ; ξ ) (2 − q − q ( − ℓ q ( x ; ξ )) − a ) 1 X j =0 b j ( − ℓ q ( x ; ξ )) − j = − X j =0 b j ∂∂s ℓ q ( x ; ξ ) · ∂∂s ℓ q ( x ; ξ ) (cid:12)(cid:12)(cid:12)(cid:12) ( ξ,ξ ) · ( − ℓ q ( x, ξ )) − j p q ( x ; ξ ) q − ! auge freedom of entropies on q -Gaussian measures 15 for s i ∈ { µ i , σ i } , where we used Lemma 1 in the case n = 2.Let us generalize Lemma 3. Lemma 5.
Fix n ∈ N and γ ≥ . Then exp q ( − x ) ( n − q − q · x γ ∈ L ( dx ) ifand only if either q = 1 or q > with γ <
12 + 1 q − n − . Proof.
The lemma trivially holds for q = 1. Assume q >
1. There exist c, C, R > q such that cx ( n − q − q − q +2 γ < exp q ( − x ) ( n − q − q · x γ = (cid:8) − (1 − q ) x (cid:9) ( n − q − q − q · x γ < Cx ( n − q − q − q +2 γ for x > R . This yields that exp q ( − x ) ( n − q − q x γ ∈ L ( dx ) if and only if2 ( n − q −
1) + q − q + 2 γ < − ⇔ γ <
12 + 1 q − n − . ⊓⊔ Corollary 6.
For n ∈ N , ≤ γ ≤ n, j ∈ Z ≥ and ξ ∈ R × Σ q,a , then p q ( x ; ξ ) ( n − q − q · x γ · ( − ℓ q ( x ; ξ )) − j ∈ L ( dx ) . Proof.
Since we have that n <
12 + 1 q − n − < q < , we apply Lemme 5 together with the change of variables to have that p q ( x ; ξ ) ( n − q − q · x γ ∈ L ( dx ) for 0 ≤ γ ≤ n. Moreover, the fact that − ℓ q ( x ; ξ ) ≥ − ln q (cid:18) Z q σ (cid:19) > ⊓⊔ Combining the computation that ∂∂µ ℓ q ( x ; µ, σ ) = 2(3 − q ) · Z q σ ) − q σ x − µσ ,∂∂σ ℓ q ( x ; µ, σ ) = − Z q σ ) − q σ ( − (cid:18) x − µσ (cid:19) ) (4.1) with Corollary 6 in the case n = 2, we conclude that x ∂∂s ∂∂s ρ ( q,a ) ( x ; ξ , ξ ) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) is integrable on R for ξ ∈ R × Σ q,a . Since the function x ρ ( q,a ) ( x ; ξ , ξ ) isintegrable on R for ( ξ , ξ ) ∈ ( R × Σ q,a ) , the dominated convergence theoremimplies that ∂∂s ∂∂s D ( q,a ) ( p q ( · ; ξ ) , p q ( · ; ξ ) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) = − Z R ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂s ln q,a ( p q ( x ; ξ ) · exp ′′ q,a (ln q,a ( p q ( x ; ξ )) (cid:12)(cid:12)(cid:12) ( ξ,ξ ) dx = − X j =0 b j Z R (cid:18) ∂∂s ℓ q ( x ; ξ ) · ∂∂s ℓ q ( x ; ξ ) (cid:19) (cid:12)(cid:12)(cid:12)(cid:12) ( ξ,ξ ) · ( − ℓ q ( x, ξ )) − j p q ( x ; ξ ) q − dx for s i ∈ { µ i , σ i } . This quantity evaluated at the diagonal set { ( ξ , ξ ) | ξ = ξ } provides a Riemannian metric on S q,a . Definition 7.
For s, t ∈ { µ, σ } , define a function g ( q,a ) st : R × Σ q,a → R by g ( q,a ) st ( ξ ) := Z R ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂t ln q,a ( p q ( x ; ξ )) · exp ′′ q,a (ln q,a ( p q ( x ; ξ ))) dx. Theorem 2.
For ξ ∈ R × Σ q,a and s, t ∈ { µ, σ } , g ( q,a ) (cid:18) ∂∂s , ∂∂t (cid:19) ( p q ( · ; ξ )) := g ( q,a ) st ( ξ ) determines a Riemannian metric on S q,a .Proof. It is enough to show that g ( q,a ) µµ , g ( q,a ) σσ > g ( q,a ) µσ = 0 on R × Σ q,a . The positivities of g ( q,a ) µµ , g ( q,a ) σσ follows from that of ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂s ln q,a ( p q ( x ; ξ )) · exp ′′ q,a (ln q,a ( p q ( x ; ξ ))) for s ∈ { µ, σ } . We derive g ( q,a ) µσ = 0 from the fact that ∂∂µ ln q,a ( p q ( x ; ξ )) · ∂∂σ ln q,a ( p q ( x ; ξ )) · exp ′′ q,a (ln q,a ( p q ( x ; ξ )))is an odd function in x ∈ R with respect to x = µ according to (4.1). ⊓⊔ auge freedom of entropies on q -Gaussian measures 17 Remark 7.
The Riemannian metric g ( q, coincides with the Fisher metric up toscalar multiple. The third order derivatives of ( q, q -Gaussian densities induce a pair of affine connections. The cubic tensorwhich expresses the difference between the two affine connections is called the Amari– ˇCencov tensor . In a similar way, a cubic tensor C ( q,a ) is defined by C ( q,a ) (cid:18) ∂∂s , ∂∂t , ∂∂u (cid:19) ( p q ( · ; ξ )):= Z R ∂∂s ln q,a ( p q ( x ; ξ )) · ∂∂t ln q,a ( p q ( x ; ξ )) · ∂∂u ln q,a ( p q ( x ; ξ )) × exp ′′′ q,a (ln q,a ( p q ( x ; ξ ))) dx = Z R ∂∂s (cid:26) − a ( − ℓ q ( x ; ξ )) a (cid:27) · ∂∂t (cid:26) − a ( − ℓ q ( x ; ξ )) a (cid:27) · ∂∂u (cid:26) − a ( − ℓ q ( x ; ξ )) a (cid:27) × p q ( x ; ξ ) (3 − q − q ( − ℓ q ( x ; ξ )) − a ) 2 X j =0 b j ( − ℓ q ( x ; ξ )) − j = X j =0 b j Z R ∂∂s ℓ q ( x ; ξ ) · ∂∂t ℓ q ( x ; ξ ) · ∂∂u ℓ q ( x ; ξ ) · ( − ℓ q ( x ; ξ )) − j p q ( x ; ξ ) q − dx for s, t, u ∈ { µ, σ } and ξ ∈ R × Σ q,a . The above improper integral converges dueto Corollary 6 in the case n = 3.The Fisher metric (resp. the Amari– ˇCencov tensor) is a unique invariantquadric (resp. cubic) tensor under Markov embeddings up to scalar multiple(see [2, Chapter 5]). We compute the exact value of g ( q,a ) µµ ( ξ ) = 4(3 − q ) X j =0 b j ( Z q σ ) − q ) σ Z R (cid:18) x − µσ (cid:19) p q ( x ; ξ ) q − ( − ℓ q ( x, ξ )) j dx = 4(3 − q ) X j =0 b j ( Z q σ ) − q ) σ Φ ( q, , , j ; ξ ) ,g ( q,a ) σσ ( ξ ) = X j =0 b j ( Z q σ ) − q ) σ Z R ( − (cid:18) x − µσ (cid:19) ) p q ( x ; ξ ) q − ( − ℓ q ( x, ξ )) j dx = X j =0 b j ( Z q σ ) − q ) σ X k =0 (cid:18) k (cid:19) ( − k Φ ( q, , k, j ; ξ ) (4.2)for ξ ∈ R × Σ q,a , where we set Φ ( q, n, k, j ; ξ ) := Z R (cid:18) x − µσ (cid:19) k p q ( x ; ξ ) ( n − q − q ( − ℓ q ( x, ξ )) j dx. Lemma 6.
For n ∈ N , k ∈ { , , . . . , n } and ξ = ( µ, σ ) ∈ R × Σ q,a , then Φ ( q, n, k, ξ )= σ ( Z q σ ) ( n − q − q (cid:18) − qq − (cid:19) k + B (cid:18) − q q −
1) + n − k,
12 + k (cid:19) if q > , (2 k − if q = 1 , where by convention (2 · − .Proof. We apply the change of variables with y = 12 (cid:18) x − µσ (cid:19) if q = 1, and y = q − − q (cid:18) x − µσ (cid:19) otherwise . For q = 1, we observe that Φ (1 , n, k, ξ ) = Z R p ( x ; ξ ) ( n − − (cid:18) x − µσ (cid:19) k dx = 2 Z ∞ √ πσ exp − (cid:18) x − µσ (cid:19) ! ( n − − (cid:18) x − µσ (cid:19) k dx = 2 k √ π Z ∞ e − y y k − dy = 2 k √ π Γ (cid:18) k + 12 (cid:19) = 2 k √ π (2 k − k √ π = (2 k − , where Γ ( · ) stands for the Gamma function, that is Γ ( s ) := Z ∞ e − x x s − dx for s > . For q >
1, it tuns out that Φ ( q, n, k, ξ )= Z R p q ( x ; ξ ) ( n − q − q (cid:18) x − µσ (cid:19) k dx = 2 Z ∞ Z q σ ) ( n − q − q " q − − q (cid:18) x − µσ (cid:19) ( n − q − q − q (cid:18) x − µσ (cid:19) k dx = σ ( Z q σ ) ( n − q − q (cid:18) − qq − (cid:19) k + Z ∞ y k − (1 + y ) n − qq − dy = σ ( Z q σ ) ( n − q − q (cid:18) − qq − (cid:19) k + B (cid:18) − q q −
1) + n − k,
12 + k (cid:19) . ⊓⊔ auge freedom of entropies on q -Gaussian measures 19 Proposition 2.
For a = 1 and ξ = ( µ, σ ) ∈ R × Σ q,a , we have that g ( q, µµ ( ξ ) = 1 σ , g ( q, σσ ( ξ ) = 3 − qσ . Proof.
It follows from Lemma 6 that Φ (1 , , , ξ ) = 1 , Φ (1 , , , ξ ) = 1 , Φ (1 , , , ξ ) = 3 , implying g (1 , µµ ( ξ ) = b ( q,
1) 1 σ = 1 σ , g (1 , σσ ( ξ ) = b ( q, X j =0 σ (1 − σ . Assume q >
1. By the property that B ( s + 1 , t ) = sts + t B ( s, t ) for s, t > , we have that Φ ( q, , k, ξ ) = σ ( Z q σ ) (2 − q − q (cid:18) − qq − (cid:19) k + B (cid:18) − q q −
1) + 2 − k,
12 + k (cid:19) = σ ( Z q σ ) ( q − q (cid:18) − qq − (cid:19) k + f ( k )( q − + 1) · q − B (cid:18) − q q − , (cid:19) = 1( Z q σ ) q − (cid:18) − qq − (cid:19) k ( q − f ( k ) q , where we set f (0) : = (cid:18) − q q −
1) + 1 (cid:19) · − q q −
1) = ( q + 1)(3 − q )4( q − ,f (1) : = 3 − q q − ·
12 = 3 − q q − ,f (2) : = 32 ·
12 = 34 . This leads to that g ( q, µµ ( ξ ) = 4(3 − q ) b ( q, Z q σ ) − q ) σ Φ ( q, , , ξ ) = 1 σ ,g ( q, σσ ( ξ ) = b ( q, Z q σ ) − q ) σ X k =0 (cid:18) k (cid:19) ( − k Φ ( q, , k, ξ )= 1 σ X k =0 (cid:18) k (cid:19) ( ( − k (cid:18) − qq − (cid:19) k ( q − f ( k ) ) = 3 − qσ . ⊓⊔ Fix n, j ∈ N , k ∈ { , , . . . , n } and ξ = ( µ, σ ) ∈ R × Σ q,a . Let us compute Φ ( q, n, k, j ; ξ ) with the use of the residue theorem. Note that Φ ( q, n, k, j ; µ, σ ) = Φ ( q, n, k, j ; 0 , σ ) . Define a complex valued function φ q,n,k,j ; σ on C by φ q,n,k,j ; σ ( z ) := (cid:16) zσ (cid:17) k p q ( z ; 0 , σ ) ( n − q − q ( − ℓ q ( z ; 0 , σ )) j = (cid:16) zσ (cid:17) k p q ( z ; 0 , σ ) ( n − q − q (cid:26) z + r ( q, σ ) ( Z q σ ) − q (3 − q ) σ (cid:27) − j , where we set r ( q, σ ) := s − ln q (cid:18) Z q σ (cid:19) · ( Z q σ ) − q (3 − q ) σ . The function φ q,n,k,j ; σ has poles of order j at ± ı r ( q, σ ). For R > r ( q, σ ), let L R and C R be smooth curves in C defined respectively by L R := { z : [ − R, R ] → C | z ( θ ) = θ } , C R := { z : [0 , π ] → C | z ( θ ) = Re ı θ } . The residue theorem yields that Z L R ∪ C R φ q,n,k,j ; σ ( z ) dz = 2 π ı · Res( φ q,n,k,j ; σ ; ı r ( q, σ )) , (4.3)where Res( φ q,n,k,j ; σ ; ı r ( q, σ )) stands for the residue of φ q,n,k,j ; σ at z = ı r ( q, σ ). Lemma 7.
For n, j ∈ N , k ∈ { , , . . . , n } and ( µ, σ ) ∈ R × Σ q,a , then Φ ( q, n, k, j ; µ, σ ) = 2 π ı · Res( φ q,n,k,j ; σ , ı r ( q, σ )) . Proof.
If we show that lim R →∞ Z C R φ q,n,k,j ; σ ( z ) dz = 0 , then we have the desired result by letting R → ∞ in (4.3).Take R > r ( q, σ ) large enough. We calculate that (cid:12)(cid:12)(cid:12)(cid:12)Z C R φ q,n,k,j ; σ ( z ) dz (cid:12)(cid:12)(cid:12)(cid:12) ≤ R Z π (cid:12)(cid:12) φ q,n,k,j ; σ ( Re ı θ ) (cid:12)(cid:12) dθ = R Z π (cid:18) Rσ (cid:19) k (cid:12)(cid:12) p q ( Re ı θ ; 0 , σ ) (cid:12)(cid:12) ( n − q − q (cid:12)(cid:12)(cid:12)(cid:12) R e θ + r ( q, σ ) ( Z q σ ) − q (3 − q ) σ (cid:12)(cid:12)(cid:12)(cid:12) − j dθ ≤ CR k − j )+1 Z π (cid:12)(cid:12)(cid:12)(cid:12) exp q (cid:18) − R e θ (3 − q ) σ (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ( n − q − q dθ, auge freedom of entropies on q -Gaussian measures 21 where the constant C depends on q and σ .In the case q = 1, we have that (cid:12)(cid:12)(cid:12)(cid:12) exp (cid:18) − R e θ (3 − σ (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ( n − − q = exp (cid:18) − R cos 2 θ σ (cid:19) , consequently (cid:12)(cid:12)(cid:12)(cid:12)Z C R φ q,n,k,j ; σ ( z ) dz (cid:12)(cid:12)(cid:12)(cid:12) ≤ CR k − j )+1 Z π exp (cid:18) − R cos 2 θ σ (cid:19) dθ R →∞ −−−−→ . In the case q >
1, we observe that (cid:12)(cid:12)(cid:12)(cid:12) exp q (cid:18) − R e θ (3 − q ) σ (cid:19)(cid:12)(cid:12)(cid:12)(cid:12) ( n − q − q = (cid:12)(cid:12)(cid:12)(cid:12) q − − q R e θ σ (cid:12)(cid:12)(cid:12)(cid:12) ( n − q − q − q ≤ C ′ R − n + − q , where the constant C ′ depends on q and σ . This yields that (cid:12)(cid:12)(cid:12)(cid:12)Z C R φ q,n,k,j ; σ ( z ) dz (cid:12)(cid:12)(cid:12)(cid:12) ≤ C · C ′ R k − j )+1 − n + − q · π. The right-hand side converges to 0 as R → ∞ since we have2( k − j ) + 1 − n + 21 − q ≤ − − q < k ≤ n and j ≥ ⊓⊔ Proposition 3.
For ξ = ( µ, σ ) ∈ R × Σ q,a , then g ( q,a ) µµ ( ξ ) = b ( q, a ) b ( q, σ − − q πb ( q, a )( Z q σ ) − q σ r ( q, σ ) ,g ( q,a ) σσ ( ξ ) = (3 − q ) b ( q, a ) b ( q, σ + π (3 − q ) b ( q, Z q σ ) − q r ( q, σ ) ( (cid:18) r ( q, σ ) σ (cid:19) ) . Proof.
It follows from Lemma 7 that Φ ( q, n, k, ξ )= 2 π ı · Res( φ q,n,k, σ , ı r ( q, σ ))= 2 π ı lim z → ı r ( q,σ ) { ( z − ı r ( q, σ )) · φ q,n,k,j ; σ ( z ) } = 2 π ı lim z → ı r ( q,σ ) (cid:16) zσ (cid:17) k p q ( z ; 0 , σ ) ( n − q − q (cid:26) z + ı r ( q, σ )( Z q σ ) − q (3 − q ) σ (cid:27) − = 2 π ı · (cid:18) ı r ( q, σ ) σ (cid:19) k ( Z q σ ) − q (3 − q ) σ r ( q, σ )= ( − k π ( Z q σ ) − q (3 − q ) σ k − r ( q, σ ) k − , where we used p q (ı r ( q, σ ); 0 , σ ) = 1. This with Proposition 2 and (4.2) concludesthe proof of the proposition. ⊓⊔ Remark 8.
In the case a = 1, the Riemannian manifold ( S q, , g ( q, ) has a con-stant curvature − / (3 − q ). This means that all ( S q, , g ( q, ) for 1 ≤ q < a = 1. In this note, we presented gauge freedom of entropies on the subset S q of all q -Gaussian densities for 1 ≤ q <
3. We showed that a constant multiple of each( q, a )-entropy coincides with the Boltzmann–Shannon entropy if q = 1, and theTsallis entropy otherwise. However, any constant multiple of the ( q, a )-relativeentropy differs from the ( q, a = 1. We remark that the( q, q = 1,and the Tsallis relative entropy of the Csisz´ar type otherwise.In information geometry, the Kullback–Leibler divergence projection fromobserved data to a statistical model attains the maximum likelihood estimator(see [1, Chapter 4]). The terminology “maximum” depends on a criterion. It isknown that higher-order asymptotic theory of estimation and Bayesian statisticsimprove the maximum likelihood estimator in another criterion. Ishige, Salaniand the second named author showed in [3, Theorem 3.2] that the concavity re-lated to the case ( q, a ) = (1 , /
2) is the strongest concavity among all admissibleconcavities preserved by the heat flow in Euclidean space. We expect that the(1 , / References
1. Amari, S., Nagaoka, H.: Methods of information geometry, Translations of Math-ematical Monographs, vol. 191. American Mathematical Society, Providence, RI;Oxford University Press, Oxford (2000), translated from the 1993 Japanese originalby Daishi Harada2. Ay, N., Jost, J., Lˆe, H.V., Schwachh¨ofer, L.: Information geometry, Ergebnisse derMathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Math-ematics [Results in Mathematics and Related Areas. 3rd Series. A Series of ModernSurveys in Mathematics], vol. 64. Springer, Cham (2017)3. Ishige, K., Salani, P., Takatsu, A.: To logconcavity and beyond. Commun. Contemp.Math. (in press)4. Matsuzoe, H., Wada, T.: Deformed algebras and generalizations of independence ondeformed exponential families. Entropy (8), 5729–5751 (2015)5. Naudts, J.: Estimators, escort probabilities, and φ -exponential families in statisticalphysics. JIPAM. J. Inequal. Pure Appl. Math. (4), Article 102, 15 (2004)6. Naudts, J., Zhang, J.: Rho-tau embedding and gauge freedom in information geom-etry. Inf. Geom. (1), 79–115 (2018)7. Suyari, H.: Generalization of Shannon-Khinchin axioms to nonextensive systems andthe uniqueness theorem for the nonextensive entropy. IEEE Trans. Inform. Theory50