Velimir M. Ilic

Generalized Shannon–Khinchin axioms and uniqueness theorem for pseudo-additive entropies☆

Abstract We consider the Shannon–Khinchin axiomatic systems for the characterization of generalized entropies such as the Sharma–Mittal and Frank–Daffertshofer entropies. We provide the generalization of the Shannon–Khinchin axioms and give the corresponding uniqueness theorem. The previous attempts at such axiomatizations are also discussed.

A unified characterization of generalized information and certainty measures

In this paper we consider the axiomatic characterization of information and certainty measures in a unified way. We present the general axiomatic system which captures the common properties of a large number of the measures previously considered by numerous authors. We provide the corresponding characterization theorems and define a new generalized measure called the Inforcer, which is the quasi-linear mean of the function associated with the event probability following the general composition law. In particular, we pay attention to the polynomial composition and the corresponding polynomially composable Inforcer measure. The most common measures appearing in literature can be obtained by specific choice of parameters appearing in our generic measure and they are listed in tables.

Comments on “Generalization of Shannon–Khinchin Axioms to Nonextensive Systems and the Uniqueness Theorem for the Nonextensive Entropy”

In this comment, we consider a generalization of the Shannon–Khinchin axioms (GSK axioms) and the uniqueness theorem for the entropy determined by GSK axioms proposed in [H. Suyari, IEEE Trans. Inf. Theory, vol. 50, pp. 1783–1787, Aug. 2004]. It is shown that the class of entropy functions determined by the axioms from the mentioned paper is wider than the one proposed in this paper, and a counterexample is given. We also derive a new class of entropies by fixing the incorrectness which occurs in the mentioned paper.

Comments on “On q-non-extensive statistics with non-Tsallisian entropy”

Abstract Recently, in Jizba and Korbel (2016), four generalized Shannon–Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we show that the unique class of the entropies derived in the aforementioned paper is not correct, as it violates the fourth GSK axiom, and we derive the correct one. Nevertheless, the class of entropies proposed in the commented paper still can serve as a basis for generalized statistical mechanics. We propose a new axiomatic system which characterizes the class of entropies.

Gradient computation in linear-chain conditional random fields using the entropy message passing algorithm

The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of enhancing memory efficiency when applied to long observation sequences. Unlike the traditional algorithm based on the forward-backward recursions, the memory complexity of our algorithm does not depend on the sequence length. The experiments on real data show that it can be useful for the problems which deal with long sequences.

Computation of cross-moments using message passing over factor graphs

This paper considers the problem of cross-moments computation for functions which decompose according to cycle-free factor graphs. Two algorithms are derived, both based on message passing computation of a corresponding moment-generating function (

Optimal signal constellation design for nonlinear chromatic dispersion optical channel

MGF

CROSS-MOMENTS COMPUTATION FOR STOCHASTIC CONTEXT-FREE GRAMMARS

). The first one is realized as message passing algorithm over a polynomial semiring and represents a computation of the

On a General Definition of Conditional Rényi Entropies

MGF

An Axiomatic Characterization of Generalized Entropies under Analyticity Condition

Taylor coefficients, while the second one represents message passing algorithm over a binomial semiring and a computation of the