Armando B. Matos

Conditional Rényi Entropies

There is no generally accepted definition of conditional Rényi entropy. The (unconditional) Rényi entropy depends on a parameter α, which for the case of min-entropy takes the value ∞. Even for this particular case, there are several proposals for the definition of conditional entropy. This paper describes three general definitions of conditional Rényi entropy that were found or suggested in the literature. Their properties are studied and their values, as a function of α, are compared. The particular case of min-entropy is widely used in cryptography as a security parameter; this case is studied in some detail.

Entropy Measures vs. Kolmogorov Complexity

Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for R´enyi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we show that, for some distributions we have a similar result. We prove that, for universal time-bounded distribution mt(x), Tsallis and Renyi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.

Periodic sets of integers

Abstract Consider the following kinds of sets: • the set of all possible distances between two vertices of a directed graph; • any set of integers that is either finite or periodic for all n greater or equal to some n0 (such a set is called ultimately periodic); • a context-free language over an alphabet with one letter (such a language is also regular); • the set of all possible lengths of words of a context-free language. All these sets are isomorphic relatively to the operations of union (or sum), concatenation and Kleene (or transitive) closure. Furthermore, they all share a particularly important property which is not valid in some similar algebraic structure - the concatenation is commutative. The purpose of this paper is to investigate the representation and properties of these sets and also the algorithms to compute the operations mentioned above. The concepts of linear number and Δ-sum are developed in order to provide convenient methods of representation and manipulation. It should be noted that although Δ-sums and regular expressions (or finite automata) over a one-letter alphabet denote essentially the same sets, the corresponding algebras are quite different. For example, it is always possible to eliminate the closure and concatenation operations from a Δ-sum by expanding it as a sum of linear numbers. No such elimination is possible for regular expressions (although special forms of regular expressions or finite automata are sufficient to denote regular sets over one-letter alphabets). The algorithms using Δ-sums are often faster and simpler than those based on finite automata or regular expressions over a one-letter alphabet. We think that this improvement comes from the fact that a set of words over one letter is represented by the set of their lengths and manipulated by arithmetic operations. We apply these methods to the first kind of sets listed above and present new algorithms dealing with a variety of problems related to distances in directed graphs.

Linear programs in a simple reversible language

Very simple reversible programming languages can be useful for the study of reversible transformations. For this purpose we define simple reversible language (SRL), a very simple reversible language, and analyse its properties. The language SRL is similar to the “loop” languages that have been used by several authors to characterise the set of primitive recursive functions. There are, however, important differences: SRL has domain Z instead of N and only reversible programs can be written in SRL. The reversibility of linear homogeneous SRL programs is related to the fact that the corresponding set of matrices has the algebraic structure of a group. We show that such programs implement exactly the linear transformations corresponding to the group of integer positive modular matrices, while in ESRL, an extended version of SRL, the set of transformations that can be implemented by linear homogeneous programs corresponds exactly to the group of integer modular matrices.

Entropy measures vs. algorithmic information

Algorithmic entropy and Shannon entropy are two conceptually different information measures, as the former is based on size of programs and the later in probability distributions. However, it is known that, for any recursive probability distribution, the expected value of algorithmic entropy equals its Shannon entropy, up to a constant that depends only on the distribution. We study if a similar relationship holds for Rényi and Tsallis entropies of order α, showing that it only holds for Rényi and Tsallis entropies of order 1 (i.e., for Shannon entropy). Regarding a time bounded analogue relationship, we show that, for distributions such that the cumulative probability distribution is computable in time t(n), the expected value of time-bounded algorithmic entropy (where the alloted time is nt(n) log(nt(n))) is in the same range as the unbounded version. So, for these distributions, Shannon entropy captures the notion of computationally accessible information. We prove that, for universal time-bounded distribution mt(x), Tsallis and Rényi entropies converge if and only if a is greater than 1.

Monadic logic programs and functional complexity

Problems related to the complexity and to the decidability of several languages weaker than Prolog are studied in this paper. In particular, monadic logic programs, that is, programs containing only monadic functions and monadic predicates, are considered in detail. The functional complexity of a monadic logic program is the language of all words ƒ1… ƒk such that the literal p(ƒ1(…(ƒk(a))…)) is a logical consequence of the program. The relationship between several subclasses of monadic programs, their functional complexities and the corresponding automata is studied. It is proved that the class of monadic programs corresponds exactly to the class of regular languages. As a consequence, the “SUCCESS” problem is decidable for that class. It is also proved that the success set of a specific subclass of monadic programs (“simple” programs) corresponds exactly to regular languages with star-height not exceeding 1.

A matrix model for the flow of control in Prolog programs with applications to profiling

In Prolog the flow of control is relatively complex; four counts (which we call currents)—call, fail, succeed and redo—and two intrinsic properties—the failure and the ‘alternative’ probabilities—can be associated with every literal in a clause body. In this work we describe a new matrix model where those currents and properties are related at the literal, clause and predicate definition levels. This model is useful for predicate classification, execution profiling and program debugging. The application to profilers is discussed in detail.

Ackermann and the superpowers

The Ackermann function a(m, n) is a classical example of a total recursive function which is not primitive recursive. It grows faster than any primitive recursive function. It is usually defined by a general recurrence together with two “boundary” conditions. In this paper we obtain a closed form of a(m, n), which involves the Knuth superpower notation, namely a(m, n) = 2 m−2 ↑ (n + 3) − 3. Generalized Ackermann functions, that is functions satisfying only the general recurrence and one of the boundary conditions are also studied. In particular, we show that the function 2 m−2 ↑ (n + 2)− 2 also belongs to the “Ackermann class”.

Regular Languages and a Class of Logic Programs

We show that the success set of a specific family of logic programs (having only monadic functors and monadic predicates) can be characterized by a regular language with star-height 0 or 1 and reciprocally that for every such set S there is a logic program belonging to the family whose success set is characterized by S.

Distinguishing Two Probability Ensembles with One Sample from each Ensemble

AbstractWe introduced a new method for distinguishing two probability ensembles called one from each method, in which the distinguisher receives as input two samples, one from each ensemble. We compare this new method with multi-sample from the same method already exiting in the literature and prove that there are ensembles distinguishable by the new method, but indistinguishable by the multi-sample from the same method. To evaluate the power of the proposed method we also show that if non-uniform distinguishers (probabilistic circuits) are used, the one from each method is not more powerful than the classical one, in the sense that does not distinguish more probability ensembles. Moreover we obtain that there are classes of ensembles, such that any two members of the class are easily distinguishable (a definition introduced in this paper) using one sample from each ensemble;there are pairs of ensembles in the same class that are indistinguishable by multi-sample from the same method.