Ryo Nomura

Second-Order Resolvability, Intrinsic Randomness, and Fixed-Length Source Coding for Mixed Sources: Information Spectrum Approach

The second-order achievable asymptotics in typical random number generation problems such as resolvability, intrinsic randomness, and fixed-length source coding are considered. In these problems, several researchers have derived the first-order and the second-order achievability rates for general sources using the information spectrum methods. Although these formulas are general, their computations are quite hard. Hence, an attempt to address explicit computation problems of achievable rates is meaningful. In particular, for i.i.d. sources, the second-order achievable rates have earlier been determined simply by using the asymptotic normality. In this paper, we consider mixed sources of two i.i.d. sources. The mixed source is a typical case of nonergodic sources and whose self-information does not have the asymptotic normality. Nonetheless, we can explicitly compute the second-order achievable rates for these sources on the basis of two-peak asymptotic normality. In addition, extensions of our results to more general mixed sources, such as a mixture of countably infinite i.i.d. sources or Markov sources, and a continuous mixture of i.i.d. sources, are considered.

Second-order Slepian-Wolf coding theorems for non-mixed and mixed sources

The second-order achievable rate region in Slepian-Wolf source coding systems is investigated. The concept of second-order achievable rates, which enables us to make a finer evaluation of achievable rates, has already been introduced and analyzed for general sources in the single-user source coding problem. Accordingly, in this paper, we first define the second-order achievable rate region for the Slepian-Wolf coding system and establish the source coding theorem for general sources in the second-order sense. Moreover, we compute the explicit second-order achievable rate region for i.i.d. correlated sources with countably infinite alphabets and mixed correlated sources, respectively, using the relevant asymptotic normality.

A Note on the ε-Overflow Probability of Lossless Codes

In this letter, we generalize the achievability of variable-length coding from two viewpoints. One is the definition of an overflow probability, and the other is the definition of an achievability. We define the overflow probability as the probability of codeword length, not per symbol, is larger than ηn and we introduce the e-achievability of variable-length codes that implies an existence of a code for the source under the condition that the overflow probability is smaller than or equal to e. Then we show that the e-achievability of variable-length codes is essentially equivalent to the e-achievability of fixed-length codes for general sources. Moreover by using above results, we show the condition of e-achievability for some restricted sources given e.

First- and Second-Order Coding Theorems for Mixed Memoryless Channels With General Mixture

This paper investigates the first- and second-order maximum achievable rates of codes with/without cost constraints for mixed channels whose channel law is characterized by a general mixture of (at most) uncountably many stationary and memoryless discrete channels. These channels are referred to as mixed memoryless channels with general mixture and include the class of mixed memoryless channels of finitely or countably memoryless channels as a special case. For the mixed memoryless channels with general mixture, the first-order coding theorem which gives a formula for the

Information spectrum approach to fixed-length lossy source coding problem with some excess distortion probability

\varepsilon

Single-letter characterization of epsilon-capacity for mixed memoryless channels

-capacity is established, and then a direct part of the second-order coding theorem is provided. A subclass of mixed memoryless channels whose component channels can be ordered according to their capacity is introduced, and the first- and second-order coding theorems are established. It is shown that the established formulas reduce to several known formulas for restricted scenarios.

Variable-length coding with epsilon-fidelity criteria for general sources

This paper deals with a fixed-length lossy source coding problem with some excess distortion probability called the source coding problem with ε-fidelity criterion. In this problem, the rate-distortion function and the distortion-rate function have already been characterized for i.i.d. sources with an additive distortion measure in the sense of first-order and second-order coding rates. However, general formulas for these functions have not been revealed up to present. Hence, in this paper we derive general formulas for the rate-distortion function and the distortion-rate function with the ε-fidelity criterion in both of the first-order and second-order cases. A relationship between our general formulas and previous results are also discussed.

Variable-Length Coding with Cost Allowing Non-Vanishing Error Probability

For the class of mixed channels decomposed into stationary memoryless channels, single-letter characterizations of the ε-capacity have not been known except for restricted classes of channels such as the regular decomposable channel introduced by Winkelbauer. This paper gives single-letter characterizations of ε-capacity for mixed channels decomposed into at most countably many memoryless channels with a finite input alphabet with/without cost constraints. It is shown that the given characterization reduces to the one for the channel capacity given by Ahlswede when ε is zero. Some properties of the function of the ε-capacity are analyzed.

Variable-length lossy source coding allowing some probability of union of overflow and excess distortion

This paper addresses two problems of variablelength coding with a fidelity criterion for general sources, in which either the probability of codeword length overflow or that of excess distortion for a given threshold is allowed up to ε. In each problem, a general formula for the achievable region of coding rates and distortion levels is established via information spectrum methods. It is shown that there is a tight connection between the two problems, and the achievable regions are the same under a mild condition on the distortion measure. As a consequence, it turns out that superficially different general formulas for the two coding problems coincide with each other.

First- and second-order coding theorems for mixed memoryless channels with general mixture

We derive a general formula of the minimum achievable rate for fixed-to-variable length coding with a regular cost function by allowing the error probability up to a constant ε. For a fixed-to-variable length code, we call the set of source sequences that can be decoded without error the dominant set of source sequences. For any two regular cost functions, it is revealed that the dominant set of source sequences for a code attaining the minimum achievable rate with a cost function is also the dominant set for a code attaining the minimum achievable rate with the other cost function. We also give a general formula of the second-order minimum achievable rate.