Kenric P. Nelson

Nonlinear statistical coupling

By considering a nonlinear combination of the probabilities of a system, a physical interpretation of Tsallis statistics as representing the nonlinear coupling or decoupling of statistical states is proposed. The escort probability is interpreted as the coupled probability, with Q=1−q defined as the degree of nonlinear coupling between the statistical states. Positive values of Q have coupled statistical states, a larger entropy metric, and a maximum coupled-entropy distribution of compact-support coupled-Gaussians. Negative values of Q have decoupled statistical states and for −2<Q<0 a maximum coupled-entropy distribution of heavy-tail coupled-Gaussians. The conjugate transformation between the heavy-tail and compact-support domains is shown to be Qˆ=−2Q2+Q for coupled-Gaussian distributions. This conjugate relationship has been used to extend the generalized Fourier transform to the compact-support domain and to define a scale-invariant correlation structure with heavy-tail limit distribution. In the present paper, we show that the conjugate is a mapping between the source of nonlinearity in non-stationary stochastic processes and the nonlinear coupling which defines the coupled-Gaussian limit distribution. The effects of additive and multiplicative noise are shown to be separable into the coupled-variance and the coupling parameter Q, providing further evidence of the importance of the generalized moments.

A Risk Profile for Information Fusion Algorithms

E.T. Jaynes, originator of the maximum entropy interpretation of statistical mechanics, emphasized that there is an inevitable trade-off between the conflicting requirements of robustness and accuracy for any inferencing algorithm. This is because robustness requires discarding of information in order to reduce the sensitivity to outliers. The principal of nonlinear statistical coupling, which is an interpretation of the Tsallis entropy generalization, can be used to quantify this trade-off. The coupled-surprisal, -lnκ(p)≡-(pκ-1)/κ , is a generalization of Shannon surprisal or the logarithmic scoring rule, given a forecast p of a true event by an inferencing algorithm. The coupling parameter κ=1-q, where q is the Tsallis entropy index, is the degree of nonlinear coupling between statistical states. Positive (negative) values of nonlinear coupling decrease (increase) the surprisal information metric and thereby biases the risk in favor of decisive (robust) algorithms relative to the Shannon surprisal (κ=0). We show that translating the average coupled-surprisal to an effective probability is equivalent to using the generalized mean of the true event probabilities as a scoring rule. The metric is used to assess the robustness, accuracy, and decisiveness of a fusion algorithm. We use a two-parameter fusion algorithm to combine input probabilities from N sources. The generalized mean parameter ‘alpha’ varies the degree of smoothing and raising to a power Νβ with β between 0 and 1 provides a model of correlation.

A definition of the coupled-product for multivariate coupled-exponentials

The coupled-product and coupled-exponential of the generalized calculus of nonextensive statistical mechanics are defined for multivariate functions. The nonlinear statistical coupling is indexed such that κd=κ/1+dκ, where d is the dimension of the argument of the multivariate coupled-exponential. The coupled-Gaussian distribution is defined such that the argument of the coupled-exponential depends on the coupled-moments but not the coupling parameter. The multivariate version of the coupled-product is defined such that the output dimensions are the sum of the input dimensions. This enables construction of the multivariate coupled-Gaussian from univariate coupled-Gaussians. The resulting construction forms a model of coupling between distributions, generalizing the product of independent Gaussians.

Probabilistic graphs using coupled random variables

Neural network design has utilized flexible nonlinear processes which can mimic biological systems, but has suffered from a lack of traceability in the resulting network. Graphical probabilistic models ground network design in probabilistic reasoning, but the restrictions reduce the expressive capability of each node making network designs complex. The ability to model coupled random variables using the calculus of nonextensive statistical mechanics provides a neural node design incorporating nonlinear coupling between input states while maintaining the rigor of probabilistic reasoning. A generalization of Bayes rule using the coupled product enables a single node to model correlation between hundreds of random variables. A coupled Markov random field is designed for the inferencing and classification of UCI’s MLR ‘Multiple Features Data Set’ such that thousands of linear correlation parameters can be replaced with a single coupling parameter with just a (3%, 4%) reduction in (classification, inference) performance.

Use of the geometric mean as a statistic for the scale of the coupled Gaussian distributions

Abstract The geometric mean is shown to be an appropriate statistic for the scale of a heavy-tailed coupled Gaussian distribution or equivalently the Student’s t distribution. The coupled Gaussian is a member of a family of distributions parameterized by the nonlinear statistical coupling which is the reciprocal of the degree of freedom and is proportional to fluctuations in the inverse scale of the Gaussian. Existing estimators of the scale of the coupled Gaussian have relied on estimates of the full distribution, and they suffer from problems related to outliers in heavy-tailed distributions. In this paper, the scale of a coupled Gaussian is proven to be equal to the product of the generalized mean and the square root of the coupling. From our numerical computations of the scales of coupled Gaussians using the generalized mean of random samples, it is indicated that only samples from a Cauchy distribution (with coupling parameter one) form an unbiased estimate with diminishing variance for large samples. Nevertheless, we also prove that the scale is a function of the geometric mean, the coupling term and a harmonic number. Numerical experiments show that this estimator is unbiased with diminishing variance for large samples for a broad range of coupling values.

Fractional generalizations of Zakai equation and some solution methods

Abstract The paper discusses fractional generalizations of Zakai equations arising in filtering problems. The derivation of the fractional Zakai equation, existence and uniqueness of its solution, as well as some methods of solution to the fractional filtering problem, including fractional version of the particle flow method, are presented.

Assessing Probabilistic Inference by Comparing the Generalized Mean of the Model and Source Probabilities

An approach to the assessment of probabilistic inference is described which quantifies the performance on the probability scale. From both information and Bayesian theory, the central tendency of an inference is proven to be the geometric mean of the probabilities reported for the actual outcome and is referred to as the “Accuracy”. Upper and lower error bars on the accuracy are provided by the arithmetic mean and the −2/3 mean. The arithmetic is called the “Decisiveness” due to its similarity with the cost of a decision and the −2/3 mean is called the “Robustness”, due to its sensitivity to outlier errors. Visualization of inference performance is facilitated by plotting the reported model probabilities versus the histogram calculated source probabilities. The visualization of the calibration between model and source is summarized on both axes by the arithmetic, geometric, and −2/3 means. From information theory, the performance of the inference is related to the cross-entropy between the model and source distribution. Just as cross-entropy is the sum of the entropy and the divergence; the accuracy of a model can be decomposed into a component due to the source uncertainty and the divergence between the source and model. Translated to the probability domain these quantities are plotted as the average model probability versus the average source probability. The divergence probability is the average model probability divided by the average source probability. When an inference is over/under-confident, the arithmetic mean of the model increases/decreases, while the −2/3 mean decreases/increases, respectively.

Fractional generalizations of filtering problems and their associated fractional Zakai equations

In this paper we discuss fractional generalizations of the filtering problem. The ”fractional” nature comes from time-changed state or observation processes, basic ingredients of the filtering problem. The mathematical feature of the fractional filtering problem emerges as the Riemann-Liouville or Caputo-Djrbashian fractional derivative in the associated Zakai equation. We discuss fractional generalizations of the nonlinear filtering problem whose state and observation processes are driven by time-changed Brownian motion or/and Lévy process.