Pouya D. Tafti

An Introduction to Sparse Stochastic Processes

1. Introduction 2. Roadmap to the book 3. Mathematical context and background 4. Continuous-domain innovation models 5. Operators and their inverses 6. Splines and wavelets 7. Sparse stochastic processes 8. Sparse representations 9. Infinite divisibility and transform-domain statistics 10. Recovery of sparse signals 11. Wavelet-domain methods 12. Conclusion Appendix A. Singular integrals Appendix B. Positive definiteness Appendix C. Special functions.

A Unified Formulation of Gaussian Versus Sparse Stochastic Processes—Part I: Continuous-Domain Theory

We introduce a general distributional framework that results in a unifying description and characterization of a rich variety of continuous-time stochastic processes. The cornerstone of our approach is an innovation model that is driven by some generalized white noise process, which may be Gaussian or not (e.g., Laplace, impulsive Poisson, or alpha stable). This allows for a conceptual decoupling between the correlation properties of the process, which are imposed by the whitening operator L, and its sparsity pattern, which is determined by the type of noise excitation. The latter is fully specified by a Lévy measure. We show that the range of admissible innovation behavior varies between the purely Gaussian and super-sparse extremes. We prove that the corresponding generalized stochastic processes are well-defined mathematically provided that the (adjoint) inverse of the whitening operator satisfies some Lp bound for p ≥ 1. We present a novel operator-based method that yields an explicit characterization of all Lévy-driven processes that are solutions of constant-coefficient stochastic differential equations. When the underlying system is stable, we recover the family of stationary continuous-time autoregressive moving average processes (CARMA), including the Gaussian ones. The approach remains valid when the system is unstable and leads to the identification of potentially useful generalizations of the Lévy processes, which are sparse and non-stationary. Finally, we show that these processes admit a sparse representation in some matched wavelet domain and provide a full characterization of their transform-domain statistics.

Stochastic Models for Sparse and Piecewise-Smooth Signals

We introduce an extended family of continuous-domain stochastic models for sparse, piecewise-smooth signals. These are specified as solutions of stochastic differential equations, or, equivalently, in terms of a suitable innovation model; the latter is analogous conceptually to the classical interpretation of a Gaussian stationary process as filtered white noise. The two specific features of our approach are 1) signal generation is driven by a random stream of Dirac impulses (Poisson noise) instead of Gaussian white noise, and 2) the class of admissible whitening operators is considerably larger than what is allowed in the conventional theory of stationary processes. We provide a complete characterization of these finite-rate-of-innovation signals within Gelfands framework of generalized stochastic processes. We then focus on the class of scale-invariant whitening operators which correspond to unstable systems. We show that these can be solved by introducing proper boundary conditions, which leads to the specification of random, spline-type signals that are piecewise-smooth. These processes are the Poisson counterpart of fractional Brownian motion; they are nonstationary and have the same 1/ω-type spectral signature. We prove that the generalized Poisson processes have a sparse representation in a wavelet-like basis subject to some mild matching condition. We also present a limit example of sparse process that yields a MAP signal estimator that is equivalent to the popular TV-denoising algorithm.

A Unified Formulation of Gaussian Versus Sparse Stochastic Processes—Part II: Discrete-Domain Theory

This paper is devoted to the characterization of an extended family of continuous-time autoregressive moving average (CARMA) processes that are solutions of stochastic differential equations driven by white Lévy innovations. These are completely specified by: 1) a set of poles and zeros that fixes their correlation structure and 2) a canonical infinitely divisible probability distribution that controls their degree of sparsity (with the Gaussian model corresponding to the least sparse scenario). The generalized CARMA processes are either stationary or nonstationary, depending on the location of the poles in the complex plane. The most basic nonstationary representatives (with a single pole at the origin) are the Lévy processes, which are the non-Gaussian counterparts of Brownian motion. We focus on the general analog-to-discrete conversion problem and introduce a novel spline-based formalism that greatly simplifies the derivation of the correlation properties and joint probability distributions of the discrete versions of these processes. We also rely on the concept of generalized increment process, which suppresses all long range dependencies, to specify an equivalent discrete-domain innovation model. A crucial ingredient is the existence of a minimally supported function associated with the whitening operator L; this B-spline, which is fundamental to our formulation, appears in most of our formulas, both at the level of the correlation and the characteristic function. We make use of these discrete-domain results to numerically generate illustrative examples of sparse signals that are consistent with the continuous-domain model.

Invariances, Laplacian-Like Wavelet Bases, and the Whitening of Fractal Processes

In this contribution, we study the notion of affine invariance (specifically, invariance to the shifting, scaling, and rotation of the coordinate system) as a starting point for the development of mathematical tools and approaches useful in the characterization and analysis of multivariate fractional Brownian motion (fBm) fields. In particular, using a rigorous and powerful distribution theoretic formulation, we extend previous results of Blu and Unser (2006) to the multivariate case, showing that polyharmonic splines and fBm processes can be seen as the (deterministic vs stochastic) solutions to an identical fractional partial differential equation that involves a fractional Laplacian operator. We then show that wavelets derived from polyharmonic splines have a behavior similar to the fractional Laplacian, which also turns out to be the whitening operator for fBm fields. This fact allows us to study the probabilistic properties of the wavelet transform coefficients of fBm-like processes, leading for instance to ways of estimating the Hurst exponent of a multiparameter process from its wavelet transform coefficients. We provide theoretical and experimental verification of these results. To complement the toolbox available for multiresolution processing of stochastic fractals, we also introduce an extended family of multidimensional multiresolution spaces for a large class of (separable and nonseparable) lattices of arbitrary dimensionality.

Fractional Brownian Vector Fields

This work puts forward an extended definition of vector fractional Brownian motion (fBm) using a distribution theoretic formulation in the spirit of Gel′fand and Vilenkins stochastic analysis. We introduce random vector fields that share the statistical invariances of standard vector fBm (self-similarity and rotation invariance) but which, in contrast, have dependent vector components in the general case. These random vector fields result from the transformation of white noise by a special operator whose invariance properties the random field inherits. The said operator combines an inverse fractional Laplacian with a Helmholtz-like decomposition and weighted recombination. Classical fBms can be obtained by balancing the weights of the Helmholtz components. The introduced random fields exhibit several important properties that are discussed in this paper. In addition, the proposed scheme yields a natural extension of the definition to Hurst exponents greater than one.

Variational enhancement and denoising of flow field images

In this work we propose a variational reconstruction algorithm for enhancement and denoising of flow fields that is reminiscent of total-variation (TV) regularization used in image processing, but which also takes into account physical properties of flow such as curl and divergence. We point out the invariance properties of the scheme with respect to transformations of the coordinate system such as shifts, rotations, and changes of scale. To demonstrate the utility of the reconstruction method, we use it first to denoise a simulated phantom where the scheme is found to be superior to its quadratic (L2) variant both in terms of SNR and in preservation of discontinuities. We then use the scheme to enhance the quality of pathline visualizations in an application to 4D (3D+time) flow-sensitive magnetic resonance imaging of blood flow in the aorta.

On Regularized Reconstruction of Vector Fields

In this paper, we give a general characterization of regularization functionals for vector field reconstruction, based on the requirement that the said functionals satisfy certain geometric invariance properties with respect to transformations of the coordinate system. In preparation for our general result, we also address some commonalities of invariant regularization in scalar and vector settings, and give a complete account of invariant regularization for scalar fields, before focusing on their main points of difference, which lead to a distinct class of regularization operators in the vector case. Finally, as an illustration of potential, we formulate and compare quadratic (L2) and total-variation-type (L1) regularized denoising of vector fields in the proposed framework.

Spatio-temporal regularization of flow-fields

We introduce a novel variational framework for the regularized reconstruction of time-resolved volumetric flow fields. Our objective functional takes the physical characteristics of the underlying flow into account in both the spatial and the temporal domains. For an efficient minimization of the objective functional, we apply a proximal-splitting algorithm and perform parallel computations. To demonstrate the utility of our variational method, we first denoise a simulated flow-field in the human aorta and show that our method outperforms spatial-only regularization in terms of signal-to-noise ratio (SNR). We then apply the scheme to a real 3D+time phase-contrast MRI dataset and obtain high-quality visualizations.

A dual algorithm for L 1 -regularized reconstruction of vector fields

Recent advances in vector-field imaging have brought to the forefront the need for efficient denoising and reconstruction algorithms that take the physical properties of vector fields into account and can be applied to large volumes of data. With these requirements in mind, we propose a computationally efficient algorithm for variational de-noising and reconstruction of vector fields. Our variational objective combines rotation- and scale-invariant regularization functionals that permit one to tune the algorithm to the physical characteristics of the underlying phenomenon. In addition, these regularization terms involve L1 norms in the spirit of total-variation (TV) regularization, which, as in the scalar case, leads to better preservation of discontinuities and superior SNR performance compared to its quadratic alternative. Some experimental results are provided to illustrate and verify the proposed scheme.