Ali Mohammad-Djafari

A Matlab Program to Calculate the Maximum Entropy Distributions

The classical maximum entropy (ME) problem consists of determining a probability distribution function (pdf) from a finite set of expectations μ n = E {o n (x)} of known functions o; n (x), n = 0,…, N. The solution depends on N + 1 Lagrange multipliers which are determined by solving the set of nonlinear equations formed by the N data constraints and the normalization constraint. In this short communication we give three Matlab programs to calculate these Lagrange multipliers. The first considers the case where o n (x) can be any functions. The second considers the special case where o n (x) = x n , n = 0,..., N. In this case the µ n are the geometrical moments of p(x). The third considers the special case where o n (x) = exp(−jnωx), n = 0,..., N. In this case the µ n are the trigonometrical moments (Fourier components) of p(x). We give also some examples to illustrate the usefullness of these programs.

Joint estimation of parameters and hyperparameters in a Bayesian approach of solving inverse problems

We propose a joint estimation of the parameters and hyperparameters (the parameters of the prior law) when a Bayesian approach with maximum entropy (ME) priors is used to solve the inverse problems which arise in signal and image reconstruction and restoration problems. In particular we propose two methods: one based on the expectation maximization (EM) algorithm who aims to find the marginalized MAP (MMAP) estimate and the second based on a joint MAP estimation (JMAP). We discuss and compare these methods and give some simulation results in image restoration to show the relative performances of the proposed methods.

On the estimation of hyperparameters in Bayesian approach of solving inverse problems

The author proposes a new view on the estimation of hyperparameters (the parameters of the prior law) when a Bayesian approach with maximum entropy (ME) priors is used to solve the inverse problems which arise in signal and image reconstruction and restoration problems. In particular, he compares two methods; the expectation maximization (EM) algorithm which aims to find the marginalized maximum likelihood (MML) estimate and the generalized maximum likelihood (GML). Some simulation results with application in image restoration are provided to show the performance of the GML method. The convergence of the present implementation of the GML method depends essentially on the initialization of the hyperparameters and the image. If one starts with good initial values, the GML works satisfactorily.<<ETX>>

Probabilistic Methods for Data Fusion

The main object of this paper is to show how we can use classical probabilistic methods such as Maximum Entropy (ME), maximum likelihood (ML) and/or Bayesian (BAYES) approaches to do microscopic and macroscopic data fusion. Actually ME can be used to assign a probability law to an unknown quantity when we have macroscopic data (expectations) on it. ML can be used to estimate the parameters of a probability law when we have microscopic data (direct observation). BAYES can be used to update a prior probability law when we have microscopic data through the likelihood. When we have both microscopic and macroscopic data we can use first ME to assign a prior and then use BAYES to update it to the posterior law thus doing the desired data fusion. However, in practical data fusion applications, we may still need some engineering feeling to propose realistic data fusion solutions. Some simple examples in sensor data fusion and image reconstruction using different kind of data are presented to illustrate these ideas.

Inversion of large-support ill-conditioned linear operators using a Markov model with a line process

We propose a method for the reconstruction of an image, only partially observed through a linear integral operator. As such an inverse problem is ill-posed, prior information must be introduced. We consider the case of a compound Markov random field with a non-interacting line process. In order to maximise the posterior likelihood function, we propose an extension of the graduated non convexity principle pioneered by Blake and Zisserman (1987) which allows its use for ill-posed linear inverse problems. We discuss the role of the observation scale and some aspects of the implemented algorithm. Finally, we present an application of the method to a diffraction tomography imaging problem.<<ETX>>

A SCALE INVARIANT BAYESIAN METHOD TO SOLVE LINEAR INVERSE PROBLEMS

In this paper we propose a new Bayesian estimation method to solve linear inverse problems in signal and image restoration and reconstruction problems which has the property to be scale invariant. In general, Bayesian estimators are nonlinear functions of the observed data. The only exception is the Gaussian case. When dealing with linear inverse problems the linearity is sometimes a too strong property, while scale invariance often remains a desirable property. As everybody knows one of the main difficulties with using the Bayesian approach in real applications is the assignment of the direct (prior) probability laws before applying the Bayes’ rule. We discuss here how to choose prior laws to obtain scale invariant Bayesian estimators. In this paper we discuss and propose a family of generalized exponential probability distributions functions for the direct probabilities (the prior p(x) and the likelihood p(y|x)), for which the posterior p(x|y), and, consequently, the main posterior estimators are scale invariant. Among many properties, generalized exponentials can be considered as the maximum entropy probability distributions subject to the knowledge of a finite set of expectation values of some known functions.

Scale Invariant Markov Models for Bayesian Inversion of Linear Inverse Problems

In a Bayesian approach for solving linear inverse problems one needs to specify the prior laws for calculation of the posterior law. A cost function can also be defined in order to have a common tool for various Bayesian estimators which depend on the data and the hyperparameters. The Gaussian case excepted, these estimators are not linear and so depend on the scale of the measurements. In this paper a weaker property than linearity is imposed on the Bayesian estimator, namely the scale invariance property (SIP).

Eddy current tomography using a binary Markov model

Abstract The non-destructive evaluation (NDE) problem we treat is the testing of a globally homogeneous conductive medium for anomalies such as cracks and notches. The medium is illuminated with a monochromatic electric field; the anomalies induce eddy currents and they modify the total field which can be measured. The tomographic approach, aimed to draw up an image of the medium, is recent in this area. It corresponds to an extremely difficult ill-posed inverse problem and its resolution needs the use of pertinent prior information. The considered anomalies can be represented using images whose pixels can only take the values 0 and 1. Our main contribution lies in the regularization of a large-support ill-posed observation operator using a locally constant binary image Markov random field. The resulting high-dimensional combinatorial optimization problem is tedious: neither exact resolution nor simulated annealing are feasible. Instead, we establish an equivalent continuous-valued optimization problem. A nearly optimal solution is then calculated using a graduated non-convexity algorithm adapted for this purpose. The proposed inversion technique surpasses the particular NDE problem and can be applied whenever a binary image is observed using a linear system and corrupted by Gaussian noise.

Hierarchical Markov modeling for fusion of X ray radiographic data and anatomical data in computed tomography

We consider an X ray computed tomography (CT) image reconstruction problem where we want to include some geometrical information coming from an anatomical atlas and propose new methods based on hierarchical Markov modeling and a Bayesian estimation approach. We use two kinds of anatomical information: partial knowledge of values in some regions and partial knowledge of the edges of some other regions. We show the advantages of using such information on increasing the quality of reconstructions. We also show some results to analyze the effects of some errors in anatomical data on the reconstructed results.

Shape Reconstruction in X-Ray Tomography from a Small Number of Projections Using Deformable Models

X-ray tomographic image reconstruction consists of determining an object function from its projections. In many applications such as nondestructive testing, we look for a fault region (air) in a homogeneous, known background (metal). The image reconstruction problem then becomes the determination of the shape of the default rexad gion. Two approaches can be used: modeling the image as a binary Markov random field and estimating the pixels of the image, or modeling the shape of the fault and estimating it directly from the projections. In this work we model the fault shape by a deformable polygonal disc or a deformable polyhedral volume and propose a new method for directly estimating the coordinates of its vertices from a very limited number of its projections. The basic idea is not new, but in other competing methods, in general, the fault shape is modeled by a small number of parameters (polygonal shapes with very small number of vertices, snakes and deformable templates) and these parameters are estimated either by least squares or by maximum likelihood methods. We propose modeling the shape of the fault region by a polygon with a large number of vertices, allowing modeling of nearly any shape and estimation of its vertices coordinates directly from the projections by defining the solution as the minimizer of an appropriate regularized criterion. This formulation can also be interpreted as a maximum a posteriori (MAP) estimate in a Bayesian estimation framework. To optimize this criterion we use either a simulated anxad nealing or a special purpose deterministic algorithm based on iterated conditional modes (leM). The simulated results are very encouraging, especially when the number and the angles of projections are very limited.