Petros Maragos

Energy separation in signal modulations with application to speech analysis

An efficient solution to the fundamental problem of estimating the time-varying amplitude envelope and instantaneous frequency of a real-valued signal that has both an AM and FM structure is provided. Nonlinear combinations of instantaneous signal outputs from the energy operator are used to separate its output energy product into its AM and FM components. The theoretical analysis is done first for continuous-time signals. Then several efficient algorithms are developed and compared for estimating the amplitude envelope and instantaneous frequency of discrete-time AM-FM signals. These energy separation algorithms are used to search for modulations in speech resonances, which are modeled using AM-FM signals to account for time-varying amplitude envelopes and instantaneous frequencies. The experimental results provide evidence that bandpass-filtered speech signals around speech formants contain amplitude and frequency modulations within a pitch period. >

Pattern spectrum and multiscale shape representation

The results of a study on multiscale shape description, smoothing and representation are reported. Multiscale nonlinear smoothing filters are first developed, using morphological opening and closings. G. Matheron (1975) used openings and closings to obtain probabilistic size distributions of Euclidean-space sets (continuous binary images). These distributions are used to develop a concept of pattern spectrum (a shape-size descriptor). A pattern spectrum is introduced for continuous graytone images and arbitrary multilevel signals, as well as for discrete images, by developing a discrete-size family of patterns. Large jumps in the pattern spectrum at a certain scale indicate the existence of major (protruding or intruding) substructures of the signal at the scale. An entropy-like shape-size complexity measure is also developed based on the pattern spectrum. For shape representation, a reduced morphological skeleton transform is introduced for discrete binary and graytone images. This transform is a sequence of skeleton components (sparse images) which represent the original shape at various scales. It is shown that the partially reconstructed images from the inverse transform on subsequences of skeleton components are the openings of the image at a scale determined by the number of eliminated components; in addition, two-way correspondences are established among the degree of shape smoothing via multiscale openings or closings, the pattern spectrum zero values, and the elimination or nonexistence of skeleton components at certain scales. >

Morphological filters--Part I: Their set-theoretic analysis and relations to linear shift-invariant filters

This paper examines the set-theoretic interpretation of morphological filters in the framework of mathematical morphology and introduces the representation of classical linear filters in terms of morphological correlations, which involve supremum/infimum operations and additions. Binary signals are classified as sets, and multilevel signals as functions. Two set-theoretic representations of signals are reviewed. Filters are classified as set-processing (SP) or function-processing (FP). Conditions are provided for certain FP filters that pass binary signals to commute with signal thresholding because then they can be analyzed and implemented as SP filters. The basic morphological operations of set erosion, dilation, opening, and closing are related to Minkowski set operations and are used to construct FP morphological filters. Emphasis is then given to analytically and geometrically quantifying the similarities and differences between morphological filtering of signals by sets and functions; the latter case allows the definition of morphological convolutions and correlations. Toward this goal, various properties of FP morphological filters are also examined. Linear shift-invariant filters (due to their translation-invariance) are uniquely characterized by their kernel, which is a special collection of input signals. Increasing linear filters are represented as the supremum of erosions by their kernel functions. If the filters are also discrete and have a finite-extent impulse response, they can be represented as the supremum of erosions only by their minimal (with respect to a signal ordering) kernel functions. Stable linear filters can be represented as the sum of (at most) two weighted suprema of erosions. These results demonstrate the power of mathematical morphology as a unifying approach to both linear and nonlinear signal-shaping strategies.

Morphological filters--Part II: Their relations to median, order-statistic, and stack filters

This paper extends the theory of median, order-statistic (OS), and stack filters by using mathematical morphology to analyze them and by relating them to those morphological erosions, dilations, openings, closings, and open-closings that commute with thresholding. The max-min representation of OS filters is introduced by showing that any median or other OS filter is equal to a maximum of erosions (moving local minima) and also to a minimum of dilations (moving local maxima). Thus, OS filters can be computed by a closed formula that involves a max-min on prespecified sets of numbers and no sorting. Stack filters are established as the class of filters that are composed exactly of a finite number of max-min operations. The kernels of median, OS, and stack filters are collections of input signals that uniquely represent these filters due to their translation-invariance. The max-min functional definitions of these nonlinear iliters is shown to be equivalent to a maximum of erosions by minimal (with respect to a signal ordering) kernel elements, and also to a minimum of dilations by minimal kernel elements of dual filters. The representation of stack filters based on their minimal kernel elements is proven to be equivalent to their representation based on irreducible sum-of-products expressions of Boolean functions. It is also shown that median filtering (and its iterations) of any signal by convex 1-D windows is bounded below by openings and above by closings; a signal is a root (fixed point) of the median iff it is a root of both an opening and a closing; the open-closing and clos-opening yield median roots in one pass, suppress impulse noise similarly to the median, can discriminate between positive and negative noise impulses, and are computationally less complex than the median. Some similar results are obtained for 2-D median filtering.

On amplitude and frequency demodulation using energy operators

It is shown that the nonlinear energy-tracking signal operator Psi (x)=(dx/dt)/sup 2/-xd/sup 2/x/dt/sup 2/ and its discrete-time counterpart can estimate the AM and FM modulating signals. Specifically, Psi can approximately estimate the amplitude envelope of AM signals and the instantaneous frequency of FM signals. Bounds are derived for the approximation errors, which are negligible under general realistic conditions. These results, coupled with the simplicity of Psi , establish the usefulness of the energy operator for AM and FM signal demodulation. These ideas are then extended to a more general class of signals that are sine waves with a time-varying amplitude and frequency and thus contain both an AM and an FM component; for such signals it is shown that Psi can approximately track the product of their amplitude envelope and their instantaneous frequency. The theoretical analysis is done for both continuous- and discrete-time signals. >

Morphological skeleton representation and coding of binary images

This paper presents the results of a study on the use of morphological set operations to represent and encode a discrete binary image by parts of its skeleton, a thinned version of the image containing complete information about its shape and size. Using morphological erosions and openings, a finite image can be uniquely decomposed into a finite number of skeleton subsets and then the image can be exactly reconstructed by dilating the skeleton subsets. The morphological skeleton is shown to unify many previous approaches to skeletonization, and some of its theoretical properties are investigated. Fast algorithms that reduce the original quadratic complexity to linear are developed for skeleton decomposition and reconstruction. Partial reconstructions of the image are quantified through the omission of subsets of skeleton points. The concepts of a globally and locally minimal skeleton are introduced and fast algorithms are developed for obtaining minimal skeletons. For images containing blobs and large areas, the skeleton subsets are much thinner than the original image. Therefore, encoding of the skeleton information results in lower information rates than optimum block-Huffman or optimum runlength-Huffman coding of the original image. The highest level of image compression was obtained by using Elias coding of the skeleton.

Morphological systems for multidimensional signal processing

The basic theory and applications of a set-theoretic approach to image analysis called mathematical morphology are reviewed. The goals are to show how the concepts of mathematical morphology geometrical structure in signals to illuminate the ways that morphological systems can enrich the theory and applications of multidimensional signal processing. The topics covered include: applications to nonlinear filtering (morphological and rank-order filters, multiscale smoothing, morphological sampling, and morphological correlation); applications to image analysis (feature extraction, shape representation and description, size distributions, and fractals); and representation theorems, which shows how a large class of nonlinear and linear signal operators can be realized as a combination of simple morphological operations. >

Tutorial on advances in morphological image processing and analysis

This paper reviews some recent advances in the theory and applications of morphological image analysis. Regarding applications, we show how the morphological filters can be used to provide simple and systematic algorithms for image processing and analysis tasks as diverse as nonlinear image filtering, noise suppression, edge detection, region filling, skeletonization, coding, shape representation, smoothing, and recognition. Regarding theory, we summarize the representation of a large class of translation-invariant nonlinear filters (including morphological, median, order-statistic, and shape recognition filters) as a minimal combination of morphological erosions or dilations; these results provide new realizations of these filters and lead to a unified image algebra.

A representation theory for morphological image and signal processing

A unifying theory for many concepts and operations encountered in or related to morphological image and signal analysis is presented. The unification requires a set-theoretic methodology, where signals are modeled as sets, systems (signal transformations) are viewed as set mappings, and translational-invariant systems are uniquely characterized by special collections of input signals. This approach leads to a general representation theory, in which any translation-invariant, increasing, upper semicontinuous system can be presented exactly as a minimal nonlinear superposition of morphological erosions or dilations. The theory is used to analyze some special cases of image/signal analysis systems, such as morphological filters, median and order-statistic filters, linear filters, and shape recognition transforms. Although the developed theory is algebraic, its prototype operations are well suited for shape analysis; hence, the results also apply to systems that extract information about the geometrical structure of signals. >

A comparison of the energy operator and the Hilbert transform approach to signal and speech demodulation

Abstract The Hilbert transform together with Gabors analytic signal provides a standard linear integral approach to estimate the amplitude envelope and instantaneous frequency of signals with a combined amplitude modulation (AM) and frequency modulation (FM) structure. A recent alternative approach uses a nonlinear differential ‘energy’ operator to track the energy required to generate an AM-FM signal and separate it into amplitude and frequency components. In this paper, we compare these two fundamentally different approaches for demodulation of arbitrary signals and of speech resonances modeled by AM-Fm signals. The comparison is done from several viewpoints: magnitude of estimation errors, computational complexity, and adaptability to instantaneous signal changes. We also propose a refinement of the energy operator approach that uses simple binomial convolutions to smooth the energy signals. This smoothed energy operator is compared to the Hilbert transform on tracking modulations in speech vowel signals, band-pass filtered around their formants. The effects of pitch periodicity and band-pass filtering on both demodulation approaches are examined and an application to formant tracking is presented. The results provide strong evidence that the estimation errors of the smoothed energy operator approach are similar to that of the Hilbert transform approach for speech applications, but smaller for communication applications. In addition, the smoothed energy operator approach has smaller computational complexity and faster adaptation due to its instantaneous nature.