Robert A. Schowengerdt

A review and analysis of backpropagation neural networks for classification of remotely-sensed multi-spectral imagery

Abstract A literature survey and analysis of the use of neural networks for the classification of remotely-sensed multi-spectral imagery is presented. As part of a brief mathematical review, the backpropagation algorithm, which is the most common method of training multi-layer networks, is discussed with an emphasis on its application to pattern recognition. The analysis is divided into five aspects of neural network classification: (1) input data preprocessing, structure, and encoding, (2) output encoding and extraction of classes, (3) network architecture, (4) training algorithms, and (5) comparisons to conventional classifiers. The advantages of the neural network method over traditional classifiers are its nonparametric nature, arbitrary decision boundary capabilities, easy adaptation to different types of data and input structures, fuzzy output values that can enhance classification, and good generalization for use with multiple images. The disadvantages of the method are slow training time, inconsis...

Image reconstruction by parametric cubic convolution

Abstract A parametric implementation of cubic convolution image reconstruction is presented which is generally superior to the standard algorithm and which can be optimized to the frequency content of the image.

Modulation-transfer-function analysis for sampled image systems

Sampling generally causes the response of a digital imaging system to be locally shift-variant and not directly amenable to MTF analysis. However, this paper demonstrates that a meaningful system response can be calculated by averaging over an ensemble of point-source system inputs to yield an MTF which accounts for the combined effects of image formation, sampling, and image reconstruction. As an illustration, the MTF of the Landsat MSS system is analyzed to reveal an average effective IFOV which is significantly larger than the commonly accepted value, particularly in the along-track direction where undersampling contributes markedly to an MTF reduction and resultant increase in image blur.

Spatially Variant Contrast Enhancement Using Local Range Modification

Numerous adaptive contrast enhancement techniques have been developed to improve the visual utility of imagery. Most of these techniques involve the use of neighborhood operators, and some require manual input of scene-dependent functions to operate properly on different images. The local range modification (LRM) technique described in this paper does not involve neighborhood operators in the conventional sense of the word nor does it require design of scene-dependent functions, yet it yields results comparable to more computation-intensive techniques. In this paper, a unified description of the spatial filter-based adaptive contrast enhancement algorithms is presented. LRM is then described and qualitatively compared with the other algorithms in terms of efficiency, effectiveness, and artifact generation.

Image sampling, reconstruction, and the effect of sample-scene phasing

This paper is a 1-D analysis of the degradation caused by image sampling and interpolative reconstruction. The analysis includes the sample-scene phase as an explicit random parameter and provides a complete characterization of this image degradation as the sum of two terms: one term accounts for the mean effect of undersampling (aliasing) and nonideal reconstruction averaged over all sample-scene phases; the other term accounts for variations about this mean. The results of this paper have application to the design and performance analysis of image scanning, sampling, and reconstruction systems.

Lung tumor tracking in fluoroscopic video based on optical flow

Respiratory gating and tumor tracking for dynamic multileaf collimator delivery require accurate and real-time localization of the lung tumor position during treatment. Deriving tumor position from external surrogates such as abdominal surface motion may have large uncertainties due to the intra- and interfraction variations of the correlation between the external surrogates and internal tumor motion. Implanted fiducial markers can be used to track tumors fluoroscopically in real time with sufficient accuracy. However, it may not be a practical procedure when implanting fiducials bronchoscopically. In this work, a method is presented to track the lung tumor mass or relevant anatomic features projected in fluoroscopic images without implanted fiducial markers based on an optical flow algorithm. The algorithm generates the centroid position of the tracked target and ignores shape changes of the tumor mass shadow. The tracking starts with a segmented tumor projection in an initial image frame. Then, the optical flow between this and all incoming frames acquired during treatment delivery is computed as initial estimations of tumor centroid displacements. The tumor contour in the initial frame is transferred to the incoming frames based on the average of the motion vectors, and its positions in the incoming frames are determined by fine-tuning the contour positions using a template matching algorithm with a small search range. The tracking results were validated by comparing with clinician determined contours on each frame. The position difference in 95% of the frames was found to be less than 1.4 pixels (approximately 0.7 mm) in the best case and 2.8 pixels (approximately 1.4 mm) in the worst case for the five patients studied.

Airborne video registration and traffic-flow parameter estimation

This paper investigates airborne helicopter video for estimating traffic parameters. Roll, pitch, and yaw of the helicopter make the video unstable, difficult to view, and the derived parameters less accurate. To correct this, a frame-by-frame video-registration technique using a feature tracker to automatically determine control-point correspondences is proposed. This converts the spatio-temporal video into temporal information, thereby correcting for airborne platform motion and attitude errors. The registration is robust, with the residual jitter being less than a few pixels over hundreds of frames. A simple vehicle-detection scheme identifies vehicle locations in the video, which are then tracked by the feature tracker, enabling us to estimate average velocity, instantaneous velocity, and other parameters automatically to within 10% of manual measurements. The entire process of registration, detection, tracking, and estimation takes only a few seconds for each frame. A prototype multimedia geographic information system (GIS) is created as a visualization tool for viewing the registered video, other airborne or satellite imagery, and data pertaining to georeferenced locations within a base map.

On the estimation of spatial-spectral mixing with classifier likelihood functions

Abstract Conventional, hard classification algorithms that decide one class per pixel ignore the fact that many pixels in a remote sensing image represent a spatial average of spectral signatures from two or more surface categories. The mixing of signatures arises from the intrinsic, spatially-mixed nature of most natural land cover categories, the physical continuum that may exist between discrete category labels, resampling for geometric rectification, and by the spatial integration defined by the sensors point spread function. By allowing for multiple classes per pixel, each with a relative membership likelihood, soft classification algorithms have the potential to “unmix” the pixel data into the proportions of individual components. The potential and limitations of this approach are explored in this paper by empirical examples and analyses. A major conclusion is that the use of likelihood functions as estimators of mixing is valid for classes with high spectral signature separability, but problematic otherwise.

A Robust Technique for Precise Registration of Radar and Optical Satellite Images

Combining data from different sensors for visual or classification analysis is a common task in remote sensing. The first step is normally to register the images which may be considered geometric integration; the accuracy of this step is important to create a valuable final hybrid image. This paper addresses geometric integration and introduces a new method for automatically registering two dissimilar images, such as, a radar image and an optical image with high accuracy. Pre-registration of the two images to within a specified tolerance is required. In our examples, this tolerance is up to 17 pixels (at the scale of the higher resolution image) and may be achieved by, for example, visually located control points. The described approach then uses large-scale edge gradient contours in a process that automatically locates candidate control points on the contours. The points are selected using a cost function that measures the degree of match between all possible pairs of points. Numerous control points (typically around 50 pairs) are found from matched pairs of gradient contours and used in a global, rubber sheet, polynomial warp to refine the registration. This approach is applied to register a Synthetic Aperture Radar (SAR) image (ERS2, 12.5 m pixels) and a Thematic Mapper (TM) optical image (Landsat-5, 28.5 m pixels) automatically. Several examples with different scene content are shown to validate the approach and discussed in terms of residual registration error and processing efficiency.

A deformable lung tumor tracking method in fluoroscopic video using active shape models: a feasibility study

A dynamic multi-leaf collimator (DMLC) can be used to track a moving target during radiotherapy. One of the major benefits for DMLC tumor tracking is that, in addition to the compensation for tumor translational motion, DMLC can also change the aperture shape to conform to a deforming tumor projection in the beams eye view. This paper presents a method that can track a deforming lung tumor in fluoroscopic video using active shape models (ASM) (Cootes et al 1995 Comput. Vis. Image Underst. 61 38-59). The method was evaluated by comparing tracking results against tumor projection contours manually edited by an expert observer. The evaluation shows the feasibility of using this method for precise tracking of lung tumors with deformation, which is important for DMLC-based real-time tumor tracking.