Giovanni B. Marchisio

Learning bayesian classifiers for scene classification with a visual grammar

A challenging problem in image content extraction and classification is building a system that automatically learns high-level semantic interpretations of images. We describe a Bayesian framework for a visual grammar that aims to reduce the gap between low-level features and high-level user semantics. Our approach includes modeling image pixels using automatic fusion of their spectral, textural, and other ancillary attributes; segmentation of image regions using an iterative split-and-merge algorithm; and representing scenes by decomposing them into prototype regions and modeling the interactions between these regions in terms of their spatial relationships. Naive Bayes classifiers are used in the learning of models for region segmentation and classification using positive and negative examples for user-defined semantic land cover labels. The system also automatically learns representative region groups that can distinguish different scenes and builds visual grammar models. Experiments using Landsat scenes show that the visual grammar enables creation of high-level classes that cannot be modeled by individual pixels or regions. Furthermore, learning of the classifiers requires only a few training examples.

On the relative predictive value of the new spectral bands in the WorldWiew-2 sensor

We apply a comparative data mining framework to the multispectral classification of WorldView-2 (WV2) imagery. Our goal is two-fold. First, we want to identify land covers for which the combination of extended spectral coverage and high spatial resolution provide a distinctive advantage in classification accuracy. Second, we perform predictor analyses to determine which combinations of bands are more effective in resolving individual targets. This experimental approach provides a basis for building a spectral atlas that can offer guidance on the optimal combination of WV2 spectral bands for different application areas.

VisiMine: interactive mining in image databases

We describe VisiMine, a system for data mining and statistical analysis of large collections of remotely sensed images.

Image information mining utilizing hierarchical segmentation

The hierarchical segmentation (HSEG) algorithm is an approach for producing high quality, hierarchically related image segmentations. The VisiMine image information mining system utilizes clustering and segmentation algorithms for reducing visual information in multispectral images to a manageable size. The project discussed herein seeks to enhance the VisiMine system through incorporating hierarchical segmentations from HSEG into the VisiMine system.

Extracting statistical data frames from text

We present a framework that bridges the gap between natural language processing (NLP) and text mining. Central to this is a new approach to text parameterization that captures many interesting attributes of text usually ignored by standard indices, like the term-document matrix. By storing NLP tags, the new index supports a higher degree of knowledge discovery and pattern finding from text. The index is relatively compact, enabling dynamic search of arbitrary relationships and events in large document collections. We can export search results in formats and data structures that are transparent to statistical analysis tools like S-PLUSID®. In a number of experiments, we demonstrate how this framework can turn mountains of unstructured information into informative statistical graphs.

Interactive training of advanced classifiers for mining remote sensing image archives

Advances in satellite technology and availability of downloaded images constantly increase the sizes of remote sensing image archives. Automatic content extraction, classification and content-based retrieval have become highly desired goals for the development of intelligent remote sensing databases. The common approach for mining these databases uses rules created by analysts. However, incorporating GIS information and human expert knowledge with digital image processing improves remote sensing image analysis. We developed a system that uses decision tree classifiers for interactive learning of land cover models and mining of image archives. Decision trees provide a promising solution for this problem because they can operate on both numerical (continuous) and categorical (discrete) data sources, and they do not require any assumptions about neither the distributions nor the independence of attribute values. This is especially important for the fusion of measurements from different sources like spectral data, DEM data and other ancillary GIS data. Furthermore, using surrogate splits provides the capability of dealing with missing data during both training and classification, and enables handling instrument malfunctions or the cases where one or more measurements do not exist for some locations. Quantitative and qualitative performance evaluation showed that decision trees provide powerful tools for modeling both pixel and region contents of images and mining of remote sensing image archives.

Land Cover Classification with Multi-Sensor Fusion of Partly Missing Data

We describe a system that uses decision tree-based tools for seamless acquisition of knowledge for classification of remotely sensed imagery. We concentrate on three important problems in this process: information fusion, model understandability, and handling of missing data. Importance of multi-sensor information fusion and the use of decision tree classifiers for such problems have been wellstudied in the literature. However, these studies have been limited to the cases where all data sources have a full coverage for the scene under consideration. Our contribution in this paper is to show how decision tree classifiers can be learned with alternative (surrogate) decision nodes and result in models that are capable of dealing with missing data during both training and classification to handle cases where one or more measurements do not exist for some locations. We present detailed performance evaluation regarding the effectiveness of these classifiers for information fusion and feature selection, and study three different methods for handling missing data in comparative experiments. The results show that surrogate decisions incorporated into decision tree classifiers provide powerful models for fusing information from different data layers while being robust to missing data.

A Case Study in Natural Language Based Web Search

Is there a public for natural language based search? This study, based on our experience with a Web portal, attempts to address criticisms on the lack of scalability and usability of natural language approaches to search. Our solution is based on InFact®, a natural language search engine that combines the speed of keyword search with the power of natural language processing. InFact performs clause level indexing, and offers a full spectrum of functionality that ranges from Boolean keyword operators to linguistic pattern matching in real time, which include recognition of syntactic roles, such as subject/object and semantic categories, such as people and places. A user of our search can navigate and retrieve information based on an understanding of actions, roles and relationships. In developing InFact, we ported the functionality of a deep text analysis platform to a modern search engine architecture. Our distributed indexing and search services are designed to scale to large document collections and large numbers of users. We tested the operational viability of InFact as a search platform by powering a live search on the Web. Site statistics and user logs demonstrate that a statistically significant segment of the user population is relying on natural language search functionality. Going forward, we will focus on promoting this functionality to an even greater percentage of users through a series of creative interfaces.

Automated feature selection through relevance feedback

The VisiMine project aims to provide infrastructure that would enable the analysis of large databases containing satellite images. Our work addresses two issues. One is the extraction of information that enables reduction of the data from multi-spectral images into a number of features. Second is the organization and selection of the features that would allow flexible and scalable discovery of the knowledge from the databases of remotely sensed images. The VisiMine architecture distinguishes between three types of feature vectors: pixel, region and tile. One of the challenges in information retrieval is the proper choice of the set of features that are the best suited for a data mining task. The VisiMine system enables extraction of a large number of features that describe textural and spectral properties of satellite information, in addition to the analysis of image information, the system can perform data fusion of image properties with auxiliary data such as DEM. Tilton et al. (2002) presented the results of the information retrieval experiments with the Hierarchical Segmentation (HSEG) algorithm that produces a hierarchical set of image segmentations. The results presented showed that the use of HSEG features improves the precision and recall of similarity searches. However, for different types of land cover, different combinations of HSEG segmentation levels and textural features provided the best results. Image analysis applications often require different levels of image segmentation detail as well as the use of different mixes of spectral, textural and shape features combined together with auxiliary information. Furthermore, a particular application may require different features and different levels of image segmentation detail depending on how the image objects are being analyzed. Thus, an automatic selection of feature sets would be very useful for satellite image analysis. In this paper, we present algorithms that allow for automatic selection of features for region and tile similarity searches. The relevance feedback technique allows for selective choices to be made in the region(s) of interest for which a good subset of features may be found in real time. The preliminary results of the experiments with LANDSAT data show improvements in both precision and recall over previously used methods.

Applications of terrain and sensor data fusion in image mining

We describe usage of DEM data in the VisiMine system for data mining and statistical analysis of the collections of remotely sensed images.