Grant J. Scott

Entropy-Balanced Bitmap Tree for Shape-Based Object Retrieval From Large-Scale Satellite Imagery Databases

In this paper, we present a novel indexing structure that was developed to efficiently and accurately perform content-based shape retrieval of objects from a large-scale satellite imagery database. Our geospatial information retrieval and indexing system, GeoIRIS, contains 45 GB of high-resolution satellite imagery. Objects of multiple scales are automatically extracted from satellite imagery and then encoded into a bitmap shape representation. This shape encoding compresses the total size of the shape descriptors to approximately 0.34% of the imagery database size. We have developed the entropy-balanced bitmap (EBB) tree, which exploits the probabilistic nature of bit values in automatically derived shape classes. The efficiency of the shape representation coupled with the EBB tree allows us to index approximately 1.3 million objects for fast content-based retrieval of objects by shape.

A fast protein structure retrieval system using image-based distance matrices and multidimensional index

Indexing protein structures has been shown to provide a scalable solution for structure-to-structure comparisons in large protein structure retrieval systems. To conduct similarity searches against 46,075 polypeptide chains in a database with real-time responses, two critical issues must be addressed, information extraction and suitable indexing. In this paper, we apply computer vision techniques to extract the predominant information encoded in each 2D distance matrix, generated from 3D coordinates of protein chains. Distance matrices are capable of representing specific protein structural topologies, and similar proteins will generate similar matrices. Once meaningful features are extracted from distance images, an advanced indexing structure, entropy balanced statistical (EBS) k-d tree, can be utilized to index the multidimensional data. With a limited amount of training data from domain experts, namely structural classification of a subset of available protein chains, we apply various techniques in the pattern recognition field to determine clusters of proteins in the multi-dimensional feature space. Our system is able to recall search results in a ranked order from the protein database in seconds, exhibiting a reasonably high degree of precision.

ProteinDBS: a real-time retrieval system for protein structure comparison

We have developed a web server (ProteinDBS) for the life science community to search for similar protein tertiary structures in real time. This system applies computer visualization techniques to extract the predominant visual patterns encoded in two-dimensional distance matrices generated from the three-dimensional coordinates of protein chains. When meaningful contents, represented in a multi-dimensional feature space, have been extracted from distance matrices, an advanced indexing structure, Entropy Balanced Statistical (EBS) k-d tree, is utilized to index the data. Our system is able to return search results in ranked order from a database with 46 075 chains in seconds, exhibiting a reasonably high degree of precision. To our knowledge, this is the first real-time search engine for protein structure comparison. ProteinDBS provides two types of query method: query by Protein Data Bank protein chain ID and by new structures uploaded by users. The system is hosted at http://ProteinDBS.rnet.missouri.edu.

Knowledge-Driven Multidimensional Indexing Structure for Biomedical Media Database Retrieval

Today, biomedical media data are being generated at rates unimaginable only years ago. Content-based retrieval of biomedical media from large databases is becoming increasingly important to clinical, research, and educational communities. In this paper, we present the recently developed entropy balanced statistical (EBS) k-d tree and its applications to biomedical media, including a high-resolution computed tomography (HRCT) lung image database and the first real-time protein tertiary structure search engine. Our index utilizes statistical properties inherent in large-scale biomedical media databases for efficient and accurate searches. By applying concepts from pattern recognition and information theory, the EBS k-d tree is built through top-down decision tree induction. Experimentation shows similarity searches against a protein structure database of 53 363 structures consistently execute in less than 8.14 ms for the top 100 most similar structures. Additionally, we have shown improved retrieval precision over adaptive and statistical k-d trees. Retrieval precision of the EBS k-d tree is 81.6% for content-based retrieval of HRCT lung images and 94.9% at 10% recall for protein structure similarity search. The EBS k-d tree has enormous potential for use in biomedical applications embedded with ground-truth knowledge and multidimensional signatures

Fusion of Deep Convolutional Neural Networks for Land Cover Classification of High-Resolution Imagery

Deep convolutional neural networks (DCNNs) have recently emerged as a dominant paradigm for machine learning in a variety of domains. However, acquiring a suitably large data set for training DCNN is often a significant challenge. This is a major issue in the remote sensing domain, where we have extremely large collections of satellite and aerial imagery, but lack the rich label information that is often readily available for other image modalities. In this letter, we investigate the use of DCNN for land–cover classification in high-resolution remote sensing imagery. To overcome the lack of massive labeled remote-sensing image data sets, we employ two techniques in conjunction with DCNN: transfer learning (TL) with fine-tuning and data augmentation tailored specifically for remote sensing imagery. TL allows one to bootstrap a DCNN while preserving the deep visual feature extraction learned over an image corpus from a different image domain. Data augmentation exploits various aspects of remote sensing imagery to dramatically expand small training image data sets and improve DCNN robustness for remote sensing image data. Here, we apply these techniques to the well-known UC Merced data set to achieve the land–cover classification accuracies of 97.8 ± 2.3%, 97.6 ± 2.6%, and 98.5 ± 1.4% with CaffeNet, GoogLeNet, and ResNet, respectively.

EBS k-d tree: an entropy balanced statistical k-d tree for image databases with ground-truth labels

In this paper we present a new image database indexing structure - Entropy Balanced Statistical (EBS) k-d Tree. This indexing mechanism utilizes the statistical properties and ground-truth labeling of image data for efficient and accurate searches. It is particularly valuable in the domains of medical and biological image database retrieval, where ground-truth labeling are available and archived with the images. The EBS k-d tree is an extension to the statistical k-d tree that attempts to optimize a multi-dimensional decision tree based on the fundamental principles from which it is constructed. Our approach is to develop and validate the notion of an entropy balanced statistical based decision tree. It is shown that by making balanced split decisions in the growth processing of the tree, that the average search depth is improved and the worst case search depth is usually dramatically improved. Furthermore, a method for linking the tree leaves into a non-linear structure was developed to increase the n-nearest neighbor similarity search accuracy. We have applied this to a large-scale medical diagnostic image database and have shown increases in search speed and accuracy over an ordinary distance-based search and the original statistical k-d tree index.

Modeling multi-object spatial relationships for satellite image database indexing and retrieval

Geospatial information analysts are interested in spatial configurations of objects in satellite imagery and, more importantly, the ability to search a large-scale database of satellite images using spatial configurations as the query mechanism. In this paper we present a new method to model spatial relationships among sets of three or more objects in satellite images for scene indexing and retrieval by generating discrete spatial signatures. The proposed method is highly insensitive to scaling, rotation, and translation of the spatial configuration. Additionally, the method is efficient for use in real-time applications, such as online satellite image retrievals. Moreover, the number of objects in a spatial configuration has minimal effect on the efficiency of the method.

A FAST PROTEIN STRUCTURE RETRIEVAL SYSTEM USING IMAGE-BASED DISTANCE MATRICES AND MULTIDIMENSIONAL INDEX

Indexing protein structures has been shown to provide a scalable solution for structure-to-structure comparisons in large protein structure retrieval systems. To conduct similarity searches against 46,075 polypeptide chains in a database with real-time responses, two critical issues must be addressed, information extraction and suitable indexing. In this paper, we apply computer vision techniques to extract the predominant information encoded in each 2D distance matrix, generated from 3D coordinates of protein chains. Distance matrices are capable of representing specific protein structural topologies, and similar proteins will generate similar matrices. Once meaningful features are extracted from distance images, an advanced indexing structure, entropy balanced statistical (EBS) k-d tree, can be utilized to index the multidimensional data. With a limited amount of training data from domain experts, namely structural classification of a subset of available protein chains, we apply various techniques in the pattern recognition field to determine clusters of proteins in the multi-dimensional feature space. Our system is able to recall search results in a ranked order from the protein database in seconds, exhibiting a reasonably high degree of precision.

Automated object extraction through simplification of the differential morphological profile for high-resolution satellite imagery

This paper presents an approach for automated, multi-scale object extraction from high-resolution satellite imagery. Our algorithm combines techniques from mathematical morphology and principal components analysis (PCA) to identify building footprints in scenes. There are three major components in the algorithm: First, the differential morphological profile (DMP) of the image is constructed using structuring elements (SE) of varying sizes. Several preprocessing techniques are applied to the DMP. Next, the values for an entire profile at a given pixel are combined into a k-dimensional vector representing that pixel, where k is the number of resolution levels. PCA is applied to the set of pixel vectors to reduce the number of dimensions needed to represent the data of the DMP, yet capture a significant portion of their variance. We call the intermediate results of PCA the Eigen-opening image and Eigen-closing image. In the final stage of processing, candidate objects are extracted from the Eigen images. Applying a minimal amount of heuristics we can automatically merge the Eigen images into a set of objects. Additionally, spatial information is used to correlate man-made objects with their shadows from the closing profile, if they exist. The efficiency of this algorithm, coupled with it’s robustness, allow it to be useful as an online object extraction tool for geospatial applications.

Clustering of Detected Changes in High-Resolution Satellite Imagery Using a Stabilized Competitive Agglomeration Algorithm

The Geospatial Change Detection and exploitation (GeoCDX) is a fully automated system for detection and exploitation of change between multitemporal high-resolution satellite and airborne images. Overlapping multitemporal images are first organized into 256 m × 256 m tiles in a global grid reference system. The system quantifies the overall amount of change in a given tile with a tile change score as an aggregation of pixel-level changes. The tiles are initially ranked by these change scores for retrieval, review, and exploitation in a Web-based application. However, the ranking does not account for the wide variety of change types that are typically observed in the top-ranked change tiles. To automatically organize the wide variety of change patterns observed in multitemporal high-resolution imagery, we perform tile clustering using the competitive agglomeration (CA) algorithm stabilized using the fuzzy c-means (FCM) algorithm. Each resulting cluster contains tiles with a visually similar type of change. By visual inspection of these tile clusters, GeoCDX users can quickly find certain types of change without having to sift through a large number of tiles initially organized solely by their tile change score, thereby reducing the time it takes for users to discover and exploit the change pattern(s) of greatest interest to a given application (e.g., urban growth, disaster assessment, facility monitoring, etc.). The tile clusters also provide a high-level overview of the various types of change that occur between the two observations. This overview is compared with a similar yet more limited view offered by a relevance feedback tool that requires a user to select sample tiles for use as samples in the reranking process.