Carl Sable

Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs

Annotating photographs automatically with content descriptions facilitates organization, storage, and search over visual information. We present an integrated approach for scene classi cation that combines image-based and text-based approaches. On the text side, we use the text accompanying an image in a novel TF*IDF vector-based approach to classi cation. On the image side, we present a novel OF*IIF (object frequency) vector-based approach to classi cation. Objects are de ned by clustering of segmented regions of training images. The image based OF*IIF approach is synergistic with the text based TF*IDF approach. By integrating the TF*IDF approach and the OF*IIF approach, we achieved a classi cation accuracy of 86%. This is an improvement of approximately 12% over existing image classi ers, an improvement of approximately 3% over the TF*IDF image classi er based on textual information, and an improvement of approximately 4% over the OF*IIF image classi er based on visual information.

Text-Based Approaches for the Categorization of Images

The rapid expansion of multimedia digital collections brings to the fore the need for classifying not only text documents but their embedded non-textual parts as well. We propose a model for basing classification of multimedia on broad, non-topical features, and show how information on targeted nearby pieces of text can be used to effectively classify photographs on a first such feature, distinguishing between indoor and outdoor images. We examine several variations to a TF*IDF-based approach for this task, empirically analyze their effects, and evaluate our system on a large collection of images from current news newsgroups. In addition, we investigate alternative classification and evaluation methods, and the effect that a secondary feature can have on indoor/outdoor classification. We obtain a classification accuracy of 82%, a number that clearly outperforms baseline estimates and competing image-based approaches and nears the accuracy of humans who perform the same task with access to comparable information.

Text-based approaches for non-topical image categorization

Abstract.The rapid expansion of multimedia digital collections brings to the fore the need for classifying not only text documents but their embedded non-textual parts as well. We propose a model for basing classification of multimedia on broad, non-topical features, and show how information on targeted nearby pieces of text can be used to effectively classify photographs on a first such feature, distinguishing between indoor and outdoor images. We examine several variations to a TF*IDF-based approach for this task, empirically analyze their effects, and evaluate our system on a large collection of images from current news newsgroups. In addition, we investigate alternative classification and evaluation methods, and the effects that secondary features have on indoor/outdoor classification. Using density estimation over the raw TF*IDF values, we obtain a classification accuracy of 82%, a number that outperforms baseline estimates and earlier, image-based approaches, at least in the domain of news articles, and that nears the accuracy of humans who perform the same task with access to comparable information.

NLP Found Helpful (at least for one Text Categorization Task)

Attempts to use natural language processing (NLP) for text categorization and information retrieval (IR) have had mixed results. Nevertheless, there is a strong intuition that NLP is important, at least for some tasks. In this paper, we discuss a task involving captioned images for which the subject and the predicate are critical. The usefulness of NLP for this task is established in two ways. In addition to the standard method of introducing a new system and comparing its performance with others in the literature, we also present evidence from experiments with human subjects showing that NLP generally improves speed and accuracy.

Hybrid Evolution of Convolutional Networks

With the increasing trend of neural network models towards larger structures with more layers, we expect a corresponding exponential increase in the number of possible architectures. In this paper, we apply a hybrid evolutionary search procedure to define the initialization and architectural parameters of convolutional networks, one of the first successful deep network models. We make use of stochastic diagonal Levenberg-Marquardt to accelerate the convergence of training, lowering the time cost of fitness evaluation. Using parameters found from the evolutionary search together with absolute value and local contrast normalization preprocessing between layers, we achieve the best known performance on several of the MNIST Variations, rectangles-image and convex image datasets.

Using Density Estimation to Improve Text Categorization

This paper explores the use of a statistical technique known as density estimation to potentially improve the results of text categorization systems which label documents by computing similarities between documents and categories. In addition to potentially improving a systems overall accuracy, density estimation converts similarity scores to probabilities. These probabilities provide con dence measures for a systems predictions which are easily interpretable and could potentially help to combine results of various systems. We discuss the results of three complete experiments on three separate data sets applying density estimation to the results of a TF*IDF/Rocchio system, and we compare these results to those of many competing approaches.

Energy Consumption Reduction of a WSN Node Using 4-bit ADPCM

A 4-bit adaptive differential pulse-code modulation (ADPCM) scheme applied to the sensor data of a Zigbee based wireless sensor network node is shown to decrease the energy consumption of the analog front-end of the node by 58%. Simulation results from an energy model of an 802.15.4 based analog front-end show that the energy consumed by the network node is inversely proportional to the number of bits used to encode the digital data. The quantization error of a 4-bit ADPCM scheme is on average -14 dB for low frequency data and only 3 dB higher than a traditional 8-bit PCM scheme. By modifying the modulation scheme in the software with no modification of the hardware, the lifetime of the node can be increased significantly with minimal modifications.