Brian F. White

Lightweight document clustering

A lightweight document clustering method is described that operates in high dimensions, processes tens of thousands of documents and groups them into several thousand clusters, or by varying a single parameter, into a few dozen clusters. The method uses a reduced indexing view of the original documents, where only the k best keywords of each document are indexed. An efficient procedure for clustering is speci fied in two parts (a) compute k most similar documents for each document in the collection and (b) group the documents into clusters using these similarity scores. The method has been evaluated on a database of over 50,000 customer service problem reports that are reduced to 3,000 clusters and 5,000 exemplar documents. Results demonstrate efficient clustering performance with excellent group similarity measures.

Probabilistic estimation-based data mining for discovering insurance risks

IBMs underwriting profitability analysis application mines property and casualty insurance policy and claims data to construct predictive models for insurance risks. UPA uses the ProbE data-mining kernel to discover risk-characterization rules by analyzing large, noisy data sets.

Lightweight document matching for help-desk applications

For decades, researchers have been working on ways to process text for classification and queries by relevant document retrieval. The authors describe a method that uses minimal data structures and lightweight algorithms to match new documents to those stored in a database. It is a completely automated Java based document matcher that accepts an unlimited-length textural structure as input and employs a fast matching algorithm to produce, like a search engine, a ranked list of relevant documents.

Continuous prediction of manufacturing performance throughout the production lifecycle

We describe methods for continual prediction of manufactured product quality prior to final testing. In our most expansive modeling approach, an estimated final characteristic of a product is updated after each manufacturing operation. Our initial application is for the manufacture of microprocessors, and we predict final microprocessor speed. Using these predictions, early corrective manufacturing actions may be taken to increase the speed of expected slow wafers (a collection of microprocessors) or reduce the speed of fast wafers. Such predictions may also be used to initiate corrective supply chain management actions. Developing statistical learning models for this task has many complicating factors: (a) a temporally unstable population (b) missing data that is a result of sparsely sampled measurements and (c) relatively few available measurements prior to corrective action opportunities. In a real manufacturing pilot application, our automated models selected 125 fast wafers in real-time. As predicted, those wafers were significantly faster than average. During manufacture, downstream corrective processing restored 25 nominally unacceptable wafers to normal operation.

The Stakeholder Matrix: Supporting the Modeling of Responsibility in Situation-Oriented Directories

Situation-oriented directories help to determine the stakeholder responsible for a given situation. Defining all responsibility assignments for a directory can be challenging if situations need to be distinguished by many attributes and some of the attributes have many potential values. In this paper we present the stakeholder matrix, a compact representation of a situation-oriented directory that makes it easier for users to ensure the completeness and consistency of the assignments in the directory. Our evaluation showed that in a real world case the number of responsibility assignments could be compacted from 33700 to 128. We also review related research and show that compact representations similar to the stakeholder matrix will be useful to support validation of definitions in expert finding systems, content-based access systems and content-based routing systems.