Richard Rohwer

New Experiments in Distributional Representations of Synonymy

Recent work on the problem of detecting synonymy through corpus analysis has used the Test of English as a Foreign Language (TOEFL) as a benchmark. However, this test involves as few as 80 questions, prompting questions regarding the statistical significance of reported results. We overcome this limitation by generating a TOEFL-like test using WordNet, containing thousands of questions and composed only of words occurring with sufficient corpus frequency to support sound distributional comparisons. Experiments with this test lead us to a similarity measure which significantly outperforms the best proposed to date. Analysis suggests that a strength of this measure is its relative robustness against polysemy.

Dynamic decentralized any-time hierarchical clustering

Hierarchical clustering is used widely to organize data and search for patterns. Previous algorithms assume that the body of data being clustered is fixed while the algorithm runs, and use centralized data representations that make it difficult to scale the process by distributing it across multiple processors. Self-Organizing Data and Search (SODAS) inspired by the decentralized algorithms that ants use to sort their nests, relaxes these constraints. SODAS can maintain a hierarchical structure over a continuously changing collection of leaves, requiring only local computations at the nodes of the hierarchy and thus permitting the system to scale arbitrarily by distributing nodes (and their processing) across multiple computers.

Text Modeling for Real-Time Document Categorization

We report on experiments in adapting document categorization techniques to provide for implementation in high-speed hardware. Because resources are scarce, it is important to have a small set of robust and maximally informative variables over which learning can occur. We generate variables using information-theoretic clustering. The resulting performance is on par with general-purpose computing implementations which are able to take advantage of large amounts of time and memory. We conclude that custom high-speed hardware for document categorization can be made very accurate. We also believe that some of the strengths of information-theoretic data analysis techniques are brought out

An architecture for streaming coclustering in high speed hardware

We seek to learn the semantics of a data stream at optical line speed. We focus on text data, but the techniques developed should apply to broad modalities of network data wherever appropriate features can be computed rapidly enough. We consider a custom hardware system designed to categorize documents based on feature clusters and document clusters that have been learned offline on standard general-purpose computers, and we present a technique for extending this system to permit online learning from arbitrarily large data sets

Simmered Greedy Optimization for Co-clustering

We present a fast yet highly effective stochastic algorithm, Simmered Greedy Optimization (SG(N)) for solving the co-clustering problem: to simultaneously cluster two finite sets by maximizing the mutual information between the clusterings. (Clustering one set by this criterion is a special case.) This is a combinatorial optimization problem of great interest for deriving maximally predictive feature sets. Co-clustering has found applications in many areas, particularly statistical natural language processing and bioinformatics. We report results of tests on a suite of statistical natural language problems, comparing SG(N) with simulated annealing and a publicly available implementation of co-clustering. In all cases we obtain superior results with far less computation using SG(N).