Devang K. Naik

A novel word clustering algorithm based on latent semantic analysis

A new approach is proposed for the clustering of words in a given vocabulary. The method is based on a paradigm first formulated in the context of information retrieval, called latent semantic analysis. This paradigm leads to a parsimonious vector representation of each word in a suitable vector space, where familiar clustering techniques can be applied. The distance measure selected in this space arises naturally from the problem formulation. Preliminary experiments indicate that, the clusters produced are intuitively satisfactory. Because these clusters are semantic in nature, this approach may prove useful as a complement to conventional class-based statistical language modeling techniques.

Automatic junk e-mail filtering based on latent content

The explosion in unsolicited mass electronic mail (junk e-mail) over the past decade has sparked interest in automatic filtering solutions. Traditional techniques tend to rely on header analysis, keyword/keyphrase matching and analogous rule-based predicates, and/or some probabilistic model of text generation. This paper aims instead at deciding whether or not the latent subject matter is consistent with the users interests. The underlying framework is latent semantic analysis: each e-mail is automatically classified against two semantic anchors, one for legitimate and one for junk messages. Experiments show that this approach is competitive with the state-of-the-art in e-mail classification, and potentially advantageous in real-world applications with high junk-to-legitimate ratios. The resulting technology has been successfully released in August 2002 as part of the e-mail client bundled with the MacOS 10.2 operating system.

Using talker location to detect spurious utterances in desktop command and control

Hands-free desktop command and control speech recognition suffers from the critical drawback of improperly rejecting spurious conversation. This results in false acceptances of unintended speech commands that can inconvenience the user. A neural-network approach is proposed to detect spurious conversation by determining talker location. The approach is based on the premise that spoken utterances not directed towards the microphone source tend to be more reverberant and are likely to be spurious. The method estimates a confidence measure proportional to the amount of reverberation in the end-pointed speech signal. The measure is obtained from a neural network that determines if the speech signal was directed to the microphone or was spoken otherwise. The proposed measure can be combined with the acoustic, linguistic and semantic information to improve upon decisions taken by conventional rejection modeling schemes.