Francine R. Chen

Method of speaker clustering for unknown speakers in conversational audio data

A method for clustering speaker data from a plurality of unknown speakers. The method includes steps of providing a portion of audio data containing speech from at least all the speakers in the audio data and dividing the portion into data clusters. A pairwise distance between each pair of clusters is computed, the pairwise distance being based on a likelihood that two clusters were created by the same speaker, the likelihood measurement being biased by the prior probability of speaker changes. The two clusters with a minimum pairwise distance are combined into a new cluster and speakers models are trained for each of the remaining clusters including the new cluster. The likelihood that two clusters were created by the same speaker may be biased by a Markov duration model based on speaker changes over the length of the initial data clusters.

Multimodal browsing of images in Web documents

In this paper, we describe a system for performing browsing and retrieval on a collection of web images and associated text on an HTML page. Browsing is combined with retrieval to help a user locate interesting portions of the corpus, without the need to formulate a query well matched to the corpus. Multi-modal information, in the form of text surrounding an image and some simple image features, is used in this process. Using the system, a user progressively narrows a collection to a small number of elements of interest, similar to the Scatter/Gather system developed for text browsing. We have extended the Scatter/Gather method to use multi-modal features. With the use of multiple features, some collection elements may have unknown or undefined values for some features; we present a method for incorporating these elements into the result set. This method also provides a way to handle the case when a search is narrowed to a part of the space near a boundary between two clusters. A number of examples illustrating our system are provided.

Detection and location of multicharacter sequences in lines of imaged text

A system for detecting and locating user-specified search strings, or phrases, in lines of imaged text is described. The phrases may be single words or multiple words, and may contain a partially specified word. The imaged text can be composed of a number of different fonts and graphics. Textlines in a deskewed image are hypothesized using multiresolution morphology. For each textline, the baseline, topline and x-height are identified by simple statistical methods and then used to normalize each textline bounding box. Columns of pixels in the resulting bounding box serve as feature vectors. One hidden Markov model is created for each user-specified phrase and another represents all text and graphics other than the user-specified phrases. Phrases are identified using Viterbi decoding on a spotting network created from the models. The operating point of the system can be varied to trade off the percentage of words correctly spotted and the percentage of false alarms. Results are given using a subset of the UW English Document Image Database I.

Audio indexing using speaker identification

In this paper, a technique for audio indexing based on speaker identification is proposed. When speakers are known a priori, a speaker index can be created in real time using the Viterbi algorithm to segment the audio into intervals from a single talker. Segmentation is performed using a hidden Markov model network consisting of interconnected speaker sub- networks. Speaker training data is used to initiate sub-networks for each speaker. Sub- networks can also be used to model silence, or non-speech sounds such as musical theme. When no prior knowledge of the speakers is available, unsupervised segmentation is performed using a non-real time iterative algorithm. The speaker sub-networks are first initialized, and segmentation is performed by iteratively generating a segmentation using the Viterbi algorithm, and retraining the sub-networks based on the results of the segmentation. Since the accuracy of the speaker segmentation depends on how well the speaker sub-networks are initiated, agglomerative clustering is used to approximately segment the audio according to speaker for initialization of the speaker sub-networks. The distance measure for the agglomerative clustering is a likelihood ratio in which speed segments are characterized by Gaussian distributions. The distance between merged segments is recomputed at each stage of the clustering, and a duration model is used to bias the likelihood ratio. Segmentation accuracy using agglomerative clustering initialization matches accuracy using initialization with speaker labeled data.

Extraction of text-related features for condensing image documents

A system has been built that selects excerpts from a scanned document for presentation as a summary, without using character recognition. The method relies on the idea that the most significant sentences in a document contain words that are both specific to the document and have a relatively high frequency of occurrence within it. Accordingly, and entirely within the image domain, each page image is deskewed and the text regions of are found and extracted as a set of textblocks. Blocks with font size near the median for the document are selected and then placed in reading order. The textlines and words are segmented, and the words are placed into equivalence classes of similar shape. The sentences are identified by finding baselines for each line of text and analyzing the size and location of the connected components relative to the baseline. Scores can then be given to each word, depending on its shape and frequency of occurrence, and to each sentence, depending on the scores for the words in the sentence. Other salient features, such as textblocks that have a large font or are likely to contain an abstract, can also be used to select image parts that are likely to be thematically relevant. The method has been applied to a variety of documents, including articles scanned from magazines and technical journals.