Wei-Hao Lin

Which Side are You on? Identifying Perspectives at the Document and Sentence Levels

In this paper we investigate a new problem of identifying the perspective from which a document is written. By perspective we mean a point of view, for example, from the perspective of Democrats or Republicans. Can computers learn to identify the perspective of a document? Not every sentence is written strongly from a perspective. Can computers learn to identify which sentences strongly convey a particular perspective? We develop statistical models to capture how perspectives are expressed at the document and sentence levels, and evaluate the proposed models on articles about the Israeli-Palestinian conflict. The results show that the proposed models successfully learn how perspectives are reflected in word usage and can identify the perspective of a document with high accuracy.

Extreme video retrieval: joint maximization of human and computer performance

We present an efficient system for video search that maximizes the use of human bandwidth, while at the same time exploiting the machines ability to learn in real-time from user selected relevant video clips. The system exploits the human capability for rapidly scanning imagery augmenting it with an active learning loop, which attempts to always present the most relevant material based on the current information. Two versions of the human interface were evaluated, one with variable page sizes and manual paging, the other with a fixed page size and automatic paging. Both require absolute attention and focus of the user for optimal performance. In either case, as users search and find relevant results, the system can invisibly re-rank its previous best guesses using a number of knowledge sources, such as image similarity, text similarity, and temporal proximity. Experimental evidence shows a significant improvement using the combined extremes of human and machine power over either approach alone.

News video classification using SVM-based multimodal classifiers and combination strategies

Video classification is the first step toward multimedia content understanding. When video is classified into conceptual categories, it is usually desirable to combine evidence from multiple modalities. However, combination strategies in previous studies were usually ad hoc. We investigate a meta-classification combination strategy using Support Vector Machine, and compare it with probability-based strategies. Text features from closed-captions and visual features from images are combined to classify broadcast news video. The experimental results show that combining multimodal classifiers can significantly improve recall and precision, and our meta-classification strategy gives better precision than the approach of taking the product of the posterior probabilities.

Structuring continuous video recordings of everyday life using time-constrained clustering

As personal wearable devices become more powerful and ubiquitous, soon everyone will be capable to continuously record video of everyday life. The archive of continuous recordings need to be segmented into manageable units so that they can be efficiently browsed and indexed by any video retrieval systems. Many researchers approach the problem in two-pass methods: segmenting the continuous recordings into chunks, followed by clustering chunks. In this paper we propose a novel one-pass algorithm to accomplish both tasks at the same time by imposing time constraints on the K-Means clustering algorithm. We evaluate the proposed algorithm on 62.5 hours of continuous recordings, and the experiment results show that time-constrained clustering algorithm substantially outperforms the unconstrained version.

Clever clustering vs. simple speed-up for summarizing rushes

This paper discusses in detail our approaches for producing the submitted summaries to TRECVID, including the two baseline methods. The cluster method performed well in terms of coverage, and adequately in terms of user satisfaction, but did take longer to review. We conducted additional evaluations using the same TRECVID assessment interface to judge 2 additional methods for summary generation: 25x (simple speed-up by 25 times), and pz (emphasizing pans and zooms). Human assessors show significant differences between the cluster, pz, and 25x approaches. The best coverage (text inclusion performance) is obtained by 25x, but at the expense of taking the most time to evaluate and perceived as the most redundant. Method pz was easier to use than cluster and had better performance on pan/zoom recall tasks, leading into discussions on how summaries can be improved with more knowledge of the anticipated users and tasks.

Are These Documents Written from Different Perspectives? A Test of Different Perspectives Based on Statistical Distribution Divergence

In this paper we investigate how to automatically determine if two document collections are written from different perspectives. By perspectives we mean a point of view, for example, from the perspective of Democrats or Republicans. We propose a test of different perspectives based on distribution divergence between the statistical models of two collections. Experimental results show that the test can successfully distinguish document collections of different perspectives from other types of collections.

Which Thousand Words are Worth a Picture? Experiments on Video Retrieval using a Thousand Concepts

In contrast to traditional video retrieval that represents visual content with low-level features (e.g. color and texture), emerging concept-based video retrieval allows users to search video archives by specifying a limited number of high-level concepts (e.g. outdoors and car). Recent studies have demonstrated the feasibility of concept-based retrieval, but a fundamental question remains: what kinds of concepts should we index? We analyze a large video archive annotated with more than a thousand high-level concepts, and develop guidelines for choosing concepts of high utility to video retrieval

Revisiting the effect of topic set size on retrieval error

Evaluating retrieval systems in a controlled environment with a large set of topics has been the core paradigm in the information retrieval community. Voorhees and Buckley proposed to estimate the reliability of retrieval experiments by calculating the probability of making wrong effectiveness judgments between two retrieval systems over two retrieval experiments[2], which is called Retrieval Experiment Error Rate (REER) in this paper. They have successfully shown how the topic set sizes affect the retrieval experiment reliability. However, the REER model in the previous work was empirically justified without providing a derivation based on statistical principles. We fill this gap and show that REER can indeed be derived from statistical principles. Based on the derived model we can explain why a successful experiment design depends on factors including a sufficient number of topics, large enough measurement score difference between systems, and a homogeneous distribution of retrieval scores for topics and systems, which reduces the variance of the score differences.

A wearable digital library of personal conversations

We have developed a wearable, personalized digital library system, which unobtrusively records the wearers part of a conversation, recognizes the face of the current dialog partner and remembers his/her voice. The next time the system sees the same person and hears the same voice, it can replay parts of the last conversation in compressed form. Results from a prototype system show the effectiveness of combining of face recognition and speaker identification for retrieving conversations.

Exploring the utility of fast-forward surrogates for bbc rushes

This paper discusses in detail our approaches for producing the video summaries submitted to the TRECVID 2008 BBC rushes summarization task, including the baseline method. Empirical work produced during and after the TRECVID 2007 rushes summarization task gave strong evidence that a simple 50x method (sampling every 50th frame) provides excellent coverage (text inclusion performance). Our submissions for TRECVID 2008 investigated the effects of junk frame removal, including a comprehensible audio track, and emphasizing pans and zooms when backfilling to reclaim the space removed with the noise shots from the original 50x set. Results show that 50x based methods provide excellent coverage as expected. There were limited effects for the other strategies to improve user satisfaction, with the discussion providing some insights for future video summary development and evaluation work.