Lekha Chaisorn

Image and Video Retrieval

We have witnessed a decade of exploding research interest in multimedia content analysis. The goal of content analysis has been to derive automatic methods for high-level description and annotation. In this paper we will summarize the main research topics in this area and state some assumptions that we have been using all along. We will also postulate the main future trends including usage of long term memory, context, dynamic processing, evolvable generalized detectors and user aspects.

Story boundary detection in large broadcast news video archives: techniques, experience and trends

The segmentation of news video into story units is an important step towards effective processing and management of large news video archives. In the story segmentation task in TRECVID 2003, a wide variety of techniques were employed by many research groups to segment over 120-hour of news video. The techniques employed range from simple anchor person detector to soisticated machine learning models based on HMM and Maximum Entropy (ME) approaches. The general results indicate that the judicious use of multi-modality features coupled with rigorous machine learning models could produce effective solutions. This paper presents the algorithms and experience learned in TRECVID evaluations. It also points the way towards the development of scalable technology to process large news video corpuses.

VideoQA: question answering on news video

When querying a news video archive, the users are interested in retrieving precise answers in the form of a summary that best answers the query. However, current video retrieval systems, including the search engines on the web, are designed to retrieve documents instead of precise answers. This research explores the use of question answering (QA) techniques to support personalized news video retrieval. Users interact with our system, VideoQA, using short natural language questions with implicit constraints on contents, context, duration, and genre of expected videos. VideoQA returns short precise news video summaries as answers. The main contributions of this research are: (a) the extension of QA technology to support QA in news video; and (b) the use of multi-modal features, including visual, audio, textual, and external resources, to help correct speech recognition errors and to perform precise question answering. The system has been tested on 7 days of news video and has been found to be effective.

The segmentation of news video into story units

The segmentation of news video into single-story semantic units is a challenging problem. This research proposes a two-level, multi-modal framework to tackle this problem. The video is analyzed at the shot and story unit (or scene) levels using a variety of features and techniques. At the shot level, we employ a decision tree to classify the shot into one of 13 predefined categories. At the scene level, we perform HMM (hidden Markov models) analysis to locate the story boundaries. We test the performance of our system using two days of news video obtained from the MediaCorp of Singapore. Our initial results indicate that we could achieve a high accuracy of over 95% for shot classification, and over 89% in F/sub 1/ measure on scene/story boundary detection.

A Multi-Modal Approach to Story Segmentation for News Video

This research proposes a two-level, multi-modal framework to perform the segmentation and classification of news video into single-story semantic units. The video is analyzed at the shot and story unit (or scene) levels using a variety of features and techniques. At the shot level, we employ Decision Trees technique to classify the shots into one of 13 predefined categories or mid-level features. At the scene/story level, we perform the HMM (Hidden Markov Models) analysis to locate story boundaries. Our initial results indicate that we could achieve a high accuracy of over 95% for shot classification, and over 89% in F1 measure on scene/story boundary detection. Detailed analysis reveals that HMM is effective in identifying dominant features, which helps in locating story boundaries. Our eventual goal is to support the retrieval of news video at story unit level, together with associated texts retrieved from related news sites on the web.

The Segmentation and Classification of Story Boundaries in News Video

The segmentation and classification of news video into single-story semantic units is a challenging problem. This research proposes a two-level, multi-modal framework to tackle this problem. The video is analyzed at the shot and story unit (or scene) levels using a variety of features and techniques. At the shot level, we employ a Decision Tree to classify the shot into one of 13 predefined categories. At the scene level, we perform the HMM (Hidden Markov Models) analysis to eliminate shot classification errors and to locate story boundaries. We test the performance of our system using two days of news video obtained from the MediaCorp of Singapore. Our initial results indicate that we could achieve a high accuracy of over 95 % for shot classification. The use of HMM analysis helps to improve the accuracy of the shot classification and achieve over 89% accuracy on story segmentation.

Story Boundary Detection in News Video using Global Rule Induction Technique

Global rule induction technique has been successfully used in information extraction (IE) from text documents. In this paper, we employ the technique to identify story boundaries in news video. We divide our framework into two levels: shot and story levels. We use a hybrid algorithm to classify each input video shot into one of the predefined genre types and employ the global rule induction technique to extract story boundaries from the sequence of classified shots. We evaluate our rule induction based system on ~120-hours of news video provided by TRECVID 2003. The results show that we could achieve an F 1 accuracy of over 75%

Human Action Recognition Bases on Local Action Attributes

Human action recognition received many interest in the computer vision community. Most of the existing methods focus on either construct robust descriptor from the temporal domain, or computational method to exploit the discriminative power of the descriptor. In this paper we explore the idea of using local action attributes to form an action descriptor, where an action is no longer characterized with the motion changes in the temporal domain but the local semantic description of the action. We propose an novel framework where introduces local action attributes to represent an action for the final human action categorization. The local action attributes are defined for each body part which are independent from the global action. The resulting attribute descriptor is used to jointly model human action to achieve robust performance. In addition, we conduct some study on the impact of using body local and global low-level feature for the aforementioned attributes. Experiments on the KTH dataset and the MV-TJU dataset show that our local action attribute based descriptor improve action recognition performance.

Video retrieval - evolution of video segmentation, indexing and search

This paper discusses the history and current trends of video retrieval, focusing mainly on video segmentation, indexing and search. The objective is to share with the readers how much we have done so far as well as the current trends in the field. Unlike text documents, video contains dynamic information such as audio, motion (object and/or camera motions), etc. Thus, indexing videos for future search still remains a difficult problem. In addition, one particular problem that remains with digital videos is that it is hard to deal with copyright issues. Thus, indexing videos needs to take this into consideration. In this paper, the history and some of the-state-of-art methods that help to solve these problems will be addressed.

A simplified ordinal-based method for video signature

In this paper, we propose an algorithm for video signature using music videos and news video for our study. The algorithm was developed based on our earlier work with an improvement (reducing the steps) on the process of generating video signature. Thus, revised process reduces the amount of processing time needed tremendously. In addition, besides color as a feature, we introduce the addition of a face feature to improve the system performance for video matching. For our experimental set up, most conditions met the requirements set forth by MPEG standardization on visual signature tools. Initial results show that the algorithm is effective and robust to several transformations such as color change, brightness change, the addition of Gaussian noise, and resolution reduction.