Kuo-Yu Liu

Exploring media correlation and synchronization for navigated hypermedia documents

This paper is devoted to explore media correlation and media synchronization in a composite multimedia document, the so-called navigated hypermedia document in our language learning system, to facilitate the multimedia authoring, presentation, and access. Two levels of media correlation in temporal, spatial, and content domains are investigated: syntactic level correlation and semantic level correlation. We devise a capturing mechanism to record all the media streams and relations between them, including voice and event streams, for replaying the lecturing in a form as close as possible to the original classroom experience. The syntactic level correlation is based on specific timestamps of the media stream and used to reconstruct the recorded lecture for synchronized presentation. Furthermore, to integrate media objects with specific segments within the media stream, some computed synchronization processes are required to discover semantic content of the media. The proposed computed synchronization techniques, including speech-event binding process for temporal domain, tele-pointer (i.e. cursor) movement interpolation and adaptable handwriting presentation for spatial domain, and erasing handling for content domain, will be addressed. Experimental results show that in the speech-event binding process 74% of speech access entries for accessible visualized events are found. The acceptable rate of human perception on tele-pointer movement is higher than 85% if time interval is selected carefully. Finally, the accuracy of erasing handling for content removing is about 71%.

The WSML system: web-based synchronization multimedia lecture system

This demonstration presents a web-based multimedia lecture system that perfectly integrates multimedia lecturing with static HTML pages and enables their presentation in synchronization. We develop key techniques to reproduce vivid web-teaching scenarios for on-demand access, including events capturing scheme and synchronized mechanism. To facilitate multimedia lectures creation, the easy-to-use authoring and managing tools specially designed for teachers have been also included in the development. In addition, integrated presentation and efficient access issues are also discussed for the students aspect.

WMA: A Marking-Based Synchronized Multimedia Tutoring System for English Composition Studies

This paper presents a system, Web-based Multimedia Annotation (WMA) system, for english as foreign language learning in writing skills. The whole correcting process, including the instructors voice and navigation events (i.e., tele-pointer (cursor), highlight, pen strokes, markings and annotations), can be captured through our system for later access. We address the issues of exploring involved media correlation to benefit adaptable presentation in a synchronization manner from temporal, spatial and content domains. The proposed computed synchronization techniques include speech-event binding process in the temporal domain, tele-pointer movement interpolation and adaptable handwriting presentation in the spatial domain, and visualized annotation erasing in the content domain. The experimental results show that in the speech-event binding process 74% of speech access entries for accessible visualized annotations are found. The acceptable rate of human perception of tele-pointer movement is higher than 85% if time interval is selected carefully. The accuracy of visualized annotation erasing for content removal is about 71%. Our user study shows that students can devote their efforts to writing practice because they can better understand their own mistakes corrected by the instructors using this multimedia presentation.

Sharing Web-based multimedia learning objects using NNTP news architecture

This paper presents a multimedia objects sharing framework on the basis of Internet news infrastructure (NNTP) to facilitate e-learning content access and distribution. Through this framework, a Web-based e-learning system can publish sharable learning objects by posting them to a news server nearby, so that the objects can be automatically spread to world-wide news servers through NNTP forwarding mechanism. Instead of hunting for the desired materials from many different e-learning systems, users can access learning objects shared from different e-learning platforms easily, and in a unified way. Moreover, we deal with severe data redundancy problem occurring especially when the shared object contains multimedia data (e.g., instructors speech, image, and video). A system prototype has been implemented in our university to share English lectures with others.

Adaptive synchronization framework for navigated hypermedia document presentation

Synchronization anomalies, or so-called out-of-sync problems, may occur in a composite multimedia document presentation especially when a presentation tool receives unexpected events that destroy intermedia relations in the document. This work proposes an adaptive synchronization framework (ASF) to solve this problem by adopting a resynchronization procedure. In contrast to the existing approaches focusing on temporal synchronization only, spatial and content relations for a more general presentation are also considered and incorporated into the proposed synchronization core. The experimental results show that the proposed framework can adapt a document to an unsupervised presentation environment and the performance is satisfactory (85%) under some synchronization-killer operations.

A marking-based synchronized multimedia tutoring system for composition studies

In this paper we propose a marking-based synchronized multimedia tutoring system (MSMT) to enhance our previous work, the Web-based synchronized multimedia lecturing system (WSML), for language learning in writing skill. For non-native users, writing is usually a barrier when they are learning a foreign language. Therefore we simulate several symbols which are common use in the traditional paper-based writing correction by state-of-the-art technologies. The system not only provides an easy-to-use interface for instructors to simplify the composition editing tasks, but also provides visualized editing markings for learners to grasp the writing mistakes. A system prototype has been implemented to support ESL composition studies. An overview page that presents the system features with demonstrations can be accessed at http://english-csie.ncnu.edu.tw/OWL/.

An Application System to Enable Multimedia on English Composition Studies

Nowadays, many applications support the capturing of lecturing information such as speech/video and the writings on the whiteboard to integrate with the presentation slides for later access. This method provides a meaningful way to keep a piece of information presented to audience in a classroom or seminar because they usually have to spend much time on perceiving the complex concepts conveyed by the instructor or speaker. With the successful integration of captured materials and access interfaces, students could review the lectures again and again. In this paper, we keep the core ideas of automated capture for classroom experience and structural access for multimedia documents in mind to develop an application which can be used to record the lecturing process when teaching/learning a language. All recorded materials of a lecture are packaged in the form of Web documents which can be accessed via the Internet (online) or from the local computers (offline).

Efficient Methods for Skimming the Web-Based Synchronization Multimedia Lectures

With the development of streaming technologies and e-learning platforms, more and more multimedia lectures integrating different media are accessible on the web. The massive multimedia lectures with huge and lengthy media decrease the learning efficiency. In this paper, two access modes are provided to help users to search significant information in multimedia lectures rapidly. We apply the temporal relationship among multimedia to search information in different granularities, and detect the significant segments by the analysis concerning the intonation and navigation events of lecturers to offer the user an abridged version of to comprehend a multimedia lecture. The integration of these two modes has been practically applied to Web-based Synchronization Multimedia Lecture system for facilitating the access and browse of multimedia lecture.

Web-Based Multimedia Authoring and Presentation Framework for Children's Art Cultivation

This paper proposes a web-based multimedia authoring and presentation framework to facilitate online pieces creation, sharing, and reuse in e-learning applications. Through the easy-to-use interface of the customized tools, students are allowed to create their own pieces online and then share with friends in a very easy way. To make the presentation of the piece more interesting, the authoring tool provides the capturing mechanism that records all significant authoring steps as synchronization information: temporal and spatial relations between composed objects. Synchronization module in rendering function enables users (author and receiver) to replay a vivid, animated multimedia presentation in a form of as close as possible to the original sense. A prototype has been implemented in a website especially designed for young student art learning, which has being conducted by the Military of Education, Taiwan, since 2003. The experimental results show that this hands-on experience in creating and sharing indeed benefits the efficacy of web-based e-learning.