Bryan S. Maher

Clever clustering vs. simple speed-up for summarizing rushes

This paper discusses in detail our approaches for producing the submitted summaries to TRECVID, including the two baseline methods. The cluster method performed well in terms of coverage, and adequately in terms of user satisfaction, but did take longer to review. We conducted additional evaluations using the same TRECVID assessment interface to judge 2 additional methods for summary generation: 25x (simple speed-up by 25 times), and pz (emphasizing pans and zooms). Human assessors show significant differences between the cluster, pz, and 25x approaches. The best coverage (text inclusion performance) is obtained by 25x, but at the expense of taking the most time to evaluate and perceived as the most redundant. Method pz was easier to use than cluster and had better performance on pan/zoom recall tasks, leading into discussions on how summaries can be improved with more knowledge of the anticipated users and tasks.

RumbleBlocks: Teaching science concepts to young children through a Unity game

RumbleBlocks was developed at the Entertainment Technology Center (ETC) to teach engineering principles of tower stability to children ages 4-7. The game features tower construction, tower piece removal, and tower comparison levels which were designed with feedback from early childhood educators and learning researchers, and iteratively improved with feedback from child play tests. This paper emphasizes the development process, and initial formative play tests with children. It was developed using the Unity3D game engine, allowing for export as a stand-alone application, web player, or to mobile devices. First results are promising in terms of educational effectiveness, with more studies planned for the future.

XSLT for tailored access to a digtal video library

Surrogates, summaries, and visualizations have been developed and eval uated for accessing a digital video library containing thousands of documents and terabytes of data. These interfaces, formerly implemented within a monolithic stand-alone application, are being migrated to XML and XSLT for delivery through web browsers. The merits of these interfaces are presented, along with a discussion of the benefits in using W3C recommendations such as XML and XSLT for delivering tailored access to video over the web.

Exploring the utility of fast-forward surrogates for bbc rushes

This paper discusses in detail our approaches for producing the video summaries submitted to the TRECVID 2008 BBC rushes summarization task, including the baseline method. Empirical work produced during and after the TRECVID 2007 rushes summarization task gave strong evidence that a simple 50x method (sampling every 50th frame) provides excellent coverage (text inclusion performance). Our submissions for TRECVID 2008 investigated the effects of junk frame removal, including a comprehensible audio track, and emphasizing pans and zooms when backfilling to reclaim the space removed with the noise shots from the original 50x set. Results show that 50x based methods provide excellent coverage as expected. There were limited effects for the other strategies to improve user satisfaction, with the discussion providing some insights for future video summary development and evaluation work.

Evaluating audio skimming and frame rate acceleration for summarizing BBC rushes

For the first time in 2007, TRECVID considered structured evaluation of automated video summarization, utilizing BBC rushes video. In 2007, we conducted user evaluations with the published TRECVID summary assessment procedure to rate a cluster method for producing summaries, a 25x (sampling every 25th frame), and pz (emphasizing pans and zooms). Data from 4 human assessors shows significant differences between the cluster, pz, and 25x approaches. The best coverage (text inclusion performance) is obtained by 25x, but at the expense of 25x taking the most time to evaluate and judged as being the most redundant. Method pz was easier to use than cluster and rated best on redundancy. A question following the TRECVID workshop was whether simple speed-ups would still work at 50x or 100x, leading to a study with 15 human assessors looking at pzA (pz but with better audio), 25x, 50x, and 100x summaries (these latter 3 with an unsynchronized more comprehensive audio track as well). 100x gives the fastest time on task but with poor usability and performance. PzA gives the best usability measures but poor time on task and performance. 25x does well on performance as before, with 50x doing just as well but with much less time on task and better ease of use and redundancy scores. Based on these results, 50x with its audio skimming is recommended as the best way to summarize video rushes materials.

Beanstalk: A unity game addressing balance principles, socio-emotional learning and scientific inquiry

Beanstalk is an educational game for children ages 6-10 teaching balance-fulcrum principles while folding in scientific inquiry and socio-emotional learning. This paper explores the incorporation of these additional dimensions using intrinsic motivation and a framing narrative. Four versions of the game are detailed, along with preliminary player data in a 2×2 pilot test with 64 children shaping the modifications of Beanstalk for much broader testing.

Enhanced exploration of oral history archives through processed video and synchronized text transcripts

A digital video library of over 900 hours of video and 18000 stories from The HistoryMakers was used by 266 students, faculty, librarians, and life-long learners interacting with a system providing multiple search and viewing capabilities over a trial period of several months. User demographics and actions were logged with this multimedia collection, providing quantitative and qualitative metrics on system use. These transaction logs were complemented with heuristic evaluation, interviews, and contextual inquiry with representative users. Collectively, these mixed methods informed the development of the next generation web-based interface for the HistoryMakers video oral histories to improve access to and dissemination of this rich cultural resource. In particular, the feature of a synchronized text transcript in the video player for the narratives merited further investigation. Such an interface has not seen widespread use in digital video players available on the web, yet was valued highly by oral history archive viewers. A user study with 27 participants measured the utility of the HistoryMakers web interface incorporating the synchronized transcript video player for stated fact-finding and open-ended tasks. For life oral histories, an aligned text transcript is valued for both tasks, with the video rated significantly more useful for open-ended tasks over fact-finding. These results suggest a task-dependent role of modality in presentation of oral histories, with synchronized transcripts rated highly across tasks.

Analysis of transaction logs for insights into use of life oral histories

A digital video library of over 900 hours of video and 18000 stories from The HistoryMakers was used by 214 students, faculty, librarians, and life-long learners interacting with a system providing multiple search and viewing capabilities over a trial period of several months. User demographics and actions were logged, providing metrics on how the system was used. This poster overviews a few highlights from these transaction logs of the Informedia digital video library system for life oral histories.

Helios: An HTML5 game teaching proportional reasoning to child players

Helios was developed at the Entertainment Technology Center (ETC) to teach proportional reasoning to children ages six through ten. The game features balancing and unbalancing levels which were designed with input from early childhood educators and learning researchers, and iteratively updated based on results of child play tests. Such feedback universally recommended making the balancing exercises less repetitive and introducing opportunities for more player engagement. Helios also addresses scientific inquiry steps of hypothesis formation and explanation and socio-emotional learning in terms of discussing with an in-game peer. This paper emphasizes the development process and presents problems uncovered during the evolution of the balance game and their resolution, contextualizing the discussion with references to intrinsic motivation literature. It reports on a specific formative play test with 17 children. It was developed using the ImpactJS game engine, allowing for use across major web browsers without additional plug-ins. Lessons learned regarding the formative test are shared, culminating in a series of next steps for Helios development.

An interactive flash website for oral histories

Automatic speech alignment and natural language processing technologies provide full content search and retrieval access into oral history collections. These tools have been field-tested with The HistoryMakers and Harrisburg Living Legacy oral history archives, showing the value of an Adobe Flash front-end interface. Built with Adobe Flex 3, the interface works across browsers and operating systems, supports deep linking and browser-based navigation, provides synchronized transcripts that can be fully searched and tracked while watching the interviews, and incorporates filtering by facets, a menu bar breadcrumb interface, and a user play list to collect stories of interest. Refinements to the interface are discussed following the first six months of web deployment, with suggestions offered for other digital video libraries, particularly oral histories.