Brian C. Smith

Query by humming: musical information retrieval in an audio database

The emergence of audio and video data types in databases will require new information retrieval methods adapted to the specific characteristics and needs of these data types. An effective and natural way of querying a musical audio database is by humming the tune of a song. In this paper, a system for querying an audio database by humming is described along with a scheme for representing the melodic information in a song as relative pitch changes. Relevant difficulties involved with tracking pitch are enumerated, along with the approach we followed, and the performance results of system indicating its effectiveness are presented.

Passive capture and structuring of lectures

Despite recent advances in authoring systems and tools, creating multimedia presentations remains a labor-intensive process. This paper describes a system for automatically constructing structured multimedia documents from live presentations. The automatically produced documents contain synchronized and edited audio, video, images, and text. Two essential problems, synchronization of captured data and automatic editing, are identified and solved.

ThinStreams: an architecture for multicasting layered video

Multicast is a common method for distributing audio and video over the Internet. Since receivers are heterogeneous in processing capability, network bandwidth, and requirements for video quality, a single multicast stream is usually insufficient. A common strategy is to use layered video coding with multiple multicast groups. In this scheme, a receiver adjusts its video quality by selecting the number of multicast groups, and thereby video layers, it receives. Implementing this scheme requires the receivers to decide when to join a new group or leave a subscribed group. This paper presents a new solution to the join/leave problem using ThinStreams. In ThinStreams, a single video layer is multicast over several multicast groups, each with identical bandwidth. ThinStreams separates the coding scheme (i.e., the video layers) from control (i.e., the multicast groups), helping to bound network oscillations caused by receivers joining and leaving high bandwidth multicast groups. This work evaluates the join/leave algorithms used in ThinStreams using simulations and preliminary experiments on the MBONE. It also addresses fairness among independent video broadcasts and shows how to prevent interference between them.

MPEG Video in Software: Representation, Transmission, and Playback

A software decoder for MPEG-1 video was integrated into a continuous media playback system that supports synchronized playing of audio and video data stored on a file server. The MPEG-1 video playback system supports forward and backward play at variable speeds and random positioning. Sending and receiving side heuristics are described that adapt to frame drops due to network load and the available decoding capability of the client workstation. A series of experiments show that the playback system adds a small overhead to the stand alone software decoder and that playback is smooth when all frames or very few frames can be decoded. Between these extremes, the system behaves reasonably but can still be improved.

Characterizing User Access To Videos On The World Wide Web

Despite evidence of rising popularity of video on the web (or VOW), little is known about how users access video. However, such a characterization can greatly benefit the design of multimedia systems such as web video proxies and VOW servers. Hence, this paper presents an analysis of trace data obtained from an ongoing VOW experiment in Lulea University of Technology, Sweden. This experiment is unique as video material is distributed over a high bandwidth network allowing users to make access decisions without the network being a major factor. Our analysis revealed a number of interesting discoveries regarding user VOW access. For example, accesses display high temporal locality: several requests for the same video title often occur within a short time span. Accesses also exhibited spatial locality of reference whereby a small number of machines accounted for a large number of overall requests. Another finding was a browsing pattern where users preview the initial portion of a video to find out if they are interested. If they like it, they continue watching, otherwise they halt it. This pattern suggests that caching the first several minutes of video data should prove effective. Lastly, the analysis shows that, contrary to previous studies, ranking of video titles by popularity did not fit a Zipfian distribution.

CU-SeeMe VR immersive desktop teleconferencing

Current video-conferencing systems provide a “video-ina-window” user interface. This paper presents a distributed video-conferencing system called CU-SeeMe VR that embeds live video and audio conferencing in a virtual space. This paper describes a prototype implementation of CU-SeeMe VR, including the user interface, system architecture, and a detailed look at the enabling technologies. Future directions and the implications of the virtual reality meta phor are discussed.

Compressed domain transcoding of MPEG

Current video compression formats optimize for either compression or editing. For example, motion-JPEG (MJPEG) provides excellent random access and moderate overall compression, while MPEG optimizes for compression at the expense of random access. Converting from one format to another, a process called transcoding, is often desirable over the life of a video segment. The paper shows how to transcode MPEG-1 video to motion-JPEG without fully decompressing the MPEG-1 source. The described technique for compressed domain transcoding differs from previous work because it uses a new approximation approach that is optimized for software implementations. This new approach is 1.5 to 3 times faster than spatial domain transcoders and offers an additional degree of freedom: higher transcoding speeds can be obtained at the price of lower picture quality. This speed/quality trade-off is useful in many real time applications such as offline editing and video gateways.

Experiment to characterize videos stored on the Web

The design of file systems is strongly influenced by measuring the use of existing file systems, such file size distribution and patterns of access. We believe that a similar characterization of video stored on the Internet will help network engineers, codec designers, and other multimedia researchers. We therefore executed an experiment to measure how video data is used on the Web today. In this experiment, we downloaded and analyzed over 57,000 AVI, QuickTime and MPEG files stored on the Web -- approximately 100 gigabytes of data. Among our more interesting discoveries, we found that the most common video technology in use today is QuickTime, and that the image resolution and frame rate of video files that include audio are much more uniform than video-only files. The majority of all audio/video files have dimensions of CIF or QCIF (or very similar) at 10, 12, 15, or 30 fps, whereas the dimensions and frame rates of video-only files are more uniformly distributed. We also experimentally verified the conjecture that current Internet bandwidth is at least an order of magnitude too slow to support streaming playback of video. We present these results and other statistical information characterizing video on the web in this paper.

The PICASSO applications framework

PICASSO is a graphical user interface development system that includes an interface toolkit end an application framework. The application framework provides high-level abstractions including modal dialog boxes end non-modal frames end partels simh.r to conventional programming language procedures and co-routines. These abstractions can be used to define objects that have local variables and that can be called with parameters. PICASSO also has a constraint system that is used to bind program variables to widgets, to implement triggered behaviors, and to itnplement multiple views of data. The system is implemented in Common Lisp using the Common Lisp Object System and the CLX interface to the X Window System.

Compressed Domain Processing of JPEG-encoded imaages☆

Abstract This paper addresses the problem of processing motion-JPEG video data in the compressed domain. The operations covered are those where a pixel in the output image is an arbitrary linear combination of pixels in the input image, which includes convolution, scaling, rotation, translation, morphing, de-interlacing, image composition, and transcoding. This paper further develops an approximation technique calledcondensationto improve performance and evaluates condensations in terms of processing speed and image quality. Using condensation, motion-JPEG video can be processed at near real-time rates on current generation workstations.