Harrick M. Vin

Designing file systems for digital video and audio

We address the unique requirements of a multimedia file system such as continuous storage and retrieval of media, maintenance of synchronization between multiple media streams, and efficient manipulation of huge media objects. We present a model that relates disk and device characteristics to the recording rate, and derive storage granularity and scattering parameters that guarantee continuous access. In order for the file system to support multiple concurrent requests, we develop admission control algorithms for determining whether a new request can be accepted without violating the realtime constraints of any of the requests.We define a strand as an immutable sequence of continuously recorded media samples, and then present a multimedia rope abstraction which is a collection of individual media strands tied together by synchronization information. We devise operations for efficient manipulation of multi-stranded ropes, and develop an algorithm for maintaining the scattering parameter during editing so as to guarantee continuous playback of edited ropes.We have implemented a prototype multimedia file system, which serves as a testbed for experimenting with policies and algorithms for multimedia storage. We present our initial experiences with using the file system.

Communication architectures and algorithms for media mixing in multimedia conferences

The problem of media mixing that arises in teleconferencing applications such as teleorchestra is addressed. The mixing algorithm presented minimizes the difference between generation times of the media packets that are being mixed together in the absence of globally synchronized clocks, but in the presence of jitter in communication delays on packet switched networks. In order to support applications, such as teleorchestra, that involve a large number of participants, hierarchical mixing architectures are proposed, and it is shown that they are an order of magnitude more scalable than purely centralized or distributed architectures. Furthermore, mechanisms for minimizing the delays incurred by mixing in various communication architectures are presented. The mixing algorithms are implemented on a network of workstations connected by Ethernets, and the performance of various mixing architectures is experimentally evaluated. The results reveal the maximum number of participants that can be supported in a conference. >

Hierarchical conferencing architectures for inter-group multimedia collaboration

Advances in computer and communication technologies have stimulated the integration of digital video and audio with computing, leading to the development of various computer-assisted collaborations. In this paper, we propose a multi-level conferencing paradigm (called super conferences) for supporting collaborative interactions between geographically separated groups of users, with each group belonging to possibly a different organization. Hierarchical communication architectures are naturally suited for carrying out media transmission in super conferences. We study the performance of hierarchical communication architectures, and present algorithms for bounding end-to-end delays of real-time media traffic in them. We derive some interesting limits on the number of participants in a group and the number of groups within a super conference, so as not to violate bandwidth and delay requirements of multimedia. At the Multimedia Laboratory at the University of California, San Diego, we have implemented a conferencing system on an environment of Sun SPARCstations and PCATS equipped with digital video and audio processing hardware. As an interesting application of the conferencing system, we have developed a tele-presenter by which users can remotely attend lectures in progress. We present our initial experiences with using the system.

A window-based editor for digital video and audio

The authors present a window-based editor for manipulating digital video and audio. The editor supports real-time recording, playback, and editing (cut, copy, and paste) of several multimedia objects. Using the X Window system, the authors have implemented the editor on an environment of Sun SPARCstations, and PC-ATs equipped with video compression hardware. The user interface of the multimedia editor consists of a main editing window for each display device, and rope windows, which represent synchronized sequences of digital video and audio being accessed, called ropes. Experience shows that the editor provides a simple and easy to use, but powerful system for multimedia document preparation, and it can act as a basis for supporting applications such as multimedia mail, electronic distribution of television news and video entertainment, etc.<<ETX>>

Research: Techniques for multimedia synchronization in network file systems

One of the unique features that distinguishes digital multimedia from traditional computer data is the presence of multiple media streams, whose display must proceed in a mutually synchronized manner. The design of techniques for synchronization of multimedia data at the time of storage, and retrieval from network file servers is the subject matter of this paper. We present algorithms by which a file server can create a relative time system and synchronize media units transmitted by different sources on a network to construct a multimedia object. These algorithms stay robust in the absence of global clocks, presence of transmission jitter and generation rate mismatches. We develop a feedback technique using which the file server can detect asynchronies in display devices during retrieval of multimedia objects, and even restore synchrony by deleting or duplicating media units destined for asynchronous destinations. We then present strategies by which the file server can actually predict the time in future when the asynchrony of a device is expected to exceed the permitted bound, and take gradual preventive action to nullify the asynchrony in advance. These algorithms can be generalized to heterogeneous multimedia networks in which there may be variations in sizes of media units generated, differences in network locations of sources and destinations, etc. We are currently implementing these techniques in a digital multimedia on-demand storage server being developed at the UCSD Multimedia Laboratory.

Optical communication architectures for multimedia conferencing in distributed systems

The design of techniques and protocols for media mixing and communication architectures that optimize the performance of media mixing are discussed. The performance of monostage and multistage techniques for mixing are compared. Centralized and distributed architectures are analyzed for their suitability to both monostage and multistage media mixing, and hierarchical architectures that significantly reduce bandwidth consumption are presented. Algorithms for designing hierarchies that optimize real-time end-to-end delays and a packet train protocol in which the routing nodes can be used as mixers are proposed.<<ETX>>

Admission Control Algorithm for Multimedia On-Demand Servers

Future advances in networking coupled with the rapid advances in storage technologies will make it feasible to build a multimedia on-demand server on a metropolitan-area network. However, the economic viability of such a multimedia on-demand server is dependent on the ability to amortize its operational costs over a large number of clients. In this paper, we present a taxonomy of policies for servicing multiple clients, and propose round robin and quality proportional admission control algorithms for determining whether a retrieval request from a new client can be admitted without violating the real-time requirements of any of the clients already being serviced. We evaluate the performance of various servicing policies, and show that they are an order of magnitude scalable compared to straightforward multiplexing techniques such as servicing one subscriber per disk head.

Integrating virtual reality, tele-conferencing, and entertainment into multimedia home computers

It is pointed out that the integration of virtual reality techniques with multimedia teleconferencing leads to the development of tele-virtual conferencing systems, which synthesize panoramic, life-like three-dimensional video images and stereophonic audio. The authors investigate the architectural requirements of such systems and propose a high-level design of an intelligent multimedia interface unit (IMIU) capable of supporting tele-virtual conferencing in multimedia home computers. The IMIU is capable of supporting panoramic, life-like three-dimensional video images and stereophonic audio, so as to synthesize tele-virtual conferencing environments. >

Frame-Induced Packet Discarding: An Efficient Strategy for Video Networking

In order to provide efficient frame loss guarantees for video communication over ATM-like fast packet switched networks, we propose a simple to implement, yet effective, strategy called Frame-Induced Packet Discarding (FIPD), in which, upon detection of loss of a threshold number (determined by an applications video encoding scheme) of packets belonging to a video frame, the network attempts to discard all the remaining packets of that frame. Performance simulations are shown to demonstrate the efficacy of the FIPD strategy; networks employing FIPD exhibit close to two-fold increase in the number of video channels that they can support.

Media synchronization in distributed multimedia file systems

One of the unique features that distinguishes digital multimedia from traditional computer data is th e presence of multiple media streams (such as audio and video), whose display must proceed in a mutuall y synchronized manner . The design of techniques for synchronization of multimedia data at the time of storag e and retrieval from network file servers, which are essential in designing multimediaPon-demand servers ove r metropolitan area networks, is the subject matter of this paper . A multimedia object being stored on a network file server may he constituted of media component s that may be generated by different sources on the network . Furthermore, the media components may b e generated at different times, but their display may be required to be simultaneous (e .g ., audio dubbing i n movies) . Hence, it, is convenient to place all the media units constituting a multimedia object on a relative time scale, with the media units that are at the beginning of the object placed at zero on the scale . The position of a media unit on the relative time scale defines its relative time stamp (RTS) . Each media unit (such as a video frame or an audio sample) should be associated with a RTS . During retrieval, all thos e media units with the same RTS must be displayed simultaneously . Assignment of RTSs to media units is straight-forward in the absence of both rate mismatches betwee n 1/0 devices used for generation of media units and network jitter . However, in future integrated networks , I/O devices that lack the sophistication to run elaborate time rate synchronization protocols may be directl y (as opposed to via a host computer) to the network, e .g ., Etherphone, ISDN Telephone, etc . Furthermore , even in environments in which clocks are synchronized, compression may yield variations in sizes of medi a units, as a result of which the actual period of each media unit. may vary between p* (1 — c) and p * (1 + c) . where p is the nominal value and c the maximum fractional variation . The relative time scale is is based c m a master media source (all other sources are slave sources) whose choice is application dependent . The fi e server, when it receives media units from the master and slave devices, determines sets of media units tha t are generated within a tolerable window of asynchrony, A and assigns the the same RTS to media units that are in the same set . Such sets of media units are called synchronization sets : media units nm and n, fro m the master and slave devices, respectively, can form a synchronization set iff their generation times g(n,,, ) and g(n,) are such that Ig(nm) g(n,)I <,A . Owing to non-deterministic variations in transmission delays (which are assumed to be bounded betwee n Amin and Amax) of media units, the exact generation times of media units are not known to the file server . Determination of synchronization sets is therefore based on the the file servers estimates of the earliest an d latest possible generation times of media units from the master and slave sources . When media unit n i s received by the file server at time r, the file server can determine the earliest and latest possible generatio n times of media unit n, denoted by g e (n), and g t (n), t .o be : g e (n) = — ,mar, and g t (n) = r — z min . Give n the generation intervals [gn,(nm),g ;,ti(n,,,)] and [g ;(n,),g;(n,)] of the latest media units n m and n, receive d from the master and slave sources, respectively, the file server can determine that ti,,, and n, belong to a synchronization set if 6 gmar( n m, n ,) < .A (1 )