Chapter 33: Streaming Multimedia Presentations in Distributed Database Environments

Aidong Zhang, Ramazan Savaş Ayg n, and Yuqing Song
Department of Computer Science and Engineering
State University of New York at Buffalo
Buffalo, NY, USA
{azhang,aygun,ys2}@cse.buffalo.edu

1. Introduction

There has been increasing interest in flexibly constructing and manipulating heterogeneous presentations from multimedia data resources [34] to support sophisticated applications [26]. In these applications, information from multimedia data resources at one location must be made available at other remote locations for purposes of collaborative engineering, educational learning and tutoring, interactive computer-based training, electronic technical manuals and distributed publishing. Such applications require that basic multimedia objects be stored in multimedia databases/files. The multimedia objects, such as audio samples, video clips, images and animation, are then selectively retrieved, transmitted and composed for customized presentations. In order to support these advanced applications, novel approaches to network-based computing are needed to position adaptive quality-of-service (QoS) management as its central architectural principle. One of the key problems is to design the end system software to be coupled to the network interface. Also, in the best-effort network environment such as Internet, the transmission of media data may experience large variations in available bandwidth, latency and latency variance. Consequently, network delays may occur in the delivery of media data. To meet the demands of the advanced applications over the Internet in the real-world, end system software must have the ability to efficiently store, manage, and retrieve multimedia data. Fundamental principles and advanced techniques need be developed with which one can design an end system software to be integrated with the network to support sophisticated and customized multimedia applications.

Various research has been conducted to develop techniques for distributed multimedia systems. Collaboration between clients and servers has been addressed to support a globally integrated multimedia system [49,30]. In particular, buffering and feedback control for multimedia data transmission in the distributed environment have been studied. In [41,42], server-centric feedback strategies are proposed to maintain synchronized transmission and presentation of media streams. In [38], feedback was used to design robust resource management over the network. In [30], a backward feedback control approach is designed to maintain loosely-coupled synchronization between a server and a client. In [22], an approach is given to use the buffer, a priori information and the current network bandwidth availability to decide whether one should increase the quality of the video stream when more network bandwidth becomes available. An approach on the integration of buffering and feedback control can be found in [37].

Substantial research has been conducted to reduce the burstiness of variable-bit-rate streams by prefetching data [17,43,23]. Especially many bandwidth smoothing algorithms have been proposed for this task. These algorithms smoothen the bandwidth requirements for retrieval of the video stream from the video server and transmission across the network. Given a fixed client buffer, it is possible to minimize the peak bandwidth requirement for the continuous delivery of the video stream while minimizing the number of bandwidth rate increases [24]. Another scheme aims at minimizing the total number of bandwidth changes [21]. It is also possible to minimize the variability of the bandwidth [44], the buffer utilization [18] or adhering to a buffer residency constraint [19].

Flow and congestion control for multimedia data transmission in best effort networks has also been investigated [46,8,10,25]. These approaches assume no direct support for resource reservation in the network and attempt to adapt the media streams to current network conditions. Best effort transmission schemes adaptively scale (reduce or increase) the bandwidth requirements of audio/video streams to approximate a connection that is currently sustainable in the network. Best effort schemes split into TCP-friendly and non-TCP-friendly mechanisms. TCP-friendly algorithms avoid starvation of TCP connections by reducing their bandwidth requirements in case of packet loss as an indicator of congestion [46,25,47,13,11]. Non-TCP-friendly schemes reduce rates only to guarantee a higher QoS to its own stream, thereby forcing TCP connections to back off when competing for insufficient network bandwidth [16,31,7,12].

Recent research also addresses the need for a middleware framework in transmitting multimedia data. The middleware is located between the system level (operating system and network protocol) and the applications [6]. A good conceptual model of the middleware is described in [40]. It proposes a real-time middleware for asynchronous transfer mode (ATM) systems. It provides virtual connection setup, bandwidth reservation and session synchronization. A middleware control structure is described in [35], which requires applications to implement observation methods for respective system resources and proper adaptation policies within the application. However, there is a lack of a middleware structure, which can effectively put all component techniques together as multimedia middleware services and provide a mapping from the high-level requests of the application to the low-level implementation of the system to support the above mentioned customized multimedia applications.

Multimedia presentation and organization have also been studied at different levels with various models. The proposed synchronization models usually have their own specification, limitations on the user interactions and flexibility on the presentation. Time-based models do not provide the necessary flexibility that is needed for network presentations. Timed Petri Nets are first introduced for multimedia presentations in OCPN [36]. Gibbs [27] proposed a way of composing objects through BLOB stored in the databases and created objects by applying interpretation, derivation and temporal composition to a BLOB. The temporal composition is based on the start times of media objects on a timeline. NSync [5] is a toolkit that manages synchronous and asynchronous interactions, and finegrained synchronization. The synchronization requirements are specified by synchronization expressions having syntax When {expression} {action}. The synchronization expression semantically corresponds to "whenever the expression becomes true, invoke the corresponding action." Time-based models usually keep the start time and duration of each stream and these models modify the duration and start time after each interaction of the system or the user. In an event-based model, the start of a stream depends on an event signal. SSTS [39] is a combination of Firefly's [9] graph notation and transition rules of OCPN [36]. FLIPS [45] is an event-based model that has barriers and enablers to satisfy the synchronization requirements at the beginning and the end of the streams. PREMO [29] presents an event-based model that also manages time. They have synchronization points, which may also be AND synchronization points to relate several events. Time for media is managed with clock objects and time synchronizable objects, which contain a timer. Multimedia synchronization using ECA rules is covered in [2,3,4]. These projects show how synchronization requirements can be modeled using ECA rules. A hierarchical synchronization model that has events and constraints is given in [14].

This chapter presents the design strategies of a middleware for client-server distributed multimedia applications. The middleware, termed NetMedia, provides services to support synchronized presentations of multimedia data to higher level applications. In the NetMedia environment, an individual client may access multiple servers to retrieve customized multimedia presentations, and each server simultaneously supports multiple clients. NetMedia is capable of flexibly supporting synchronized streaming of continuous media data across best-effort networks. To achieve this, we design a multi-level buffering scheme to control the collaborations between the client and server for flexible data delivery in distributed environments, an end-to-end network delay adaptation protocol, termed DSD, to adjust the sending rate for each stream according to the optimum network delay [50] and a flow and congestion control protocol, termed PLUS, which utilizes probing of the network status to avoid congestion rather than react to it. The DSD and PLUS schemes integrate transmission support strategies and robust software systems at both the server and client ends to dynamically handle adaptive quality-of-service (QoS) management for customized multimedia presentations. The organization of streams and inter-stream synchronization are maintained by using synchronization rules, which are based on event-condition-action rules. The synchronization rules are handled by receivers, controllers and actors to provide consistent presentations over networks. This chapter demonstrates how these schemes are used to enforce the interactive control (such as fast forward/backward playback, pause, seek and slow motion) with immediate response for all multimedia streams. This chapter provides an integrated solution to buffer management, congestion control, end-to-end delay variations, interactivity support and synchronization of streams.

The chapter is organized as follows. Section 2 presents the overall architecture of the NetMedia and its module design. Section 3 proposes the buffer management at both server and client sites. Section 4 introduces two end-to-end flow control schemes to address network delay and data loss. Section 5 discusses the inter-stream synchronization. Section 6 gives conclusions.