1.1 Multimedia Content: Context

< Day Day Up >

Exhibit 1.3 shows an average picture.

Exhibit 1.3: Average picture.

click to expand

It is not so much that a picture is worth a thousand words, for many fewer words can describe a still picture for most retrieval purposes, the issue has more to do with the fact that those words vary from one person to another.^[4],

A picture is worth a thousand words.

The perceptual quality of an image differs from one viewer to another, and thus the adage is certainly true. Sometimes it is just impossible to summarize the content or the message or the characteristics of a picture in words—not to mention to do so in just a few words.

… the issue has more to do with the fact that those (thousands) words vary from one person to another.

To complicate matters, the same picture may elicit different responses from different viewers, who may use very different words to describe the picture. Therefore, in an image database the use of words or keywords may fail to be an effective way of retrieving the data desired or of classifying the images stored. Thus, one needs a metadata model of multimedia data that describes the semantics of the multimedia objects and how they are related to each other.

A picture is worth a thousand words, but how do you find it?

Finding multimedia objects by their content in a database means searching on the basis of content descriptions and similarity measures. Currently available multimedia database management systems (MMDBMSs) promise to efficiently find multimedia objects by their content. They often claim that they can manage huge quantities of multimedia data and can support many different applications. ^[6], ^[7], ^[8], ^[9] For instance, if one investigates Oracle's Technet Network at http://technet.oracle.com for applications using multimedia data through the use of their integrated search engine and through browsing, one may find out that Oracle 9i promises to support video on demand, home shopping, geographic information systems, computer-aided design, photographic libraries, art gallery and museum management, and educational systems as well as E-applications including E-business, E-commerce, and so forth.

Less poetic: Can I get and view this image with my mobile phone?

A number of mechanisms have been proposed to meet the demands of multimedia delivery and presentation in the (mobile) Internet. Discrete media transport is less critical to real-timeliness, but gives only a partial view of the complete multimedia world. It is conjectured that continuous media and, in particular, video flow will make up a substantial part of future network traffic, as it is an increasingly important media type in distributed information, education, and entertainment systems. ^[10]

The key for the effective use of multimedia in our usage and terminal environment is to guarantee the required (by user, terminal) level of QoS. QoS support in distributed multimedia systems has been a vivid research field for many years and has led to the development of sophisticated QoS architectures that address the issue of end-to-end QoS support for multimedia communications (rather than investigating individual architectural layers). A survey is available in Aurrecoechea. ^[11] The relationship between content and media is yet not very clear. MPEG-21 ^[12] is currently establishing a common multimedia framework for the resource (the media itself) and its conceptual content described by metadata.

We are still at the beginning of the development of techniques to guarantee efficiency and effectiveness in content-based multimedia systems. The premature state of these technologies is caused by the peculiar nature of multimedia; that is, heterogeneity of the multimedia data (continuous vs. static data); higher storage, processing, and communication requirements than needed for traditional data; difficulty of semantic indexing; complex multimedia data modeling; and difficulty of adaptation to different applications and resource availabilities such as network bandwidth. To make content-based multimedia systems fully available for commercial applications, one has to consider the following topics:

Retrieval and indexing. Hardly any available content-based retrieval (CBR) techniques can answer a request like, "give me all video sequences showing Roger Moore laughing." ^[13], ^[14] Video indexing schemes, based on similarity matches relying on low-level color and texture information, can only be the first step for really useful retrieval techniques. ^[15], ^[16] Semantically meaningful indexing methods must be developed and linked to low-level indexing and retrieval. ^[17]
Storage, communication, and performance. Some multimedia data (especially video) has several orders of magnitude higher storage and network bandwidth requirements than does text (or even images). ^[18] Compression techniques have reached a mature level for most applications and have been standardized in various communities (e.g., the ISO-IEC MPEG [http://www.chiariglione.org/mpeg/] and ITU-T Video Coding Experts Group [VCEG; http://www.itu.int/ITU-T/]). Nevertheless, the adaptation of the digitally stored representation of the multimedia data to different applications, to different network bandwidths, and to different presentation devices is an open issue. This is, however, the key to achieving a level of performance that enables satisfactory usage of commercial applications including multimedia content. ^[19], ^[20]
Streaming and resource scheduling. Continuous data (such as audio and video) needs continuous transport over the network and continuous presentation to the user. This demands resource-scheduling techniques providing guaranteed QoS at the server, as well as in the network and at the client side. ^[21]
Authoring. Authoring is mostly an unsolved problem. ^[22] There are some tools for special applications, such as video cutting, but even these have not yet reached a mature status. ^[23] There are some tools for annotating videos (e.g., that provided by Virage, http://www.virage.com), but there are no general tools for creating complex scenarios, such as an advertisement containing text, images, animated and natural video, and audio.
Presentation. Similar to authoring, presentation of complex scenarios not just consisting of all types of data but also requiring a set of spatial and temporal constraints on their presentation is another interesting issue. ^[24] In an advertisement, for example, we may be interested in presenting first a background image and music, and then showing a video clip, which is subsequently enhanced by an animation and some blinking text. Especially interesting is the mapping of complex presentations onto widely differing devices ranging from mobile phones to high-performance workstations. Moreover, the management of presentation delivery in video servers is an open research question. ^[25], ^[26]

^[4]Taken from Keister, Lucinda H., User types and queries: impact on image access systems, in Challenges in Indexing Electronic Text and Image, Fidel, Raya, et al., Eds. ASIS Monograph Series, Learned Information, Medford, NJ, 1994, pp. 7–22. ^[5]

^[4]Apers, P.M.G., Blanken, H.M., and Houtsma, M.A.W., Multimedia Databases in Perspective, Springer-Verlag, Heidelberg, 1997.

^[6]Apers, P.M.G., Blanken, H.M., and Houtsma, M.A.W., Multimedia Databases in Perspective, Springer-Verlag, Heidelberg, 1997.

^[7]Subrahmanian, V.S., Principles of Multimedia Database Systems, Morgan Kaufman Press, San Francisco, January 1998.

^[8]Lu, G., Multimedia Database Management Systems, Artech House, Norwood, MA, 1999.

^[9]Chen, S.-C., Kashyap, R.L., and Ghafoor, A., Semantic Models for Multimedia Database Searching and Browsing, Kluwer Academic, Boston, 2000.

^[10]Xiao, X. and Ni, L.M., Internet QoS: a big picture, IEEE Network, 13, 8–18, 1999.

^[11]Aurrecoechea, C., Campbell, A.T., and Hauw, L., A survey of QoS architectures, ACM Multimedia Syst., 6, 128–151, 1998.

^[12]Hill, K. and Bormans, J., Overview of the MPEG-21 Standard. ISO/IECJTC1/SC29/WG11 N4041 (Shanghai Meeting), October 2002, http://www.chiariglione.org/mpeg/.

^[13]Park, Y., Kim, P., Golshani, F., and Panchanathan, S., Concept-based visual information management with large lexical corpus, in Proceedings of the Internation Conference on Database and Expert Applications (DEXA), Munich 2001, Springer-Verlag, Heidelberg, pp. 350–359, LNCS 2113.

^[14]Li, W.-S., Selçuk Candan, K., Hirata, K., and Hara, Y., Supporting efficient multimedia database exploration, VLDB J., 9, 312–326, 2001.

^[15]Chen, S.-C., Kashyap, R.L., and Ghafoor, A., Semantic Models for Multimedia Database Searching and Browsing, Kluwer Academic, Boston, 2000.

^[16]Brewer, E.A., When everything is searchable, Comm. ACM, 44, 53–54, 2001.

^[17]Zhao, R, and Grosky, W.I., Negotiating the semantic gap: from feature maps to semantic landscapes, Pattern Recognition, 35, 51–58, 2002.

^[18]Steinmetz, R., Multimedia Technology, 2nd ed., Springer-Verlag, Heidelberg, 2000.

^[19]Gecsei, J., Adaptation in distributed multimedia systems, IEEE MultiMedia, 4, 58–66, 1997.

^[20]Rejaie, R., On design of Internet multimedia streaming applications: an architectural perspective, in Proceedings of the IEEE International Conference on Multimedia and Exhibition, New York, July 2000, pp. 327–330.

^[21]Dan, A., Feldman, S.I., and Serpanos, D.N., Evolution and challenges in multimedia, IBM J. Res. Dev., 24, 177–184, 1998.

^[22]Böszörményi, L., Hellwagner, H., and Kosch, H., Multimedia technologies for e-business systems and processes, in Proceedings of Elektronische Geschäftsprozesse (E-Business Processes), Klagenfurt, Austria, September 2001, IT Verlag für Informationstechnik, pp. 471–481.

^[23]Anderson, D.P., Device reservation in audio/video editing systems, ACM Trans. Computer Syst., 15, 111–133, 1997.

^[24]Raymond, T.N. and Paul, S., Optimal clip ordering for multi-clip queries, VLDB J., 7, 239–252, 1998.

^[25]Raymond, T.N. and Paul, S., Optimal clip ordering for multi-clip queries, VLDB J., 7, 239–252, 1998.

^[26]Bouras, C., Kapoulas, V., Miras, D., Ouzounis, V., Spirakis, P., and Tatakis, A., On-demand hypermedia/mutimedia service using pre-orchestrated scenarios over the Internet, Networking Inf. Syst. J., 2, 741–762, 1999.

< Day Day Up >