6.2 Semantics in Multimedia Databases

< Day Day Up >

The management of semantics in multimedia databases is an important concern. To this end, an important contribution of this book the demonstration that it is worthwhile to use semantic information at different levels of abstraction for querying a multimedia database. Examples of high-level abstraction queries are, "give me all sport news from the available TV channels about a penalty caused by a severe foul committed to Ronaldo," or, "give me all videos where Arnold Schwarzenegger is laughing." An example of a low-level semantic abstraction query is, "give me all scores by Klose where he wore the traditional German black and white dress" (match on color feature).

Semantics querying attracted much more attention with the emerging ISO/IEC MPEG-7 Standard. MPEG-7 (see Chapter 2) provides a standardized means of describing audiovisual data content in multimedia environments. Its scope is to facilitate the description of the content of multimedia data so that these data can be searched in a database and browsed, filtered, or interpreted by search engines, filter agents, or any other program. Integrating the semantic descriptors introduced in MPEG-7 into current retrieval tools is an important task. ^[2]

Semantic querying presupposes efficient semantic indexing. The best technique is a clever combination of manual and automatic indexing methods. We have shown that in many domains (e.g., sports), event detection methods are good enough to make a rough predetermination of the multimedia content and to render manual adaptation simpler. ^[3]

Using semantics in a MMDBMS is, however, difficult, as the semantics of multimedia data depend on the context in which it is used. Here the meaning of context is twofold. On the one hand, it represents the use of multimedia data in the current application, and on the other hand, it represents the overall placement of the content in the domain to which the application is applied.

Using semantic information equally enhances the performance of distributed multimedia database systems. For instance, in Kosch et al., ^[4] the efficiency of prefetching in a multimedia streaming server was improved by using semantic information on the contextual correlations of clips.

However, extracting semantic information cannot be done for free and, hence, involves annotation cost. Therefore, one has to balance the costs of extracting semantic information with the benefits to be obtained from using them. This balance should be determined by qualitative and quantitative analyses. For instance, the contextual information can easily be obtained during the annotation process through video shot detection and by considering the metadata of the already stored and annotated clips, and thus, it is nearly free.

Describing the semantics of multimedia is a complex problem. MPEG-7 provides a solution for generic multimedia content description. Describing the semantics of content is also domain dependent. For instance, the semantic entities differ if we consider a basketball clip (where persons are players and referees) and a political news clip (where persons are politicians, journalists, etc.). As a consequence, the definition of semantic entities should be left to the industrial bodies standardizing domain metadata; for example, NewsML (http://www.newsml.org) for multimedia content used in the news domain. To permit interoperability between the various domains where multimedia is used as the major exchange material, we have to supply a syntactical framework to integrate the different multimedia data models. MPEG-21 provides such a container with the Digital Item Model and enhances descriptions with their digital rights and intellectual properties. An example of this integration is given in Section 4.3. Related to this is the generic ontology by Hunter, ^[5] which is able to integrate information from multiple genres of multimedia content.

^[2]Lux, M. and Becker, J., XML and MPEG-7 for interactive annotation and retrieval using semantic meta-data, J. Universal Comput. Sci., 8, 965–985, 2003.

^[3]Kosch, H., Tusch, R., Böszörményi, L., Bachlechner, A., Dörflinger, B., Hofbauer, C., Riedler, C., Lang, M., and Hanin, C., SMOOTH—a distributed multimedia database system, in Proceedings of the International VLDB Conference, Rome, September 2001, pp. 713–714.

^[4]Kosch, H., Moustefaoui, A., and Brunie, L., Semantic based prefetching in news-on-demand video servers, Multimedia Tools Appl., 18, 169–179, 2002.

^[5]Hunter, J., Towards a core ontology for information integration, J. Digital Inf., 4(1), 2003.

< Day Day Up >