Agma Juci M. Traina and Caetano Traina Jr.
Department of Computer Science
University of S o Paulo at S o Carlos
S o Carlos, S o Paulo, Brazil
A noticeable characteristic of the current information systems is the ubiquitous use of multimedia data, such as image, audio, video, time series and hypertext. The processing power demanded by the increasing volume of information generated by business and scientific applications has motivated researchers of the data base, artificial intelligence, statistics and visualization fields, who have been working closely to build tools that allow to understand and to efficiently manipulate this kind of data. Besides the multimedia data being complex, they are voluminous as well. Thus, to deal with the multimedia data embedded in the current systems, much more powerful and elaborate apparatuses are required as compared to the needs of the simple data handled by the old systems.
The main challenges concerning complex data are twofold. The first one is how to retrieve the relevant information embedded inside them, allowing the systems to process the data in a straightforward manner. The second challenge refers to how to present the data to the users, allowing them to interact with the system and to take advantage of the data semantics. Therefore, a system dealing with multimedia data has to tackle these two issues: feature extraction and data presentation and browsing. In this chapter, we discuss more deeply the first issue and give directions to the readers for the second issue.
A common problem regarding multimedia or complex data is how to compare and group them using only their embedded information, that is, how to compare and group video, images, audio, DNA strings, fingerprints, etc., based on how similar their contents are. Thus, content-based retrieval systems have been extensively pursued by the multimedia community during the last few years. Initially the specialists in content-based retrieval started working on images. Hence there are many works on content-based image retrieval tackling different approaches  , such as those based on color distribution   , shape   and texture  .
With the growing amount of video data available in multimedia systems, such as digital libraries, educational and entertaining systems among others, it became important to manage this kind of data automatically, using the intrinsic characteristics of video. Consequently, content-based video retrieval systems use different views or parts of video data, that is, emphasizing on the spatial aspect of video and annotations  or on the audio portion  of them.
Obtaining the main characteristics or properties of multimedia data is not a simple task. There are many aspects in each kind of data to be taken in account. For example, when dealing with images, it is well known that the main characteristics or features to be extracted are color distribution, shape and texture  . The features extracted are usually grouped into vectors, the so-called feature vectors. Through the features, the content-based image retrieval system can deal with the meaningful extracted characteristics instead of the images themselves. Feature extraction processes correspond in some way to data reduction processes, as well as consist of a manner to obtain a parametric representation of the data that is invariant to some transformations. Recalling the previous example of images, there are techniques that allow comparison on images represented in different sizes, positioning, brightness during acquisition and even different shapes  . However, a common problem, when reducing the information through feature vectors, is that some ambiguity is introduced in the process. That is, two or more distinct objects can be represented by the same feature values. When this happens, other steps of comparisons, using other features of the data must be done to retrieve the desired data.
This chapter discusses techniques for searching multimedia data types by similarity in databases storing large sets of multimedia data. The remainder of this chapter is organized as follows. Section 2 discusses the main concepts involved in supporting search operations by similarity on multimedia data. Section 3 presents a flexible architecture to build content-based image retrieval in relational databases. Section 4 illustrates the application of such techniques in a practical environment allowing similarity search operations. Finally, section 5 presents conclusions regarding future developments in the field, tackling the wide spread of content-based similarity searching in multimedia databases.