Chapter 18: Video Intelligent Exploration

Chabane Djeraba
IRIN, Polytech'Nantes, Nantes University, France
<[e-mail]>

Younes Hafri and Bruno Bachimont
INA, Bry-Sur-Marne, France
<[e-mail]>, <[e-mail]>

1. Introduction

Content-based video exploration (retrieval, navigation, browsing) deals with many applications such as news broadcasting, video archiving, video clip management, advertising, etc.

The different applications mean different exploration parameters. In advertising applications, for example, scenes are generally short. An example of exploration may concern the fact of retrieving all document related to a style of shooting. In video clips, an example of retrieving documents may be based on some dance step described by, e.g., a sketch. In distant learning applications, we distinguish two types of educational documents. In the first one, audio stream presents the main content of tutorials. In the second one, both visual contents and audio stream highlight practical courses. Finally, in broadcasting applications, real-time interactive detection and identifications are of importance.

Different applications mean, too, different user behaviors. Any exploration system should help final users to retrieve scenes within large video bases. Three different user behaviors may be distinguished. In the first one, the user knows that the targeted scene is in the video base; the user will keep exploration until he finds the document. He should be able to describe the scene he looks for, and would be able to see at the first glance whether a suggested scene corresponds to his query. In the second one, the objective is to provide an accurate exploration tool, so that the user may decide as soon as possible whether the target is in the video base or not. In this case, the user does not know if the target scene exists in the video base. In the third one, the user has not a specific scene in mind and the search is fuzzy. The user simply explores video bases based on topics or some event occurring within it. In this case, the exploration should be highly structured, in order to drive the user through the big amounts of video bases. Since the user is not supposed to know exactly what he looks for, he should be able to scroll all responses quickly for selecting the relevant or irrelevant ones. Relevance feedback may be interesting in this case, since it permits the user to ameliorate interactively the quality of explorations. It is therefore essential that exploration systems have to represent retrieval units by compressed, complete and comprehensive descriptions. For all cases, textual annotations or visual characteristics, stored on video bases, should support the content exploration of video documents.

Based on these different applications, exploration parameters and user behaviors, it is evident that there are strong dependencies between content-based video exploration and different domains and usages. The fundamental requirements that ensure the usability of such systems are: obtaining compressed and exhaustive scene representations, and providing different exploration strategies adapted to user requirements.

The scope of the paper deals with the second requirement by investigating a new form of exploration based on historical exploration of video bases and user profiles. This new form of exploration seems to be more intelligent than traditional content-based exploration. The notion of intelligent exploration depends strongly on user-adaptive (profiling) exploration notion.

This new form of exploration induces the answer to difficult problems. An exploration system should maintain over time the inference schema for user profiling and user-adapted video retrieval. There are two reasons for this. Firstly, the information itself may change. Secondly, the user group is largely unknown from the start, and may change during the usage of exploration processes. To address these problems, the approach, presented in this paper, models profile structures. The video server should automatically extract and represent in Markov models user profiles in order to consider the dynamic aspect of user behaviors.

Thus, the main technical contribution of the paper is the notion of probabilistic prediction and path analysis using Markov models. The paper provides solutions, which efficiently accomplish such profiling. These solutions should enhance the day-to-day video exploration in terms of information filtering and searching.

The agreement calls for the participating video bases to track their users so the advertisements can be precisely aimed at the most likely prospects for scenes and shots. For example, a user who looks up a tourist scene about Paris on video bases in the video base might be fed ads for images or scenes of hotels in Paris.

The paper contains the following sections. Section 2 situates our contribution among state-of-the-art approaches. Section 3 describes user-profiling based video exploration. Section 4 highlights the general framework of the system. Finally, section 5 concludes the paper.