Video Key Framing System Process

We have developed a prototype system addressing multi-format video visualization and analysis. The system aims to discover the different cut changes within a video file with high accuracy and better performance. Figure 14.8 depicts the architecture of the video segmentation and key framing system. The system has four main components. They are:

1. The Media Preparation module recognizes and verifies the input file format, size, resolution, etc. We also use this module to handle different video formats seamlessly. For instance, it handles compressed video formats through low-level function calls using the software driver for the corresponding format. Therefore, we implemented this function to unify the next processing functions irrespective of the input format. A further responsibility of this module is browsing with different VCR capabilities such as play, stop, pause, resume, rewind, forward, go to certain frame, etc.

2. The Media Analysis module implements two different key framing algorithms, namely the "six most significant RGB bits with the use of blocks intensity difference" both through blind search and through binary penetration, as explained. This allows their relative merits to be studied. This module also updates the key framing index list of the required input segment and provides the updated index list to the Key Frames Handling module.

3. The Frames Management module uses the media format segment for the analysis. The system uses this module to extract certain frames from the input segment. The Media Analysis module requests certain frames within a defined segment and specifies the region of the requested frames, as well as the temporal skip, frames quality, color/gray condition and frames format. The Frames Management module executes the frame extraction function itself. It then replies to the Frames Analysis module request with the extracted frame references (i.e., their locations on the local storage). To compensate for the storage requirements, we partition the file into discrete regions for separate analysis.

4. The Key Frames Handling module could use the Frames Management module services again to extract the actual frames from the video file by checking their index, received in the key frames list from the Media Analysis module. The Key Frames Handling module is responsible for providing the user with the key frames result report, including the total processing time and the configuration of the parameters used.

Figure 14.8: Video segmentation process.

We tried to separate the different modules according to their functions in order to facilitate the future update of any module without affecting the other ones. This is done by a module requesting services from the others through consistent and coordinated interfaces. For example, we could process new media formats without the need to change the current video analysis algorithm. Another example is the possibility of changing the video analysis algorithm itself for the current media formats supported.