Multimedia Service Integration

Trying to utilize the developed subsystem of the media key framing algorithm to generate low-level video indexing and summary reports, a batch service program is developed. Figure 14.9 represents the described subsystem of media key framing as a black box and its interface to the corresponding service agent. The process accepts an input request file, which is submitted from a service representative agent. This input mission file contains the parameters of the applied request and it generates a result report that contains the detected key frames along with the confidence percentage of each key frame. Then the service agent utilizes the results and reformats it, using XML as we will show later, before passing this report to the user or storing it for later retrieval requests.

Figure 14.9: The use of the media key framing as a batch process.

Listing 14.1 depicts an example of an input XML file prepared by the service agent before submitting this mission request to the service process. There are only two mandatory parameters while the other ones are optional. These mandatory parameters are the reference number of the request and the video part itself. The other optional parameters contain some other input parameters about the request. They include as well some parameters that could affect the behaviour of the media key framing algorithm itself such as the threshold for cut detection, the temporal and spatial skips used, start and end of the requested segment of the media file, etc. However, these optional parameters have default values that are included in the algorithm itself, which could be overridden by the input mission request if necessary.

Listing 14.1: An example of video service mission request in XML format.

  <?xml version="1.0" ?>  <mission>         <ReferenceNumber>624</ReferenceNumber>         <InputMediaFile>video122.avi</InputMediaFile>         <UserID>[e-mail]</UserID>         <iDate>17 Aug 1999</iDate>         <iTime>14:26:47 ET</iTime>         <StartF>1/4</StartF>         <EndF>3/4</EndF>         <TemporalSkip>5</TemporalSkip>         <SpatialSkip>5</SpatialSkip>         <Threshold>25</Threshold>         <Operation>KeyFraming</Operation>         <KeyframingMethod>6MSB_Blocks</KeyframingMethod>         <PerformanceMethod>Binary_Penetration</PerformanceMethod>         <NumberOfBlocks>9</NumberOfBlocks>         <WorkingDirectory>vb5\for_integration\outputs</WorkingDirectory>         <FramesFormat>JPG</FramesFormat>         <ColorGrey>Color</ColorGrey>         <CompressionRatio>30</CompressionRatio>         <InitialFormSize>Normal</InitialFormSize>  </mission> 

Upon a successful completion of the key framing process, the system generates a result report in XML format. Figure 14.10 shows the adopted report structure of our video indexing service. Each report has the four shown main components. First, we will provide the high-level description of these components. Then, a detailed XML result file will be explained later.

Figure 14.10: Adopted video indexing service structure.

The components of the service report are:

-Request Component:

It has the details of the submitted request. It should uniquely describe a certain request. This is guaranteed through a unique serial number that is incremented sequentially each time a new request is submitted to the system.

-User Component:

The details of the user of the system who submitted the request are stored within this part. This could be useful for accounting and billing reasons. It could be useful as well for future access to this report and other submitted queries by this user after authentication.

-Service Component:

This part describes the properties of the selected video service for the mentioned request. The system elects the appropriate service according to accepted level of service for this request after the proposal counter proposal procedure among our system agents [15].

-Result Component:

The results themselves of the video service reside within this component. The detailed description of this component will differ according to the elected service described within the service component. This component should preserve some indication about the processing time or power that have been undergone to process this request for billing and accounting reasons.

Listing 14.2 shows a result record in a well-formed and validated XML format generated by the service agent for the corresponding output results, which are shown in Figure 14.11-b. We chose XML (eXtensible Markup Language) as a result format language because first XML is extensible language while the tags used to markup HTML documents and the structure of HTML documents are predefined. The author of HTML documents can only use tags that are defined in the HTML standard. In contrast, XML allows the author to define his own tags and his own document structure. Second, it is strongly and widely believed that XML will be as important to the future of the Web as HTML has been to the foundation of the Web. XML is regarded as the future for all data transmission and data manipulation over the Web especially among heterogeneous environments and devices. In summary, XML provides an application-independent way to share information. With a DTD (Document Type Definition), independent groups of people or organizations can agree to use a common certain DTD for interchanging and understanding the meaning of the shared data. Thus, any application could use a standard DTD file to verify that data that you receive from the outside world is valid. In addition, you can also use a DTD to verify your own data before delivering it to the user or another organization.

Figure 14.11: Samples of user interface to browse tourist destinations in Egypt. (a) In the case of highest quality level— Video Streaming; (b) In the case of medium quality level— Color and High Resolution Key Frames; (c) In the case of lowest quality level— Gray and Low Resolution Key Frames.

Listing 14.2: An example of a result report for the video service in XML format.

  (1)   <?xml version="1.0" ?>  (2)   <!DOCTYPE report SYSTEM "KF_report.dtd">  [3]   <report>  (4)          <user>  (5)                <id>[e-mail]</id>  (6)                <title>PhD student</title>  (7)                <vlocation>Mitel</vlocation>  (8)                <hlocation>UoOttawa</hlocation>  (9)          </user>  (10)         <service>  (11)               <name>MediABS</name>  (12)               <location>UoOttawa</location>  [13]               <parameter>  [14]                      <key>cost</key>  [15]                      <value>15</value>  (16)               </parameter>  (17)               <parameter>  (18)                      <key>quality-of-service</key>  (19)                             <value>colorKeyFrames</value>  (20)               </parameter>  (21)         </service>  (22)         <request>  (23)                      <reference>624</reference>  (24)                      <input>video122.avi</input>  (25)                      <startframe>1/4</startframe>  (26)                      <endframe>3/4</endframe>  [27]               <operation>key framing</operation>  (28)               <submitdate>17 Aug 1999</submitdate>  (29)               <submittime>14:26:47 ET</submittime>  (30)         </request>  (31)         <result>  (32)                      <processing>ok</processing>  (33)               <time>8.19 s</time>  [34]               <initial_size>2.310 Mb</initial_size>  (35)               <totalframes>295</totalframes>  (36)               <processedframes>28</processedframes>  (37)               <frames>  (38)                            <number>4</number>  (39)                      <fvalue>73</fvalue>  (40)                      <fvalue>120</fvalue>  (41)                      <fvalue>185</fvalue>  (42)                      <fvalue>221</fvalue>  (43)                      <total_size>18.428 kb</total_size>  (44)                            <format>JPG</format>  (45)                            <dimension>160x120</dimension>  (46)               </frames>  (47)               <dlocation>  (48)                            <protocol>http</protocol>  (49)                            <host>altair.genie.uottawa.ca</host>  (50)        <path>\mediabs\outputs\624_result\</path>  (51)               </dlocation>  (52)               </result>  [53]         </report> 

The following XML result report is well formed and validated against the associated DTD structure, presented later, using a third-party software tool called XMLwriter [14].

As mentioned, the XML result file contains four main parts, namely: user, service, request and result as illustrated in Figure 14.10.

The elements used within the user part are:

Line 5:	the identity that uniquely describes the user who submitted the request.
Line 6:	title or affiliation of the user.
Line 7:	visiting site or server of the user where he submitted the request.
Line 8:	original home site or server of the user.

The elements used within the service part are:

Line 11:	name of the service.
Line 12:	location of the main service.
Line 14–15:	cost attribute name and value of the request.
Line 18–20:	the accepted level of service of the request.

The elements used within the request part are:

Line 23:	the request unique reference number.
Line 24:	the name of the media file that has been processed.
Line 25–26:	the start and end points of the media segment within the video.
Line 27:	the name of the operation or function used to process this request.
Line28–29:	the date and time of the request.

The elements used within the result part are:

Line 32:	the end status of the request.
Line 33:	the CPU processing time of serving the request.
Line 34:	the initial size of the media file.
Line 35:	total number of video segment frames.
Line 36:	number of processed frames.
Line 38:	count of key frames for this request.
Line 39–42:	key frames numbers or references within the media file.
Line 44:	format of the result key frames.
Line 45:	width and height of the key frames in pixel units.
Line 48:	the protocol that could be used to retrieve and browse this report.
Line 49:	the server address of the XML and frames files.
Line 50:	the path of the XML and frames files within the server given in line 39–42.

In addition, we defined a Document Type Definition (i.e., DTD) description file associated with all the XML result documents. The main purpose of a DTD description is to define and validate the structure of an XML document when an XML parser within a browser parses the XML document. It defines the XML document structure with a list of approved elements. A DTD can be declared inline within the XML document itself, or as an external reference file. Then, we can use an associated XSLT transformation/ filtering template or a cascade style sheet (CSS) to present the report document on the user's web browser.

In our system, the external DTD file, KF_report.dtd, associated with the provided XML result file is shown in Listing 14.3.

Listing 14.3: The DTD file used to validate the XML service's result report.

  <?xml version="1.0"?>  <!ELEMENT report (user,service,request,result)>  <!ELEMENT user (id,title,vlocation,hlocation)>  <!ELEMENT id (#PCDATA)>  <!ELEMENT title (#PCDATA)>  <!ELEMENT vlocation (#PCDATA)>  <!ELEMENT hlocation (#PCDATA)>  <!ELEMENT service (name,location,parameter+)>  <!ELEMENT name (#PCDATA)>  <!ELEMENT location (#PCDATA)>  <!ELEMENT parameter (key,value)>  <!ELEMENT key (#PCDATA)>  <!ELEMENT value (#PCDATA)>  <!ELEMENT request  (reference,input,startframe,endframe,operation,submitdate, submittime)>  <!ELEMENT reference (#PCDATA)>  <!ELEMENT input (#PCDATA)>  <!ELEMENT startframe (#PCDATA)>  <!ELEMENT endframe (#PCDATA)>  <!ELEMENT operation (#PCDATA)>  <!ELEMENT submitdate (#PCDATA)>  <!ELEMENT submittime (#PCDATA)>  <!ELEMENT result  (processing,time,initial_size,totalframes,processedframes,frames,dlocati on)>  <!ELEMENT processing (#PCDATA)>  <!ELEMENT time (#PCDATA)>  <!ELEMENT initial_size (#PCDATA)>  <!ELEMENT totalframes (#PCDATA)>  <!ELEMENT processedframes (#PCDATA)>  <!ELEMENT frames (number,fvalue*,total_size,format,dimension)>  <!ELEMENT number (#PCDATA)>  <!ELEMENT fvalue (#PCDATA)>  <!ELEMENT total_size (#PCDATA)>  <!ELEMENT format (#PCDATA)>  <!ELEMENT dimension (#PCDATA)>  <!ELEMENT dlocation (protocol,host,path)>  <!ELEMENT protocol (#PCDATA)>  <!ELEMENT host (#PCDATA)>  <!ELEMENT path (#PCDATA)>