Section 5.7. Creating Synchronized Presentations

5.7. Creating Synchronized Presentations

One of the most compelling features of FlashCom-enabled applications is the way Flash content and streaming audio and video can be merged into a seamless and full-featured presentation. With some careful scripting, you can create a presentation that includes optional viewing of closed captions, animated slides, dynamic repositioning of video, and other effects all synchronized to video playback. There are two approaches to synchronizing events, such as displaying closed captions, to stream time.

The first approach is to embed information about events directly in a stream using the NetStream.send( ) method. The advantages of send( ) are that synchronization is as precise as it can be and that it often requires less scripting. The disadvantages are that send( ) must be called on the publishing stream while it is being recorded and, if the user is allowed to seek to different parts of a stream, additional scripting is required to replay or reestablish the events up to the time the user seeks to. In the future, FLV editors may become available that allow postproduction embedding and editing of remote method calls.

The second approach is to synchronize a list of instructions to a stream by regularly checking (polling) the stream time to see if an instruction should be executed. An instruction can be used to display a caption, resize or hide a video object, or display an animation. For presentations, especially when prerecorded video and audio from multiple sources must be assembled , synchronizing lists of instructions to a stream by regularly checking stream time is the most practical approach.

5.7.1. Adding Closed Captions

Closed captions are a good place to experiment with synchronizing presentation events to a stream. When someone is speaking, the caption containing the correct words must be displayed. A simple and flexible way to provide closed captioning data is to provide a list of caption start times along with the text of each caption in an XML file:

 <closedCaptions>   <caption seconds="0.0"><![CDATA[]]></caption>   <caption seconds="3.0"><![CDATA[Welcome to my presentation.]]></caption>   <caption seconds="7.0"><![CDATA[I hope you enjoy it.]]></caption> </closedCaptions> 

Each <caption> tag contains a seconds attribute, which is the stream time at which the caption should first be displayed. Within each <caption> tag, a <![CDATA[]]> tag is used to contain unencoded text. Any unencoded text, including the & and < characters , can safely be placed in a CDATA section. When Flash parses the XML, the contents of the CDATA section will be converted to a text node. The first <caption> tag contains no text so that when the stream starts, no caption text will be visible. Other empty captions can be added to a list of captions to pad the list for intervals when no one is speaking.

When the XML data is loaded into a Flash movie, it can be used to create an array of objects. Each object can have a time property and a text property.

During playback of the stream, the stream time must be checked at regular intervals against the caption that is currently displayed. If the wrong caption is displayed, the correct one must be found and displayed. Either setInterval( ) or onEnterFrame( ) can be used to regularly call a function to check the stream time against the current caption. Unless a very low frame rate is used, checking with onEnterFrame( ) is a good practice as it will be called just before the next frame is redrawn. There is often little point in choosing an interval that checks several times in between frames or more often than the movie's frame rate.

Since the stream time will be checked repeatedly, the process of finding the correct caption should be as efficient as possible. A simple and efficient way to check the caption is to store caption objects in an array sorted by time, as depicted in Figure 5-2. An index into the array can be used to keep track of the current caption and easily get the time of the next caption.

Figure 5-4. An array of caption objects laid out against time

Assuming the stream begins to play at 0 seconds and proceeds forward linearly, the first caption can be displayed, and the time of the next captionin the illustration, 3 secondskept in a variable. As long as the current stream time is less than the next caption time, nothing has to be done. When the time finally progresses to being equal to or greater than the time of the next caption, the next caption can be displayed and the time of the one following it retrieved. Later, when time becomes equal to or greater than 7, the next caption is shown. Assuming that an array named captionList containing caption objects with time properties is available, a simple onEnterFrame( ) function might look something like this:

 // Start with caption number 0. currentIndex = 0; // Get the time of the next caption. nextCaptionTime = captionList[1].time; // Deal with the case where there is no next caption. if (!nextCaptionTime) nextCaptionTime = Number.POSITIVE_INFINITY; // Show the current caption. showCaption(currentIndex); // As the stream plays, keep checking the stream time. function onEnterFrame ( ) {   // If the stream time has progressed far enough, show the next caption.   if (ns.time >= nextCaptionTime) {     currentIndex++;     nextCaptionTime = captionList[currentIndex + 1].time;     if (!nextCaptionTime) nextCaptionTime = Number.POSITIVE_INFINITY;     showCaption(currentIndex);   } } 

The showCaption( ) function is not shown but is responsible for displaying the text of a caption. A strategy like this works well if the stream only plays through from the beginning to the end. If the user is allowed to navigate to any part of the stream at any time, another approach must be used to find the current caption in the array. Since the captions are stored in time order, a binary search can be performed on the array to quickly find the current caption when necessary. When the stream begins to play again, the script can go back to checking the time of the next caption.

Captions are much like other events that have to be synchronized to stream time, so it makes sense to create a general purpose container to hold events and keep them synchronized to a stream.

The book's web site includes a complete tech note that describes the design and development of a SynchronizedEventsList class, which can store, find, and return events given a stream time. The tech note contains a complete sample presentation and source code. The sample combines closed captioning with repositioning, hiding, and showing of video as the presentation progresses and synchronized animations. For the sake of brevity, the entire application, which includes a NetStream subclass; the ClosedCaptionsManager , CaptionEvent , and PlaybackDirector classes; and methods to read and parse an XML file, is not listed here.

5.7.2. Adding Synchronized Slides

The presentation on the book's site uses the SynchronizedEventsList class from within other objects to synchronize events to a stream. A SynchronizedEventsList instance can be used by another object to store events and then to find an event for a specific time. For example, another object that has the responsibility of responding to stream events by always displaying the correct caption can create its own SynchronizedEventsList instance and then use it to store captions and find the right one at any time. In fact, it is a good idea to use a separate events list because it makes searching for the correct event much simpler than trying to mix caption and other types of events within a single array.

The presentation implements custom SlideManager and SlideEvent classes. As the slideshow stream plays, the playhead of the client movie's timeline must be moved to a labeled frame representing each slide at the appropriate time. The example uses an XML file to define the start times in seconds when each frame label should be played .