8.7 Display Model

There are some specialized requirements for running applications within a consumer electronic environment as opposed to the simpler situation that occurs when an applet is displayed within a PC. Most notably, the screen dimensions and aspect ratios are significantly different between PCs and devices. In the On-Screen Display (OSD) graphics model of today's set-top boxes, output video may be in a number of different configurations, including traditional 4:3 TV display, as well as 16:9 wide screen displays. The graphics resolution and aspect ratio are often locked to the video resolution and aspect ratio. If the video aspect pixel ratio changes, then the graphics pixel aspect ratio may also change. Thus, the key requirements are to determine the resolution and physical characteristics of the current display device, and detect modifications to the resolution and physical characteristics of the current display device.

8.7.1 Screen Reference Model

A model is provided that is designed to best fit the needs of consumer electronics devices. There are four layers of graphic display (see Figure 8.2): (a) the background plane, containing a static image or uniform pattern/ color ; (b) the video plane displaying the video-stream for the current virtual channel; (c) the application graphics plane displaying the GUI; and (d) the cursor plane displaying icons used for selection and status notification.

Figure 8.2. The four planes of the display model.

The four planes are overlaid so that the content of the cursor plane is always visible (i.e., on top); the content of the application graphics plane is visible only in locations where the cursor plane is transparent; the video plane is visible only in locations where both the graphics and cursor planes are transparent; the background plane is visible only at locations where the video, graphics, and cursor planes are transparent. The blending is done through hardware using an Alpha channel.

On reception , the transport stream is split into the video and data portions. The data portion is eventually decoded into an application, which is executed to produce a GUI, which is used to generate the graphics output. The video portion of the transmission is decoded separately and combined with the graphics output. The background information is not present in the transport stream, yet may originate in (or be controlled by) various sources such as receiver configuration, downloaded applications, or user -defined background.

An interesting aspect of the graphic model is that it allows for non-uniform and non-square pixels. Although the pixels on a single plane are modeled uniform, it is possible for the pixel dimensions to differ among the four planes. For example, pixels on the background plane would have a width greater than their height, whereas the pixels on the graphics plane could have their height greater then their width.

Instances of the class HScreen encapsulate independent final video output signals from a device. Each independent final video output signal is made up of the sum of graphics devices, video devices, and backgrounds. These are represented by instances of the classes HGraphicsDevice , HVideoDevice , and HBackgroundDevice , respectively. All of these classes inherit from the common parent class HScreenDevice .

The HAVi L2 GUI specification provides limited support for applications to be displayed so that they are split across multiple concurrent display devices. The HSceneFactory class allows the HGraphicsDevice to be specified in the HSceneTemplate used to generate the HScene 's for the application.

8.7.2 Screen Representation

The HScreen class describes the final output composition of a device. It ties together all the (MPEG) video decoders, and all the graphics subsystems and backgrounds which are all combined together before finally being displayed. A platform with two independent displays would support two instances of this class. Where a device outputs audio closely bound with video, that audio output can also be represented through this class.

An HScreen represents a single video output signal from a device. Therefore, all the devices which contribute to that signal have certain properties in common. It is not possible to select conflicting configurations for different devices on the same HScreen . For example, it is not possible to select a video device with a 4:3 picture aspect ratio and a graphics device with a 16:9 picture aspect ratio. This specification intentionally does not define configurations, or which configurations would be conflicting, because these are essentially region or market dependent.

8.7.3 Screen Device Representation

An instance of the HScreen class represents a single independent video output signal from a device. Devices with multiple independent video output signals should support multiple instances of this class. Figure 8.3 depicts the HAVi screen device representation model.

Figure 8.3. Class diagram for HAVi 1.1 ScreenDevice.

A video output signal is created by adding together the contributions from the devices represented by a number of objects inheriting from the HScreenDevice class. These can be HGraphicsDevice objects, HVideoDevice objects, and HBackgroundDevice objects. A given HScreen may support any number of any of these objects.

8.7.4 Screen Device Discovery

Application can discover the range of display devices available using the classes of GraphicsDevice , GraphicsConfiguration , and GraphicsConfigTemplate . For each display device class ( HVideoDevice , HGraphicsDevice , and HBackgroundDevice ) there are classes that represent distinct possible configurations of a single device. Applications may obtain a list of all possible configurations of a particular device. Applications may also obtain the current configuration using the getCurrentConfiguration() method. Subject to security and resource management issues, applications may also set the configuration of a device using setCurrentConfiguration() methods found on each device class.

Applications that are interested in a particular configuration of a device can request configurations matching a specific set of constraints. The first step in this process is to construct objects whose name ends in "ConfigTemplate." Instances of these classes can then be populated with the properties by the application and then used to request a configuration supporting those properties. Properties can also have priorities attached to allow applications to express whether support for that property is required by the application, whether support for that property is only preferred by that application, whether support for that property is required to be absent, or whether support for that property is preferred to be absent. In some cases, properties such as PIXEL_ASPECT_RATIO require extra information. This extra information can be provided as part of the method used to add the property to the configuration template.

A description of a configuration retrieved by getCurrentConfiguration() can be obtained using the getConfigTemplate() method that yields a ConfigTemplate that uniquely identifies the given Configuration. Individual properties in this ConfigTemplate can then be examined using the getPreferencePriority() and getPreferenceObject() methods. Features that are implemented should return REQUIRED whereas features that are not implemented should return REQUIRED_NOT. Values of some properties may also be obtained through a limited set of query methods provided on HScreenConfiguration .

8.7.5 Compatibility

The java.awt.Toolkit.getScreenSize() method should be equivalent to the pixel resolution of the current configuration of the default screen device returned by HScreen.getDefaultGraphicsDevice() .

The java.awt.Toolkit.getScreenResolution() method should return the resolution (in dots per inch) of the current configuration of the default screen device returned by HScreen.getDefaultGraphicsDevice() . In the case that the horizontal and vertical resolutions differ, then the lower value should be returned.

Where the screen aspect ratio is unknown (as in the case where a set-top box is connected to an analog display), the default aspect ratio is 4:3. In the case where an analog monitor is used with a HAVi compliant set-top box the resolution returned should be based on the raster of the set-top box, ignoring any interpolation or other processing that may be present in the monitor.

The java.awt.Toolkit.getNativeContainer() method should return null; interoperable applications should not rely on this method.

8.7.6 Detecting Configuration Changes

It is important for devices to be able to detect variations in their settings because they may be subject to " on-the-fly " modifications of these settings, for example, they may be heavily influenced by the nature of some input video streams. Hence, the HScreenDevice class provides support for detecting when its configuration (settings) have been changed, using the HScreenDevice.addScreenConfigurationListener() methods and the HScreenConfigurationListener and HScreenConfigurationEvent classes.

When an HScreenDevice 's configuration is modified, then an HScreenConfigurationEvent is generated. After a HScreenConfigurationEvent is obtained any HScreenConfiguration (or HScreenConfigTemplate ) associated with that HScreenDevice is re- acquired to obtain the current settings for the device.

In general, a modification to the HScreenDevice might require that the displayed user-interface be modified. For example, if the resolution has changed, or the pixel aspect ratio has been modified.

8.7.7 Screen Interfaces

Recall the HAVi API model for screen interfaces (depicted in Figure 8.3). The HScreen class describes the final output composition of a device. It ties together all the (MPEG) video decoders, all the graphics subsystems, and backgrounds that are all combined together before finally being displayed. A platform with two independent displays would support two instances of this class. Where a device outputs audio closely bound with video, that audio output can also be represented through this class.

Each HScreenDevice can have multiple settings specified by multiple HScreenConfigurations , one of which can be active at any point in time. The current configuration can be determined on the HScreenDevice subclasses using their specific getCurrentConfiguration() methods, and modified using their specific setCurrentConfiguration() methods ( assuming sufficient rights).

Applications may select the best of these configurations for them by creating an instance of HScreenConfigTemplate and populating that with a number preferences, each with a priority. The implementation then matches this template against the range of possible configurations and attempts to find one that matches the template provided. Priorities REQUIRED and REQUIRED_NOT are always respected.

8.7.7.1 HGraphicsDevice

The HGraphicsDevice class describes the raster graphics devices that are available for a particular HScreen . Each HGraphicsDevice has one or more HGraphicsConfiguration objects associated with it. These objects specify the different configurations (settings) in which the HGraphicsDevice can be used.

8.7.7.2 HVideoDevice

The HVideoDevice class describes the logical video devices that can contribute to the appearance of a particular screen. Each HVideoDevice has one or more HVideoConfiguration objects associated with it. These objects specify the different configurations (settings) in which the HVideoDevice can be used. This class represents the presentation only of video and does not provide for the selection of which video is to be presented.

8.7.7.3 HBackgroundDevice

This class represents the ultimate background of a screen. The background is the very back of the video and graphics composition stack. It can potentially cover the entire area of a screen. Where a device supports multiple applications on screen at the same time (or even a window manager), the background is not constrained by any particular application or window. The right to control the background of a screen is a scarce resource and managed as such.

8.7.7.4 HScreenConfiguration

The HScreenConfiguration class describes the characteristics (settings) of an HScreenDevice . There can be many HScreenConfiguration objects associated with a single HScreenDevice .

8.7.7.5 HScreenConfigTemplate

This class describes a configuration of a screen device in terms of various properties and their importance to the application. It is used to request a valid instance of a configuration conforming to the description provided. Subclasses of this define additional constants which may be used for additional properties. Those classes can be subclassed in turn to add further properties by systems using the HAVi L2 GUI.

In the event of conflicting preferences between HScreenConfigTemplates (e.g., where a configuration is requested that conflicts with an existing configuration), the following rules are applicable in all cases:

• When all the conflicting preferences have priority PREFERRED, PREFERRED_NOT, or DONT_CARE, the conflicting preferences are discarded from the template.

• When all the conflicting preferences have priority REQUIRED or REQUIRED_NOT, configuration setting fails.

In situations other than these, all preferences with priority PREFERRED, PREFERRED_NOT, or DONT_CARE are discarded. Subsequently, if there is still a conflict, the configuration setting fails.

8.7.8 Emulation

The HAVi L2 GUI introduces extra subclasses that are used to indicate that a device may perform emulations of other device capabilities: HEmulatedGraphicsDevice , HEmulated-GraphicsConfiguration .

Instances of these classes can be returned by the same methods that would return the corresponding class without "Emulated" in the class name. Returning the subclass indicates that the implementation is emulating the requested configuration on one of its actual supported configurations. The class HEmulatedGraphicsConfiguration includes methods to allow applications to compare the configuration actually being emulated and the actual underlying configuration being really used. The extent of support for emulated configurations is a profile issue. All possible emulated configurations are not required nor guaranteed to be supported. Emulated configurations may have a significant performance penalty with respect to those supported natively on the device.

8.7.8.1 Mapping Coordinates

A special case of device emulation is the emulation of various graphics coordinate systems on a single physical device. The HAVi L2 GUI provides mechanisms that allow devices to perform such emulations, for example, by down-sampling a high-resolution system to match the limitations of a standard definition display. Thus, authors can rely on seamless mapping between authoring and device coordinates by the use of the HEmulatedGraphicsDevice class. Authors may determine an appropriate graphics device and request the best configuration that matches their requirements, as in a standard device discovery mechanism, or examine configurations themselves (both emulation and implementation) to determine appropriate settings. The extent to which devices are required to support emulation of other coordinate systems is profile dependent.

8.7.8.2 Integrating Video Support

The HAVi specification includes several classes to represent video in the user interface system. This representation of video devices only includes the display of video. The setup of the video decoder and the video pipeline is not included in the HAVi L2 GUI.

The class HVideoComponent is intended to be returned by the JMF method Player .getVisualComponent() . The class HVideoDevice provides two hooks to platform specific APIs for this setup, the methods getVideoController() and getVideoSource() . The JMF getVideoController() method should return a JMF Player. The getVideoSource() method should return a JMF java.net.URL or javax.media.MediaLocator .

8.7.8.3 Background

The HAVi specification includes several classes to represent the background plane, namely the area that is behind the running graphics and video and not covered by those. Using the same naming convention as video and graphics, these are called HBackgroundDevice , HBackgroundConfiguration , and HBackgroundConfigTemplate . The basic HBackgroundConfiguration allows applications to control a single full-screen background color; note that this should be an RGBA JavaTV color as opposed to an RGB AWT color.

The HAVi specification includes support for more sophisticated backgrounds comprising of still images. Applications wishing to use these should request an HBackgroundConfiguration supporting them by using the STILL_IMAGE property in an HBackgroundConfigTemplate . If this feature is supported by the platform concerned , when such a configuration is requested, an instance of the class HStillImageBackgroundConfiguration should be returned. This class adds two extra methods over the standard background configuration, which support the loading of background images. This loading is done through the HBackgroundImage class.

Using the HBackgroundImage class rather than the standard java.awt.Image allows for image formats that are decoded using hardware outside of the graphics system. One specific example of this is the decoding of still MPEG I frames using an MPEG video decoder. This is a commonly used feature in some devices because it provides good quality backgrounds without using software decoders or system memory. In systems where the same underlying MPEG video decoder can be used to decode both video and MPEG I frames , this decoder should be represented both by an HVideoDevice instance and by an HBackgroundDevice instance for each application. The sharing of the single underlying decoder between these two instances should be managed using the reserveDevice() and releaseDevice() methods that both classes inherit from HScreenDevice . This applies both for resource transfers between two different applications and for resource transfers within the same application. The behavior of this should be as described in the specification for ResourceClient .

For example, on such a system, in the situation where an application has the right to control an HVideoDevice and that same application wishes to request control of a HBackgroundDevice currently configured as an HStillImageBackgroundConfiguration , the following steps are followed. First, the application calls reserveDevice() on the HBackgroundDevice . This method should block as the following steps complete. The implementation calls the method ResourceClient.requestRelease() on the ResourceClient provided by the application when it called HVideoDevice.reserveDevice() . The implementation of that method should decide whether to release the underlying decoder for use in decoding backgrounds or to continue with decoding video. If the application decides that the use of the background is more important to it than the use of the video then the application should conform to the behavior defined for this method, namely call HVideoDevice.releaseDevice() and then return true. At this point, the original call to reserveDevice() should return true. On systems where there are separate underlying decoders for MPEG video and MPEG stills, clearly calling HBackgroundDevice.reserveDevice() should return immediately if no other applications are using that device.

8.7.8.4 Controlling Configurations

The HAVi specification is silent about whether a single display device is shared between multiple applications or not. When a display device may be shared between applications, it provides a mechanism for applications to assert control over the right to change the configuration of the display device. The HScreenDevice class includes methods to allow applications to reserve and release the right to control this configuration. It also allows an application to register and remove listeners for events that are generated when the applications reserve and release this right.

Applications wishing to control the configuration of an HScreenDevice define a class implementing the ResourceClient interface and pass an instance of this class to the reserveDevice() method of the HScreenDevice that they wish to control. If the reserveDevice() method succeeds then the application obtains control over the device. The HScreenDevice.getClient() method should return the ResourceClient passed in to the last call to the reserve method on that HScreenDevice instance.

Where there is a conflict between applications, the HAVi L2 GUI includes a mechanism to allow the platform to arbitrate between conflicting applications. The policy for this arbitration is intentionally not defined in HAVi L2 GUI. When it is decided to remove the right to control a screen from an application, it is notified through the ResourceClient . notifyRelease() method. The requestRelease() method should only be called when the existing owner of the resource and the application requesting the resource is authenticated. The HAVi L2 GUI specification is silent about the details of this authentication.