HUD and GUI

The complexities of HUDs vary largely from game to game. Games of certain genres, such as RTS, are guaranteed to have rather involved HUDs. For example, a game such as WarCraft III has an interactive HUD, which understands mouse and keyboard input. There are buttons on the left side of the screen that can be used to select the heroes. There is also an interactive overhead map and even a viewport that shows a close-up of the currently selected unit or building. In addition, there is a text field that acts as the in-game console and is used for passing messages to the players. The HUD is practically a fully functional window that has different components such as buttons, text fields, and progress bars. Even though FPS games tend to have very little, if any, interactive HUDs, the in-game console is still an interactive GUI component.

Besides HUDs that are rendered in a 3D context, many games render their menus in a 3D context as well. The main menu of WarCraft III and Quake III are in a 3D context. In this section, we will build a GUI system that can be used to create interactive HUD and menus. The code that comes with this chapter demonstrates the infamous spinning cube game, with main menu, options menu, HUD, and in-game console.

By creating your GUIs as demonstrated here, you will be able to have nice menus that can even be transparent and rendered right onto the 3D context. In addition, the menus and dialog boxes will be local to your game. That is, pressing Alt + Tab will not cause the focus to be stolen from the game and given to a top-level container. In addition, clicking the wrong window will not cause another window to be inadvertently hidden from the user. The approach presented here is also extremely light and will not require much startup time or resources.

 On the CD   The code for this chapter can be found on the book’s companion CD-ROM in Code/Chapter 14/HGUI.

Components

The GUI system that we implement here has to model some of the functionalities of AWT and Swing. We define a component as an object, such as a button or a text field. All the components we define here inherit from the HComponent base class. Some of the more important members of this class are shown in Figure 14.1.

Figure 14.1: The more important members of the HComponent class.

The members x, y, w, and h are used to define the position and the boundary of the component. They are also used to render the component and serve as a bounding box used to find out if the mouse is on the component. The coordinates of this box are integer values that assume a coordinate where the origin is the upper-left corner of the screen. Note that even though the boundary is a rectangle, it does not mean that a component cannot be round. The boundary can simply be an estimated bounding region.

The member visible is used to indicate whether a component should be rendered. This can be useful when, for example, a simple image or text has to flash or pop up as the result of the user’s interaction. As another example, when a button is triggered, additional components may need to become visible on the same screen. The member enabled is used to indicate whether the component should receive input. A component that is not enabled should typically be rendered so that it appears grayed out, indicating that the component is there but not active. The member focusable is used to indicate whether the component can gain focus. A focused component is the component to which the keyboard input is sent. Note that at most, one component can have the keyboard focus at any given time.

The member listener is a reference to a listener whose actionPerformed method is called to handle the events generated by this component. As with a typical listener, it is notified when input to the component translates into actions. For example, a button can be assigned a listener so that when it is clicked, the listener can start the game. For the sake of simplicity, a component can have a single listener reference as opposed to a multiple list of them.

The member text stores a string that is rendered at coordinates (xText, yText), which are relative to the component’s position. The member textureIdFont is the texture that contains the font used for rendering the text member of this component.

There are generally two different types of events in a GUI system. The first type is known as low-level events, which are composed of events such as keyboard and mouse events. The second type is known as semantic (or high-level) events. High-level events are those that are typically generated by a component and have a higher level of meaning. For example, a “triggered” event can be generated by a button and passed to its listeners to indicate that the button was clicked by the mouse or the Enter key was pressed when the component had focus.

A component can receive low-level events through its processInputEvent method. Every component can choose to call its super class’s processInputEvent method to get the functionality supported by the super class.

The paint method of a component is called to allow the component to render itself. The paint method receives the GL object, which is used to invoke OpenGL methods. A component can change the OpenGL state and do just about anything it wants, as long as it restores the previous state when it is done. For example, a component may decide to change the blend function or even change the projection matrix to render a complex geometry.

Figure 14.2 shows a hierarchy of components implemented in the example that comes with this chapter.

Figure 14.2: The component types defined and used here.

An instance of the HComponent can be used as a simple component that displays an image or some text. It does not generate any high-level events and ignores low-level events. As mentioned previously, HComponent is the base class for other components.

HButton is used as a button or check box. It contains a member called toggleBotton that can specify whether the button is a typical one or one that can be toggled. A toggle button can be used as a check box component. An HButton can generate CLICKED, CHECKED, or UNCHECKED events. When the button is pressed, the default paint method of this component renders a smaller textured quad with a different texture. By doing so, there is a sense that the button is pushed away from the camera.

The HSlider component is a simple slider. It is an example of a component that cares about the MOUSE_DRAGGED in addition to the MOUSE_PRESSED event. A floating-point value is used to store the current value of the slider. It is also used to compute the position at which the slider knob is rendered. The value can be changed by clicking or dragging the mouse, or by using the page up and page down keys. This component generates VALUE_CHANGED events.

HTextComponent is the base class of components that need to do text editing. The processInputEvent of this component can distinguish typeable characters and concatenate them. It also defines the behavior for keys that are important for text editing, such as backspace, Home, End, or Event Copy and Paste.

HTextField component extends HTextComponent and catches the Enter key so that it can generate an ENTER event. If you want to be able to validate the text being entered, you can generate TEXT_CHANGED events to make sure the text is valid. After this component separates the input events in which it is interested, it sends the events to its super class, which can do the text editing.

HTextList is a component that keeps track of a list of strings and renders the most recent strings. You can easily catch behaviors, such as page up/page down. You can also set up an HSlider to scroll the strings that are rendered.

Containers

Now that we know the components we will be dealing with, let’s see how we can put them together to build a menu. To put a menu together, we need to have some sort of containers that group the components together. A container is an object that manages a list of components. Dialog boxes, windows, and panels are examples of containers. The more important members of the container class are shown in Figure 14.3.

Figure 14.3: The more important members of the HContainer class.

A container has many members and methods that are similar to those of a component. For example, a container needs to have a position, processInputEvent method, and a paint method. In fact, AWT’s java.awt.Container class extends the java.awt.Component class. By doing so, a container can have other container objects in its list. For example, an AWT Panel can have other components as well as another Panel in its list. Swing takes this one step further. A JComponent actually extends the AWT container class. This means that every Swing component is a container. For example, the JScrollPane component is a container that has JScrollBars and a JViewPort component.

Unlike AWT and Swing, the container class we use here does not inherit from the component class or vice versa. We want to keep our hierarchy flat, instead of having a tree of components and containers. Having a list instead of a tree greatly simplifies the problem. Even though this may seem overly simplified, it is sufficient for the HUD and GUI of most games.

The order of components in a container’s list dictates their draw order. The list is drawn from head to tail. That way, if some components overlap, the one that was added last will be drawn last. This means that the last component will appear on top of the others.

The processInputEvent method of the container has to forward the events to the appropriate components. To pass the keyboard events, the container must keep track of the component in focus. The focused component is the component to which the key events are sent. Intuitively, the focused component can be changed by clicking another component. Alternatively, a key such as the Tab key can be used to cycle the focus by simply walking the component list. As a side note, if a mechanism is set up to navigate through the menus by using only the keyboard, a controller’s input can be translated to equivalent keyboard events. For example, a button of the controller can be translated to the Tab key and another to the Escape key.

Mouse events are a little different. MOUSE_MOVED events should be forwarded to the component that has the mouse on it. Similarly, MOUSE_PRESSED events should be sent to the component over which the mouse was pressed. On the other hand, MOUSE_DRAGGED and MOUSE_RELEASED events should be forwarded to the component that received the corresponding MOUSE_PRESSED event. In other words, if you click a component and drag the mouse so that it is no longer on the component, the drag events should still be sent to the same component. Moreover, when the mouse button is eventually released, the component should still be notified about the MOUSE_RELEASED event, regardless of whether the mouse is directly over the component. Another relevant concern arises when the mouse is on overlapping components. In such cases, the topmost component should receive the input.

The processInputEvent of the container can also trap some keys and define a default action for them. For example, by default, the Escape key requests to close the container.

Most GUI components used in games have some sort of rollover effect to emphasize the component that is directly under the mouse. To implement rollovers, a container can draw a highlight texture on the component or allow the component to draw the rollover effect itself. By testing the position of the mouse against the bounds of the components, the component that is directly under the mouse can be determined. It is also useful to notify a component when this state changes. The onMouseEntered and onMouseExited methods of a component are used to notify it. Because our components are not managed by AWT or Swing, we must emulate these events ourselves.

In addition to rollovers, it is also useful to highlight the component that currently has the keyboard focus. Note that just because the mouse is over a text field does not mean that the keyboard input should be sent to that component.

Container Management

There are times when you may need to have multiple containers showing at the same time. Take, for example, the scenario where you are at the main menu and you want to quit the game. It may be appropriate to display another container that displays a message asking whether you really want to quit the game. The ability to show multiple containers at the same time is particularly important if you want to use the menu system to display a HUD. This is because while the HUD container is showing, you may want to show a console container or some sort of dialog box.

As its name implies, the HContainerManager class is responsible for managing the containers. It is actually a simple class. Figure 14.4 shows the more important members of this class.

Figure 14.4: The more important members of the HContainerManager class.

The most important member is the stack, which is an ArrayList. When the pushContainer method is called, the container is passed in as the parameter is pushed using the ArrayLists’s add method. The popContainer method pops the topmost container from the stack.

When a container is pushed on the stack, its onShow method is invoked. Similarly, when a container is popped, its onHide method is called. In addition, the onFocusGained and onFocusLost methods are invoked when a container becomes the topmost component or loses its spot as the topmost component. Note that a container may gain or lose focus if it is either pushed or popped from the stack or another component is pushed or popped immediately in front of a container. In other words, the onShow and onFocusGained methods are not necessarily called the same number of times. When a container’s onFocusGained or onFocusLost method is called, any component that has the mouse over it or has the keyboard focus should also be notified. This is useful, for example, when a component is rendering its own highlighting effect.

The paint method of the container manager calls the paint method of all the containers in its stack. The paint method is performed from the bottom of the stack to the top. In other words, the topmost container is painted last. In addition, the paint method of this class is responsible for setting up the frustum so that the components can draw themselves in a convenient 2D coordinate system. Such settings that are global across the GUI components can be set up in this method. This method is also responsible for storing the render states and states such as enabling textures and disabling lighting. After this paint method calls the paint method of all the containers in the stack, it should restore the render state. Note that the paint method of the container manager should be called after the game has been rendered. By restoring the render state, the game will not be affected as a side effect of rendering the GUI.

The paintFocus method draws a highlight texture on top of the component that is under the mouse, as well as a texture to emphasize the component that has the keyboard focus.

The processInputEvent method simply directs the events to the topmost container.

Unlike menus and GUIs used for most applications, game menus are typically animated. For example, the menus may slide in from the side, or the in-game-console may slide down from the top. Animation support can be easily added to the container manager. If every container has a position, and the position of the components are relative to their container’s position, by simply moving the container, all of its components will move accordingly.

To handle the animation, every container stores its initial position, destination position, and a delta vector. The delta vector is simply a pair of x- and y-values that specify the velocity of the container when it is being animated. In addition, the container manager needs to know the state of the animation for purposes such as ignoring user input while a container is being animated. The container manager can act as a simple state machine with the following states:

static final int  STATE_IDLE          = 0,                   STATE_PUSH_ENTER    = 1,                   STATE_PUSH          = 2,                   STATE_PUSH_EXIT     = 3,                   STATE_POP_ENTER     = 4,                   STATE_POP           = 5,                   STATE_POP_EXIT      = 6; int state = STATE_IDLE;

Even though there are a total of seven states, there are only three general states. The states are IDLE, PUSH, and POP. The IDLE state does not do much. The PUSH state indicates that the topmost container should be animated from its initial position to its destination position. The POP state indicates that the topmost container should be animated from its destination position to its initial position.

Before the state of the container manager is set to push or pop, the corresponding ENTER state is set so that some initializations can be performed before the animation starts. Likewise, the corresponding EXIT state is set to indicate that the animation has completed.

When a container is pushed, the state is set to STATE_PUSH_ENTER. This state is responsible for notifying the topmost container that it is losing focus, calling the onShow method of the container being pushed and adding the container to the stack. This state is also used to decide when to set the current position of the container to its initial position. Typically, the initial position of a container is set such that it is not on the screen. After the aforementioned changes are made, the state is set to STATE_PUSH, which simply adds the delta values to the x- and y-position of the container. It also checks to see if the destination position has been reached so that it can set the state to STATE_PUSH_EXIT. This state is used to notify the new topmost container that it has gained focus. The POP states perform the same tasks in reverse order, with some minor differences.

If you want to hide the components when the container is being animated, you can set the visibility flag of the components to false in the onShow method, and set them to true in the onFocusGained method.

When the container manager is not in the IDLE state, the processInputEvents method simply ignores the input events. Depending on the requirements of your menu system, you may have to set up the container manager so that it queues up push and pop requests. That is, when multiple calls to the push and pop methods of the container manager are made, the container manager can store the request and process them one at a time.

Demo

 On the CD   The following code can be found on the book’s companion CD-ROM in Code/Chapter 14/HGUI.

The demo discussed in this section uses a spinning cube as the placeholder for any 3D content you may want to have in the background while the menus are displayed. Using geometry to render the menus or display graphics in the background is a common practice. For example, both WarCraft III and Quake III use this approach. WarCraft III renders a world with weather effects, flying crows, and such in the background. Most games, especially console games, have menus more like Quake III. The menus are basically geometry right in front of the camera, as we have been doing here. For example, Quake III uses polygons to render the fire effect on the main menu. Some games may have a selection screen where a spinning model such as a vehicle or character is rendered. Either way, the spinning cube represents some 3D geometry that may be displayed during the menus or on a specific screen.

To create a menu, we can extend the container class. This allows us to create the components relevant to the specific menu in the constructor, catch any interesting keys by overwriting processInputEvent, perform any resetting by overwriting methods such as onShow, and, most important, define the actions that should be performed when the user interacts with the components.

The MainMenuContainer has a few member variables that are instances of different components. An instance of HComponent is used to draw a background for the container, which is just a textured quad. Instances of HButton are used for the Start, Options, and Quit buttons. The components are created in the constructor of the container. Each component is set up and added to the container using the addComponent method. The add method ends up adding the component to the component list of the container, which, as mentioned already, is used for tasks such as rendering the components. By making the MainMenuContainer implement HListener, the container can serve as a listener for its components. When the components are created, their listener is set to the instance of the MainMenuContainer. When the actionPerformed method of the container is called, we can distinguish which component generated the event and carry out the appropriate action.

When the application is launched, the main menu is animated such that it slides down from the top. This action is accomplished by setting the initial position of the container to some negative number. As the container manager calls the paint method of the container, it also updates the position of the container using the deltaX and deltaY values of the container.

If you move the mouse over the components of the menu, you will see that a texture is used as the rollover texture to emphasize the component that is immediately underneath the mouse. There is also another texture used to indicate the component that has the keyboard focus. You can press the Ctrl key to cycle the keyboard focus or click a component to potentially trigger a change in keyboard focus. You can click a button to trigger it, or press the Enter key while the component has the keyboard focus.

When the Start button is triggered by the user, the main menu is popped from the container manager’s container stack, and the HudContainer is pushed on. Again, the main menu animates upward until it disappears. When the main menu is completely out of sight, the background color is changed as an indication that the game has started. This action is done by calling glClearColor through the overwritten onHide method of the MainMenuContainer.

You have to be careful about invoking GL calls through the GUI methods. It is important to note that you cannot invoke the glClearColor method (or other GL methods) from the processInputEvent or the actionPerformed methods. This is because these methods are indirectly called through the mouse and keyboard listeners on the GLCanvas, which are called by the AWT event thread. Keep in mind that GL calls should not be made by arbitrary threads. Instead, we must notify the render thread that we want to change the clear color so that it can change it for us. The notification can be accomplished by simply flipping a boolean value. Even though unlike actionPerformed and processInputEvent the onHide method is called by the render thread (through the paint method of container manager), it is still a good practice to request the GL operation.

The HUD container is similar to the main menu container. Because the initial position of the container is the same as its destination position, it appears without any animation. Note that clicking the components of the HUD does not cause them to gain the keyboard focus. None of the components of the HUD is focusable—even pressing the Ctrl key does not cause them to gain the keyboard focus. The Escape key is trapped by the HUD container’s processInputEvent method, which in turn pushes the main menu without popping the HUD. Note that at this point there are two containers in the stack of container manager.

The Options button on the main menu causes the main menu to be popped and the options container to be pushed on the stack. The components of the Options menu simply affect the behavior of the cube that is spinning in the background. The Options container emulates a container with multiple tabs. The tabs are triggered by triggering the toggle buttons on the left of the menu. One of the buttons on the first tab resets the rotation of the cube. The other button, which is also a toggle button, scales down the rotation velocity when it is toggled to be on and undoes the effect when the button generates an UNCHECKED event. Clicking the second tab causes the component of the first tab to be hidden and the components of the second tab to become visible. The second tab has a slider bar that controls the rotation velocity of the cube about its x-axis. See the OptionsMenuContainer class for additional details.

Triggering the Quit button from the main menu causes the program to exit. If the game has not been started by triggering the Start button, the application terminates immediately. However, if the main menu has been brought up after that game has started, a dialog box opens to confirm that you really want to exit. In that situation, three containers will be in the container stack.

Unlike the main menu, Options menu, and the HUD container, the Quit dialog box was not implemented by extending the container class. Instead, the background and yes and no buttons are instantiated and added to an instance of the HContainer class. Similarly, the listeners are created separately as anonymous inner classes. They are then added to the yes and no buttons. This is to show you that you do not always have to make a custom container class just to show a simple container. It is important to keep in mind that the startup time and memory footprint of the game will be increased by using anonymous inner classes. However, they are convenient, because they allow you to place the action code next to where the component is created. As long as you do not abuse them, you will be okay.

As mentioned previously, pressing the C key while the HUD is showing brings up the in-game console. The console is a container that has a text field and a text list. The text list component is set up so that it is not focusable. Moreover, the text field is set up to be in focus when the container is shown. When the Enter key is pressed, the corresponding listener receives an event, reads the text from the text field, and adds it to the text list. The text list always renders as many of the most recent entries as possible.