Video 15 Puzzle

Now that you have seen the general outlines of writing a compositor, it s time for some specific examples. For our first example, we ll write a video 15 puzzle.

A 15 puzzle, also called a slide puzzle , is a set of 15 tiles placed in a square frame, with an empty slot where the 16th tile would be. The tiles can be shuffled by sliding them horizontally or vertically, the object being to return the tiles to their original configuration. Typically, there is a picture on the tiles; when you solve the puzzle, you get to see the completed picture. Using the VMR, we can add a modern twist to an old game by putting video on the tiles instead of a static image. All we need is a compositor that divides the video into rectangles and draws them on the puzzle tiles. To see the video 15 puzzle in action, run the FifteenPuzzle sample that is in the AVBook\bin directory. (See Plate 8.)

The UI portion of the FifteenPuzzle application is essentially the same as the Mangler application from Chapter 9, minus the controls for setting the mixing preferences (because our compositor will ignore them anyway). The graph-building code is almost the same as well, except that we call IVMRFilterConfig9::SetImageCompositor to set the compositor, as described in the previous section. The compositor is an instance of the CPuzzleMixer class, declared as follows .

 class CPuzzleMixer : public IVMRImageCompositor9  {  public:      CPuzzleMixer(void);      ~CPuzzleMixer(void);      void HandleMouseClick(POINT &pt);      void ShufflePuzzle() { m_puzzle.Shuffle(); }      // IUnknown methods (not listed).      // IVMRImageCompositor9 methods (not listed).  private:      long m_nRefCount;           // Reference count.      CCriticalSection m_CritSec; // Critical section.      CPuzzle m_puzzle;           // Manages the puzzle.      ScaledRect m_SrcRect;       // Scales the source rectangle.      ScaledRect m_DestRect;      // Scales the destination rectangle.  }; 

The CPuzzle object (declared as the m_puzzle variable) manages the state of the puzzle. This class contains an abstract representation of the puzzle tiles ” it knows nothing about the video surfaces that are drawn on the tiles. Tile coordinates, as defined by this class, range from 0 to 1. The m_SrcRect and m_DestRect variables , which are both instances of the ScaledRect class, are used to map the tile coordinates to pixels. For example, if the destination rectangle is 320 pixels wide, then tile coordinate 1.0 maps to pixel 320, as shown in Figure 11.1.

Each tile has a starting position and a current position. The starting position is the tile s location before the puzzle is shuffled. The current position reflects the tile s location as the puzzle is shuffled and tiles are moved around by the user. For example, in a 4 — 4 puzzle, the tile in the upper-left corner has a starting position of (0, 0). If the user slides that tile one square to the right, the tile s new position becomes (0.25, 0). The tiles are animated: when the user clicks on a tile, the tile is given a velocity and a target position. The tile s current position is updated each frame until the tile reaches the target position.

The image on a tile is always calculated relative to its starting position, which never changes. This position is used to derive the source rectangle on the video surface when the mixer draws that particular tile. The tile s placement within the application window is determined by the tile s current position.

Figure 11.1: Source and target rectangles in the video 15 puzzle.

The compositor for this application is reasonably simple, so we can do all of the work inside the CompositeImage method. The other IVMRImageCompositor9 methods simply return S_OK . Here is the code for the CompositeImage method.

 STDMETHODIMP CPuzzleMixer::CompositeImage(      IUnknown *pD3DDevice, IDirect3DSurface9 *pRenderTarget,      AM_MEDIA_TYPE *pmtRenderTarget, REFERENCE_TIME rtStart,      REFERENCE_TIME rtEnd, D3DCOLOR dwClrBkGnd,      VMR9VideoStreamInfo *pVideoStreamInfo, UINT cStreams)  {      // Hold the critical section.      CLock lock(&m_CritSec);      CComQIPtr<IDirect3DDevice9> pDevice(pD3DDevice);      if (!pDevice)      {           return E_FAIL;      }      // Use the first video stream, and ignore the others.      const VMR9VideoStreamInfo *p = pVideoStreamInfo;      // Set the scaling sizes for the puzzle coordinates.      RECT rcTarget;      GetTargetRectangle(pmtRenderTarget, &rcTarget);      m_DestRect.SetScaleSize(rcTarget.right, rcTarget.bottom);      m_SrcRect.SetScaleSize(p->dwWidth, p->dwHeight);      // Clear the background.      // Note: We could just clear the empty tile space instead.      pDevice->Clear(0, NULL, D3DCLEAR_TARGET, dwClrBkGnd, 0, 0);      // Draw each tile.      for (int y = 0; y < m_puzzle.Columns(); y ++)      {          for (int x = 0; x < m_puzzle.Rows(); x++)          {              CTile *pTile = m_puzzle.GetTileAt(x,y);              if (pTile)              {                  pTile->Update();  // Update the tile position.                  RECT rcSrc, rcDest;                  NORMALIZEDRECT rcCurrentPos, rcStartingPos;                  // Get the current and starting position of the tile.                  m_puzzle.GetStartingPosition(pTile, &rcStartingPos);                  m_puzzle.GetCurrentPosition(pTile, &rcCurrentPos);                  // Map these values to surface pixel coordinates.                  m_SrcRect.ScaleRect(rcStartingPos, &rcSrc);                  m_DestRect.ScaleRect(rcCurrentPos, &rcDest);                  // Align to even pixels.                  AlignRect(rcSrc);                  AlignRect(rcDest);                  // Stretch part of the video image to the destination surface.                  pDevice->StretchRect(p->pddsVideoSurface, &rcSrc,                      pRenderTarget, &rcDest, D3DTEXF_NONE );              }          }      }      return S_OK;  } 

The puzzle compositor only uses the first video stream in the pVideoStreamInfo array. Even if the application rendered several streams, the puzzle compositor would ignore all the rest. (In fact, you could use this feature to switch the puzzle video on the fly by rendering two streams and switching the z-order .)

For each tile in the puzzle, we map the tile s starting position to the video surface, and the current position to the render target. The GetStartingPosition function returns the starting position in tile coordinates [0 1], and the dwWidth and dwHeight fields of the VMR9VideoStreamInfo structure give the size of the video surface. The m_SrcRect object does the scaling. For example, if the source video is 320 — 240 pixels and the tile s starting position is (0.25, 0), the tile is drawn at pixel (80, 0). Similarly, GetCurrentPosition returns the tile s current position, GetTargetRectangle returns the region of the render target where the video should appear, and m_DestRect does the scaling.

The resulting rectangles, rcSrc and rcDest , are used as source and destination rectangles in the call to StretchRect . Note that StretchRect is called once for each tile, because the tiles are jumbled up after the puzzle has been shuffled.

Another approach would be to use the tile coordinates as texel coordinates and then texture the video image onto the render target. This approach is slightly more complicated than using StretchRect , but it opens up some interesting possibilities. For example, you could give the tiles a 3-D look with rounded corners and beveled edges. The next example uses texturing; by the time you are finished with this chapter, you ll know how to make these improvements in the FifteenPuzzle application.

As you can see, the puzzle compositor disregards all of the application s mixing preferences except for background color . However, the application still needs to set the position of the video by calling IVMRWindowlessControl9::SetVideoPosition . This method tells the VMR where to place the composited image in the application window. The compositor has no control over that aspect of rendering.

The FifteenPuzzle application shows that you can do some interesting things without too much code. But there s a limit to the effects that you can achieve with StretchRect. Textures are much more powerful, and you can use them to create almost any kind of effect that you can imagine.