Creating a Stencil Buffer

[Previous] [Next]

Before creating a stencil buffer, you need to determine what stenciling capabilities the target system supports. To do this, call the IDirect3DDevice7::GetCaps method. The dwStencilCaps flags specify the stencil-buffer operations that the device supports. The reported flags are valid for all three stencil-buffer operation render states: D3DRENDERSTATE_STENCILFAIL, D3DRENDERSTATE_STENCILPASS, and D3DRENDERSTATE_STENCILZFAIL. Direct3D defines the following flags for dwStencilCaps:

• D3DSTENCILCAPS_DECR Indicates that the D3DSTENCILOP_DECR operation is supported
• D3DSTENCILCAPS_DECRSAT Indicates that the D3DSTENCILOP_DECRSAT operation is supported
• D3DSTENCILCAPS_INCR Indicates that the D3DSTENCILOP_INCR operation is supported
• D3DSTENCILCAPS_INCRSAT Indicates that the D3DSTENCILOP_INCRSAT operation is supported
• D3DSTENCILCAPS_INVERT Indicates that the D3DSTENCILOP_INVERT operation is supported
• D3DSTENCILCAPS_KEEP Indicates that the D3DSTENCILOP_KEEP operation is supported
• D3DSTENCILCAPS_REPLACE Indicates that the D3DSTENCILOP_REPLACE operation is supported
• D3DSTENCILCAPS_ZERO Indicates that the D3DSTENCILOP_ZERO operation is supported

Direct3D embeds the stencil-buffer information with the depth-buffer data. To determine what formats of depth buffers and stencil buffers the target system's hardware supports, call the IDirect3D7::EnumZBufferFormats method, which has the following declaration:

 HRESULT IDirect3D7::EnumZBufferFormats (     REFCLSID riidDevice,     LPD3DENUMPIXELFORMATSCALLBACK lpEnumCallback,     LPVOID lpContext ); 

Parameter	Description
riidDevice	A reference to a globally unique identifier (GUID) for the device whose depth-buffer formats you want enumerated
lpEnumCallback	The address of a D3DEnumPixelFormatsCallback function you want called for each supported depth-buffer format
lpContext	Application-defined data that is passed to the callback function

If the method succeeds, it returns the value D3D_OK. If it fails, the method returns one of these four values:

• DDERR_INVALIDOBJECT
• DDERR_INVALIDPARAMS
• DDERR_NOZBUFFERHW
• DDERR_OUTOFMEMORY

The following code determines what stencil buffer formats are available and what operations are supported and then creates a stencil buffer. As you can see, this code notes whether the stencil buffer supports more than 1 bit—some stenciling techniques must be handled differently if only a 1-bit stencil buffer is available.

 HRESULT CMyD3DApplication::CreateStencilBuffer() {     g_bCanOnlyDoOneBitStencil=FALSE;     DWORD dwStencilCaps = m_pDeviceInfo->ddDeviceDesc.dwStencilCaps;     if( (!(dwStencilCaps & D3DSTENCILCAPS_INCR) &&           !(dwStencilCaps & D3DSTENCILCAPS_INCRSAT)) ||         (!(dwStencilCaps & D3DSTENCILCAPS_DECR) &&          !(dwStencilCaps & D3DSTENCILCAPS_DECRSAT)))     {         // Must do 1-bit stencil buffer.         g_bCanOnlyDoOneBitStencil=TRUE;     }     else     {         // Prefer sat ops that cap at 0/max, but can use other          // ones as long as enough stencil bits.         g_StencIncOp=(dwStencilCaps & D3DSTENCILCAPS_INCRSAT)?              D3DSTENCILOP_INCRSAT:D3DSTENCILOP_INCR;         g_StencDecOp=(dwStencilCaps & D3DSTENCILCAPS_DECRSAT)?              D3DSTENCILOP_DECRSAT:D3DSTENCILOP_DECR;     }     m_pddsRenderTarget->DeleteAttachedSurface( 0,NULL );     // Get z-buffer dimensions from the render target.     // Set up the surface description for the z-buffer.     DDSURFACEDESC2 ddsd;     D3DUtil_InitSurfaceDesc( ddsd );     m_pddsRenderTarget->GetSurfaceDesc( &ddsd );     ddsd.dwFlags         = DDSD_WIDTH | DDSD_HEIGHT | DDSD_CAPS |                             DDSD_PIXELFORMAT;     ddsd.ddsCaps.dwCaps  = DDSCAPS_ZBUFFER;     ddsd.ddsCaps.dwCaps2 = 0;     ddsd.ddsCaps.dwCaps3 = 0;     ddsd.ddsCaps.dwCaps4 = 0;     ddsd.ddpfPixelFormat.dwFlags = DDPF_ZBUFFER | DDPF_STENCILBUFFER;     if( m_pDeviceInfo->bHardware )         ddsd.ddsCaps.dwCaps  |= DDSCAPS_VIDEOMEMORY;     else         ddsd.ddsCaps.dwCaps  |= DDSCAPS_SYSTEMMEMORY;     // Get an appropriate pixel format from an enumeration of     // the formats.     m_pD3D->EnumZBufferFormats( (*m_pDeviceInfo->pDeviceGUID),                                 EnumZBufferFormatsCallback,                                 (VOID*)&ddsd.ddpfPixelFormat );     assert(ddsd.ddpfPixelFormat.dwStencilBitDepth!=0);     g_bCanOnlyDoOneBitStencil=g_bCanOnlyDoOneBitStencil ||          ((1<<ddsd.ddpfPixelFormat.dwStencilBitDepth)<NUM_SHADOWS);     g_dwMaxStencilValue=(1<<ddsd.ddpfPixelFormat.dwStencilBitDepth)-1;     // Leave g_bUseOneBitStencil set for the window-resize case.     if( !g_bUseOneBitStencil )         g_bUseOneBitStencil=g_bCanOnlyDoOneBitStencil;     SetMenuStates();     // Create and attach a z-buffer.     if( FAILED( m_pDD->CreateSurface( &ddsd, &m_pddsDepthBuffer,                                       NULL ) ) )         return E_FAIL;     if( FAILED(m_pddsRenderTarget->AddAttachedSurface(                                         m_pddsDepthBuffer ) ) )         return E_FAIL;     // The SetRenderTarget() call is needed to rebuild internal      // structures for the newly attached z-buffer.     return m_pd3dDevice->SetRenderTarget( m_pddsRenderTarget, 0L ); } //------------------------------------------------------------------- // Name: EnumZBufferFormatsCallback // Desc: Enumeration function to report valid pixel formats for //       z-buffers //------------------------------------------------------------------- static  HRESULT WINAPI EnumZBufferFormatsCallback( DDPIXELFORMAT* pddpf,                                            VOID* pddpfDesired ) {     if( NULL==pddpf || NULL==pddpfDesired )         return D3DENUMRET_CANCEL;     // If the current pixel formats match the desired ones      // (DDPF_ZBUFFER and possibly DDPF_STENCILBUFFER), copy it and      // return. This function isn't choosy--it accepts the first valid     // format that comes along.     if( pddpf->dwFlags == ((DDPIXELFORMAT*)pddpfDesired)->dwFlags )     {         memcpy( pddpfDesired, pddpf, sizeof(DDPIXELFORMAT) );         return D3DENUMRET_CANCEL;     }     return D3DENUMRET_OK; }