You control the various settings for the stencil buffer using the IDirect3DDevice7::SetRenderState method. Here are the stencil-related members of the D3DRENDERSTATETYPE enumerated type:
typedef enum _D3DRENDERSTATETYPE { D3DRENDERSTATE_STENCILENABLE = 52, // Enable or disable // stenciling D3DRENDERSTATE_STENCILFAIL = 53, // Stencil operation D3DRENDERSTATE_STENCILZFAIL = 54, // Stencil operation D3DRENDERSTATE_STENCILPASS = 55, // Stencil operation D3DRENDERSTATE_STENCILFUNC = 56, // Stencil comparison // function D3DRENDERSTATE_STENCILREF = 57, // Reference value for // stencil test D3DRENDERSTATE_STENCILMASK = 58, // Mask value used in // stencil test D3DRENDERSTATE_STENCILWRITEMASK = 59, // Stencil-buffer write // mask } D3DRENDERSTATETYPE; |
These are the definitions for the stencil-related render states:
The D3DSTENCILOP enumerated type describes the stencil operations for the D3DRENDERSTATE_STENCILFAIL, D3DRENDERSTATE_STENCILZFAIL, and D3DRENDERSTATE_STENCILPASS render states. Here's the definition of D3DSTENCILOP:
typedef enum _D3DSTENCILOP { D3DSTENCILOP_KEEP = 1, D3DSTENCILOP_ZERO = 2, D3DSTENCILOP_REPLACE = 3, D3DSTENCILOP_INCRSAT = 4, D3DSTENCILOP_DECRSAT = 5, D3DSTENCILOP_INVERT = 6, D3DSTENCILOP_INCR = 7, D3DSTENCILOP_DECR = 8, D3DSTENCILOP_FORCE_DWORD = 0x7fffffff } D3DSTENCILOP; |
These members serve the following purposes:
Let's walk through some code that uses the stencil buffer while rendering a scene. This code is from a sample that shows how to draw shadows. For now, don't worry about how all this code generates shadows—the algorithm is described later in the chapter.
The shadow-rendering code starts out by disabling the depth buffer and enabling the stencil buffer:
//------------------------------------------------------------------- // Name: RenderShadow // Desc: //------------------------------------------------------------------- HRESULT CMyD3DApplication::RenderShadow() { // Turn off depth buffer and turn on stencil buffer. m_pd3dDevice->SetRenderState( D3DRENDERSTATE_ZWRITEENABLE, FALSE ); m_pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILENABLE, TRUE ); |
Next the code sets the comparison function that performs the stencil test by calling the IDirect3DDevice7::SetRenderState method and setting the first parameter to D3DRENDERSTATE_STENCILFUNC. The second parameter is set to a member of the D3DCMPFUNC enumerated type. In this code, we want to update the stencil buffer everywhere a primitive is rendered, so we use D3DCMP_ALWAYS:
// // Set up stencil comparison function, reference value, and masks. // Stencil test passes if ((ref & mask) cmpfn (stencil & mask)) // is true. // m_pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILFUNC, D3DCMP_ALWAYS ); |
In this sample, we don't want the stencil buffer to change if either the stencil buffer test or the depth buffer test fails, so we set the appropriate states to D3DSTENCILOP_KEEP:
m_pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILZFAIL, D3DSTENCILOP_KEEP ); m_pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILFAIL, D3DSTENCILOP_KEEP ); |
The next settings are different depending on whether a 1-bit or a multibit stencil buffer is present. If the stencil buffer has only 1 bit, the value 1 is stored in the stencil buffer whenever the stencil test passes. Otherwise, an increment operation (either D3DSTENCILOP_INCR or D3DSTENCILOP_INCRSAT) is applied if the stencil test passes.
if(g_bUseOneBitStencil) { pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILREF, 0x1 ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILMASK, 0x1 ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILWRITEMASK, 0x1 ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILPASS, D3DSTENCILOP_REPLACE ); } else { pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILREF, 0x1 ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILMASK, 0xffffffff ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILWRITEMASK, 0xffffffff ); pd3dDevice->SetRenderState( D3DRENDERSTATE_STENCILPASS, g_StencIncOp ); } |
At this point, the stencil state is configured and the code is ready to render some primitives.