PIXEL SHADER MATH PRECISION IN DIRECTX 8 CLASS HARDWARE

Registers in pixel shaders are not full 32-bit floating point values. In fact, they are severely limited in their range. The minimum precision is 8 bits, which usually translates to an implementation of a fixed point number with a sign bit and 7–8 bits for the fraction. Since the complexity of pixel shaders will only grow over time, and hence the ability to do lengthy operations, you can expect that you'll rarely run into precision problems unless you're trying to do something like perform multiple lookup operations into large texture spaces. Only on older cards (those designed in 2001) will you find the 8-bit minimum. As manufacturers figure out how to reduce the size of the silicon and increase the complexity, they'll be able to squeeze more precision into the pixel shaders.

As the number of bits increase in the pixel shader registers, so will the overall range. You'll need to examine the D3DCAPS8 value MaxPixelShaderValue in order to see the range to which pixel registers are clamped. For PS 1.4 and 2.0, the texture registers have a valid range of ± MaxTextureRepeat. Figure 8.3 shows DirectX 8's 8-bit pixel register construction.

Figure 8.3: Pixel shader register construction.

In DirectX 6 and DirectX 7, this value was 0, indicating an absolute range of [0, 1]. In later versions of DirectX, this value represented an absolute range; thus in DirectX 8, you might see a value of 1, which would indicate a range of [−1, 1], or 8, which would indicate a range of [−8, 8].