Handling Post-Screen Shaders

The first thing you have to get for any post-screen shader is the rendered scene and put it into a render target (see Figure 8-1). The image illustrates some example post-screen shaders in FX Composer, which can be used to test out the post-screen effects before you include them into your graphics engine.

Figure 8-1

Before getting into the more complex post-screen shaders and how you can get the scene render target with help of the new RenderToTexture helper class, you will first test out the Pre-Screen Sky Cube Mapping shader and get it to work in your engine the same way it works in FX Composer.

Pre-Screen Sky Cube Mapping

First of all you need a sky cube map, which consists of six faces like a cube. Each of these faces should be in high resolution because in the game you only look in one direction. It’s okay to use 512×512 for each face if you use a lot of post-screen shaders; 1024×1024 is even better, because the sky on high-resolution screens will look much better this way.

It is not easy to create such a sky cube texture. You can either use the DirectX DxTexture tool and put together the cube map with six separate faces yourself (or do the same thing in your code, but it is much easier to load one single cube map file that contains all six faces instead of loading six separate textures and putting them together yourself). Alternatively a program like Photoshop can be used to store a big 6*512×512 texture and save it as a cube map dds with help of the Nvidia DDS Exporter plug-in for Photoshop (see Figure 8-2).

Figure 8-2

If you don’t have a good looking cube map, search the Web for one; there are some example textures available. Most of them have bad resolutions, but for testing they are fine. Or just use the cube map from this book if you like it. For the Rocket Commander game you use a space cube map that looks a little different on each of the six sides to help you with navigating through the 3D space levels of the game (see Figure 8-3).

Figure 8-3

Take a look at the shader to put this cube texture on the screen. You call this shader even before rendering the scene, and because it will fill all background buffer pixels you don’t have to clear the background color target anymore (but you probably still want to clear the z-buffer). To render this shader you should also disable depth comparing because you don’t care how far away the sky is, so it should always be rendered.

The following code is from the PreScreenSkyCubeMapping.fx shader:

  struct VertexInput {   // We only need 2d screen position, that's all.   float2 pos      : POSITION; }; struct VB_OutputPos3DTexCoord {   float4 pos      : POSITION;   float3 texCoord : TEXCOORD0; }; VB_OutputPos3DTexCoord VS_SkyCubeMap(VertexInput In) {   VB_OutputPos3DTexCoord Out;   Out.pos = float4(In.pos.xy, 0, 1);   // Also negate xy because the cube map is for MDX (left handed)   // and is upside down   Out.texCoord = mul(float4(-In.pos, scale, 0), viewInverse).xyz;   // And fix rotation too (we use better x, y ground plane system)   Out.texCoord = float3(     -Out.texCoord.x*1.0f,     -Out.texCoord.z*0.815f,     -Out.texCoord.y*1.0f);   return Out; } // VS_SkyCubeMap(..) float4 PS_SkyCubeMap(VB_OutputPos3DTexCoord In) : COLOR {   float4 texCol = ambientColor *     texCUBE(diffuseTextureSampler, In.texCoord);   return texCol; } // PS_SkyCubeMap(.) technique SkyCubeMap < string Script = "Pass=P0;"; > {   pass P0 < string Script = "Draw=Buffer;"; >   {     ZEnable = false;     VertexShader = compile vs_1_1 VS_SkyCubeMap();     PixelShader  = compile ps_1_1 PS_SkyCubeMap();   } // pass P0 } // technique SkyCubeMap 

For rendering the shader you only need a Vector2 screen position, which can be converted to the sky cube position by multiplying it with the inverse view matrix. The scale variable can be used to tweak the zooming a little bit, but usually this value should be 1.0. After getting the texture coordinate for the sky cube map, you need to convert it from the standard left-handed system cube that maps are usually stored in. To do that you have to swap the y and z coordinates and you also invert the direction the cube map texture coordinate is pointing to. And finally you multiply the y value by a custom value of 0.815 to make the sky rendered in a 1:1 aspect ratio mode look correct on a 4:3 aspect ratio of the screen. All these tweaking variables were tested with the unit test presented next until the background sky just looked correct.

If you want to build your own sky cube map, you can use the DirectX Texture Tool from the DirectX SDK to load normal 2D images on each of the six cube map faces of a cube texture. Additionally, many tools are available to directly render 3D scenes into cube maps such as Bryce, 3D Studio Max, and many other 3D content creation applications.

Then you rotate the sky cube map for XNA compatibility because it was created for MDX and uses a left-handed system. Rendering the sky cube map is pretty easy - just grab the cube texture color value and multiply it with the optional ambient color value, which is usually white. The shader is not using any advanced Shader Model 2.0 functionality and it is absolutely no problem to compile it to Shader Model 1.1.

To get it to work in your XNA graphic engine you use the following unit test:

  public static void TestSkyCubeMapping() {   PreScreenSkyCubeMapping skyCube = null;   TestGame.Start("TestSkyCubeMapping",     delegate     {       skyCube = new PreScreenSkyCubeMapping();     },     delegate     {       skyCube.RenderSky();     }); } // TestSkyCubeMapping() 

The class is derived from the ShaderEffect class from the previous chapters and it just loads the PreScreenSkyCubeMapping.fx shader in the constructor; all other shader effect parameters are already defined in the ShaderEffect class. The important new method in this class is RenderSky, which basically executes the following code:

  AmbientColor = setSkyColor; InverseViewMatrix = BaseGame.InverseViewMatrix; // Start shader // Remember old state because we will use clamp texturing here effect.Begin(SaveStateMode.SaveState); for (int num = 0; num < effect.CurrentTechnique.Passes.Count; num++) {   EffectPass pass = effect.CurrentTechnique.Passes[num];   // Render each pass   pass.Begin();   VBScreenHelper.Render();   pass.End(); } // foreach (pass) // End shader effect.End(); 

Like in any other shader you wrote before you set the effect parameters first, which is only the ambient color and the inverse view matrix because the shader does not use anything else. Then the shader is started and you render all passes (well, you only have one pass) with the help of the VBScreenHelper Render method, which is also used for all post-screen shaders.

VBScreenHelper generates a very simple screen quad for you and handles all the vertex buffer initialization and rendering. The following code initializes the vertex buffer with just VertexPositionTexture vertices. You only have one static instance of the helper class and it is used only in pre and post-screen shaders.

  public VBScreen() {   VertexPositionTexture[] vertices = new VertexPositionTexture[]     {       new VertexPositionTexture(         new Vector3(-1.0f, -1.0f, 0.5f),         new Vector2(0, 1)),       new VertexPositionTexture(         new Vector3(-1.0f, 1.0f, 0.5f),         new Vector2(0, 0)),       new VertexPositionTexture(         new Vector3(1.0f, -1.0f, 0.5f),         new Vector2(1, 1)),       new VertexPositionTexture(         new Vector3(1.0f, 1.0f, 0.5f),         new Vector2(1, 0)),     };   vbScreen = new VertexBuffer(     BaseGame.Device,     typeof(VertexPositionTexture),     vertices.Length,     ResourceUsage.WriteOnly,     ResourceManagementMode.Automatic);   vbScreen.SetData(vertices);   decl = new VertexDeclaration(BaseGame.Device,     VertexPositionTexture.VertexElements); } // VBScreen() 

As discussed in Chapters 5 and 6 the view space uses –1 to +1 as the minimum and maximum values for the screen borders. Everything above these values will not be on the screen. The projection matrix then puts the pixels at the real screen positions. You use simple VertexPositionTexture vertices here. For the sky cube mapping shader you don’t even need the texture coordinates, but they will be used for the post-screen shaders later. Please note that the position in screen coordinates of +1, +1 is the lower-left corner of the screen and –1, –1 is the upper-right corner. If you want the top-left corner of a screen texture to be displayed on the top left, the texture coordinates of 0, 0 should be in the –1, +1 corner.

You should always specify WriteOnly and Automatic when creating VertexBuffers; this way the hardware can access them much faster without having to worry about read access from the CPU, and Automatic makes it easier to get rid of the vertex buffer and let XNA handle re-creation issues when the vertex buffer gets lost and has to be re-created (which can happen when the user presses Alt+Tab in a fullscreen application and then returns).

With all these new classes up and ready you should now be able to execute your TestSkyCubeMapping unit test and see the result shown in Figure 8-4.

Figure 8-4

Writing a Simple Post-Screen Shader

Writing post-screen shaders is no picnic. It involves a lot of testing, handling render targets correctly, and it is not always easy to get the shader to work in both FX Composer and your engine and behave the same way. To make it a little simpler for the first post-screen shader you are just going to add a very simple effect to your scene. Instead of really modifying the rendered scene you are just going to darken down the screen borders to give the game a TV-like look. To change something that would not be possible without a post-screen shader, you also turn the whole screen to black and white.

Start with the PostScreenDarkenBorder.fx shader file in FX Composer. After the header comment and the description of the shader you define a script, which is only used for FX Composer to indicate that this shader is a post-screen shader and should be rendered a certain way. All variables starting with an uppercase letter are considered to be constant for your purposes.

Neither your engine nor FX Composer care about the casing, but this way you can easily spot which variables should be constant and not be changed in the app and which ones have to be set from the application. ClearColor and ClearDepth are both used only in the FX Composer. In your application you don’t have to worry about that because you handle your own clearing. For testing you can set other colors as black for ClearColor or 1.0 for depth if you like, but most of the time, these values will always be the same and they default to the same values for all upcoming post-screen shaders.

  // This script is only used for FX Composer, most values here // are treated as constants by the application anyway. // Values starting with an upper letter are constants. float Script : STANDARDSGLOBAL <   string ScriptClass = "scene";   string ScriptOrder = "postprocess";   string ScriptOutput = "color";   // We just call a script in the main technique.   string Script = "Technique=ScreenDarkenBorder;"; > = 0.5; const float4 ClearColor : DIFFUSE = { 0.0f, 0.0f, 0.0f, 1.0f}; const float ClearDepth = 1.0f; 

Post-screen shaders need to know about the screen resolution and you need access to the scene map render target. The following code is used to allow you to assign the window size and the scene map. The window size is the resolution you render into and the scene map render target contains the fully rendered scene as a texture. Please note the semantics like VIEWPORTPIXELSIZE used here to help FX Composer to assign the correct values for the preview rendering automatically. If you don’t use them FX Composer will not be able to use the correct window size and you would have to set it yourself in the properties panel.

  // Render-to-Texture stuff float2 windowSize : VIEWPORTPIXELSIZE; texture sceneMap : RENDERCOLORTARGET <   float2 ViewportRatio = { 1.0, 1.0 };   int MIPLEVELS = 1; >; sampler sceneMapSampler = sampler_state {   texture = <sceneMap>;   AddressU  = CLAMP;   AddressV  = CLAMP;   AddressW  = CLAMP;   MIPFILTER = NONE;   MINFILTER = LINEAR;   MAGFILTER = LINEAR; }; 

For darkening down the screen border you also use a little helper texture with the name ScreenBorderFadeout.dds (see Figure 8-5):

  // For the last pass we add this screen border fadeout map to darken the borders texture screenBorderFadeoutMap : Diffuse <   string UIName = "Screen border texture";   string ResourceName = "ScreenBorderFadeout.dds"; >; sampler screenBorderFadeoutMapSampler = sampler_state {     texture = <screenBorderFadeoutMap>;     AddressU  = CLAMP;     AddressV  = CLAMP;     AddressW  = CLAMP;     MIPFILTER = NONE;     MINFILTER = LINEAR;     MAGFILTER = LINEAR; }; 

Figure 8-5

Tip 

This texture has to be added to your content directory as well as the .fx shader file. XNA will not automatically add this texture like for models that contain materials and textures.

To achieve the darkening effect every pixel from the scene map is simply multiplied by the color value in the screen border fadeout texture. This way most pixels stay the same (white middle); pixels at the screen border will get darker and darker, but not completely black (or else the border would be black), so you just want to make it a little darker.

Take a look at the very simple vertex shader, which just passes the texture coordinates to the Pixel Shader. To make the shader compatible to Pixel Shader 1.1 you have to duplicate the texture coordinates because you need to access them twice, once for the scene map and once for the screen border fadeout texture. Please also note that you are adding half a pixel to the texture coordinates to fix a very common problem of rendering textures on the screen. In DirectX (and XNA) all the pixels have an offset of 0.5 pixels; you just fix this by moving the pixel to the correct location.

You can read more about this issue online. You don’t have to worry about it normally because all the helper classes (for example, SpriteBatch) take care of this issue for you, but if you render your own post-screen shader you might want to fix this. For this shader it does not matter so much, but if you have a very precise shader that shadows certain pixels you want to make sure to completely hit the pixel position and not draw somewhere else.

  struct VB_OutputPos2TexCoords {   float4 pos      : POSITION;   float2 texCoord[2] : TEXCOORD0; }; VB_OutputPos2TexCoords VS_ScreenQuad(   float4 pos      : POSITION,   float2 texCoord : TEXCOORD0) {   VB_OutputPos2TexCoords Out;   float2 texelSize = 1.0 / windowSize;   Out.pos = pos;   // Don't use bilinear filtering   Out.texCoord[0] = texCoord + texelSize*0.5;   Out.texCoord[1] = texCoord + texelSize*0.5;   return Out; } // VS_ScreenQuad(..) 

The position is already in the correct space and will just be passed over. The Pixel Shader now accesses the scene map, and for testing if everything is set up correctly you can just return the scene map colors here:

  float4 PS_ComposeFinalImage(VB_OutputPos2TexCoords In) : COLOR {   float4 orig = tex2D(sceneMapSampler, In.texCoord[0]);   return orig; } // PS_ComposeFinalImage(...) 

After adding the ScreenDarkenBorder technique you should see the normal scene as always in FX Composer (see Figure 8-6). Please note that all the annotations here are required for FX Composer only! For rendering you use the pre-screen sky cube mapping shader described earlier and the standard sphere.

  // ScreenDarkenBorder technique for ps_1_1 technique ScreenDarkenBorder <   // Script stuff is just for FX Composer   string Script = "RenderColorTarget=sceneMap;"     "ClearSetColor=ClearColor; Clear=Color;"     "ClearSetDepth=ClearDepth; Clear=Depth;"     "ScriptSignature=color; ScriptExternal=;"     "Pass=DarkenBorder;"; > {   pass DarkenBorder   <     string Script = "RenderColorTarget0=; Draw=Buffer;";   >   {     VertexShader = compile vs_1_1 VS_ScreenQuad();     PixelShader  = compile ps_1_1 PS_ComposeFinalImage(sceneMapSampler);   } // pass DarkenBorder } // technique ScreenDarkenBorder 

Figure 8-6

Improvements

Just to test if the shader is activated (which can be done in FX Composer by clicking the shader in the materials panel and selecting “Apply to Scene”), you can now easily modify the output. Just change the last line in the Pixel Shader:

  return 1.0f - orig; 

This will subtract the color value from 1.0 for each component (shaders will automatically convert from float to float3 or float4 if required; you don’t have to specify float4 (1, 1, 1, 1)). You can probably guess that this formula will invert the whole image (see Figure 8-7), which looks funny, but is not really useful.

Figure 8-7

Okay, back to your initial mission: Apply the screen border texture. To darken down the edges you first load the screen border texture and then multiply the original scene map color value with it, that’s it. You just have to return the result of that and you are almost done (see Figure 8-8):

  float4 orig = tex2D(sceneSampler, In.texCoord[0]); float4 screenBorderFadeout =   tex2D(screenBorderFadeoutMapSampler, In.texCoord[1]); float4 ret = orig; ret.rgb *= screenBorderFadeout; return ret; 

Figure 8-8

You could now also convert the image to black and white by applying the Luminance formula, which just tells you which component to weight how much (green is always the most visible color):

  // Returns luminance value of col to convert color to grayscale float Luminance(float3 col) {   return dot(col, float3(0.3, 0.59, 0.11)); } // Luminance(.) 

This method can be applied to the original scene map by just modifying one line in your Pixel Shader to finally achieve the expected post-screen shader effect: Darken down the screen borders and apply a black and white effect (see Figure 8-9):

  float4 ret = Luminance(orig); 

Figure 8-9

Writing post-screen shaders can be a lot of fun once you have your basic setup, but you should try to implement the shader first into your engine before moving on to even more cool post-screen shaders and possibilities.