After Effects ships with two 3D rendering plug-ins that controlling how 3D layers interact. You can access these by choosing Composition>Composition Settings or pressing Cmd+K (Ctrl+K). From the Composition Settings Window, click on the Advanced Tab and choose one of the following Rendering Plug-Ins.
Advanced 3D: By default, AE6 uses the Advanced 3D rendering plug-in. This plug-in accurately handles layer intersections (including blend modes and adjustment layers). It also uses shadow maps to give more control over shadow rendering.
Standard 3D: This was the original method that was introduced with AE5. You can use this if there are no intersecting layers. The benefit is that it's faster for some compositions. If rotated layers appear jaggy, click the Options button and choose the More Accurate anti-aliasing option.