When you turn on a layer's 3D switch, the most noticeable change is the appearance of the "manipulators" at the center of the layer. Dragging these red, green and blue arrows allow you to Constrain-Drag (or Constrain-Rotate, if you're using the Rotate tool) a layer in the X, Y or Z directions.
The three Axis Mode buttons on the Tool Palette dictate which way is X, which way is Y and which way is Z. The default Axis mode is Local, which means that each layer has its own X, Y and Z. Rotate a layer around its Y axis (so that you're staring at its edge, like a postcard), and its Z axis will no longer point straight at you. It will point to the right or to the left (like the X axis used to), depending on which way you rotated the layer. In Local Axis Mode, you can think of the manipulator as being attached to the layer, and rotating with it.
Contrast that with World Axis Mode, which is the middle Axis Mode button on the Tool Palette: in World Axis Mode, no matter how you rotate the layer, the manipulators stay put. This is because World Axis Mode uses set X, Y and Z directions. This is similar to the real world, in which North is always North. If you change your view by moving the active camera or selecting one of the view options in the Comp Window (i.e., Top or Custom View 1), the manipulators will change direction. But they will still point to "true" X, Y and Z. Similarly, if you were in a space ship, hovering over the North Pole, north would be below you.
The final option, View Axis Mode, orients to your screen. So that no matter how you move the camera or layer, X always points left and right, Y always points up and down, and Z always points in and out.