If you are using interlaced video (most video is) you may have to decide which field to show first. After Effects will normally do this for you by reading that metadata from the footage upon import. But if things don't look right (or you want to double-check ), do the following:
Select the footage in the Project window.
Choose File>Interpret Footage>Main or press Cmd+F (Ctrl+F).
Choose Upper Field First from the Separate Fields pop-up menu. Then click OK.
You now must open the item into a footage window. Choose it in the Project window and hold down Option (Alt) while you double-click the footage.
Select the Time Controls Window by pressing Cmd+3 (Ctrl+3).
Scroll through the Footage window until you find a section with a lot of movement.
Click the frame advance button several times. If the motion appears to jump backward every other frame, you have interpreted field wrong. Go back to footage Interpretation and choose Lower Field First.
While many analog capture cards vary, Digital Video (DV) is always lower field first.