When you click quickly on a layer and make a vertex, you make a corner point vertex. If you click and drag one way or another, the vertex point becomes a smooth point, otherwise known as a Bezier Vertex Point. Bezier vertexes have direction handles that you can tweak while drawing or after you've closed your masks.
You can change a vertex from point to smooth (or vise versa) by using the Convert Vertex Tool, which is located beneath the Pen Tool in the Tools Palette.
You can also press G on the keyboard to toggle between the Pen, Add Vertex, Delete Vertex, or Convert Vertex Tools.
Hold down the Option+Cmd (Alt+Ctrl) keys to change the Selection Tool into the Convert Vertex Tool.
Be careful not to have all your points selected because you will convert them all.
Hold down the Cmd key (Ctrl) over a vertex point to turn the Selection Tool in a Remove Vertex Tool.
Hold down the Cmd key (Ctrl) anywhere else on the mask to change the Selection Tool to the Add Vertex Tool.
Option+Click (Alt+Click) to select all points in a mask.
To invert a mask, press Cmd+Shift+I (Ctrl+Shift+I).
For a new mask, press Cmd+Shift+N (Ctrl+Shift+N).