Some products have been designed to assist you in the optimization and tuning process. There are code analyzers and profilers, which help discover both bottlenecks and slow sections in your code. There are also optimizers that simply try to write faster code using a variety of techniques.
An example of an optimization tool is Codeplay's VectorC. This package can automatically analyze your code and generate machine language, which is as close to optimal as possible. Usually, we would write our routine using a general-purpose compiler and, once we have detected a loop that is on the critical path to performance, use VectorC to write tighter code. VectorC can optimize for a variety of platforms (PCs and PS2s are supported) and specific processor types. Thus, a code snippet can be analyzed for a Pentium 4, an Athlon, or many other CPU types. Under specific tests on mission-critical routines, I have consistently seen VectorC outperform standard compilers by a factor of 25 40 percent.