General Techniques

In this section, I'll describe several general ways to improve performance, none of which should be especially surprising. The most important factor in determining a program's performance is to make sure that it uses efficient data structures and algorithms to accomplish time-consuming tasks. For the many tasks that are not time-consuming, the best data structures and algorithms are usually those that are easiest for you. Interestingly, of the common collections of linked lists, arrays, and hash tables, linked lists are by far the least efficient. The problem is that they allocate a separate piece of memory for each node, which leads to poor memory usage (poor locality of reference) and possible page faults. When the CPU must retrieve data from virtual memory stored on your hard disk, page faults are pure evil in terms of performance. A modern CPU can execute about a million instructions in the time it takes to handle a page fault, so an "efficient" binary search that results in page faults can be far slower than an "inefficient" linear search that doesn't. Check "Tips for Improving Time-Critical Code" in the Microsoft Visual C++ documentation for more information.

Profiling is an important technique for finding inefficiencies in your programs. In my distant past, my experiences with profiling were a big waste of time—the information I obtained was totally misleading. Either the profiling tools have improved or my luck has changed, but all of my recent attempts at profiling have been worthwhile. I find the profiling in Visual C++ easy to do and the results valuable. If you have had bad luck with profiling in the past, try it again. Maybe your luck will change as well.

You should always ship a release version of your program that is optimized and has the debug information removed. Note that the debug version of a program is not optimized at all. In fact, debug code is the opposite of optimized—it is terribly inefficient. While shipping the release build might seem like an easy and obvious thing to do, there is a catch. Most programmers do their development and quality assurance (QA) using the debug version. Since you must test the program you ship, you need to make the transition in the QA process from testing the debug version to testing the release version. Whatever you do, don't assume that you don't need to test the release version if you have carefully tested the debug version—although they share the same source code, they are not the same program. A test of the release version might reveal bugs not present in the debug version. Also note that not all optimizations are safe, especially aggressive speed optimizations. If you are concerned about optimization safety, optimize for size instead. You can obtain significantly better performance by optimizing for size than by not optimizing at all, and small code is generally fast code. All executable files that come with Microsoft Windows are optimized for size.

Lastly, here's an assortment of miscellaneous performance techniques to consider:

• Perform time-consuming operations on an as-needed basis. For example, don't create a complex object or read a large file until you need to. If possible, load large objects incrementally.
• Use file caching when accessing data from a slow network or CDROM.
• Offer the user previews of the results of time-consuming operations. For example, good special effects programs show thumbnails of the results of the different effects so that the user doesn't have to waste time viewing an effect only to decide he doesn't like it.
• Avoid performing unnecessary tasks in any functions that are frequently called, such as the MFC CWinApp::OnIdle and CWinApp::PreTranslateMessage functions.
• Use timers strategically. For example, suppose you have a list box with a list of options and selecting an option from the list results in an action that takes several seconds to perform. If you perform the action every time the user changes a selection, scrolling the list will be torture. Instead, use a timer and perform the action only after the user hasn't made a change for a few seconds.
• Don't do all your testing on hardware that is significantly better than your average target user's. Evaluating your program's performance using state-of-the-art hardware might cause you to draw misleading conclusions.