Scalability and Performance

[Previous] [Next]

In the early days of programming, developers had limited system resources. This forced developers to implement crafty and un-maintainable algorithms to eke out as much system performance as possible. Today, computer system performance and storage capabilities have increased enormously, allowing developers to design simpler and more maintainable algorithms.

Unfortunately, these advances have also allowed developers to become lazy. I know developers who don't even think twice about allocating megabytes of storage for tasks that need no more than a few kilobytes, tops. I also know developers who use mutex objects where critical sections would more than suffice. These developers simply don't care that functions that reference a mutex require a user-mode to kernel-mode round-trip transition, which requires about 1000 CPU cycles—and that doesn't even include the code that must execute once in kernel mode. In contrast, critical sections usually stay entirely in user mode and require about 100 CPU cycles to execute.

More and more people are using computers because servers offer the information that improves our quality of living. Studies show that users become easily frustrated with an unresponsive server and seek out desired information elsewhere, which translates into loss of business and revenue. Similarly, an enterprise with an unresponsive server frustrates employees and ultimately affects productivity—this also translates into a loss of business and revenue.

In some situations, you can improve server responsiveness by adding more computers. However, for many reasons you are usually better off running your server on a single machine when you can. First, writing server software that has various parts of itself executing on different machines is much harder than writing software that executes on a single machine. Second, the complexity of administering multiple machines often increases at a greater-than-linear rate. Third, you introduce several more potential points of failure, making errors much more difficult to locate and correct. Using only a single-machine server is not always achievable, but you should keep this ideal in mind while you implement your server's code.

On a single-machine server, performance can be gained by adding RAM, processors, disk storage, network cards, and so on, but only if your server's code is implemented to use these resources in an efficient manner. For example, the performance of a server using a one-thread-per-client connection usually won't double simply by doubling the amount of RAM in the machine. However, if the same server were designed using an efficient thread-pooling algorithm, the server's performance would scale well with additional resources.

Writing highly scalable applications requires a great deal of discipline. At every step, you must consider the ramifications of the code you write:

• Am I writing this code so that a user-mode-to-kernel-mode transition is avoided?
• Am I aligning my memory access so that it doesn't span a cache-line boundary?
• Am I ensuring that my variables are properly aligned?
• Am I replacing processes with threads to reduce the use of system resources?
• Am I reducing the number of runnable threads to avoid wasteful context switching?
• Am I having threads do useful work while waiting for device I/O operations to complete?
• Am I abandoning ANSI strings in favor of Unicode strings to improve the performance of Windows functions?
• Am I exploiting the cool features that Windows has to offer?

I mention these questions here in the hope that you will keep them in mind while you implement your code. Every item is discussed in detail either in Programming Applications for Microsoft Windows, Fourth Edition, or in this book.