Chapter 14. Scalability and the Portal

What happens if you build the best portal ever, and hundreds, thousands, or millions of people flock to it day after day to get the information for which they hunger? This is precisely the problem we all seek, the reason this book was written, and the motivation for your making it all the way to the final chapter. If the portal cannot support the volume of its users or provide the performance they expect, it will fail. Scalability is a way to describe the impact of rising traffic, users, data volume, transactions, and other metrics on the performance of the portal.

High availability is the close companion of scalability. An infrastructure that supports a large number of users and high volume of transactions cannot be so brittle that it fails frequently. Fortunately, the same technologies and architecture that support scalability lend themselves to improving availability, such as clustering, fault-tolerant hardware, and distributed systems.

To achieve the highest levels of availability, you need three things:

• The right technology . Hardware and software components are the building blocks of your architecture. They should be of the correct performance category and be compatible with one another. It makes no sense to hire a crew of construction workers to build a skyscraper out of straw. Spend your resources according to your business priorities and invest adequately where failure of a system is not an option.

• The right architecture . Quality building materials do not guarantee a comfortable, habitable space. Emulate the successful architectures of others that achieve goals similar to yours. Microsoft, for instance, shares many details of its information technology infrastructure, as do other vendors such as Dell and Hewlett-Packard. These are great case studies for large organizations.

• The right operations . Maintenance procedures must match the quality of the materials and the plan to achieve greatness. This is the area most likely to be shortchanged. Money may have already been spent on gee-whiz hardware and the high-priced architect consultants when it comes time to operate and maintain the network or server farm. High availability is every bit as much a function of the vigilance of the people who monitor the network as of the underlying hardware and software.

Failure in any one of these portal infrastructure pillars will cause the entire structure to wobble or perhaps topple. Be sure to pay equal attention to all three, as they are interdependent. A great architecture with top- drawer hardware does little good if no one is trained on how to operate it, and you don't have to look far to find people who have employed today's technology with yesterday 's architecture to produce underwhelming results.

In this chapter we take a look at the scalability and availability goals for a portal and options for scaling. We also survey hardware, software, and network techniques and best practices for achieving the performance you need. Special attention is paid to .NET scalability and the unique requirements of different servers in the Microsoft portal platform. An approach that produces optimal performance with Content Management Server, for instance, may not be appropriate for beefing up the search engine, or making catalog order transactions quicker.

The best this chapter can do is to scratch the surface of the topic, as scalability and related issues are both broad and deep. Volumes have been written on each of the areas discussed here, including entire books devoted to scalability of a single Microsoft product such as SQL Server or Windows 2000.