Software for the New Century

As expected, the development team didn't stop working on SQL Server. Several planned features that had not made it into the SQL Server 7.0 release needed to be incorporated into the product. Other new features were already in the later stages of development in preparation for a version after 7.0. Two new code names were already in use: Shiloh was to be the version after 7.0, commonly assumed to be just a "dot" release like 7.5, and Yukon was the name given to the next major revision of SQL Server.

Initially, the SQL Server product managers were reluctant to predict widespread, immediate acceptance of SQL Server 7.0. Because it was basically a complete rewrite of the core engine code, many people might consider it a 1.0-level release. It was also anticipated that many potential adopters and upgraders would not be anxious to go to any release that ended in dot-zero and would want to wait for at least the first service pack. So initially, the Shiloh release was planned as just a "Super Service Pack." It would include perhaps a few new features that didn't get into 7.0 because of the shipping schedule, plus any bug fixes that were deemed too big for a normal, unnumbered service pack. The plan was to have a quick turnaround and release Shiloh no more than a year after the initial release of SQL Server 7.0.

Several factors came into play to change this initial vision of Shiloh. First, there seemed to be very little hesitation among the already installed user base to upgrade to the new SQL Server 7.0. Also, sales figures for new customers far exceeded even the most optimistic predictions. People were using SQL Server 7.0, and they liked it. Even after the public release of the product, Microsoft continued running the ISV lab in the building where the SQL Server developers worked, so feedback was ongoing and readily available to the development team. In general, the issues that came up as problems were easily fixable and a service pack for SQL Server 7.0 was released in May 1999. A second service pack was released in March 2000. There seemed to be no need for the "Super Service Pack" that Shiloh was intended to be.

A second factor that influenced a change in vision for the Shiloh release was customer requests. After referential integrity with cascading updates and deletes failed to make it into SQL Server 7.0, the product team heard loud and clear from the customer base that any further delay in adding this feature would not be tolerable. Customers also wanted better support for partitioned views and optimizer support for the star schema design used in data warehousing applications.

Finally, there was competitive pressure to make the next release bigger and better than originally planned. The Million Dollar Challenge issued by Larry Ellison of the Oracle Corporation pointed out a major piece of functionality that "they" had and "we" didn't. Adding materialized views to the product, which would allow SQL Server to meet this challenge, would be more than just a simple fix.

So the decision was made to have Shiloh be a full-blown release with at least an 18-month development cycle, but the version number would still be 7.5. The magnitude of the changes that would eventually find their way into this release was not immediately clear because the only absolute "must-have" at the outset was the cascading updates and deletes. It soon became clear that the release would grow beyond initial expectations. The team itself had grown substantially, spilling out of their original building on the main Microsoft campus into parts of two adjacent buildings. With a larger group of developers, a greater number of small to medium-size new features could be added to the product without seriously jeopardizing the planned release date. This book describes many of these new features.

In addition to having to add a substantial amount of new functionality, the development team also imposed upon themselves something they called "stretch goals." They declared an objective of a 20 percent increase in performance across the board, on all types of applications, but to give themselves something concrete to aim for, they extended this for specific applications. The main example of this was their goal to improve performance on the SAP R/3 Sales and Distribution benchmark by at least 40 percent. To do this, they had to make very specific changes in the optimizer that would directly affect the SAP queries but also benefit many other applications. At the Windows 2000 launch event in San Francisco on February 17, 2000, benchmark results were announced that demonstrated 6700 users for the Sales and Distribution benchmark, compared to 4500 users in SQL Server 7.0 for the same benchmark on the same hardware (an eight-way Pentium III-550). This represented a 48 percent improvement, and the goal had been well met.

After the development period was extended to a full 18 months, the decision was made to add another new feature. This decision was held in the strictest secrecy and wasn't even discussed with many high-level executives at Microsoft. The feature wasn't even addressed until after the release of Beta 1 in November 1999, and it was not announced publicly until the February launch of Windows 2000. Referred to internally by the code name Coyote, this secret project allowed SQL Server 2000 to support distributed partitioned views to allow unprecedented scalability in data management operations. It was this new feature that allowed the world-record-breaking benchmark results to be achieved and announced in San Francisco in February 2000.

Originally, these scalability features were scheduled for the version after Shiloh, but a close investigation revealed that many of the necessary components were already in place. These features included extensions to the optimizations of union views and the ability to update union views. Details of distributed partitioned views will be discussed later in the book.

The initial Beta 1 of Shiloh was released to a very small group of Early Adopters and serious beta testers in September 1999. Shortly after that, Microsoft announced that the official name of the product would be SQL Server 2000. There were two main reasons for this change. First, because of everything that was changing about the product, it was unrealistic to make it just a dot release (7.5); it really needed a whole new number. Second, if the product had gone with the name 8.0, it would be the only Microsoft BackOffice product that didn't use the name 2000. To conform to this companywide naming standard, the name SQL Server 2000 was announced. However, if you look at the internal version number using the @@VERSION function, you'll see that it returns the number 8.00.194.

It turns out that from the user's perspective, SQL Server 2000 introduces even more new functionality than did its immediate predecessor. SQL Server 7 had a completely rewritten engine, including brand-new storage structures, data access methods, locking technology, recovery algorithms, transaction log architecture, memory architecture, and optimizer. These, of course, are the things that this book describes in detail. But for the end user, developer, or part-time database administrator (DBA), the external changes and language enhancements in SQL Server 7 were minimal. SQL Server 2000 adds dozens of new language features as well as drastic changes to existing objects such as table constraints, views, and triggers that all developers and most DBAs will need to know about.

Because the internal engine changes are minimal, only two betas were planned. Beta 2, released in April 2000, was the widespread public beta and was sent to thousands of interested users, conference attendees, ISVs, and consultants. The SQL Server 2000 development team froze the code at build 8.00.194.01, on August 6, 2000, and released the product to manufacturing on August 9, 2000.

Internal SQL Server development is a never-ending story. Just as after the release of SQL Server 7, new capabilities were already well past the design stage, ready to be implemented. Features that didn't make it into SQL Server 2000, as well as continued improvements in scalability, availability, interoperability, and overall performance, continue to be addressed. The development team is using feedback from the beta forums and from early users of the released version of SQL Server 2000 to shape the future of Microsoft SQL Server.