1.3 Exchange third generation

 < Day Day Up > 



1.3 Exchange third generation

The third generation of Exchange is still in the design phase and the first functional beta releases will not appear until 2005 at the earliest. Microsoft calls the development of the third generation of Exchange the Kodiak project.

Since its inception, Exchange has used its own storage engine. The current Store uses the ESE (Extensible Storage Engine) database engine, variants of which also appear in the Active Directory and SharePoint Portal Server 2001 Store. In turn, ESE builds on Joint Engine Technology (JET), a generalized database engine that Microsoft has used for many purposes from Access to the database used to synchronize directory information between Exchange and Microsoft Mail.

The logic to justify using a separate database engine for Exchange is that the needs of a messaging system are very different from those of general database applications. A typical database attempts to place known limits around transactions in terms of the size and data, whereas a messaging system has to be able to handle messages that vary in size and content. The range in message type goes from a single-line message sent to a single recipient up to messages such as those with several multimegabyte attachments (of various types) sent to a distribution list, which is, in turn, composed of multiple nested distribution lists. Microsoft designed ESE to handle the huge variance in message traffic, and it has been successful in meeting this challenge. Microsoft addressed the problems that have been encountered as Exchange evolved, including new attachment types (streaming audio and video); an ever-increasing average message size; and the tendency of users to retain far more email than they need to, by increasing the maximum size of a database, store partitioning, and the inclusion of the streaming file as part of the Store. Instead of the original 16-GB limit, you can grow a database now to fill available disk space. Exchange is able to deal with the largest Storage Area Network available today, a situation very different from the original fixed disk configurations seen in 1996. Indeed, given the size of the engineering team that works on ESE compared with the SQL development team, the amount of development achieved to build Store features is remarkable.

Even as ESE evolved, Microsoft still faced the fact that it has been developing two database engines with all the attendant costs involved in design, engineering, bug testing, support, and release. Microsoft's long-term goal— expressed many times, including as a feature of the Cairo release—has been to move to a unified storage model that handles all types of data, including files, so it did not make sense to continue with two databases. One or the other had to give, and while ESE was successful, it did not have the strategic influence carried by SQL, so Microsoft established the long-term strategic goal in mid-2001 to move Exchange over and to use SQL as its future database engine.

Setting a strategic goal is but one small step along a path. Figuring out how to bring a hugely successful product with millions of users forward to use a new database engine is quite another. Exchange cannot use the current generation of the SQL database engine, because the current SQL platform did not take the needs of Exchange into account when Microsoft built this version. After an enormous internal debate, Microsoft therefore took the decision in 2001 that Exchange will use the Yukon next-generation SQL engine, and the work began to engineer in the necessary hooks and features to allow Yukon to support Exchange.

As with ESE, Yukon is a database engine and only cares about low-level data management that makes raw data useful to applications. It does not know how to use the data, since that is the responsibility of the application. Microsoft has an enormous amount of work to do to provide access to the data for server and client components, to process the data correctly, and to link up with other applications such as IIS. Remember, the second generation of Exchange that you see today is the result of ten years of engineering effort by a very large team; the creation of the third generation can leverage much of the experience and knowledge developed through work on previous versions, but there are new challenges to face. For example, interoperability between second-and third-generation servers, migration and upgrade of second-generation servers to third, moving mailboxes from one server type to another, interoperability and backward compatibility of APIs, and so on. Microsoft has to do all of this work in a very precise manner, because the sheer size of the Exchange installed base will magnify any mistake. The amount of engineering and design work for Microsoft to do is the basic reason why the first servers in the third generation of Exchange will not appear until 2005 at the earliest.

Because of overlapping development schedules for operating system, client, and email server, we may enter a period of coexistence while Microsoft develops the functionality levels of the Kodiak-based server to match the functionality available in Exchange 2003. As with the switchover from the first to second generation, when many organizations were happy to run Exchange 5.0, 5.5, and 2000 servers together for a number of years, we can expect to have Kodiak servers deployed alongside Exchange 2003 servers. In other cases, enterprises will wait to deploy Kodiak servers until after they are happy that Kodiak meets their functionality requirements. Because Kodiak is unlikely to have the same focus on MAPI clients, Microsoft may initially direct Kodiak-based Exchange servers at ISP/ASP customers while continuing to offer the second-generation servers to corporate customers that demand higher levels of functionality. Over time, Microsoft will phase out the second-generation servers and Kodiak-based servers will be the only version of Exchange that is available. This will not happen overnight or even in a couple of years. Based on experience with other corporate email systems, you can expect second-generation servers to be running for many years yet, with their lifetime only gated by Microsoft's willingness to provide support.

There are obvious advantages for Microsoft in pursuing the Yukon strategy. Already, it has moved the vast majority of engineering effort and expenditure into Yukon and away from ESE, so it is gaining the benefit of a single integrated development plan for database engines. Support and release engineering costs will not begin to fall until Microsoft releases Kodiak-based Exchange servers, and support costs for the second generation will continue until Microsoft terminates support. You can expect to incur additional support costs during the transition period, especially when the first set of customers migrates to Kodiak servers. Eventually, Microsoft will have a single database engine and perhaps a unified storage strategy if NTFS is integrated, all contributing to a reduced cost structure.

The potential benefits to customers are far more uncertain. It is true that SQL offers some features that ESE does not deliver today such as better transaction rollback and higher scalability, but perhaps not enough to justify waiting for a fully functional release of a Kodiak-based Exchange server to warrant the pain of migration. Every migration, even from one version of Exchange to another within the same generation, causes expense and pain for administrators and users alike. You cannot expect that the migration to Kodiak will be much different.

Microsoft realizes the issues and understands that it has to do an excellent job of communicating the benefits of the migration to customers while also continuing to add features and functionality to the second generation of Exchange. It will certainly be interesting to watch and see how Microsoft approaches this task as the time comes to release Kodiak and begin the migration to the third generation of Exchange.

1.3.1 The competition

Microsoft's own figures put the total number of Exchange client licenses sold at around 115 million. This does not mean that 115 million people use Exchange. The true number is less than 70 million, because Microsoft tends to oversell many seats to large corporations under the terms of enterprise licenses (where everyone in the company receives an Exchange client license). However, to put these figures into context, Exchange is the accepted market leader for messaging deployments, with about 60 percent of the market, and enjoys a healthy growth in its installed base year after year. Interestingly, 40 percent of Microsoft's Exchange business comes from the small to medium business market (under 500 seats), so while the large deployments receive the publicity, smaller installations deliver a lot of the base.

Exchange's competition has fluctuated dramatically over its lifetime. Exchange 4.0 competed against mainframe-based systems such as IBM PROFS and LAN-based systems such as Lotus cc:Mail. These systems had largely faded into the background by the time Exchange 5.5 shipped and the major competitor was Lotus Notes (Domino), a trend that continued into Exchange 2000. Today, Domino is still a major competitor for Exchange in the corporate arena (and especially so in Japan, where Domino is approximately six times as popular as Exchange), but it has been joined by systems based on "pure" Internet protocols. Exchange supports all of the Internet protocols, such as SMTP, MIME, IMAP, POP, and so on, but it seems to make a difference to some people when these protocols are associated with an SMTP MTA running on a UNIX or Linux server. The competitive landscape is completed by companies that attempt to leverage the strength of Outlook in the market and the number of desktops it occupies by offering an alternative to Exchange that allows companies to retain Outlook as the client. The best example of this is Oracle's Collaboration Suite, which is available for UNIX or Linux.

The Exchange development group has always worried about its competition and tried to learn from its strengths and weaknesses. Paranoia (a trait shared by many Microsoft development groups) has helped the group stay ahead of the competition, sometimes by adding features, sometimes by improving areas where it needed to, such as manageability and scalability. Microsoft has also focused on great interoperability with other messaging systems through SMTP and X.400 and the provision of migration tools to allow companies to move more easily to Exchange than to any other platform. For example, Microsoft has a Lotus Notes connector for Exchange that allows the smooth transfer of message and directory data between the two servers. It even has an Outlook 2002 connector for Lotus Notes, so that you can transfer mailbox data using a client rather than going anywhere near the server (pushing the work to migrate data down to users is a great idea, if you can get away with it). The net result is that Microsoft has always been able to stay ahead and grow market share for Exchange.

The jump from one technology generation to another always creates opportunity for the competition. For example, the move to Exchange 2000 forced customers to completely revamp their Windows infrastructure to deploy the Active Directory; implement a new security model; establish new administrative and management routines; and probably install new software packages for antivirus, cloning user accounts, directory synchronization, and so on. It was a lot of work and the bigger the company, the larger the price tag.

Two things flowed from the complexity Microsoft introduced in the changeover to Exchange 2000. First, companies were slow to deploy Exchange 2000 and were happy to stay with Exchange 5.5 (which works very well on Windows 2000); second, competitors had the chance to demonstrate that it was easier to move platform than upgrade all the components necessary to deploy Exchange 2000. In my experience, not many companies actually took the chance to move platform, but certainly a lot of angst was generated as competitors took every chance to inform customers of the work required to deploy Windows 2000 and then Exchange 2000. All of the fuss resulted in a situation where Exchange 5.5 became the largest competitor to Exchange 2000, as the majority of installations stubbornly stayed loyal to Exchange 5.5 and resisted the temptation to move to Exchange 2000 during the 2001–2003 time frame.

If you run Exchange 2000, then it is much easier to move to Exchange 2003, since you did all the hard work to establish Active Directory when you deployed Exchange 2000. The rule is, therefore, that upgrades within a generation of Exchange are easy; upgrades between Exchange generations take more work.

With this rule in mind, we can predict that similar angst will exist when Kodiak appears. Microsoft will face the undoubted technical challenge of making the migration as easy as possible for all concerned, but it will also face the challenge of holding off the competitors who will gather again to take advantage of any Microsoft weakness. It will be interesting to watch.



 < Day Day Up >