Improvements in Windows SharePoint Services 3.0

The client components of the 2007 Office system work with and use the server components in Windows SharePoint Services. Improvements in Windows SharePoint Services have resulted in new features and functionality at the client end. Let's start by looking at the data platform improvements.

Data Platform Improvements

One of the more common customer scenarios is a company that wants to store a large quantity of documents in a single document library and yet have the ability to quickly find and filter those documents. In addition, users want to host different document types with different schema inside the same document library and be able to sort and search against the differences in the document schemas. And users want to be able to retrieve documents after they have deleted them without having to go to IT personnel for a restore service. In Windows SharePoint Services, all this and more is now possible.

Storage Improvements

The largest area of improvement for this version of Windows SharePoint Services is in the data storage and query lookup features. The first feature that was added was large list indexing, which gives the user the ability to search across large lists (document libraries are included in this discussion) for specific information. For example, assume that you have a document library with one million documents in the library and you want to search for all the documents authored by Bill English. Windows SharePoint Services will allow you to do this quickly because there are now indexes stored on the data in fields for the list items. In this example, the author field has its own index that can be queried to enumerate quickly all the authors in the document library.

Multivalued Lookups

Another new feature is multivalued lookups. This feature allows your users to use a list that is a lookup to another list wherein the user can specify multiple values. For example, you have a central list that contains all the status types that you might want to assign to your documents across the enterprise. You want to find all the documents that have had a technical and copy editorial review but not a legal review. In this version of Windows SharePoint Services, you can execute this type of query and receive back all the documents that match your query for which you have permissions to see. The ability to have a one-to-many lookup from within one SharePoint list to another SharePoint list is a new, exciting feature that will provide endless possibilities for better content management and methods to find information.

Because you have the ability to do lookups from one list to another list, the product also introduced a link fix-up component that allows you to learn which lists and list items have dependencies in other lists across the enterprise.

Windows SharePoint Services will also include a slimmed-down version of the SharePoint Server 2007 search engine. Replacing the SQL Full-Text Indexing engine with the SharePoint Server 2007 search engine allows a similar fidelity of results at the site level that are enjoyed in the portal search results while also allowing users a more seamless upgrade to SharePoint Server 2007, should that be desirable at some point. The Windows SharePoint Services version of this engine does not allow for the creation of external content sources, but it allows for a more consistent user experience across sites and portals in SharePoint Server 2007.

There is a change log that keeps track of changes to a library, list, or item that can be polled either via the change log API or Web services so that you can track what changes have been executed to a particular item.

Windows SharePoint Services includes a new feature called AutoCopy. This feature allows you to take an individual list item and copy it to one or more other locations on a regular, scheduled basis. The autocopy action can occur when an item is updated, when a document is updated, or when a document is published. This feature is a core feature in the new publishing model in Windows SharePoint Services.

Windows SharePoint Services includes a new transformation feature. Assume you need the ability to automatically save documents in PDF format. Now all you need to do is install the .doc to .pdf transformer, and that transformer will show up on all the menus for documents in Windows SharePoint Services so that it can be leveraged across the enterprise. This feature is installed and managed centrally, but it's consumed (potentially) across the sites in the enterprise.

Windows SharePoint Services supports more fields in a Web Part. In the previous version of Windows SharePoint Services, there was a limit of 64 columns or fields that could be made a part of that list. In this version, you can create up to 2000 fields for an individual list.

Overcoming the inability of the previous version of Windows SharePoint Services to enumerate large lists in the browser has been an ongoing effort for the product team. For example, assume you have a list with 15,000 or more items. Enumerating such a large number of items in one page in the browser can often overload the HTML engine and make it literally impossible to enumerate the entire list in the browser. Work-arounds to solve this problem continue to focus on using the Data View Web Part, using Views to filter the list, or placing the list items into folders to break up the list's enumeration. You can also build indexes around the metadata in the list and search for items in the list based on their metadata assignments.

List and Library Alignment

The Windows SharePoint Services team has also worked hard to ensure that there is list and document library alignment across all sites in the enterprise. For example, in the previous version of Windows SharePoint Services, document libraries had folders but lists did not. Version history on list items were not available, whereas version histories were available on documents. You could also assign permissions to individual list items, but you were unable to do this in a document library. Moreover, the Discussions Web Part implemented a complex folder scheme in which each discussion was housed inside a folder-one where each post in the overall thread represented a child list item in that folder.

Content Types

Content types is another new feature that has been added into Windows SharePoint Services. Content types are a set of fields that you group together to form metadata, behaviors, and workflow for a particular content item.

If you're a knowledge worker and you want to manage the metadata for documents inside of a SharePoint list, and if those documents are stored in multiple lists, maintaining a consistent metadata list can be difficult because you need to visit each list and ensure that it has the same column structure and naming convention as the others. In this version of Windows SharePoint Services, content types solve that problem. Content types allow you to centrally manage the individual metadata that is attached to a document across the enterprise.

For example, let's say that you want all the organization's press releases to have a copy edit and a legal review before being published and you want to enforce this across all document libraries in your enterprise. Content types will enable you to do this by configuring a Press Release content type that requires a document to have both a copy edit and a legal review before it can be published.

Content types can also be hierarchical. That is, you can set global content types that are managed centrally, create local content types that inherit their information from the global content type, and still have the freedom to add, remove, or modify characteristics and behaviors of the local content type that were inherited from the global content type. For example, you could create a content type called Public Documents and then create another one at a document-library level called Press Releases that inherits from Public Documents but also modifies in various ways the inherited properties of the Public Documents content type. You could also add behaviors, workflows, and characteristics to the Press Releases content type. This architecture allows for great flexibility in how schemas are defined, while still allowing centralized output of an initial schema for the enterprise.

Content types also facilitate the ability to do large list indexing because you know that the lists that are using a particular content type are all storing the data in that field (or set of fields) in the same way across all the lists, which enables easier, faster indexing of the content.

Data Restore

Probably the most anticipated new feature in this version of Windows SharePoint Services is the Recycle Bin. This bin is actually a two-stage recycle bin, which has its first stage at the library level and second stage at the site-collection level.

At the document-library level, the content owner is able to both delete and restore the document without the involvement of either the site collection owner or the system administrator. At the site-collection level, the site collection owners are able to restore documents without the involvement of the system administrator.

The site collection administrator is able to set policies for all the document library recycle bins, such as a clean-up interval of two weeks at the document-library level and three months at the site-collection level. As long as the deleted document is in either of these two bins, the document can be restored without the help of the system administrator.

The robust auditing features of Windows SharePoint Services will track when and who deleted the document, even if the document no longer exists. The audit trail for actions on a document is not stored with the document. It is stored separately from the document for future reference, even though the document might no longer exist or might have been renamed or repurposed in some manner.

Windows SharePoint Services also provides three types of backups: one is for catastrophic events, one is for individual sites and the third is for individual documents in a document library. Microsoft Office SharePoint Designer 2007 will give end users the ability to back up entire site collections and take them away from the network for offline use. This functionality is similar to that provided by smigrate.exe, but now it's at a site-collection level as well as at an individual-site level.

Catastrophic restores are provided by enterprise backups. These backups are provided by SQL backups as well as by a Volume Shadow Copy Service (VSS) writer so that they can participate in enterprise backup routines. Site backup and restores are provided by using the command-line tool, stsadm.exe. Individual documents can be deleted and restored because of the two-stage recycle bin feature.

Security Improvements

Security is another big area of improvement in Windows SharePoint Services. The most requested feature improvement from the last version of Windows SharePoint Services to this version was document-level permissions in a document library. That feature is now available in this version. Now you not only have item-level security in the document library, but you also have a security-trimmed user interface. The interface a user sees will expose only links to actions that the user has permissions to perform. Actions that a user does not have permissions to perform will not be displayed to the user.

Windows SharePoint Services now ships with a robust auditing engine that can audit every action a user performs within SharePoint. There is also a full auditing object model against which you can develop your own custom applications.

And each SharePoint Server 2007 site will have an explicit login/logout function. This function will not apply to Readers in the site, meaning that Readers will not be required to log in to the site to view the contents of the site. However, those who navigate to the site with elevated privileges will need to log in to use their elevated privileges. Because the user interface is trimmed for security, this means that the same site that hosts the development of documents can also present "finished" documents to the enterprise, providing those with read-only access the ability to read a document from that site without requiring a login to that site.

Administrator Platform Improvements

There were some key scenarios that the product team wanted to address. The most important scenario was the extranet scenario. In this scenario, administrators seek to collaborate with vendors, partners, and customers and control the authentication methods to their collaboration sites without having to add external accounts to their Active Directory. There were reverse proxy and perimeter network (also known as DMZ or demilitarized zone) configurations that the previous version of Windows SharePoint Services didn't support very well. In addition, not all administrators wanted to use NTLM authentication in their extranet scenarios.

A second common scenario that the product team wanted to address and improve upon was the process of upgrading from the previous version of Windows SharePoint Services to the current version of Windows SharePoint Services. This section discusses both scenarios, but first let's take a look at the new key components for the Administration Platform.

Key Components of the Administration Platform

This section discusses several key components and outlines what the product team has done to improve the overall Administration Platform in Windows SharePoint Services. These items include the following:

• The product team has taken all the configuration links for Administrators and placed them into a common location in Central Administration.

• An extensible time service has been implemented in this version of Windows SharePoint Services.

  Earlier versions had the timer service, but now it is extensible. The service, which must be running on each physical server in the SharePoint Server 2007 farm, is used to automatically pull all the configuration information out of the configuration database and update all the servers in the farm automatically. You'll also be able to register your own objects with the timer service and expose those in the administration interface so that you can run your own timer jobs on top of the built-in jobs in Windows SharePoint Services.

• The Central Administration tool has been greatly enhanced.

  All configuration changes are made under the Operations tab, and all Web site and virtual server activities will be managed under the Application Management tab. You'll also find that you can group administration activities together and customize the administration interface to meet you needs. The tool is extensible because the Central Administration tool is now just another Windows SharePoint Services site that is based on the Team Site template. Because the underlying code base allows for site extensibilities, these extensibilities are available to the administrator in Central Administration.

• The navigation structure has been improved and flattened in this version of Windows SharePoint Services.

• There is a new administration task list that appears as soon as you open Central Administration to inform you of the tasks that SharePoint believes are the most important tasks to be accomplished in your farm.

  This task list is literally just that-a Windows SharePoint Services task list that can be extended by you with recurring tasks or one-time tasks that can be assigned to you or other members of the farm administration team.

• There is a new Farm Topology information screen that will list each server in the SharePoint Server 2007 farm, along with the server's status and the services that the server is running.

  Also, you can manage the servers from this interface. One especially noteworthy feature is that if the status shows "error," you'll can click on the "error" link to bring up the logs from the remote server so that you can see what events are being recorded and learn about the problems associated with that server.

Extranet Scenario Improvements

The product team added support for more extranet configurations with the ASP.NET-pluggable authentication model. This means that form-based authentication and Webbased Single Sign-On (SSO) are now supported methods of authentication in Windows SharePoint Services. These two supported methods are not shipped "in the box" in Windows SharePoint Services, but they are supported and can be purchased from third-party sources or custom coded for your own environment.

Upgrade Improvements

The product team had four main goals related to upgrade paths and methods that they wanted to accomplish in this version of Windows SharePoint Services. First, they wanted to ensure that you could upgrade over time, meaning that you weren't forced into upgrading your entire SharePoint deployment all at once. The goal was to allow you to upgrade your current implementation in "weekend-size chunks" so that a disparate, discrete portion of your overall deployment can be upgraded and finished without having to upgrade other segments of your current deployment.

Second, the product team understood that most people can't take down their entire deployment all at once to perform an upgrade to the next version of Windows SharePoint Services. Having all your users sitting around, unproductive, while you're performing an upgrade is not the scenario that Microsoft wants you to experience. Therefore, the product team has assured that you'll be able to leave all parts of your deployment up and running while you take down a smaller part of your deployment for the upgrade action.

Third, the product team wanted you to be able to write your own upgrade code and plug it into the upgrade path provided by Microsoft. To accomplish this goal, they made the upgrade path itself extensible. When you adopt this approach, upgrade.exe will simply run your code along with its own code to ensure that all parts of your deployment are upgraded to the current version of Windows SharePoint Services.

Fourth, the product team wanted a consistent set of experiences between the previous version and the current version of Windows SharePoint Services, even during the upgrade efforts. This has been achieved by the product team.

Upgrade Approaches

There might be some reading this who will ask the following question: "Why didn't Microsoft write an upgrade path for everyone?" The answer is that this is, quite literally, impossible. At the time of this writing, there are over 20,000,000 team sites in production spread out over nearly every country in the world. There is no possible way that the product team could know the custom coding that everyone has performed on their sites. It is literally impossible to write an upgrade path whose starting point is not always known and whose end point might or might not be what the customer wants. It's really impossible to write an upgrade path that will take into consideration every possible permutation of customization that might have taken place and then ensure that all that custom code is preserved when the upgrade effort is completed.

Therefore, the product team opted to allow the upgrade path to be extensible. By doing this, administrators of environments who performed many customizations across many sites can write their own upgrade code to move their sites off of version 2.0 to version 3.0.

There are three main upgrade approaches. The first is an in-place upgrade, in which the administrator selects Upgrade when running the Windows SharePoint Services setup program and the program performs the upgrade without further intervention from the administrator. When the upgrade is completed, the administrator is informed and the users have been moved to the current Windows SharePoint Services platform. This method is ideal for single-server or small-farm implementations.

The second approach is a Gradual upgrade, which is intended to be used by medium or large farms. Characteristics of this upgrade approach include running version 3.0 code alongside your version 2.0 code on the same physical server. In addition, you'll get the version 3.0 farm up and running. You then select which site collections you want to migrate and perform the migration of those sites at a time that fits your schedule and pace. If you can do 10 site collections each night, that's what you'll do. If you want to do 1,000 site collections each weekend, you can do that too. Note that if you upgrade more than 100 site collections at a time, you'll need to use the command-line feature because the user interface will display only 100 site collection at a time.

A very nice characteristic is that the URL the user types in the browser doesn't change from version 2.0 to version 3.0. So, if they were typing http://trainsbydave in version 2.0, they'll still type http://trainsbydave in version 3.0 and receive the version 3.0 site after it has been upgraded.

Finally, let's assume you upgrade a site to version 3.0 and then realize you want the site moved back to version 2.0. You'll be happy to learn that you can revert sites back to version 2.0 on a per-site basis.

Microsoft Office FrontPage customizations in version 2.0 will be preserved when the site is upgraded to a version 3.0 site. You'll also have the option to convert the site's template from a version 2.0 template to a version 3.0 template, but if you choose to do this, you'll lose your version 2.0 FrontPage customizations.

The third upgrade approach is to build a new farm, upgrade the content databases, and manually reconfigure the entire farm and recode all the customizations you need in the new farm. This book contains three chapters that discuss upgrades from the present version of SharePoint Products and Technologies to SharePoint Server 2007.