Section 5.6. Windows XP and Windows Server 2003 Volume Shadow Copy Service

   

---

5.6 Windows XP and Windows Server 2003 Volume Shadow Copy Service

With Windows XP and Windows Server 2003, Microsoft has implemented the volume shadow copy service to provide a framework that allows a coordinated and consistent point-in-time copy of disk volumes . For legal reasons, Microsoft has chosen to refer to the functionality as a volume shadow copy , which is really no different from the more popularly known term , snapshots . Volume shadow copy service has been implemented via a filter driver (called volsnap.sys) beneath the file system.

Microsoft makes a volume shadow copy software development kit available on a nondisclosure basis. This SDK appears to be aimed at three broad categories:

1. ISVs who might wish to develop volume shadow copy writers, including Microsoft Exchange, SQL Server, Oracle, SAP, Sybase, and others.

2. ISVs developing backup and storage management applications. These vendors would develop requestors for the volume shadow copy service.

3. Independent hardware and software vendors (IHVs and ISVs) developing hardware and software for backup, fault tolerance, and data integrity. Examples include VERITAS, EMC, and their competitors in this area. These vendors would develop snapshot providers.

When an application does not have code to support the snapshot service, the application data will still be backed up in a state that will be consistent to the same degree as if the system had failed to shut down gracefully. When an application has code to support snapshot services, the application is expected to provide those services in a restore operation as well. This is logical because the application is now expected to furnish some data (such as what files need to be backed up, and a backup and restore methodology) when a backup is requested , and it is expected to interpret and act on this same data when presented with the data upon a restore operation.

The major advantage is that backup or restore, as it exists in Windows operating systems, is a tricky business that may not work reliably 100 percent of the time. The new snapshot service will facilitate total reliability while also enabling more complicated scenarios that until now were not possible in Windows.

The volume shadow copy architecture shipping with Windows XP and Windows Server 2003 consists of four types of modules, as shown in Figure 5.8:

1. Writers

2. Requestors

3. Volume shadow copy service

4. Providers

Figure 5.8. Volume Shadow Copy Architecture

These modules are described in detail in Sections 5.6.1 through 5.6.4.

As far as the different modules that constitute the snapshot functionality are concerned , snapshot provider 2 has a kernel mode component, and snapshot provider 1 does not. The snapshot provider service is shown in gray in Figure 5.8 to signify that Microsoft supplies this functionality and that vendors do not have to bother writing such a service. Microsoft also provides some of the other components shown, but their functionality is limited either in terms of working with a specific application or in terms of the range of features provided; hence vendors are expected and encouraged to write these components using the snapshot SDK.

Writers and providers need to implement a Component Object Model (COM) out-of-proc provider as described in the volume shadow copy SDK. ^[1] A provider is typically implemented as a finite state machine. The state machine transitions from one state to another upon receipt of an event that is generated by the shadow copy service. A provider will receive an event (generated by the shadow copy service), but exactly which event depends on the current state of the provider and whether an error has occurred. To state this differently, a provider has a preferred event that it expects to arrive , allowing it to transition to the next normally expected state. However, an error may occur and the provider may receive a different (from the expected) event, and the provider should be able to handle that as well.

^[1] For the benefit of readers unfamiliar with COM, suffice it to say that COM is an extremely important architecture in the Windows world that allows software to be constructed as a series of objects that interact with each other. These COM objects can be written in a variety of programming languages, such as C, C++, and Visual Basic, to name just a few. COM allows two interacting objects to be locally situated on the same machine or to be running on two separate machines altogether. Either these two interacting objects can both be running in the same process (if they are both on the same local machine), or they may be running in two separate processes. In the case of separate processes, COM provides mechanisms for interprocess communication. To summarize, an out-of-proc provider is simply a provider, implemented as a COM object, that runs in a separate process context.

The Microsoft shadow copy framework in Windows Server 2003 and Windows XP provides some core functionality needed for storage management, including the following:

• Defining the point in time at which the snapshot is triggered

• Providing synchronization services among applications, databases, operating systems, and file systems to flush cached data, suspend writers, create the snapshot, and provide notification to applications, databases, and operating system elements that they can resume normal mode of operation

• Providing a single API that can be used by backup/restore applications

• Providing a platform for managing the snapshot

The Microsoft shadow copy framework can handle a set of volumes for which a snapshot must be done as a single set. The only behavior supported is "all or none"; that is, if any one operation fails, all of them fail. The Microsoft shadow copy framework also issues a request (to the snapshot provider) to delete the snapshot when the requestor application is done with the snapshot. If the snapshot needs to be available later, the snapshot provider or requestor application must provide the necessary functionality. ISVs can build applications that can create, catalog, and manage multiple snapshots using the shadow copy architecture; however, such applications are not part of the Windows Server 2003 product available from Microsoft.

5.6.1 Writers

Snapshot writers are simply the applications that write data (applications simply reading data are not much of a problem). Examples of snapshot writers include Microsoft Exchange, Microsoft SQL Server 2000, SAP, and Oracle. Microsoft and third-party ISVs are both expected to develop snapshot writer “aware applications. Snapshot writers should be developed with the snapshot SDK. In particular, snapshot writers will receive two events from the snapshot service indicating that they should freeze all write activity, and a different thaw event indicating that write activity can now happen (hopefully signifying successful creation of the snapshot). There are other events that the writers can get from the snapshot service, and details can be obtained from the snapshot SDK. Because the applications can define the consistency of data they expect to achieve, they should be able to perform the quiesce operation fairly quickly.

Further, it is worthwhile understanding one big advantage of the volume shadow copy service architecture over traditional hardware-based snapshot mechanisms used with Windows 2000 and previous Windows versions. With these classic versions of creating a snapshot, the hardware-based mechanism had no means of determining the state of the application and of the operating system software and cache in particular. This meant that an appreciable percentage of the snapshots were inconsistent. Further, the only way of determining the health of the snapshot created required running an application data consistency checker, which could run for hours.

By comparison, the volume shadow copy “based architecture not only attempts to flush caches and hold writes , but it can also determine in a matter of a minute or so, whether the created snapshot is consistent or not. Once the thaw event has been signaled, the volume shadow copy service simply interrogates the writers as to whether they successfully managed to hold their write operations between the freeze and thaw events. If any of the writers failed, the snapshot is deemed to have failed.

Snapshot writers are also expected to provide data required for backup and restore ”for example, what files need to be copied , what files need to be excluded, what collection of objects needs to be treated collectively as a single set. This data is stored by the snapshot service in a writer metadata document that is in XML format. Writers can also use this document to store data that is of interest to them. To restore data, the application simply hands the collection of data to the writer application to accomplish the restore operation.

Microsoft has announced that it will ship a SQL Server 2000 and Exchange writer, as well as writers for various other components of the Windows Server operating system. Microsoft is cooperating with ISVs to develop writers for other applications, including Active Directory.

5.6.2 Requestors

Requestors are typically backup applications that request the creation of a snapshot by making the appropriate API call to the volume shadow copy service provided by Microsoft. What's interesting here is the fact that the model remarkably simplifies some issues for the backup application writer. The backup application no longer has to solve the difficult issue of where the data to be backed up is, or what part of the data consists of application log files and what special treatment those files merit.

The appropriate writer (e.g., Microsoft SQL Server) is responsible for specifying the files and directories that should be included in the backup. Restore operations also become a lot simpler because the restore application again does not have to locate the data and figure out what files to pass to what API of the application (such as Exchange or SQL). The restore application simply hands the collection of data to the writer (application) and lets it accomplish the restore operation.

5.6.3 Volume Shadow Copy Service

The Windows NT volume shadow copy service , written by Microsoft, coordinates the activities for all the snapshot components. In particular, it provides the following:

• A single interface for backup applications or snapshot requestors to deal with, rather than multiple APIs from multiple applications.

• A single interface to produce, manipulate, and delete a crash-consistent volume snapshot or ghost volume.

• A single interface to allow different applications and snapshot providers to register and deregister as snapshot writers or snapshot providers.

• Synchronization and coordination among the various components to accomplish creation, deletion, or movement of snapshots, as well as backup and restore operations. The service prioritizes snapshot providers like this: Hardware providers have the highest priority, software providers are next, and the default Microsoft-provided snapshot provider has the lowest priority.

ISVs do not need to worry about writing a volume shadow copy service. Think of the snapshot service provided by Microsoft as being akin to a print spooler. You need only one print spooler. Some vendors (such as the provider vendors) simply need to write the equivalent of a printer driver. Other vendors need to write the equivalent of a printing application.

5.6.4 Providers

Snapshot providers are expected to be written by ISVs and IHVs to create, delete, and manipulate snapshots. As described earlier in this chapter, snapshot providers need to be written as a COM out-of-proc provider, via the snapshot SDK.

The provider may also have a kernel mode component ”for example, a filter driver that is located between the file system and the Logical Disk Manager (LDM). This kernel mode functionality may also be optionally implemented in hardware instead. Note that even a hardware-based provider will still leverage the rest of the functionality provided by the framework ”for example, definition of the point in time, I/O synchronization, and platform for building storage management applications, including backup, restore, and snapshot management applications.

One prime example of a snapshot provider is the volsnap.sys driver that ships with Windows XP and is also expected to ship with Windows Server 2003. This provider uses copy-on-write technology to create the necessary minimal data in a side store to be able to re-create the volume at a given point in time. The big assumption is that the required amount of free disk space is indeed available. This provider can handle NTFS, FAT32, and raw volumes on Windows Server 2003. However, this snapshot provider can provide only read-only snapshots and handle only one snapshot per volume. This limitation is in the provider itself and not the infrastructure. ISVs and IHVs can build richer functionality in their providers and writers if they so desire .

A complete description of all the events that are signaled to the snapshot provider via its COM provider is available in the snapshot SDK. A couple of important events are discussed here:

• PreCommitSnapshot . When the snapshot provider receives a PreCommitSnapshot event, it should start all I/O operations that take a long time ”for example, synchronizing of the mirror.

• CommitSnapshot . When the provider receives a CommitSnapshot event, it should recognize that the snapshot service will time out the operation within 10 seconds. Hence the functionality here should be extremely fast. Further, until the snapshot is completed, Windows NT will wait to send any write operations to the volumes for which snapshots are being made. This means the provider should not do any I/O on this volume, and if it does, it should not expect the I/O to complete until the snapshot is either complete or is aborted.

Snapshot providers must provide certain mandatory functionality and may provide functionality that exceeds the mandatory functionality. The mandatory functionality is as follows :

• Providers are responsible for locating the storage needed to create the snapshot. The framework provided by Microsoft does not provide any such functionality.

• The provider must mount the snapshot in a different namespace and not have the snapshot mounted as a separate volume. An inspection of Windows XP shows that the Microsoft snapshot provider mounts the snapshot at \Device\HarddiskSnapshotX.

5.6.5 Windows NT I/O Subsystem Modifications

Though the modifications to the Windows NT I/O subsystem are not explicitly part of the snapshot environment, it is worthwhile noting that a fair amount of work was needed in the file systems, I/O stack, and file system filter drivers to accomplish a consistent and reliable point-in-time snapshot. In particular, Microsoft added two IOCTLs that all file systems and file system filter drivers need to implement:

1. IOCTL_VOLSNAP_FLUSH_AND_HOLD_WRITES , which should be both chained on and acted upon. The chaining is to allow drivers farther down the stack chain to also act on the IOCTL. The action consists of flushing and holding all file system metadata. Once all data has been flushed, no further writes should be issued until the outstanding IRPs issued to flush all data and metadata have completed.

2. IOCTL_VOLSNAP_RELEASE_WRITES also needs to be chained on and acted upon. This IOCTL indicates either successful completion of the snapshot or abandonment of the snapshot operation.

Some relevant portions of the Windows NT operating system have also been modified to trigger these IOCTLs at the appropriate time. Although Microsoft has already modified the file system and file system filter drivers that it ships to provide this functionality, ISV-shipped filter drivers need to do the same.

Section 5.8 describes an industry standard called Network Data Management Protocol (NDMP). But before that topic is discussed, it is worthwhile noting the relationship between volume shadow copy architecture in Windows XP/Windows Server 2003 and NDMP. The shadow copy architecture is a means of creating a clone of the data that needs to be backed up; NDMP can be used to move the data from the clone to tape or other backup media.

---

Top