Section 8.4. Planning, Implementing, and Maintaining Server Availability

8.4. Planning, Implementing, and Maintaining Server Availability

Every server deployment should be planned, implemented, and maintained with availability in mind. Availability refers to the server's ability to withstand hardware, application, or service outage. No server, application or service should be deployed in the enterprise without planning the deployment to meet a specific availability goal.

Availability goals will differ depending on how critical the server, application, or service is to the organization:

Most noncritical systems, applications, or services have a moderate availability goal of 99 percent. This means the system, application, or service is expected to be available to users 99 percent of the time that it is needed. In a 24/7 environment with 365-days-a-year operations, this means the server can have about 88 hours of downtime a year, or about 100 minutes of downtime a week.
Most critical systems, applications, or services have a high-availability goal of 99.9 percent. This means the system, application, or service is expected to be available to users 99.9 percent of the time that it is needed. In a 24/7 environment with 365-days-a-year operations, this means the server can have about 9 hours of downtime a year, or about 10 minutes of downtime a week.

As you can see, there's a huge difference in uptime expectations between 99 percent availability and 99.9 percent high availability. There's also a huge difference in planning, implementation, and maintenance. To meet a 99 percent availability goal, you'll need to:

Identify the initial hardware requirements for memory, processors, disks, and networking.
Plan a monitoring strategy that identifies potential and actual system bottlenecks.
Plan a backup and recovery strategy that meets the availability goal in terms of recovery time.

To meet a 99.9 percent high-availability goal, you'll need to:

Identify the initial hardware requirements for memory, processors, disks, and networking.
Identify high-availability software solutions, such as network load balancing (NLB) or clustering, that can help you meet availability goals.
Plan a monitoring strategy that identifies potential and actual system bottlenecks.
Plan a backup and recovery strategy that meets the availability goal in terms of recovery time, and that also works with your chosen availability software solution.

8.4.1. Planning Network Traffic Monitoring and Identifying System Bottlenecks

As discussed in the exam overview, Exam 70-293 builds on the areas of study from the previous exams. With regard to server availability planning and server maintenance, you are expected to know how to monitor network traffic and identify system bottlenecks.

In "Monitoring Network Traffic" in Chapter 5, I discussed the key tools used for monitoring network traffic, including Task Manager's Networking tab, the Performance console's Network Interface performance object, and Network Monitor. As part of availability planning, you should determine current network traffic levels and how new servers and services added to the network could possibly impact network traffic levels. As part of long-term planning and maintaining server availability, you should routinely and periodically monitor network traffic.

In "Monitoring and Optimizing a Server Environment for Performance" in Chapter 2, I discussed how to use monitoring tools to identify system bottlenecks. Memory, processor, disk, and network bottlenecks can adversely affect system performance. As discussed in Chapter 2, the primary tool for detecting system bottlenecks is System Monitor. Use System Monitor to view real-time performance data or to log performance data for later review.

8.4.2. Planning Services for High Availability

Windows Server 2003 includes built-in functionality to support three high-availability software solutions:

Network Load Balancing (NLB): Provides high availability for IP-based applications and services, such as those running on a web or Internet application server.
Component Load Balancing (CLB): Provides high availability for COM+ application components.
Server clusters: Provide high availability for business-critical applications and services.

Because component load balancing is primarily implemented by programmers, Exam 70-293 covers only network load balancing and server clusters.

8.4.2.1. Planning a high-availability solution that uses network load balancing

All editions of Windows Server 2003 support network load balancing, which is used to distribute incoming IP traffic across a cluster of servers that share a single virtual IP address. You can balance the load across as few as 2 systems and up to 32 systems.

Any IP-based application that uses TCP, UDP, or GRE can be used with network load balancing. This means network load balancing is ideally suited to use with web servers, Internet application servers, and media servers. Applications that are load balanced include:

FTP over TCP/IP
HTTP over TCP/IP
HTTPS over TCP/IP
IMAP4 over TCP/IP
POP3 over TCP/IP
SMTP over TCP/IP

Load balancing ensures that there is no single point of failure by directing client requests to a virtual IP address. If one of the load balanced servers fails, the remaining servers handle the workload of the failed server. When the failed server comes back online, the server can rejoin the group automatically and start handling requests.

Network load balancing monitors the status of each load-balanced server, referred to as a node, using heartbeats. If a node fails to send a heartbeat message to other nodes within a specified time, the node is considered to be unavailable, and the remaining servers take over the failed servers workload.

Clients using the failed server automatically retry the failed connection, in most cases, within several seconds, and are then redirected to another server. It is important, however, to point out that there is no shared data between the nodes, so any work that is stored only on the failed server is lost. To avoid this, clients can store data locally prior to submitting it in final form for processing and storage.

8.4.2.2. Implementing network load balancing

Use the Network Load Balancing Manager, shown in Figure 8-21, to implement and manage network load balancing. To start Network Load Balancing Manager, click Network Load Balancing Manager on the Administrative Tools menu or type nlbmgr at a command prompt.

Figure 8-21. Use Network Load Balancing Manager to implement and manage network load balancing.

To install and configure load balancing, use the following technique:

Open Network Load Balancing Manager.
Right-click Network Load Balancing Clusters, and then click New Cluster.
On the Cluster Parameters page, shown in Figure 8-22, type the virtual IP address and subnet mask for the cluster. The same virtual IP address is used for all NLB nodes. This IP address is fixed and cannot be a dynamic DHCP address.

Figure 8-22. Configure the NLB cluster parameters.
In the Full Internet Name field, type the fully qualified domain name for the NLB cluster. Click Next.
If the cluster will have additional virtual IP addresses, click Add, enter the virtual IP address and subnet mask information, then click OK. Repeat this step for each additional virtual IP address that will be used, and then click Next.
Using the Port Rules page, shown in Figure 8-23, specify how network traffic on a port is filtered. By default, all TCP and UPD traffic is load balanced across all members of the cluster, based on the load weight of each cluster member. Click Next.

Figure 8-23. Set the port rules for NLB.
Enter the name or IP address of the first host that will be a member of the cluster. Click Connect to connect to the server and display a list of available network interfaces. Select the network interface to use for network load balancing, and then click Next.
On the Host Parameters page, set the host priority, which indicates the order in which traffic is routed among members of the cluster, and the dedicated IP address, which is used for private node-to-node traffic (as opposed to the public traffic for the cluster). Then set the default state of the host after system startup to Started.
Click Finish.

You can then add hosts into the cluster as appropriate. If you need to change the cluster parameters later, right-click the cluster in the left pane and select Cluster Properties. You are then able to change the cluster IP configuration, operation mode, and port rules.

After you create a cluster and add the initial host, you can add other hosts to the cluster at any time, up to a maximum of 32. Additional hosts use the cluster port rules from the initial host. To add a host to a cluster, follow these steps:

Open Network Load Balancing Manager. If the cluster you want to work with isn't shown, connect to it by right-clicking the Network Load Balancing Clusters node and selecting Connect To Existing. Then enter the domain name or IP address of any host in the cluster and click Connect.
Right-click the cluster to which you want to add a node and select Add Host To Cluster.
Enter the domain name or IP address of the host to add. Click Connect.
Select the network interface for network load balancing. The IP address configured on this network adapter will be the dedicated IP address used for the public traffic of the cluster. Click Next.
On the Host Parameters page, set the unique priority for this host in the cluster and the dedicated IP address for private, node-to-node traffic. Then set the default state of the host after system startup to Started.
Click Finish.

8.4.2.3. Planning a high-availability solution that uses clustering services

Windows Server 2003 Enterprise Edition and Windows Server 2003 Datacenter Edition support clustering for up to eight nodes using Microsoft Cluster service. As Table 8-16 shows, this is different from the clustering support in Windows 2000. All nodes in a server cluster must be running the same version of Windows. You cannot mix server versions.

Table 8-16. Support for clustering in Windows 2000 and Windows Server 2003
Operating system version	Cluster service nodes supported
Windows 2000 Advanced Server	2-nodes
Windows 2000 Datacenter Server	4-nodes
Windows Server 2003 Enterprise	8-nodes
Windows Server 2003 Datacenter	8-nodes

Server clustering with Microsoft Cluster service has many similarities and differences from clustering with Network Load Balancing. With Windows Server 2003, three types of server clusters can be used:

Single-node server clusters: A cluster configuration with a single node that can be configured to use internal storage or an external cluster storage device. Single-node clustering allows automatic restart of applications and dependent services using Cluster service.
Single quorum device server clusters: A cluster configuration with two or more nodes that are all connected to the same cluster storage devices. All members in the cluster use the same cluster quorum device as well. The quorum device stores cluster configuration and recovery data.
Majority node server clusters: A cluster configuration with two or more nodes that don't have to be connected to shared storage devices. Each node can have its own cluster storage device and its own local quorum device, which means the cluster configuration and recovery data is stored on multiple disks across the cluster.

Most organizations use either single-node server clusters or single quorum device server clusters. Majority node server clusters typically are implemented for large-scale cluster deployments where cluster members of geographically separated. For example, you might use majority node server clusters to allow a backup site to handle failover from a primary site.

With server clustering, nodes can be either active or passive:

Active nodes: Actively handle requests from clients
Passive nodes: Wait on standby for another node to fail

With server clustering, scalability is as important as availability. Prior to deploying a cluster, you'll need to determine how many nodes you'll need, what hardware requirements must be met, and whether those nodes will be configured as active or passive nodes. Active and passive nodes are used in different ways:

When all nodes in a cluster are active, you need to configure nodes with sufficient resources to handle the additional processing load of a failed server. For example, if you determine that each node in the cluster must have four processors and 4 GB of RAM to handle the expected workload, you need to double these resources to ensure that any single node could handle the additional processing load should one of the nodes fail. In this configuration, up to two nodes could fail and the workload would be supported.
When a cluster includes passive nodes, you need to configure the passive node to handle the additional processing load of a failed server. For example, if you determine that each active node in the cluster must have eight processors and 32 GB of RAM to handle the expected workload, you need to configure passive nodes with this same configuration so that they can handle the processing load should one of the active nodes fail. In this configuration, up to two nodes could fail and the workload would be supported.

Microsoft Cluster Service requires that each node in a single quorum cluster be connected to the same cluster storage devices. Connecting the cluster to the same storage devices allows nodes in the cluster to share the same data. In the event of failure, the data is available to the server that assumes the failed server's workloadand the availability of data after failure is an important distinction between NLB and cluster service.

Prior to deploying clustering, you should prepare all the hard drives that the cluster will use and format all partitions appropriately using NTFS. For single quorum clusters, all nodes use the same quorum resource. With 32-bit editions of Windows Server 2003, you can use SCSI or fibre channel to share storage devices. However, fibre channel is preferred. Fibre channel is required with 64-bit editions of Windows Server 2003.

Once clustering is implemented, any cluster-aware application or service can be easily clustered using resource groups. Resource groups are units of failover that are configured on a single node. When any of the resources in the resource group fail, failover is initiated for the entire resource group according to the failover policy. Cluster-aware applications and services include:

Distributed File System (DFS)
DHCP
Exchange Server
Folder shares
IIS
Printer shares
SMTP
SQL Server
WINS

Server clusters can only use TCP/IP. They cannot use AppleTalk, IPX, NWLINK, or NetBEUI. However, clients should have NetBIOS enabled so they can browse to a virtual server by name.

Cluster service tracks the status of each node in the cluster using state flags. The five possible state flags are:

Down: Indicates a node is not active in the cluster
Joining: Indicates the node is in the process of becoming an active participant in the cluster
Paused: Indicates the node is active in the cluster but cannot or has not taken ownership of any resource groups
Up: Indicates the node is active in the cluster
Unknown: Indicates the node's state cannot be determined

Server clusters send heartbeat messages on dedicated network adapters, referred to as the cluster adapters . The heartbeat is used to track the availability of each node in the cluster. If a node fails to send a heartbeat message within a specified time, Cluster Service assumes the node has failed and initiates failover of resources. When failover occurs, another server takes over the workload, according to the failover policy. When the failed resource is back online, the original server is able to regain control of the resource.

8.4.2.4. Implementing a cluster server

You use the Cluster Administrator, shown in Figure 8-24, to implement and manage server clustering. To start Cluster Administrator, click Cluster Administrator on the Administrative Tools menu or type cluadmin at a command prompt.

Figure 8-24. Use Cluster Administrator to create and manage server clusters.

To install and configure server clustering, use the following technique:

Open Cluster Administrator.
In the Open Connection To Cluster dialog box, select Create New Cluster, and then click OK. Or click File New Cluster.
On the Select Computer page, enter the name or IP address of the first computer in the cluster. Click Next.
The wizard analyzes the configuration and details any problems found. Any fatal errors must be corrected. Click Next.
Enter the IP address that will be used by cluster management tools to connect to the cluster. Click Next.
Specify the logon information for the account under which the cluster service will run. Click Next.
Click the Quorum button to choose the quorum type and configure the quorum device. Click Next to start configuring the cluster.

After you create a cluster and add the initial node, you can add other nodes to the cluster at any time, up to a maximum of eight. Additional nodes use the quorum and resource configuration from the initial node. To add a node to a cluster, follow these steps:

Open Cluster Administrator.
In the Open Connection To Cluster dialog box, select Add Nodes To Cluster, and then click OK.
Enter the domain name or IP address of the host to add to the cluster. Click Next.
The wizard analyzes the configuration and details any problems found. Any fatal errors must be corrected. Click Next.
Follow the prompts, which are very similar to those for configuring the new node.

8.4.3. Planning a Backup and Recovery Strategy

A key part of availability planning is ensuring that you create and then implement a comprehensive backup and recovery strategy. For Exam 70-293, you'll need to be able to identify the appropriate backup type to use in a given situation, to plan a backup strategy that includes volume shadow copy, and to plan system recovery that uses Automated System Recovery (ASR).

As part of your preparation, you should review the section "Managing and Implementing Disaster Recovery" in Chapter 2. Exam 70-293 expects you to know all the details in this section for using Backup, working with Volume Shadow Copy, and using Automated System Recovery (ASR). Additionally, the exam expects you to have a more detailed understanding of the appropriate backup types to use in a given situation.

Windows Server 2003 includes the Backup utility for backing up and recovering servers. Like most third-party backup solutions, Backup supports five backup types:

Normal backups, also called full backups: Use normal backups to back up all selected files or folders. Normal backups do not use the archive flag to determine whether to back up files or folders. Always back up everything you've selected for backup. During a normal backup, the Backup utility clears the archive flag on all selected files and folders to indicate that the files and folders have been backed up. You should regularly perform or regularly schedule normal backups to run and supplement normal backups with other backups as necessary.
Copy backups: Use copy backups to back up all selected files or folders. Like normal backups, copy backups do not use the archive flag to determine whether to back up files or folders, and always back up everything you've selected for backup. During a copy backup, the Backup utility does not clear the archive flag on all selected files and folders to indicate that the files and folders have been backed up. Copy backups should be used when you want to make a current backup of data but do not want to disrupt the backup rotation.
Daily backups: You use daily backups to backup all selected files or folders that have changes during a particular day. Daily backups do not use or reset the archive flag. If you use daily backups with normal backups, you must create a daily backup every day thereafter until the next normal backup to be able to fully recover the system.
Differential backups: Use differential backups to back up all selected files or folders with an archive flag. During a differential backup, the Backup utility does not clear the archive flag. Because of this, each differential backup after a normal backup contains the full set of changes since the normal backup was created.
Incremental backups: Use incremental backups to back up all selected files or folders with an archive flag. During an incremental backup, the Backup utility clears the archive flag. Because of this, each incremental backup contains only the changes since the last incremental or full backup.

For all servers and all critical workstations, you should create normal backups at least once a week and supplement these with daily incremental backups or daily differential backups. Because incremental backups contain only changes since the last incremental or full backup, incremental backups are smaller than differential backups and can be created more quickly. However, since each differential backup contains all the changes since the last normal backup, differential backups allow you to recover a system more quickly. Therefore, the decisive factors in whether to use incremental or differential backups in addition to normal backups are the required backup time and the required recovery time.

For planning purposes, keep the following in mind:

For recovery of a system that uses normal and incremental backups, you must apply the last normal backup and then each incremental backup in order up to the day of failure. If a system fails on a Thursday after daily incremental backup, and the last full backup was the previous Sunday, then recover the system by applying the last full backup, the Monday incremental backup, the Tuesday incremental backup, the Wednesday incremental backup, and the Thursday incremental backup.
For recovery of a system that uses normal and differential backups, you must apply the last normal backup and then apply the last differential backup. If a system fails on a Thursday after daily differential backup, and the last full backup was the previous Sunday, then recover the system by applying the last full backup and the Thursday differential backup.

Following this, in cases where you are backing up large data sets, you may need to use daily incremental backups to ensure that all the changes can be backed up within the allotted time. In cases where speed of recovery is critically important, you may need to use daily differential backups to ensure systems can be recovered more quickly.