With the Event Viewer, you can monitor the events that take place on your systems and detect activities that may require your attention. Analyzing these events can help you resolve issues related to security, resource allocation, and system problems. When saved, log files also provide a historical view of these events. It is a good idea to archive the log files from each of your servers for future reference.

Event Log Categories

Three categories of events are generated in the System and Application logs and two additional event categories in the Security logs. It is important to understand what each of the categories pertain to and what they say about the server or application. The System and Application logs include the following event categories:

• Information events Provide status information and details about 'good' things such as the successful operation of a service, driver, or application. Typically, there are far more of these than any other type. Information events are represented by a blue 'i'.

• Warning events Provide information on potential problems, such as low disk space, services that have not been configured, and lost or timed-out connections. These may not be significant but should be examined carefully and, if possible, corrected immediately. Warning events are represented by a yellow triangle with an exclamation point (!) inside.

• Error events These are indicative of more serious conditions and logged when there is a failure such as a service failing to start, a duplicate IP address, a malfunctioning hardware device, or system crash. Error events are represented by red circles with white Xs through them, as shown in Figure 13-1.

Figure 13-1: The Windows 2000 Event Viewer

The Security Log contains events that are generated when auditing has been implemented. Auditing can be enabled on events such as logons, resources, or file access. The two types of Security Log events are

• Success Audit events Provide information on successful logins or access.

• Failed events Provide information on failed login attempts, failed access to resources, or other security threats.

Event Information

As you scroll through the log, each event listed provides several pieces of information:

• The type of event, whether information, warning, error, success or failure.

• The date and time the event was logged.

• The source of the event. The event source is the service, driver or application that reported the event to the log.

• The category of event, if applicable , is a number used internally by the component that reported the event.

• The event ID number that describes the event to Windows 2000.

To learn more about an event, double-click the entry to open its property sheet, as shown in Figure 13-2. The property sheet reiterates the preceding information and may also include a username, computer name, and a description of the event. The Source, Event ID, and description will often be referenced in the Microsoft and/or Citrix Knowledge Base and provide explanations and solutions to the problem.

Figure 13-2: The Event Properties dialog box

Managing the Event Viewer

Because the event logs record information on a daily basis, they could become enormous if left unmanaged. Of course, Microsoft took care to set defaults to prevent this from happening, but you can adjust those defaults to meet your own needs. To change settings on a log, right-click the log and select Properties. As shown in Figure 13-3, there are two tabs: the General tab and the Filter tab. The General tab provides information about the log such as its location, size, creation date, last modified date, and last accessed date. It also allows you to specify the maximum size of the log, what to do once the log has reached its maximum size .

It even lets you clear the log manually so you can start fresh. If left at the default setting, the log maxes out at 512KB. If you choose to change the default size, however, you have the option to overwrite the events as needed, to overwrite those events older than a specified number of days, or to forego overwriting and clear the log manually. Unless you are absolutely certain you will never need those logs again, take a moment to save them before clearing. Citrix recommends you set the maximum log size to 1024KB and to overwrite events as needed. The reasoning behind this is to keep the event logs from filling up and generating errors of their own. The only problem with this configuration is that the older events will be wiped out and unavailable for reference in the future. The General tab also allows you to configure the event logs for a low-speed connection. This option can be used when monitoring a computer across a slow WAN link or dial-up connection.

Figure 13-3: The General tab under System Log Properties

Filtering and Searching for Events

The Filter tab (Figure 13-4) allows you to select the type of events displayed in the Event Viewer. By default, the Event Viewer displays all events in a log sorted by most recent date. The Filter tab allows you to select events based on the following criteria shown in Table 13-2.

Figure 13-4: The Filter tab under System Log Properties

Windows 2000 provides a Find utility for searching through event logs. Find can be used to locate related events to determine how frequently the event has occurred. To use Find, open the Event Viewer, click View, and select Find. Under Types, select the type of events you want to find. On both operating systems, events can be sorted by clicking the column heading you wish to sort by. For example, if you want to see all events related to a particular service, click on the Source column heading and scroll down until you find the service you are tracking. All other events relating to this service will be grouped together and easy to locate.

The Event Viewer is typically the tool used in troubleshooting Windows systems. Learning to use it is relatively easy, the hard part is figuring out what the information means and how to resolve the issues it reports . We'll get to that in the sections that follow, but for now, try Exercise 13-1 to familiarize yourself with the Event Viewer and its configuration options.

Exercise 13-1: Changing Event Log Settings

1. Choose Start Programs Administrative Tools.

2. Double-click Event Viewer.

3. Right-click the System Log and select Properties. This opens up the System Log's Property sheet. At the top, you should see two tabs, the General tab and the Filter tab.

4. Select the General tab and move down to Log Size. Change Maximum Log Size to 1024 KB.

5. Below the maximum log size, select Overwrite Events As Needed.

6. Click once on the Filter tab.

7. By default, all event types are selected. Deselect all but the Warning and Error event types.

8. Click OK to finish.

9. Now, open the System Log. There should be no Information events displayed. If you're very lucky, you will not have any error or warning events, but if you do, double-click them to open their Information Properties sheet. Take note of the information provided, especially the Event ID, Source, and Description.

Now that you are more familiar with the Event Viewer, it's time for a pop quiz. Test your knowledge by trying to answer the questions that follow. Try not to look at the answers first!

System Information Tool

In Windows 2000, most of the system management tools have been centralized within the Microsoft Management Console or MMC. To open the console, you can either right-click My Computer and select Manage, or go to Start Programs Administrative Tools Computer Management. From here, you can view and monitor your local system or a remote system in your domain. To monitor a remote system, right-click Computer Management (Local) and select Connect To Another Computer.

System Information Tool contains vital information relating to your server in a read-only format. It cannot be used to test or change server hardware or software, only to display the servers configuration and components. Information contained in the folders can be saved as a text or .inf file and kept as historical data or e-mailed to a vendor's technical support. To save folder information, right-click the folder and select Save As.

If you do need to test or change hardware devices or drivers, use the Device Manager. From within Device Manager, devices can be tested , disabled, uninstalled , or updated. For problem services, use Services, located under Applications and Services. Both the Device Manager and Applications and Services are found under the Computer Management Console.

There are three main sections within Computer Management: System Tools, Storage and Services and Applications. The System Information Tool is appropriately located under System Tools. You can also get to System Information by typing winmsd from the command prompt. Within the System Information Tool there are five subfolders that provide a wealth of information about your system. The five  subfolders are: System Summary, Hardware Resources, Components, Software Environment, and Internet Explorer 5.

The System Summary displays a list of the system's basic configuration. This is where you'll find, among other things, the version and build of your operating system, the BIOS version installed memory, available memory, virtual memory, available virtual memory and page file size. Don't be surprised if you are asked to look here if you ever call a vendor for support.

The Hardware Resources folder has several subfolders devoted to information about the system's hardware (Figure 13-5).

Figure 13-5: Hardware Resources

Don't overlook this one. It is one of the most important pieces of the System Information Tool because it provides information on hardware conflicts and IRQ sharing. If you've ever installed new hardware only to find that either it will not work, or another device has stopped working, you will appreciate the importance of this resource. Table 13-3 explains some of the information found in these subfolders.

The Components folder is another that you should become familiar with. It displays a list of components, all of which may not be installed on your system, and shows the resources they are using. If a component is actually installed, you can right-click it and view its property sheet. The property sheets provide all the details relevant to the device, including the driver versions, I/O, and IRQs it is using. The Component folder has eleven subfolders that represent the different classes of components that may be installed. One of the most interesting is the Problem Devices folder. Inside, devices that are not performing as expected or devices that have been removed incorrectly will be listed. The other ten folders are as follows :

• Multimedia Audio and video codecs, CD-ROMs, sound devices

• Display Adapter information, such as name, type, resolution, and pixels

• Infrared Properties of any infrared devices installed

• Input Keyboard and pointing device information

• Modem Information on installed modems

• Network Adapter, protocols, and WinSock information

• Ports Serial and parallel port information

• Storage Drivers associated with storage devices

• Printing Any print drivers installed

• USB Information on USB devices

The Software Environment folder contains ten subfolders that detail the software running on your system, as well as the files and services in use and who's using them. Table 13-4 lists the subfolders and describes the information contained in each.

The Internet Explorer 5 folder is fairly self-explanatory and contains those settings unique to Internet Explorer (IE). This is a great place to view paths, file versions, proxy settings, encryption, and other IE configuration information. The subfolders include

• Summary Version, build, path, cipher strength, content advisor and Internet Explorer Administration Kit (IEAK)

• File Versions Information on all IE files, version, date, vendor

• Connectivity Dial-up, Local Area Network (LAN) or Proxy connection information

• Cache Summary, list of objects in cache, page refresh, temp folder

• Content Summary, personal and other peoples certificates, publisher

• Security Local intranet, trusted sites, Internet security and restricted site

Now that you've gotten more familiar with the type of information found in the System Information Tool, let's try a hands-on exercise.

Exercise 13-2: Saving System Information to a File

In this exercise, we will assume that you are the administrator of XYZ Company, and you are having some difficulties with one of your servers after installing a new application. You've tried all the tricks you know, but are stumped, so you call the vendor for assistance. The technical support person you speak with asks you to e-mail her your system information summary. Here's what to do:

1. Log on to a Windows 2000 Server as Administrator.

2. Right-click My Computer and select Manage. This will open the Computer Management Console.

3. Expand System Information by clicking on the '+' on the left.

4. Right-click System Summary. This will bring up a menu that allows you to save the System Summary as a text file or an information (.inf) file, search the information using Find, Print the information, refresh or get Help.

5. Select Save As Text File. Save the file as testlog.txt.

6. Locate the file you just saved and double-click it to open it. The information saved in the file should look similar to that in Figure 13-6 that follows. It contains information about your server's hardware and environment that may help the vendor's technical support people resolve your problem.

   
   Figure 13-6: System Information Report

Handling Events

So, you've been monitoring your Event Logs and have discovered warning and/or error events. What should you do now? What does all this stuff mean?! The dialog found in the logs is not always intuitive. Sometimes it's down right confusing. But in most cases, a cure can be found. Both Microsoft and Citrix have a Knowledge Base that contains articles relating to practically every problem you might encounter. To find the article that relates to the event, you will need to have the information found on the event's property sheet handy. Three key pieces of information are the Event ID, the Source and the Description, if there is one. Take these three pieces of information with you to Microsoft and/or Citrix's support Web site and begin your search. For Microsoft, go to http://search.support.microsoft.com/kb/c.asp, and for Citrix, go to http://www.citrix.com/support/.

Deciding which Web site to visit first is usually determined by the Source and Description information. If the information is clearly related to MetaFrame, it might save time going to Citrix first. Generally, both Knowledge Bases will provide similar information or at least a pointer to the right article on the other's Web site. Other support resources offered by Microsoft and Citrix are

• Microsoft Online TechNet A site devoted to IT professionals that provides troubleshooting tools, information about known issues, hotfix , and service pack downloads. TechNet can be found at http://www.microsoft.com/technet/.

• Microsoft TechNet Subscription Monthly TechNet subscriptions can be purchased for a mere $299 (single-user license) or $499 per year for TechNet Plus (single-user license). TechNet subscriptions provide monthly CD-ROM updates containing Knowledge Base articles, information, utilities, hotfixes, and service packs . The TechNet Plus subscription also includes software betas.

• Citrix Solution Tools Plus Citrix provides quarterly subscriptions that include the Citrix Knowledge Base and Practices CD. In addition to the CDs, Citrix sends out a monthly Solution News online newsletter and Solution Flash, which notifies the subscriber of updates to Citrix products and Knowledge Base articles.

If you are unable to locate information about your particular warning or error, you may need to contact Microsoft or Citrix support. Both provide for-fee support services to fit various organizations or needs. Microsoft even provides per-incident support for those who rarely need assistance. The pricing depends on whether or not you or your company meet certain criteria, but it usually begins at $149 per year.

Preventing Disasters

When you've found the information you need to resolve your issue, there are a few steps you should take to ensure that the fix doesn't make matters worse . It is a well-known fact that fixes, such as service packs, hotfixes, and registry and configuration changes, frequently backfire and break what was not already broken. This can be an administrator's nightmare if the right precautions are not taken. Here are a few tips that may save you if your fix backfires:

• Keep your Emergency Repair Disks (ERDs) current. Windows NT 4.0 Terminal Service Edition included the RDISK utility for creating ERDs, but because the registry is so much larger on a Terminal Server, the registry can no longer fit on a single disk. Windows 2000 has improved upon the ERD process by eliminating the System hives. The new ERD is useful if your system will not boot. It can be used to repair a damaged boot sector, a damaged Mast Boot Record (MBR) and replace or repair a damaged NT Loader (NTLDR). It is important to keep your ERD disk up-to-date. The one created when you initially install the system is no longer valid once you've added accounts and customized your system. Create a new ERD after you have changed or updated your system configuration. An ERD can be created through the Windows 2000 backup utility, found in Start Programs Accessories System Tools Backup (Figure 13-7) or by typing ntbackup at the command prompt.

Figure 13-7: Windows 2000 Emergency Repair Disk

• If the problem is reproducible, run the fix on a test server first. If your test server is a duplicate of your production servers, you should see exactly the same results.

• When installing service packs, respond YES to creating a backup of the previous system files. If the service pack produces unexpected results, you can back out of the upgrade. The same should be done for application updates and hot fixes.

• Document everything! This is one of the most important things you can do. Make sure you save the documentation where it will be easy to access and backed up.

No matter how careful you are, things can happen, it's just a matter of time. All you can do is take every precaution possible and have an arsenal of tools ready to combat whatever problem occurs. Backing up your servers and registry, testing, and backing out of upgrades are all very good practices, and I'd recommend employing them all just in case, but there are a few other tools out there that can make your life easier, or save you in a pinch . Windows 2000 comes with a few new or improved features that can really help you out of a jam.

First, Windows 2000 Backup is a big improvement over the old NT Backup. The new backup utility, located under System Tools in the Accessories folder, allows you to back up not only onto tape, but also on disks, CDs, and network-accessible volumes . A 'System State' backup will back up the registry, Component Services Class Registration database, system startup files, and Certificate Services database on a nondomain controller.

Another improvement in Windows 2000 is the advanced startup options. To access these, press f8 during the Windows startup process. The new options provide enhanced troubleshooting and resolution features such as Safe Mode, Safe Mode with Networking, and Safe Mode with Command Prompt. These features were a part of Windows 9. x operating systems, but have been improved upon in Windows 2000. If you have a driver that's gone astray and your machine blue screens at startup, try starting it in Safe Mode. It loads only the bare essentials and allows you to uninstall the problematic driver and reboot normally.

Consider this scenario: Suppose you've installed a device driver and after restarting the system, it freezes just after boot. What can you do? You've already logged in so it's too late for Last Known Good and your server locks up once you're in. Restart your server and press f8 during the Windows startup process. Select Safe Mode. Once the system starts up, right-click My Computer and select Manage. From Management Console, go into Device Manager, right-click the device in question and select Disable or Uninstall, then restart your system. An even slicker way to take care of the problem driver is to go to the command prompt and type: Listsvc . This will produce a list of drivers and services loaded, as well as their status. Once you have located the driver, type disable << drivername >> and press enter . The problem driver will now be disabled and you should be able to function normally.

One of my favorite startup options is the Enable Boot Logging feature. When selected, Enable Boot Logging creates a log file that lists all the drivers and services the system loads or fails to load. The log, called Ntblog.txt, is located under the \Winnt folder. By starting up with this option, prior to and after an upgrade or new installation, you can document the new drivers or services that were installed. Just remember to rename the log after the first boot. If problems occur, you can use these logs to locate the problem driver or service.

Windows 2000 has also added the Recovery Console (RC) that can be run from the command line to perform troubleshooting and recovery tasks. The Recovery Console can be installed from the Windows 2000 \I386 directory by running the winnt32 command with the /cmdcons switch or booting from the Windows 2000 Setup disks or CD. Running the Recovery Console allows you to start/stop services, read and write data on local drives , even if they are formatted with NTFS and to format hard disks. You can even troubleshoot and repair an NT system with RC; just run RC from the CD-ROM or install onto the NT system.

Aside from the tools that come packaged with the operating system, there are a few other tools that can round out your toolkit. Absolutely no administrator's tool kit is complete without Winternal's RegMon and FileMon. Both tools are monitoring utilities that can be used to troubleshoot application problems. Common problems seen in server-based environments such as corrupt application files, invalid file paths, or user hive entries and locked down files that need less stringent permissions, can be tracked down and corrected using these two tools. Just start up RegMon or FileMon and run the application in question. Once the error has occurred, stop the monitor and check the log. Typically you'll see an entry that says 'failed,' 'file not found,' or 'no access.' These two utilities are must haves and well worth the money. Working demos can be downloaded from http://www.winternals.com.

-Connie S. Wilson, MCSE, MCP+I, CCA, CNA

Establishing a Baseline

What is a baseline and how can it help you? A baseline helps to determine how well your system is performing and to spot potential problems before they happen. Developing a baseline will help determine how many users the system can support concurrently, and the impact of running certain applications. It's a much better plan than waiting for your users to inform you of how slow the system is! It's also the first step in assuring that your system is running at optimum performance levels and that you have enough resources to support your environment.

How do you establish a baseline for your system? There are two techniques commonly used to develop a baseline. The first is to perform benchmarks or tests on your servers before and after placing them in production. In other words, once you have configured your server and installed all the applications, you perform tests to see how the server is performing. The same tests are run again after the server is in active use. This can be done in a nonproduction, test environment as well if you can round up enough users to act as guinea pigs.

The second method of developing a baseline is to gather server resource statistics over time while the server is in use. The tests can be run at intervals, over a period of weeks, for a trend analysis. The results are documented and compared. Both methods can help you determine system throughput and where potential bottlenecks may lie. Problems occur when demand for resources exceed supply. Typically, the server's processors, memory, physical and logical disks, network interface cards and software are monitored and the results compared.

Various tools can be used to monitor performance and develop a baseline. Both NT and Windows 2000 include Performance Monitor, while Citrix MetaFrame XP e includes Resource Management, which we'll discuss in the next section. Both can be extremely helpful in monitoring and developing a baseline for your server.

Performance Monitor

Performance Monitor or PerfMon is a graphical tool incorporated into Windows NT Terminal Server Edition and the Windows 2000 family of servers. It gathers and examines information about server activity by examining system objects or components. Objects are server components that possess a set of measurable properties such as the following:

• Server

• Processors

• Memory

• Physical and logical disks

• Users

• Connections

Objects are defined that represent individual processes, sections of shared memory, and physical devices. Each object has a series of counters associated with it, which represent the measurable characteristics of objects. An example is the Processor object which has several counters associated with it, such as % Processor Time, %User Time, and so on. Certain objects exist on all systems; others only exist if the system is running the associated software.

Each object can also have several instances. An instance shows how many of an object are available in the system. For example, if a server has multiple processors, the Processor object will have multiple instances. If the server has four disk drives, the Physical Disk object will have four instances and so on. Other objects such as Memory and Server do not have instances. Objects with multiple instances produce the same counter information for each instance.

To view the objects and counters that can be used to monitor your system, go to Start Programs Administrative Tools Performance. Make sure System Monitor is selected on the left-hand side of the screen and click the '+' sign at the top of the monitor screen. (You can also right-click a blank area of the System Monitor and choose Add Counters from the context menu.) At this point, the monitor screen on the left is blank. Objects, instances, and counters must be selected and added. To select objects and their related counters, use the Performance Object drop-down menu and select the counters to monitor from the list. (See Figure 13-8.)

Figure 13-8: The Performance Monitor Add Counters window

In the following sections, we will look at some of the objects Performance Monitor tracks, as well as their associated objects.

Processor

Process threads require processor cycles to run, and if the demand exceeds the supply, long processor queues develop and system response time will degrade. Two counters that can help you monitor the overall processor load are % Processor Time and Interrupts/sec.

If processor utilization is over 75 percent on average, the processor is working pretty hard. You might consider increasing the number or the speed of the existing processors in the server. To determine processor utilization, monitor the %Processor Time counter under the Processor object. The %Processor Time shows the percentage of elapsed time that a processor is busy executing non-idle threads. If the %Processor Time counter consistently registers at or near 75 percent, the processors might be slowing the system response time.

Common causes of excessive interrupts are defective device adapters or badly designed device drivers. Interrupts degrade system performance because most of the processor time is spent handling them. A moderately busy server (32-bit hard disk adapter, network card, and about 12 users) will experience an average of 100 interrupts per second. If the interrupts per second increases noticeably without a corresponding increase in system activity, it could indicate a hardware problem or faulty drivers.

Memory

Server memory takes the biggest hit on a Windows 2000 system. As we all know, every new version of the operating system and each new version of an application requires more and more memory. If you don't have sufficient memory on your system, you will definitely feel the pain and so will your users. Table 13-5 lists a few important memory counters you may want to keep an eye on.

Physical and Logical Disks

Disk problems on MetaFrame XP servers are usually related to paging because of the memory load the applications place on the server. When the physical memory is maxed out by the load of applications and users, the server begins using the hard disks to support virtual memory. Two counters that should be monitored on both physical and logical disks to measure performance are

• % Disk Time Displays the percentage of the physical disk that is busy. If the disk is busy more than 90 percent of the time, you should probably add another disk.

• Current Disk Queue Length Displays the current number of data transfers waiting in the queue. If you are averaging more than two, you will notice the degradation. Keep this number as small as possible.