Diagnostic Commands and Tools

Cisco Secure ACS has extensive logging capability that allows an administrator to troubleshoot any issue pertaining to CS ACS Server itself (for example, replication) or an AAA requests problem (for example, an authentication problem) from NAS. This section explores these tools and how to use them efficiently.

Reports and Activity (Real-time Troubleshooting)

The Failed Attempts log under the Reports and Activity from the GUI is the quickest and best way to find out the reasons for authentication failure. Failed Attempts logs are turned on by default. However, if you want to add additional fields to the Default, you may by browsing to System Configuration > Logging > CSV Failed Attempts. In the CSV Failed Attempts File Configuration page, move desired attributes from Attributes to Logged Attributes. Then click on Submit. These additional attributes are shown under Reports and Activity. Occasionally, you might need to look at the Passed Authentications to troubleshoot authorization or NAS Access Restriction (NAR) issues. By default, the Passed Authentication log is not turned on. To turn it on, go to System Configuration > Logging > CSV Passed Authentications, and check Log to CSV Passed Authentications report under Enable Logging. There are other logs available for different services. For instance, for replication issues, there is a corresponding CSV file called Database Replication under Reports and Activity.

Radtest and Tactest

These tools are available to simulate AAA requests from the CS ACS server itself, which eliminates any possibilities of NAS configuration issues. This is especially important for troubleshooting the authentication issues with external user database authentication, for example, Microsoft Active Directory (AD) or Secure ID server. These tools are installed as part of CS ACS installation and located at C:\Program Files\CiscoSecure ACS v3.3\Utils>. More details on how to run these tools can be found at the following location: http://www.cisco.com/en/US/products/sw/secursw/ps2086/products_tech_note09186a00800afec1.shtml#auth_of

Package.cab File

Package.cab is the result of execution of the CSSupport utility, which includes all the log files for every service that we have discussed in the section entitled "CS ACS Architecture." Before running the CSSupport utility as shown in the paragraphs that follow, to capture the debug level logging, be sure to collect the "FULL" logging (on CS ACS, System Configuration > Service Control > Level of detail > Choose FULL > Restart). This is shown in Figure 13-3. Also be sure to check Manage Directory and set the appropriate option.

Figure 13-3. Turning on Full Logging on CS ACS

Once you set up the logging level to "FULL", run a few tests that are sure to fail and then run cssupport.exe as shown below:

C:\Program Files\CiscoSecure ACS v3.3\utils\CSSupport.exe 

The Package.cab file contains a good deal of meaningful information, but the amount of information may be overwhelming. So, being able to read the file efficiently is a key to success in isolating issues from the Package.cab file logs. Before getting into any more detail, you need to understand what goes into the makings of the package.cab file. Figure 13-4 shows the unzipped version of package.cab with a listing of files (icons are arranged by type).

Figure 13-4. Listing of Files in package.cab

The following are short descriptions of the files of package.cab:

• CSV Files CSV files contain the information about Audit log, Accounting, and Failed and Passed Authentication. Most of the files contain statistics, but to troubleshoot issues, Failed and Passed Authentication files are often used in conjunction with the log files that are discussed in the paragraphs that follow. The CSV files are created every day. Each file name without the date is the Active file. So, Failed Attempts active.csv is the active file, which stores the Failed Attempts information from the NAS.

• Log Files Every service discussed in the "CS ACS Architecture" section of this chapter has a corresponding log file. These files contain extensive logs about each and every service. For instance, auth.log contains all the current log information of CSAuth service. Just like CSV files, log files are created every day and the active log file is the one without the date in its name.

• User Database Files Three files go into making the CS ACS database. These files are user.dat, user.idx, and varsdb.mdb. You should not manipulate these files. Unless otherwise requested by Cisco, capturing these files is not necessary when running the CSSupport.exe utility.

• Registry File ACS.reg contains the Registry information of the CS ACS Server. Substantial CS ACS configuration (for example, NAS) goes into the Windows Registry. So, reading this file may be required for some troubleshooting. Do not import this file into another server; instead, open it with a text editor of your choice.

• Other Files Another useful file is MSInfo.txt, which contains the server and the OS information. The resource.txt file contains the resource information on the server, and SecEventDump.txt, AppEventDump.txt, and SysEventDump.txt contain an additional event dump on the server that may be used occasionally to troubleshoot any issues with the server itself.

As mentioned before, reading these files efficiently to isolate the problem is a key to success in troubleshooting CS ACS. To illustrate how to analyze the files, examine an example. The example assumes that a regular login authentication by the CS ACS Server is failing. The NAS debug does not give any conclusive output that indicates the reason for the failure.

To analyze this, first look at the Failed Attempts active.csv file to see why the user is failing. Quite often the information obtained from this file gives you the reason, so that no further analysis is needed; however, that's not always the case. For this example, assume that you have no conclusive reason for failure from the CSV file. However, you do have the username. The next step is to analyze the auth.log, because that contains more detailed information.

So, you search the username in the auth.log file. In this case, unfortunately, you receive no results from the search based on that username. So there must be a problem. It could be that CSTacacs service cannot process and forward the authentication request to the CSAuth service. Because you see the authentication failure in the Failed Attempts log, the authentication request must be reaching the CS ACS, and the first service that receives that packet is the CSTacacs, as the communication protocol configured between NAS and CS ACS is TACACS+. So, you need to analyze the TCS.log file, which contains all the activities that CSTacacs performs. As expected, you see the user request coming from the NAS. However, the user request is not being forwarded to the CSAuth service. After a little investigation, we find that NAR is configured for this user and, hence, packets are being dropped by the CSTacacs service; therefore, they are not being forwarded to the CSAuth service. Hence, you do not see the user in the auth.log. For every AAA request failure, you must look at the Failed Attempt first, and then search for the username in the auth.log. If an additional detail is needed, you need to analyze either the TCS.log or the RDS.log. Note that both CSTacacs and CSRadius form the communication bridge between the NAS and CS ACS, and CSAuth is the communication bridge between the CSTacacs/CSRadius and any external user databases such as Active Directory, NDS, and so on.