Diagnostic Commands and Tools

Most of the problems that arise with IDS/IPS MC involve importing sensors, sensor upgrades, and configuration deployments to the sensor. This section explains various logs available to troubleshoot any of these issues with IDS/IPS MC.

Audit Reports

Audit Reports provide much useful information about the IDS/IPS MC server. So, if there is any problem, either with a specific subsystem or with the IDS/IPS MC server itself, first look for information in the Audit reports. Among all the templates under Reports, the most important one is Audit Log Report, which provides information that is broad in scope about any abnormal behavior of functions on IDS/IPS MC. So to find out the cause of any problem with the functionality of IDS/IPS MC, this is the first report you must generate and view on the IDS/IPS MC. In addition, this report leads you in the right direction for generating additional reports, such as specific subsystem reports. To generate an Audit Log Report, within the IDS/IPS MC, click the Reports tab and choose Definitions. Click, for example, create of the type Audit log report for the last day. Select all for event security and choose all the options available, and click next. Run this report immediately. Then, either view the report on the IDS/IPS MC itself or export it for offline analysis. More details on audit reports can be found from the following location:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/cw2000/mgt_ids/idsmc20/ug/ch009.htm

MDCSupport File

If you want to have the comprehensive log to troubleshoot any issue on IDS/IPS MC, you must generate an MDCSupport file. Sections that follow describe how to collect the MDCSupport file.

How to Collect MDCSupport on a Windows Platform

Work through the steps that follow to collect the MDCSupport file on a Windows platform:

Step 1.	Open the MS-DOS command prompt window through Start> Run >. Then enter cmd and click OK.
Step 2.	The command prompt window opens. Enter MDCSupport at the prompt. Then press Enter.
Step 3.	The MDCSupport utility creates a .zip file that contains your CiscoWorks Common Services configuration data and log files. The default directory of the .zip file is /CSCOpx/MDC/etc.

Note

MDCSupportInformation.zip is created by default in the /CSCOpx/MDC/etc directory. To change the location of .zip file, enter MDCSupport drive:\path at the command prompt.

What to Look for and What Is Important in the MDCSupport File

Problems with the IDS/IPS MC are often unknown, and the MDCSupport file contains many files. Therefore, to identify the problem effectively, it is best to start by opening the file with WinZip, and then analyzing the file by sorting the files within the MDCSupport Zip file by file size. The largest file will always contain the most comprehensive information, so start by analyzing the largest file first. For example, if you sort by file size and see a very large IDSDbAdminAnalyzer, open it to look for errors. This file contains information on database transactions run on the IDS/IPS MC. You might get errors such as the following:

[04/06/2005 11:54:20] Looking at rule ID: 1 Exception in SystemContext - ASA Error -158: Value 72247 out of range for destination [04/06/2005 11:54:20] Rule checker error: [04/06/2005 11:54:20] com.cisco.nm.mdc.ids.common.exceptions.MdcException: ASA Error -158: Value 72247 out of range for destination - com.sybase.jdbc2.jdbc .SybSQLException: ASA Error -158: Value 72247 out of range for destination [04/06/2005 11:54:20] Reached error toleration limit 

This indicates that there may be a problem with the database index. Once you identify the problem, this can be followed up with the Cisco Support team for confirmation or for a possible fix.

Knowing the types of logs that go into different log files that are part of the package.cab file helps you analyze the log files of the package.cab file faster. Hence the list of different types of files with their contents are listed as follows:

• *.evt files Files with the .evt extension contain information about Windows Event Log. These files display the errors Windows is encountering with the IDS/IPS MC and CiscoWorks. You have to load these files with the Event Viewer on a Windows system to view their output.

• \apache\logs\error.log This log file contains information about errors on the web server for CiscoWorks. This is useful for troubleshooting issues with accessing the Web interface to CiscoWorks.

• Ciscoworks_setup00X.log This file results from CiscoWorks installer output. It includes detailed information about the installation and any error messages for problems that occur at the time of installation. The best way to analyze this file is to open it and search by the key word "error" to see what problems may have occurred during the installation. You may have several of these files included in the Zip file depending on the number of times the software has been installed and re-installed.

• stderr.log & stdout.log Process-related failures within IDS/IPS MC go into these files, so you must analyze these files. The following is an example of a database connection problem as reported by the stdout.log file:

  Exception in SystemContext - IDSSystem unable to create SQL Statement: null Database Connection creation failed!, err = JZ00L: Login failed. Examine the SQLWarnings chained to this exception for the reason(s). Detail - JZ006: Caught IOException: java.io.IOException: JZ0EM: End of data. An error occurred while trying to add a message to the Audit Log, message was IDSSystem unable to create SQL Statement: null SQL Error is null 

• AuditLogFirst.log, AuditLogLast.log These are very important files and need to be analyzed thoroughly. The files display audit log information as discussed earlier in this section. Among many other things, this file shows if IDS/IPS MC has trouble pushing a signature update out, and will give you the specific details about the error message. For example, following is the message shown in the audit log when the signature upgrade job hangs on the IDS/IPS MC:

  4644,'2005-04-06 16:06:15.703',0,0,3,11500,1004,1,3,0,'SYSTEM','sensor14: Error while pushing files to the sensor java.lang.Exception: An exception occurred during deploy, detail=The UI sensor version 4.1(4)S132 does not match the real sensor version 4.1(4)S133','' 

  Here you can see that the message clearly states that the IDS/IPS MC and sensor are not in agreement with the version that the IDS/IPS sensor is running; hence, the upgrade job hangs.

Enable Additional Debugging on IDS/IPS MC

Most of the issues with the IDS/IPS MC can be resolved with MDCSupport file. However, in some rare circumstances, getting to the root cause of the problem may not be possible without additional debug information. This information is usually analyzed by the Cisco developer. Work through the steps that follow to enable debugging on IDS/IPS MC:

Step 1.	On the GUI, go to Server Configuration > Administration > Process Management, and stop the IDS/_DeployDaemon process.
Step 2.	Back up the DeploymentConfig.xml file under Program Files/CSCOpx/MDC/etc/ids/xml into a location different from the installation base directory.
Step 3.	Edit the DeploymentConfig.xml file with Notepad. In the DeploymentConfig.xml file change "<DebugEnabled>false<DebugEnabled>" to "<DebugEnabled>true<DebugEnabled>," and under <CliLog> change "<Enabled>false<Enabled>" to "<Enabled>true<Enabled>".
Step 4.	On the GUI, go to Server Configuration > Administration > Process Management, and start the IDS_DeployDaemon process.

Import/Sigupdate will have debug messages in /log/IDS_SensorInterfaceDebug.log and deploy will have debug messages in the IDS_DeploymentDebug.log file. Also, collect the file cli-log in c or d:/documents and settings/(default user or username)/cli-log. More log files will be created under user's temp directory (show temp, from cmd line will report the exact temp directory) in Windows and under /var/tmp in Solaris.

Note

When you are finished with debugging, reset the value for DebugEnabled and CliLog back to false and the value for CleanupTempFiles to true. These settings ensure that the log files do not become too large or consume large amounts of disk space.