| IDSM-2 uses the same code base as the sensor appliances. Hence, the common issues and resolutions are the same for IDSM-2 blade as the sensor appliances, with few exceptions. Chapter 14 explains the configuration and troubleshooting aspects on the Appliance and other platforms. Hence, for IDS/IPS, software-related troubleshooting is not explained again in this section. However, some aspects of troubleshooting are unique to IDSM-2, which is the focus of this chapter as follows: Hardware issues Communication issues with IDSM-2 command and control port Issues with not getting traffic from the switch in Promiscuous mode Issues with Inline mode Not generating events issues TCP reset issues Software installation and upgrade issues
Hardware Issues After you insert the IDSM-2 blade into one of the slots on the switch, first be sure that the module is up and running correctly. Verification and troubleshooting steps are identical, but the commands are different on Native IOS and CatOS. The sections that follow explain how to troubleshoot the hardware issues on the Catalyst switch: IDSM-2 Hardware Issues on Native IOS After installing the module, first check the status of the module. If you find any problem with the module, follow the steps in the troubleshooting section to rectify the problem. Verify Hardware Operation Execute the show module command on the switch that is running Native IOS to obtain the status of the module as shown in Example 15-2. Example 15-2. show module Command Output cat6506#show module Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------- 3 48 48 port 10/100 mb RJ45 WS-X6348-RJ-45 SAL062100V7 4 6 Firewall Module WS-SVC-FWM-1 SAD08450149 ! The following line indicates that the IDSM-2 blade is in slot 5 5 8 Intrusion Detection System WS-SVC-IDSM-2 SAD074101MV 6 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAD082104HE Mod MAC addresses Hw Fw Sw Status --- ---------------------------------- ------ ------------ ------------ ------- 3 0009.1267.27d8 to 0009.1267.2807 6.1 5.4(2) 8.3(0.110)TE Ok 4 0003.e471.b758 to 0003.e471.b75f 3.0 7.2(1) 2.3(2) Ok ! The following line indicates that the version of IDS sensor software is 5.0(2). It also ! shows that the status of the blade is OK. 5 0003.feac.cb4a to 0003.feac.cb51 4.0 7.2(1) 5.0(2) Ok 6 000f.9093.cfd4 to 000f.9093.cfd7 4.0 8.1(3) 12.2(17d)SXB Ok Mod Sub-Module Model Serial Hw Status --- --------------------------- ------------------ ------------ ------- ------- 3 Inline Power Module WS-F6K-PWR 1.0 Ok ! The following line is an indication of the presence of the IDSM-2 daughter card ! as a sub-module. This sub-module works to accelerates performance on the IDSM-2. ! The status of this IDSM-2 sub-module shows OK. 5 IDS 2 accelerator board WS-SVC-IDSUPG ADEI32603225 2.2 Ok 6 Policy Feature Card 3 WS-F6K-PFC3B SAD082306Y9 1.0 Ok 6 MSFC3 Daughterboard WS-SUP720 SAD081805U0 2.1 Ok Mod Online Diag Status --- ------------------- 3 Pass 4 Pass ! The Online Diag Status shows for slot 5 is Pass. This is where the IDSM-2 blade is ! inserted 5 Pass 6 Pass cat6506#
|
The output in Example 15-2 shows that the IDSM-2 module is in a Catalyst 6506 chassis (with Supervisor 720 in slot 6) in slot 5. To obtain the same details as those contained in Example 15-2 for only the IDSM-2 blade, which is in slot 5 for Example 15-2, you can execute the command show module 5. Note The status should read ok. If it reads other, the IDSM-2 is not yet online. It may take five minutes or more to bring the module online through the switch.
Troubleshooting Steps If the IDSM-2 module is not coming on line and shows status as other, unknown, faulty, errdisable, power-deny, or power-bad in the output of the show module command, or if an amber or red status displays on the LED, work through the following steps to troubleshoot the issue: Step 1. | Be sure that you are running the supported HW and software on the switch for the IDSM-2 blade. If the IDSM-2 module is not supported in the software you are currently running, download the required software from the Cisco IOS Software Center at the following link: http://www.cisco.com/tacpage/sw-center/sw-ios.shtml
Refer to the following link for the supported software switch version: http://www.cisco.com/univercd/cc/td/doc/product/iaabu/csids/csids11/hwguide/hwIDSM-2.htm#wp712716
| Step 2. | If the status is power-deny, the switch does not have enough power available to power this module. Issue the show power command to confirm whether enough power is available. Refer to the Troubleshooting C6KPWR-4-POWRDENIED: insufficient power, module in slot x power denied Error Messages section in the following link: http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00801751d7.shtml#subtopic4b
| Step 3. | If the status is power-bad, the switch is able to see a card, but unable to allocate power. This is possible if the Supervisor Engine is not able to access the Serial Programmable Read Only Memory (SPROM) contents on the module to determine the identification of the line card. You can issue the show idprom module slot command to verify if the SPROM is readable. If SPROM is not accessible, you can reset the module.
| Step 4. | Be sure the module is properly seated and is screwed down completely. If the module still does not come online, issue the global configuration command diagnostic level complete to ensure that the diagnostic is enabled, and then issue the hw-module module slot number reset command. If the module still does not come on line, inspect the backplane connector on the module to be sure it is not damaged. If there is no visual damage, try the module in another slot or a different chassis. Also, inspect for bent pins on the slot connector on the backplane. You may want to use a flashlight when inspecting the connector pins on the chassis backplane.
| | | Step 5. | Issue the show diagnostics module slot number command to identify any hardware failures on the module. You need to have complete diagnostics enabled by issuing the global configuration command diagnostic level complete for the switch to be able to perform diagnostics on the module. If you have minimal diagnostics enabled and you change to complete diagnostics, the module needs to reset for the switch to perform the full diagnostics. In the same output that is shown in Example 15-3, the show diagnostics module command is issued, but the output is inconclusive, as many of the tests have not been performed in minimal mode. The output shows how to turn on the diagnostic level and then issue the show diagnostics module command again to see the complete results, as shown in Example 15-3.
Example 15-3. Turning on the Complete Level Logging for the IDSM-2 Module cat6506# configure terminal Enter configuration commands, one per line. End with CNTL/Z. ! The following command enables complete diagnostics. This command is hidden cat6506(config)#diagnostic level complete ! If you are running 12.2(14)SX on sup 720 for example use the following command Cat6506(config)# diagnostic bootup level complete cat6506(config)#end cat6506# 6d05h: %SYS-5-CONFIG_I: Configured from console by console ! Reset the module with the following command cat6506#hw-module module 5 reset Device BOOT variable for reset = <empty> Warning: Device list is not verified. Proceed with reload of module?[confirm] % reset issued for module 5 cat6506# 6d05h: SP: The PC in slot 5 is shutting down. Please wait ... 6d05h: SP: PC shutdown completed for module 5 6d05h: %C6KPWR-SP-4-DISABLED: power to module in slot 5 set off (Reset) ! The following line indicates a complete diagnostic test is in action 6d06h: %DIAG-SP-6-RUN_COMPLETE: Module 5: Running Complete Diagnostics... ! The following line shows the module passed the online diagnostics 6d06h: %DIAG-SP-6-DIAG_OK: Module 5: Passed Online Diagnostics 6d06h: %FABRIC-SP-5-LINECARDMODE_BUS_FORCED: The switching mode of module in slot 5 is forced to bus-mode. 6d06h: %OIR-SP-6-INSCARD: Card inserted in slot 5, interfaces are now online cat6506# ! Following shows the complete diagnostics test results. cat6506#show diagnostic module 5 ! This line indicates the complete level diagnostic is turned on Current bootup diagnostic level: complete Module 5: ! The following line shows the module pass the test Overall Diagnostic Result for Module 5 : PASS Diagnostic level at card bootup: complete ! A dot means pass, F means Fail and U means its not tested Test results: (. = Pass, F = Fail, U = Untested) 1) TestPortASICLoopback: Port 1 2 3 4 ---------------- . . . . 2) TestPCLoopback: Port 1 2 3 4 ---------------- . . . . 3) TestNetflowInlineRewrite: Port 1 2 3 4 ---------------- . . . . cat6506#
|
| Step 6. | IDSM-2 hard drive has been working for too long and has stopped functioning.
If the IDSM-2 drive is in constant use for extended periods of time (2 weeks plus), you might encounter multiple problems: inability to log in, I/O errors to the console when doing read/write operations (ls command), and commands not executing properly (cannot find the PATH to the executable). The switch might report that the module is OK, but attempts to log in to the service account and execute commands will reveal that the problem exists. The latest version (for example, 4.1[4] and above) alleviates the problem, but this is something that you may need to check.
| Step 7. | If you still have issues, collect the show tech-support and show logging commands output, and look for any error or other types of messages related to the IDSM-2 blade to troubleshoot further.
|
IDSM-2 HW Issue on CatOS Troubleshooting hardware issues with the IDSM-2 blade is very similar to the Native IOS mode. But, the syntax for show and debug commands are different. The sections that follow explain the verification of the hardware operation of the module, along with troubleshooting steps. Verify Hardware Operation Execute the show module command on the switch running CatOS to find out the status of the module as shown in Example 15-4. Example 15-4. show module Command Output cat6k> (enable) show module Mod Slot Ports Module-Type Model Sub Status --- ---- ----- ------------------------- ------------------- --- -------- 1 1 2 1000BaseX Supervisor WS-X6K-SUP1A-2GE yes ok 15 1 1 Multilayer Switch Feature WS-F6K-MSFC no ok 2 2 48 10/100/1000BaseT Ethernet WS-X6548-GE-TX no ok 3 3 16 1000BaseX Ethernet WS-X6516A-GBIC no ok ! The following line shows the IDSM-2 blade and the status shows OK. 4 4 8 Intrusion Detection Mod WS-SVC-IDSM-2 yes ok Mod Module-Name Serial-Num --- -------------------- ----------- 1 SAD041308AN 15 SAD04120BRB 2 SAD073906RC 3 SAL0751QYN0 4 SAD062004LV Mod MAC-Address(es) Hw Fw Sw --- -------------------------------------- ------ ---------- ---------------- 1 00-d0-c0-cc-0e-d2 to 00-d0-c0-cc-0e-d3 3.1 5.3.1 8.4(1) 00-d0-c0-cc-0e-d0 to 00-d0-c0-cc-0e-d1 00-30-71-34-10-00 to 00-30-71-34-13-ff 15 00-30-7b-91-77-b0 to 00-30-7b-91-77-ef 1.4 12.1(23)E2 12.1(23)E2 2 00-0d-29-f6-01-98 to 00-0d-29-f6-01-c7 5.0 7.2(1) 8.4(1) 3 00-0e-83-af-15-48 to 00-0e-83-af-15-57 1.0 7.2(1) 8.4(1) ! This line shows the version information of the IDSM-2 and the sup 4 00-e0-b0-ff-3b-80 to 00-e0-b0-ff-3b-87 0.102 7.2(1) 5.0(2) Mod Sub-Type Sub-Model Sub-Serial Sub-Hw Sub-Sw --- ----------------------- ------------------- ----------- ------ ------ 1 L3 Switching Engine WS-F6K-PFC SAD041303G6 1.1 ! The following line indicates the presence of acceleration card 4 IDS 2 accelerator board WS-SVC-IDSUPG . 2.0 cat6k> (enable)
|
The next section explains how to perform troubleshooting steps if the module status is other than OK. Troubleshooting Steps After inserting IDSM-2 blade, execute the show module command and check to see the status of the module. Also observe the light on the blade. If the show module command shows a status other than OK (the switch does not recognize the IDSM-2 blade) or the IDSM-2 blade LED status is not green, work through the following steps to troubleshoot the issue: | | Step 1. | If you run an unsupported CatOS version that runs on the Supervisor module for the IDSM-2 module, the switch may not be able to recognize the module. Execute the show version command on the switch and refer to the following link to verify whether the hardware and software are supported for the IDSM-2 blade: http://www.cisco.com/univercd/cc/td/doc/product/iaabu/csids/csids11/hwguide/hwIDSM-2.htm#wp712716
| Step 2. | If the status indicator is off on IDSM-2, turn the power on to IDSM-2 with the following command:
cat6k> (enable) set module power up module_number
| Step 3. | If the IDSM-2 blade is disabled, enable it with the following command:
cat6k> (enable) set module enable module_number
| Step 4. | If the IDSM-2 module is still not coming up, reset the module with the following command:
cat6k> (enable) reset module_number
| Step 5. | The module may not come up if there is not sufficient power in the chassis. Issue the show module command to see the status of the module. If it indicates power-deny, it is most likely not a hardware issue but a power budget issue. Issue the show environment power command to check the redundancy mode of the power supply. If you use 1 + 1 redundancy, you have these two choices:
Install two higher wattage power supplies if you still want 1+1 redundancy. Change the mode of power redundancy to combined. This means that the available power is now the sum of the two power supplies installed in the system. If you lose one of the power supplies, however, some of the modules may shut down because one of the power supplies would not be able to supply power on its own.
Refer to the following link for additional details on power management: http://www.cisco.com/en/US/partner/products/hw/switches/ps700/products_tech_note09186a008015bfa8.shtml
| Step 6. | If the IDSM-2 blade is improperly seated, the blade may not come on line. Turn off the switch and remove the module. Inspect for bent pins on the slot connector on the backplane. You may want to use a flashlight when inspecting the connector pins on the chassis backplane. Try to reseat it. Be sure that the screws on both sides are tightened, and confirm that the line card is inserted tightly into the chassis. Turn on power for the chassis and observe the status. In some cases, a badly seated card can cause symptoms that appear to be a hardware failure. A badly seated card may cause traffic corruption on the backplane. This might result in various problems occurring in the Catalyst chassis. For example, if one module corrupts traffic on the Catalyst backplane, this might cause the self-test to fail for both itself and other modules. Reseating all the cards can resolve this, and allow the self-tests to pass.
| | | Step 7. | Put the IDSM-2 module in a different slot to ensure that the slot in the chassis is not causing the problem.
| Step 8. | After reseating the IDSM-2 module, and putting that in a different slot, if the module status still shows anything other than OK, run a diagnostic test to eliminate the possibility of HW failure with the show test mod command. The show test command will show you a diaglevel entry. If this diaglevel is set to bypass or minimal, you can change this by issuing the set test diaglevel complete command, and resetting the module so that the self-test occurs. The set test diaglevel complete command executes all self-tests available, whereas the minimal and bypass options skip some or all of the tests. If you see an F in the output of the show test command, this indicates that this part might be suffering from a hardware failure. It's important to note that the STATUS LED should flash orange once and stay orange during diagnostic boot tests. It turns green when the module is operational (online).
| Step 9. | If the port status reads fail, be sure IDSM-2 is firmly connected in the switch.
| Step 10. | If the hdd status reads fail, you must re-image the application partition.
| Step 11. | If the diagnostics test turns out to be OK, and you are still having issues with the IDSM-2 blade, review Error Messages in the show logging buffer -1023 command output and show log command output around the time you encountered the module failure.
Use the Error Message Decoder, which can be found in the following link, to help decipher the output of any messages:
http://www.cisco.com/cgi-bin/Support/Errordecoder/index.cgi
In addition to reviewing the error messages, it is a good idea to use the Bug Toolkit, at the following link:
http://www.cisco.com/cgi-bin/Support/Bugtool/launch_bugtool.pl
The show version command will provide the software version information to help you run a bug search. For example, if you identify an exception in the show log command output, use the Bug Toolkit to search for bugs on your Catalyst platform, software version, and the exception from the show log.
|
Communication Issues with IDSM-2 Command and Control Port IDSM-2 module's port 2 is used as a command and control interface. If you have the module in slot 5, the port number for command and control is 5/2. To use Telnet or SSH to access the IDSM-2 blade, you must have this command and control port to connect to the rest of your network. If the IDSM-2 is managed by the IDS/IPS MC or IDM, and if the reporting tool (for example, Security Monitor) needs to pull the event, this interface must be enabled and must have connectivity to your network. This interface is also used for blocking. To establish the connectivity for the IDSM-2 blade to the rest of your network, put the Command and Control port of IDSM-2 in a VLAN that has connectivity to the rest of your network. By default, this interface is part of VLAN 1 on the switch. The sections that follow discuss how to configure the Command and Control port and how to troubleshoot it should a problem arise. Configuration Steps First, you need to map the Command and Control port to a specific VLAN, and second, you need to enable the Command and Control port on the IDSM-2 blade. The sections that follow cover these two topics: Switch Configuration Depending on whether you are running Native IOS or CatOS, use different syntax to assign the Command and Control port to a specific VLAN on the switch so that the IDS Management software or the Monitoring software can access the IDSM-2 blade. This is explained in the following list: Assigning Command and Control port to a VLAN on CatOS The command and control port must be assigned to a VLAN from which it can be accessed by external management clients (for example, IDS MC or Security Monitor). If an appropriate VLAN does not exist, you can create it with the following command: set vtp domain name set vlan vlan number clear vlan vlan_number Now, add the Command and Control port (slot#/2) to the VLAN that you have just created with the following command: set vlan vlan number slot/port For example, to create a VLAN 150 and assign the Command and Control port of the IDSM-2 module that is on slot 5, you need the following configuration on the switch: set vtp domain cisco set vlan 150 clear vlan 150 set vlan 150 5/2 Assigning a Command and Control port to a VLAN on Native IOS You can use the following command to create a VLAN and assign the management port to the VLAN: [View full width] vtp domain name vlan vlan_number state active intrusion-detection module module_number management-port access-vlan number For example, if you want to create VLAN 150 and assign the Command and Control port to  VLAN 150 on Native IOS: vtp domain cisco vlan 150 state active intrusion-detection module 5 management-port access-vlan 150
IDSM-2 Configuration Following the switch configuration, you need to log in to the IDSM-2 blade with the session command as shown in Example 15-5, or use Telnet to access it from the switch by typing the address 127.0.0.x1, where x is the slot number where the IDSM-2 blade is inserted. Example 15-5. How to Session into the Blade cat6506# session slot 5 processor 1 The default escape character is Ctrl-^, then x. You can also type 'exit' at the remote prompt to end the session Trying 127.0.0.51 ... Open ! When you login the first time, you need to use cisco/cisco for username/password. Then ! you will be prompted for the new password. In this example, the initial password is ! already changed; hence you don't see the password change prompt. login: cisco Password: Last login: Mon May 9 03:09:11 from 127.0.0.61 ***NOTICE*** This product contains cryptographic features and is subject to United States and local country laws governing import, export, transfer and use. Delivery of Cisco cryptographic products does not imply third-party authority to import, export, distribute or use encryption. Importers, exporters, distributors and users are responsible for compliance with U.S. and local country laws. By using this product you agree to comply with applicable laws and regulations. If you are unable to comply with U.S. and local laws, return this product immediately. A summary of U.S. laws governing Cisco cryptographic products may be found at: http://www.cisco.com/wwl/export/crypto/tool/stqrg.html If you require further assistance please contact us by sending email to export@cisco.com. sensor#
|
Once you are in the IDSM-2 CLI, you can enable and configure the Command and Control interface along with other initial setup requirements using the setup command. Example 15-6 shows how to enable and configure the Command and Control port using the setup command. Example 15-6. setup Command Output Continue with configuration dialog?[yes]: yes Enter host name[sensor]: IDSM-2 ! The following line assigns the Command and Control port Enter IP address[10.1.1.2]: 192.168.1.2 Enter netmask[255.255.255.0]: 255.255.255.0 ! Following line assigns the default gateway for the IDSM-2 blade Enter default gateway[10.1.1.1]: 192.168.1.1 Enter telnet-server status[disabled]: enable Enter web-server port[443]:
|
With this setup, you can log in to IDSM-2 blade using IDM, or you can connect from IDS/IPS MC. Troubleshooting Steps Once you have gone through the configuration steps that are outlined in the preceding sections, you should be able to connect to the IDSM-2 blade's Command and Control port using IDM, SSH, IDS/IPS MC, and so on. If you cannot do so, work through the following steps to troubleshoot the issue: | | Step 1. | Check the Command and Control port statistics on the switch.
Run the following command to find out if the 5-minute output rate is increasing:
Cat6509# show intrusion-detection module 4 management-port traffic
Example 15-7 shows the traffic flow for Command and Control interface on Native IOS:
Example 15-7. Finding Out If the Traffic Is Sent to the Command and Control Port of IDSM-2 Blade Cat6506# show intrusion-detection module 5 management-port traffic Intrusion-detection module 5 management-port: ! Make sure the line shows the Command & Control port UP Specified interface is up (connected)line protocol is up Hardware is C6k 1000Mb 802.3, address is 000e.3879.4961 (bia 000e.3879.4961) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Unknown duplex, Unknown Speed, media type is unknown output flow-control is unsupported, input flow-control is unsupported, 1000Mb/s input flow-control is off, output flow-control is off Last input never, output 00:00:48, output hang never Last clearing of "show interface" counters never Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) ! Make sure the following rate number increases 5 minute input rate 200 bits/sec, 10 packets/sec 5 minute output rate 300 bits/sec, 5 packets/sec Cat6506#
|
If the 5-minute output rate is not increasing, the switch is not sending the IDSM traffic at all. Check that your VLANs are correct and that there is not a misconfiguration. Example 15-8 shows how to obtain the state information of the management port on the switch.
Example 15-8. State Information of the Management Port Cat6506-A# show intrusion-detection module 5 management-port state Intrusion-detection module 5 management-port: Switchport: Enabled Administrative Mode: dynamic desirable Operational Mode: static access Administrative Trunking Encapsulation: negotiate Operational Trunking Encapsulation: native Negotiation of Trunking: On ! The port is part of VLAN 150 Access Mode VLAN: 150 (default) Trunking Native Mode VLAN: 150 (default) Trunking VLANs Enabled: ALL Pruning VLANs Enabled: 1-249,251-1001 Vlans allowed on trunk:150 Vlans allowed and active in management domain:150 Vlans in spanning tree forwarding state and not pruned: 150 Access Vlan = 150 Cat6506#
|
If you are running CatOS, execute show vlan vlan_number and show port module_number/2 to determine the statistics and VLAN configuration of the Command and Control port.
| | | Step 2. | Check to see if the Command and Control port is enabled on the IDSM-2 blade. Verify that by executing the command shown in Example 15-9.
Example 15-9. Verification of Command and Control Interface on the IDSM-2 Blade IDSM-2# show interfaces gigabitEthernet0/2 MAC statistics from interface GigabitEthernet0/2 Media Type = backplane ! The link status should show up Link Status = Up Link Speed = Auto_1000 Link Duplex = Auto_Full ! Make sure following two lines show some positive number and should increment ! over time Total Packets Received = 410270 Total Bytes Received = 33465114 Total Multicast Packets Received = 1 Total Receive Errors = 0 Total Receive FIFO Overruns = 0 Total Packets Transmitted = 64400 Total Bytes Transmitted = 6813106 Total Transmit Errors = 0 Total Transmit FIFO Overruns = 0 IDSM-2#
|
If the Command and Control interface shows that the Command and Control port is down, run the setup command to enable the Command and Control port.
| Step 3. | Wrong VLAN number and VLAN routing issues.
Be sure that the Command and Control ports are in the correct VLAN. If the management software or the Security Monitor is in a different VLAN, be sure you have the Inter-VLAN routing configured. For example, if you have Command and Control interface on VLAN 150, and the Management software server is on VLAN 160, then you must have routing between VLAN 150 and 160.
|
Failing to Get Traffic from the Switch with Promiscuous Mode There are two ports on the IDSM-2 blade that can be used to capture the traffic for analysis. On the switch side, you need to configure to send the packets to the IDSM-2 blade, and on the IDSM-2 blade, you must ensure that these sniffing ports are enabled and running. The sections that follow explain how to configure and troubleshoot the issues when IDSM-2 is running in Promiscuous mode. Configuration Steps The Promiscuous mode configuration for IDSM-2 involves configuring the switch to redirect the packets to the IDSM-2 blade, and on IDSM-2 to receive the packets from the switch. There are multiple options available to redirect the packets from the switch to the IDSM-2 blade: SPAN Configuration on Switch running Native IOS VACL Configuration on Switch running Native IOS Multilayer Switching (MLS) IP IDS Configuration on Switch running Native IOS SPAN Configuration on Switch running CatOS VACL Configuration on Switch running CatOS MLS IP IDS Configuration on Switch running CatOS IDSM-2 Blade Configuration
SPAN Configuration on Switch Running Native IOS Work through the following steps to configure SPAN on the switch running Native IOS: Step 1. | Log in to the switch and enter into global configuration mode:
Cat6506# configure terminal
| Step 2. | Configure the source interfaces for the monitor session with the following command:
Cat6506(config)# monitor session ( session_number) source interface interface/port_number [, | - | rx | tx | both]
The following command captures the traffic in both directions on interface GigabitEthernet2/10:
Cat6506(config)# monitor session 1 source interface GigabitEthernet2/10 both
You can also source the VLAN with the following command:
Cat6506(config)# monitor session ( session_number) source vlan vlan_number [, | - | rx | tx | both]
| Step 3. | Configure the IDSM-2 data port (port 1 or 2 or both) as a SPAN destination with the following command:
monitor session ( session_number) destination intrusion-detection-module module_number data-port data_port_number
The following command defines data-port 1 as the destination port for SPAN:
Cat6506(config)# monitor session 1 destination intrusion-detection- module 5 data-port 1
To disable the monitor session, use the no monitor session session_number command.
| Step 4. | To filter the SPAN session so that only certain VLANs are seen from switch port trunks, use the following command:
monitor session ( session_number) {filter vlan {vlan_ID} [, | - ]}
The following command shows that the switch will span only VLAN 140 traffic on port GigabitEthernet2/10, and forward to the IDSM-2 blade.
Cat6506(config)# monitor session 1 filter vlan 140
| | | Step 5. | Exit configuration mode and monitor the sessions using the following command:
show monitor session session_number
Example 15-10. Output of the show monitor command Cat6506# show monitor session 1 Session 1 --------- Type : Local Session Source Ports : Both : Gi2/10 Destination Ports : intrusion-detection-module 5 data-port 1 Cat6506#
|
| Step 6. | Configuring a Source Port/VLAN
You must define a source port or VLAN to capture the traffic that you want to send to the IDSM-2 module. You can use the following syntax to configure the source port or VLAN:
monitor session session source {{interface type} | {{vlan type} [rx | tx | both]} | { remote vlan rspan-vlan-id }}
|
VACL Configuration on Switch Running Native IOS If you do not want to configure SPAN as described in the previous section, use the steps that follow to configure the switch using Native IOS mode (for the 7600 and 6500 running native mode): Step 1. | Verify that the IOS version supports the IDSM-2.
| Step 2. | Place the Management Interface on the IDSM-2 into a VLAN with the following command:
intrusion-detection module module_number management-port access-vlan number
| Step 3. | The command that follows shows how to assign the management interface to VLAN 120:
cat6506(config)#intrusion-detection module 5 management-port access- vlan 120
| Step 4. | Create an ACL that will match the traffic that you want to send (capture) to the blade. In the example that follows, you want to match all Web traffic destined to the 10.1.1.0 network and send it to the IDSM. All other traffic should be passed normally. If access-list 151 is not created, all other traffic will be dropped.
cat6506(config)#access-list 150 permit tcp any 10.1.1.0 0.0.0.255 eq 80 cat6506(config)#access-list 151 permit ip any any
| | | Step 5. | Configure the router to capture the traffic in access-list 150 and pass the traffic in access-list 151, using a VLAN access-map as shown in Example 15-11:
Example 15-11. Configuring ACL to Define Interesting Traffic to be Captured cat6506(config)#vlan access-map IDSM-2 10 cat6506(config-access-map)#match ip address 150 cat6506(config-access-map)#action forward capture cat6506(config-access-map)#vlan access-map IDSM-2 20 cat6506(config-access-map)#match ip address 151 cat6506(config-access-map)#action forward
|
| Step 6. | Instruct the switch on which VLANs should be applied to the ACLs created earlier using the vlan filter command as follows:
vlan filter map_name vlan-list vlan_list
The example that follows shows VLANs 1-10, 36, and 124 tied to the access-list.
cat6506(config)#vlan filter IDSM-2 vlan-list 1-10,36,124
| Step 7. | Configure the IDSM-2 to receive capture packets on the specified VLANs. This tells the IDSM-2's sensing port to add these VLANs to the trunk with the following command:
intrusion-detection module module_number data-port data_port_number capture allowed-vlan capture_vlans
Following is an example:
cat6506(config)#intrusion-detection module 5 data-port 1 capture allowed-vlan 1-10,36,124
| | | Step 8. | Finally, you must enable the IDSM-2's sensing port to receive packets by using the command:
intrusion-detection module module_number data-port data_port_number capture
Following is an example of how to use the command:
cat6506(config)#intrusion-detection module 5 data-port 1 capture
Example 15-12 shows the complete configuration that is configured with the previous steps.
Example 15-12. Complete Configuration with VACL on Native IOS intrusion-detection module 6 management-port access-vlan 36 intrusion-detection module 6 data-port 1 capture intrusion-detection module 6 data-port 1 capture allowed-vlan 1-10,36,124 ! vlan access-map IDSM-2 10 match ip address 150 action forward capture vlan access-map IDSM-2 20 match ip address 151 action forward ! vlan filter IDSM-2 vlan-list 1-10,36,124 ! access-list 150 permit tcp any 10.1.1.0 0.0.0.255 access-list 151 permit ip any any
|
|
Note If more than one module is inserted into the chassis, you might want to segregate which traffic goes to which module. This is done by limiting the VLANs specified in Step 5 with the intrusion-detection module statement. You should not do this by restricting the VLANS in the vlan filter command. All traffic to be sent to any IDSM-2 must be marked to be captured using the vlan access-map AND vlan filter, and then traffic is limited by VLAN when sent to the module. Note that the vlan filter matches on the ingress VLAN the packets that come in on it. If it fails, delete it all and re-enter it.
MLS IP IDS Configuration on Switch Running Native IOS When you are using ports as router interfaces rather than switch ports, you cannot apply a VACL, as there is no VLAN. You can use the mls ip ids command to designate which packets will be captured. Packets that are permitted by the ACL will be captured. Those denied by the ACL will not be captured. The permit/deny parameter does not affect whether a packet is forwarded to destination ports. Packets coming into that router interface are checked against the IPS ACL to determine if they should be captured. Work through the following steps to configure mls ip ids command to capture IDS traffic: Step 1. | Go into the global configuration mode as follows:
Cat6k# configure terminal
| Step 2. | Configure an ACL to designate which packets will be captured with the following command:
Cat6k(config)# ip access-list extended MY_ACL
| Step 3. | Select the interface that carries the packets to be captured:
Cat6k(config)# interface interface_name
| | | Step 4. | Specify the capture VLANs with the following command:
Cat6k(config)# intrusion-detection module module_number data-port data_port_number capture allowed-vlan capture_vlans
The following command captures VLAN 150 traffic and sends it to the data-port 1:
Cat6k(config)# intrusion-detection module 5 data-port 1 capture allowed-vlan 150
| Step 5. | Apply the ACL created in Step 2 to the interface selected in Step 3:
Cat6k(config-if)# mls ip ids MY_ACL
|
Note For IDSM-2 to capture all packets marked by the mls ip ids command, data port 1 or data port 2 of IDSM-2 must be a member of all VLANs to which those packets are routed.
SPAN Configuration on Switch Running CatOS To configure a SPAN session in CatOS, the SPAN source port or source VLAN and the SPAN destination port must be identified. Then for the SPAN source port or VLAN, the direction of spanned traffic must be defined (that is, receive traffic only, transmit traffic only, or both receive and transmit traffic). The syntax for the SPAN configuration in CatOS is as follows: set span {src_mod/src_ports | src_vlans | sc0} {dest_mod/dest_port} [rx | tx | both] [inpkts {enable | disable}] [learning {enable | disable}][multicast {enable | disable}][ filter vlans...] [create]
The destination port can either be slot/7 or slot/8. If the IDSM-2 is seated in slot 5, the destination SPAN port will be port 4/7 or 4/8. Example 15-13 shows the setting up of a SPAN session to destination port 5/7. Example 15-13. A Simple SPAN Session Based on Source Port Console> (enable) set span 3/12 5/7 Enabled monitoring of Port 3/12 transmit/receive traffic by Port 5/7
|
This example simply sets up port 3/12 as the SPAN source port and port 5/7 as the SPAN destination port. This form of the command creates a SPAN session that sends a copy of both Receive and Transmit traffic. This assumes that the IDSM-2 is in slot 5. Example 15-14 shows how to send a copy of transmit-only traffic from VLAN 100 to the SPAN destination port 5/7 (assuming the IDSM-2 is in slot 5). Example 15-14. Simple SPAN Session Based on Source VLAN Console> (enable) set span 100 5/7 tx SPAN destination port incoming packets enabled. Enabled monitoring of VLAN 100 transmit traffic by Port 5/7 Console> (enable)
|
After you configure the SPAN on CatOS, to verify SPAN configuration on CatOS, use the command shown in Example 15-15. Example 15-15. The SPAN Sessions on CatOS cat6k> (enable) show span Destination : Port 5/7 Admin Source : VLAN 100 Oper Source : Port 3/12 Direction : receive Incoming Packets: disabled Learning : enabled Multicast : enabled Filter : - Session Number : 1 Total local span sessions: 1 cat6k> (enable)
|
To disable the SPAN session that is sending traffic to IDSM-2, use the command shown in Example 15-16: Example 15-16. Disabling a SPAN Session cat6k> (enable) set span disable session 1 This command will disable your span session. Do you want to continue (y/n) [n]? y Disabled Port 13/7 to monitor receive traffic of VLAN 650 cat6k> (enable)
|
Note Both ports slot/7 and slot/8 can be used to capture traffic. However, in the 5.0 release, the total amount of traffic is limited to a maximum of about 600 Mbps, regardless of which combinations of ports are used. As there is no advantage to using both ports to capture traffic over using just a single port, we recommend using just a single port. If both ports slot/7 and slot/8 are used to capture traffic, these ports should be configured to monitor different VLANs. Otherwise, the capture traffic will be duplicated.
VACL Configuration on Switch Running CatOS If you decide to configure VACL instead of SPAN, work through the following steps to configure VACL: | | Step 1. | Configure the VACL to define which traffic you want to capture with the following command:
set security acl ip acl_name permit ip [permit (...) | deny (...)] capture
Only permitted traffic can be captured. To permit traffic but not capture it, do not use the capture keyword. The following example captures all http traffic destined to host 10.1.1.1:
Console> (enable) set security acl ip filter_http permit ip any host 10.1.1.1 capture filter_http editbuffer modified. Use 'commit' command to apply changes. Console> (enable)
The capture keyword at the end of the statement is used to indicate that this classified traffic should be directed to the port that is configured as the capture port.
| Step 2. | Commit the VACL defined in the previous step to hardware. This process writes the VACL into Ternary Content Addressable Memory (TCAMs) on the PFC. TCAMs are a specialized piece of memory designed to provide high-speed lookups of their contents. The command to commit a VACL to memory is as follows:
Console> (enable) commit security acl filter_http ACL commit in progress. ACL filter_http successfully committed. Console> (enable)
| Step 3. | Map the VACL to the specific port (or VLAN) where the traffic is to be inspected. The following command maps the VACL to VLAN 55:
Console> (enable) commit security acl map filter_http 55 Mapping in progress. VLAN 55 successfully mapped to ACL filter_http. Console> (enable)
| Step 4. | Finally, define the capture port (either slot/7 or slot/8). The following command shows how to define the capture port (in this case 5/7, as the IDSM-2 module is slot 7).
Console> (enable) set security acl capture-ports 5/7 Successfully set the following ports to capture ACL traffic:. 5/7 Console> (enable)
| Step 5. | By default, the capture port is a trunk for all VLANs. So, to restrict the traffic for a specific VLAN, clear and reset the trunk. For example, capture port 7 could be set to be a trunk for only VLANs 2-5 using the following CatOS commands:
Console> (enable) clear trunk 5/7 1-4096 Console> (enable) set trunk 5/7 2-5
The process is now complete, and all traffic sourced from any host destined to host 10.1.1.1 in VLAN 55 will be copied and forwarded to the IDSM-2 (assuming in this example it is in slot 5) for monitoring.
|
MLS IP IDS Configuration on a Switch Running CatOS If you are running the Cisco IOS Firewall on the MSFC, you cannot use VACLs to capture traffic for IDSM-2. This is because you cannot apply VACLs to a VLAN in which you have applied an IP inspect rule for the Cisco IOS Firewall. However, you can use the mls ip ids command to designate which packets need to be captured. Packets that are permitted by the ACL are captured. Those denied by the ACL are not captured. The permit/deny parameter does not affect whether a packet is forwarded to destination ports. Packets coming into that router interface are checked against the IPS ACL to determine if they should be captured. The mls ip ids command is applied as part of the MSFC configuration instead of the supervisor configuration. The mls ip ids command only captures incoming traffic. You will need to use the mls ip ids command on both the client-side router interface and server-side router interface, so that both directions of the connection will be captured. Work through the following configuration to use the mls ip ids command to capture IPS traffic: Step 1. | Log in to the MSFC and configure an ACL to designate which packets will be captured with the following command:
msfc(config)# ip access-list extended word
| Step 2. | Select the interface that carries the packets to be captured with the following command:
msfc(config)# interface interface_name
| Step 3. | Apply the ACL created in Step 1 to the interface selected in Step 2:
msfc(config-if)# mls ip ids word
| Step 4. | Exit from MSFC to go to the switch.
| Step 5. | Add the IDSM-2 monitoring port (port 7 or 8) to the VACL capture list with the following command:
cat6k> (enable) set security acl capture module_number/port_number
|
Note For IDSM-2 to capture all packets marked by the mls ip ids command, port 7 or 8 of IDSM-2 must be members of all VLANs to which those packets are routed.
IDSM-2 Blade Configuration Once you configure the switch to send the packets to the sniffing interface of the IDSM-2 blade, you need to enable the sniffing port on the IDSM-2 blade. The best way to enable the sniffing port is to run the setup and add the sniffing port for Promiscuous mode as shown in Example 15-17. Example 15-17. Enabling a Sniffing Port When Running Promiscuous Mode IDSM-2#setup ! The additional output has been removed. Only the relevant information on how to enable ! the sniffing interfaces are added here Add Promiscuous interfaces?[no]: Interface[]: GigabitEthernet0/7 Interface[]: GigabitEthernet0/8 Interface[]: Add Inline pairs?[no]: No changes were made to the configuration. IDSM-2#
|
Troubleshooting Steps Work through the following steps to troubleshoot the issue if the IDSM-2 is not receiving the traffic from the switch: | | Step 1. | Verify on the switch whether the switch is sending any traffic to the data port with the command as shown in Example 15-18:
Example 15-18. The Output of the Data-port 1 Running Native IOS on the Switch Cat6506#show intrusion-detection module 5 data-port 1 traffic Intrusion-detection module 5 data-port 1: Specified interface is up line protocol is up (monitoring) Hardware is C6k 1000Mb 802.3, address is 0003.feac.cb50 (bia 0003.feac.cb50) MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 1000Mb/s input flow-control is off, output flow-control is unsupported Last input 00:00:31, output 2d21h, output hang never Last clearing of "show interface" counters never Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) ! You need to make sure these counters are increasing 5 minute input rate 1000 bits/sec, 40 packets/sec 5 minute output rate 1000 bits/sec, 20 packets/sec 4993 packets input, 1929584 bytes, 0 no buffer Received 4993 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 input packets with dribble condition detected 526773 packets output, 90662579 bytes, 0 underruns 0 output errors, 0 collisions, 3 interface resets 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier 0 output buffer failures, 0 output buffers swapped out cat6506#
|
This should show a 5-minute output rate increasing. If it does not, the switch is not sending the IDSM traffic at all. Check that your VLANs are correct and that there is no misconfiguration.
| Step 2. | Check to see if the switch SPAN/VACL misconfigured.
Revise the switch configuration again and verify that the switch is configured correctly to forward the traffic to the data port of the IDSM-2 blade. Check to see if the Remote SPAN (RSPAN) that is needed by SPAN is configured (refer to Chapter 14, "Troubleshooting Cisco Intrusion Prevention System" for more details). Also be sure to use mls ip ids if and when needed, instead of VACL.
| | | Step 3. | Be sure that the sensor's sniffing interface is enabled and receiving traffic.
If the switch configuration is correct and sending traffic to the IDSM-2 blade, Be sure the sniffing port on the IDSM-2 blade is enabled. From the command line of the IDSM-2, execute a show interfaces command and be sure that the sniffing ports are receiving packets, as shown in Example 15-19.
Example 15-19. The show interfaces Output on IDSM-2. sensor1-23# show interfaces Interface Statistics ! Some additional output is removed MAC statistics from interface GigabitEthernet0/7 Media Type = backplane Missed Packet Percentage = 0 ! Make sure the link status is up Link Status = Up Link Speed = Auto_1000 Link Duplex = Auto_Full ! Make sure you are receiving traffic and it's increasing Total Packets Received = 527406 Total Bytes Received = 90737051 Total Multicast Packets Received = 81228 Total Broadcast Packets Received = 25554 Total Jumbo Packets Received = 0 Total Undersize Packets Received = 0 Total Receive Errors = 0 Total Receive FIFO Overruns = 0 Total Packets Transmitted = 5003 Total Bytes Transmitted = 1933687 Total Multicast Packets Transmitted = 5003 Total Broadcast Packets Transmitted = 0 Total Jumbo Packets Transmitted = 0 Total Undersize Packets Transmitted = 0 Total Transmit Errors = 0 Total Transmit FIFO Overruns = 0 MAC statistics from interface GigabitEthernet0/8 Media Type = backplane ! Output is removed Total Transmit FIFO Overruns = 0 IDSM-2#
|
You should see that traffic is being received.
| Step 4. | If you do not see any packets in the previous step, it is likely that the traffic is not spanned correctly, or that the interface on the IDSM-2 is down. In that case, revise your switch configuration. Also, use the show interface command to check that the interface of the sniffing ports (Gig 0/7 and/or Gig 0/8) is up.
| Step 5. | Allow all VLANs (1-4094) on the list of allowed VLANs to be captured by the IDSM-2 data port to see if routing between VLANs is preventing the traffic from being captured. VLAN routing can affect VACL capture. For the routed traffic, the capture ports transmit the packets only after they are Layer 3 switched. The packets are transmitted out of a port only if the output VLAN of the Layer 3-switched flow is the same as the capture port VLAN. For example, assume that you have flows from VLAN 10 to VLAN 20, and you add a VACL on one of the VLANs permitting these flows. Then you specify a capture port. This traffic is transmitted from the capture port only if it belongs to VLAN 20, or if the port is a trunk carrying VLAN 20. If the capture port is in VLAN 10, it does not transmit any traffic. Whether a capture port transmits the traffic or not is independent of the VLAN on which you placed the VACL.
If you want to capture the traffic from one VLAN going to many VLANs, the capture port has to be a trunk carrying all the output VLANs.
For the bridged traffic, because all the traffic remains in the same VLAN, ensure that the capture port is in the same VLAN as the bridged traffic.
| Step 6. | Configure VACL capture or span/monitor to a sniffer machine so that another machine (non-IDSM-2) gets the traffic. Look at the traffic arriving on that machine to see if it's getting the traffic. This will independently verify if it's a switch or an IDSM-2 issue.
| Step 7. | Loose ribbon cable on IDSM-2.
If the IDSM-2 is producing alerts and stops suddenly, this could be because the ribbon cable between the XL (Falcon) Card and the Komodo+ motherboard has become loose. For additional details refer to Partner Field Notice 52816, at the following link: http://www.cisco.com/en/US/partner/products/sw/secursw/ps2113/products_field_notice09186a0080234f9c.shtml
This could be observed after a module has been moved or shipped.
|
Issues with Inline Mode The configuration steps on the IDSM-2 are the same for Inline as in Appliance, which is explained in detail in Chapter 14. So, the same information will not be repeated here. However, as the switch has only two sniffing ports, only one Inline pair configuration is possible. For the same reason, you cannot configure both Inline and Promiscuous mode. Not Generating Events Issues If the IDSM-2 is receiving the traffic, which is verified but unable to generate any events, work through the following steps to troubleshoot this issue: Step 1. | Check to see if you are getting events. Verify if you are not receiving any events on the IDSM-2 blade by executing the following command:
sensor# show events past 00:30:00
This will show all the events in the past 30 minutes.
| Step 2. | IDSM is getting traffic but is not triggering any events.
If you are receiving the traffic to match the triggering criteria for the signature, you may not receive any events even though the IDSM-2 may be functioning as it is configured to. To test whether the IDSM-2 is functioning properly or not, you can enable the ICMP signatures and send ping across the sensing interface to see if there are any event triggers.
| Step 3. | The sensing interfaces on the IDSM could be down. Ensure that they are up and that they are assigned to an interface group.
| Step 4. | The IDS/IPS receives events for a while. But then it stops receiving traffic.
Even though the switch is sending traffic, the virtual sensor may not be processing the traffic. This can be verified by using the show statistics virtual-sensor command. This can be because of the sensorApp loop when IP logging on a dual processor. To work around the problem, work through the steps that follow for the IDSM-2 to operate in single mode:
Log in to the service account and become root. Stop the cids applications by using the command "/etc/init.d/cids stop." Edit the mainApp.conf configuration file: "vi/usr/cids/idsRoot/etc/mainApp.conf" Add the following lines after the AnalysisEngine section: Arg01=-t Arg02=single Start the cids applications by using the command "/etc/init.d/cids start."
| | | Step 5. | A filter is blocking the events.
Be sure there is no filter that is applied to the sensor that could be blocking the events. If a filter is applied and you execute a show conf, you will see the following:
EventFilter Filters DestAddrs * Exception False SIGID * SourceAddrs * SubSig *
From the IDM on the sensor (https://sensorip), log in, and go to Configuration>Sensing Engine>Alarm Channel Configuration>Event filters and delete any event filters that you see. These could be filtering out your events.
|
TCP Reset Issues TCP reset concept, configuration, and troubleshooting are identical on the IDSM-2 blade and on Appliance. So, the same discussion is not repeated in this section. However, some specific issues are unique to IDSM-2 blade. Unlike Sensor Appliances, IDSM-2 has a separate interface for TCP reset. Port 1 is used as the TCP reset interface. The IDSM-2 has a specific TCP reset interface because it cannot send TCP resets on its sensing ports. If you have reset problems with the IDSM-2, try the following: If the sensing ports are access ports (a single VLAN), you need to configure the reset port to be in the same VLAN. If the sensing ports are dot1q trunk ports (multi-VLAN), the sensing ports and reset port all must have the same native VLAN, and the reset port must trunk all the VLANs being trunked by both the sensing ports.
|
