Diagnostic Commands and Tools

Several tools and commands are available on the PIX Firewall to troubleshoot all kinds of issues with PIX Firewall. In this section, we will attempt to go through all such tools and commands, which will be used in the rest of the chapter for troubleshooting specific issues.

show Commands

show commands on PIX Firewall are used to display statistics and information about the PIX firewall both current and past. show commands are used mainly for troubleshooting and the monitoring the health of the PIX firewall. Some of most useful show commands are shown in the sections that follow.

show xlate [detail]

This command shows the translation details through the PIX firewall. Example 3-5 shows both the summary and details of translation that are built up on the PIX firewall. It is recommended to look at the details of the translation which gives the interfaces involved the flow of a packet. This is useful to identify and correct any NAT related mis-configuration.

Example 3-5. Translation Through the PIX Firewall

PIX (config) # show xlate 3 in use, 3 most used PAT Global Local ICMP id 340 PAT Global (1024) Local PAT Global (1024) Local PIX (config) # show xlate detail 3 in use, 3 most used Flags: D DNS, d  dump, I  identity, I  inside, n  no random,            o  outside, r  portmap, s  static TCP PAT from inside: to outside: flags ri UDP PAT from inside: to outside: flags ri ICMP PAT from inside: to outside: flags ri PIX(config)# 

Table 3-4 describes the Xlate flags.

Table 3-4. Xlate Flags Table




Static Translation Slot


Dump Translation Slot on Next Clearing Cycle


Port Map Translation


No Randomization of TCP Sequence Number


Outside Address Translation


Inside Address Translation


DNS A RR Rewrite


Identity Translation from NAT 0

show connection [detail]

This command shows the connection details output on the PIX firewall. Connection will not be built up without translation. So, if you do not see any connection, you need to find out if the translation is built up. Example 3-6 shows the translation that is built up on the PIX firewall.

Example 3-6. Shows the show connection [detail] Output from the PIX Firewall

PIX (config) # show connection 2 in use, 2 most used ! Idle time, bytes transferred and the flags are shown in the following connection TCP out in idle 0:00:22 bytes 1774 flags UIO UDP out in idle 0:00:14 bytes 540 flags d ! Following command shows the interface details of the connection PIX (config) # show connection detail 2 in use, 2 most used Flags: A  awaiting inside ACK to SYN, a  awaiting outside ACK to SYN,            B  initial SYN from outside, D  DNS, d  dump,            E  outside back connection, F  outside FIN, f  inside FIN,            G  group, H  H.323, I  inbound data, M  SMTP data,            m  SIP media, O- outbound data, P  inside back connection,            q  SQL*Net data, R  outside acknowledged FIN,            R  UDP RPC, r  inside acknowledged FIN, S  awaiting inside SYN,            s  awaiting outside SYN, T  SIP, t  SIP transient, U  up TCP outside: dmz: flags UIO UDP outside: dmz: flags d PIX# 

show local-host

A local-host is an entry that is created for any source IP on a higher security level interface. The show local-host command displays the translation, connection, and AAA information together. Example 3-7 shows a local-host information for local host IP address of

Example 3-7. show local-host Command Output

PIX# show local-host Interface inside: 822 active, 823 maximum active, 0 denied local host: <>,     TCP connection count/limit = 0/unlimited     TCP embryonic count = 0     TCP intercept watermark = unlimited     UDP connection count/limit = 63/unlimited   AAA:   Xlate(s):     PAT Global Local   Conn(s):     UDP out in idle 0:01:31 flags  PIX# 

show service-policy

This command is used to see what inspection policies are applied and the packets matching them, as shown in Example 3-8.

Example 3-8. Output of the show service-policy Command

PIX# show service-policy Global policy:   Service-policy: global_policy     Class-map: inspection_default       Inspect: dns maximum-length 512, packet 0, drop 0, reset-drop 0       Inspect: ftp, packet 0, drop 0, reset-drop 0       Inspect: h323 h225, packet 0, drop 0, reset-drop 0       Inspect: h323 ras, packet 0, drop 0, reset-drop 0       Inspect: http, packet 0, drop 0, reset-drop 0       Inspect: netbios, packet 0, drop 0, reset-drop 0       Inspect: rsh, packet 0, drop 0, reset-drop 0       Inspect: rtsp, packet 0, drop 0, reset-drop 0       Inspect: skinny, packet 0, drop 0, reset-drop 0       Inspect: esmtp, packet 0, drop 0, reset-drop 0       ... Interface outside:   Service-policy: VoIP     Class-map: voice_marked       Priority:         Interface outside: aggregate drop 0, aggregate transmit 0 PIX# 

show asp drop

This command is used to identify the number of packets dropped by the PIX while processing the packet as shown in Example 3-9.

Example 3-9. show asp drop Command Output

PIX# show asp drop Frame drop:    Invalid tcp length                                        9382    Invalid udp length                                          10    No route to host                                          1009    Reverse-path verify failed                                  15    Flow is denied by access rule                         25247101    First TCP packet not SYN                                 36888    Bad option length in TCP                                   731    TCP MSS was too large                                    10942    TCP Window scale on non-SYN                               2591    TCP Dual open denied                                        11    TCP data send after FIN                                     62    TCP failed 3 way handshake                              328859    TCP SEQ in SYN/SYNACK invalid                              142    TCP ACK in SYNACK invalid                                  278    TCP packet SEQ past window                               46331    DNS Inspect packet too long                                  5    DNS Inspect id not matched                                8270    ... PIX# 

show cpu usage

This command is first introduced in PIX OS Version 6.0(1). Under normal conditions, the PIX CPU should stay below 30 percent, and can go as high as 60 percent. Anything above 60 percent is high. If the CPU reaches 100 percent, the PIX will start dropping packets. The show cpu usage command displays the CPU over time as a running average as shown below:

PIX# show cpu usage CPU utilization for 5 seconds = 1%; 1 minute: 2%; 5 minutes: 1% PIX# 


The percentage usage prints as NA (Not Applicable) if the usage is unavailable for the specified time interval; this can happen if you try to find out CPU Usage before the 5-second, 1-minute, or 5-minute intervals.

show traffic

The show traffic command displays the traffic transmitted and received on each interfaces of the PIX as shown in Example 3-10.

Example 3-10. show traffic Command Output

PIX# show traffic outside:         received (in 124.650 secs):                 295468 packets  167218253 bytes                 2370 pkts/sec   1341502 bytes/sec         transmitted (in 124.650 secs):                 260901 packets  120467981 bytes                 2093 pkts/sec   966449 bytes/sec inside:         received (in 124.650 secs):                 261478 packets  120145678 bytes                 2097 pkts/sec   963864 bytes/sec         transmitted (in 124.650 secs):                 294649 packets  167380042 bytes                 2363 pkts/sec   1342800 bytes/sec PIX# 

show blocks

The show blocks command and the show cpu usage command are useful in determining whether the PIX is being overloaded. The blocks are internal storage locations, similar to queues on a router; a packet is stored in a block until the PIX can process it and place it on the outbound interface xmit queue. Example 3-11 shows the show blocks output.

Example 3-11. show blocks Command Output

PIX# show block   SIZE    MAX    LOW   CNT      4    100     99   100     80    100     98   100    256    600    592   600   1550   1704   1362  1448   2048    100    100   100   2560     40     40    40   4096     30     30    30   8192     60     60    60  16384    104    104   104  65536     10     10    10 PIX# 

In the show blocks command output, SIZE is the block size, MAX is the total number of block available, LOW is the lowest number of blocks available on PIX since the last reboot, and the CNT is the current number of blocks available for specific task. Both LOW and CNT for any block size hit to zero indicate a low memory condition, which requires further investigation. To determine which features are responsible for memory block utilization, refer to Table 3-5, which summarizes the different memory blocks and the purpose of different sizes of blocks.

Table 3-5. Showing Different Sizes of Memory Blocks and Their Usage

Block Size

Used For

Created at boot up time



Duplicating existing blocks in DNS, isakmp, url-filtering, uauth, h323, tftp, and TCP modules




Used in TCP Intercept to generate an ACK packet, failover hello messages.




Stateful Failover, Syslog, TCP module




Ethernet Packets, buffering url filtered packets.




QoS Metrics




IKE Messages




QoS Metrics




QoS Metrics




Only used for the Livengood (i82543) Gig Ethernet cards




QoS Metrics



show output filters

Sometimes, it is very important to view the show command output to specific lines for specific information. You can do this with the Output filter option. The syntax is as follows:

show command | begin | include | exclude | grep [-v] <regular_exp> 

Following is a list of definitions for the arguments of this command:

  • begin Start displaying the output beginning at the First Match of the RegEx, and continue to display the remaining output.

  • include Display any line that matches the RegEx.

  • exclude Display any line that does not match the RegEx.

  • grep This is the same as include.

  • grep v This is the same as exclude.

For example, to display the interface stats starting with ethernet1, execute the following command:

PIX# show interface | begin ethernet1 

To display only the route statements from the running-config, execute the following command:

PIX# show running-config | include route 

To display the whole configuration except for the access-lists, you can execute the following command:

PIX# show running-config | exclude access-list 

Displaying the access-list entries that contain address can be achieved with the following command:

PIX# show access-list | grep 

To display only access-list entries that have non-zero hit counts, execute the following command:

PIX# show access-list | grep v hitcnt=0 

show tech-support

show tech-support collects output of a list of show commands. The command for show tech-support is as follows:

show tech-support [no-config | detail | tftp:] 

Example 3-12 summarizes how to use the show tech-support command.

Example 3-12. How to Use the show tech-support Command

! Following command will collect all the information including the running-config for ! Troubleshooting PIX# show tech-support ! The following command collect the same output without the running-config PIX# show tech-support no-config ! The following allows you to redirect the output into different location PIX# show tech-support file ?   flash:  Write output to flash: file system   ftp:    Write output to ftp: file system   tftp:   Write output to tftp: file system PIX# ! The following command allows you to collect the detail show tech-support information PIX# show tech-support detail . . . . . . PIX# 

Debug Commands

Although show commands are very useful to identify the problem quickly, debug commands are required to see more detailed information about the problem under some circumstances for the connectivity issues. As debug commands affect the CPU of the PIX negatively, use the debug command as a last resort. Before turning on the respective debug command, it is very important to know how much traffic is flowing through the firewall of a specific type.

In this section, we discuss several debug commands available on PIX that will help you troubleshoot connectivity problems across the PIX firewall.

debug icmp trace

This debug command is used to see the debug output of the ping going across the PIX firewall. Ping is usually used to check the IP connectivity across the firewall.

While using ping for connectivity tests, remember these points:

  • You can ping only the local interface of the PIX. For example, if your PC is on the inside network, you can ping only to the inside interface of the PIX.

  • You cannot ping the remote interface of the PIX. For example, if you are on an inside network, you cannot ping to the DMZ or the outside interface of the PIX firewall. If you are on outside, you can ping only on the outside interface.

  • ICMP echo-replies must be permitted explicitly thru the PIX unless you have ICMP inspection enabled.

Figure 3-2 demonstrates that Bob is able to ping to the inside interface, not the DMZ or outside interface.

Figure 3-2. Inside Users Ping Ability to PIX Firewall

For a successful ping across the PIX firewall, you should see the request and reply packets on the debug icmp trace command output as shown in Example 3-13.

Example 3-13. debug icmp trace Output of a Successful IP Connection

PIX# debug icmp trace PIX# ! In the following line, the ingress interface is inside, inside untranslated IP ! address, and destination address of the packet is ICMP echo-request from inside: to ID=3239 seq=4369 length=80 ! Following line shows that is translated to IP address ICMP echo-request: translating inside: to outside: ! The following lines are reply packets ICMP echo-reply from outside: to ID=3239 seq=4369 length=80 ICMP echo-reply: untranslating outside: to inside: PIX# 

The debug icmp trace command output shows enough details of the ingress, egress, source, destination IP/port, and the protocol information which can be used to define the packet flow.


You can turn off all debugs globally on the PIX firewall by issuing the no debug all or undebug all (u all is the short form) command.

debug application_protocol

To troubleshoot any application-specific issues, for example, Session Initiation Protocol (SIP) across the PIX firewall, you may run the debug for the corresponding protocol. For instance, to troubleshoot the issue pertaining to Media Gateway Control Protocol (MGCP) for voice over IP (VoIP) traffic between a phone and Call Manager, run the debug mgcp command.

debug pix process

To debug NAT within the payload of the packet, run debug pix process to see if the NAT is working correctly. For instance, if there is NAT in for MGCP, debug mgcp will not show the NAT details from the payload. For the NAT details, you must run debug pix process.

debug fixup tcp | udp

Use this command to find any inspection-related issue of a protocol. For example, to debug a problem with an FTP connection, you might need to run debug fixup tcp, which shows the FTP connection-related issue.

capture Command

The capture command (introduced in Version 6.2) allows for sniffing the packet hits at the interface of the PIX firewall. The command debug packet is deprecated by this capture command. The capture command must be executed from the enable mode (not the configuration mode), and optionally, you can configure an access-list to define the interesting traffic. Traffic can be captured both before and after it passes through the PIX; one capture on the inside interface, one capture on the outside interface. You can copy captures via TFTP or HTTPS.

Following is the syntax to enable capture on the PIX Firewall for traffic analysis:

capture capture-name [access-list acl-name] [buffer buf-size] [circular-buffer]   [ethernet-type type] [interface if-name] [packet-length bytes] 

Table 3-6 summarizes the meaning of the arguments of the capture command.

Table 3-6. capture Command Arguments



Capture name

This is the name of the capture that is used to view the information after the capture is completed.


Used to define the traffic that needs to be captured.


Capture buffer saved in RAM (default size 512kb).


The default is to stop capturing when the buffer is full. Overwrites the buffer from beginning when full. The default is non-circular.


Used to ca Ethernet packets of a particular type. The default is IP.


Used to c packets on a specific interface. The default is all interfaces.


Used to configure the maximum length to save from each packet. The default is 68 bytes.

To illustrate how to use the capture command on the PIX firewall, examine an example. Assume that an inside host ( is unable to go through Telnet to the server with IP address on the outside. Additionally, assume that the host address is translated to Work through the following steps to enable capture on the PIX firewall:

Step 1.

Create an ACL for both the inside and outside interfaces.

You must create two separate ACLs to apply with the capture for the inside and outside interfaces.

The inside interface ACL should use the untranslated source IP address and the destination IP address:

PIX(config)# access-list 100 permit tcp host host eq 23 PIX(config)# access-list 100 permit tcp host eq 23 host 

The outside interface ACL should use the translated source address and the destination IP address.

PIX(config)# access-list 101 permit tcp host host eq 23 PIX(config)# access-list 101 permit tcp host eq 23 host 

Step 2.

Create captures on both inside and outside interfaces.

PIX(config)# capture out-telnet access-list 101 interface outside   packet-length 1500 PIX(config)# capture in-telnet access-list 100 interface inside packet-   length 1500 

Step 3.

Perform the test.

Initiate a Telnet session from the inside host ( to access on the outside.

Step 4.

Copy the captures off to a TFTP Server or use HTTP server on the PIX.

You can display the capture output on the PIX firewall with the following commands:

PIX# show capture in-telnet PIX# show capture out-telnet 

To download the capture output to a TFTP server, use the following commands:

PIX# copy /pcap capture:out-telnet tftp:// PIX# copy /pcap capture:in-telnet tftp:// 

If the HTTP server is enabled on the PIX firewall for the ASDM access, you can use the following command to download the pcap files from the PIX firewall using the web browser:


Step 5.

Analyze captures with sniffer software.

After downloading the captures from the PIX firewall, you can analyze the captures with sniffer capture software such as Ethereal.

The capture command has been enhanced to capture packets dropped by security policies.

PIX# capture mycapture type asp-drop ?   acl-drop                Flow is denied by access rule   all                     All packet drop reasons   bad-crypto              Bad crypto return in packet   bad-ipsec-natt          Bad IPSEC NATT packet   bad-ipsec-prot          IPSEC not AH or ESP   bad-ipsec-udp           Bad IPSEC UDP packet   bad-tcp-cksum           Bad TCP checksum   bad-tcp-flags           Bad TCP flags   conn-limit              Connection limit reached   . . . PIX# 

Sniffer Capture

The capture command on the PIX firewall is useful only if the packets are reaching to the PIX interface. So you need to rely on external sniffer capture software. Besides, the capture command output can be converted and saved in pcap format, which later can be opened and analyzed by sniffer capture software. Ethereal is very popular free downloadable sniffer software (www.ethereal.com).


Syslog is the best troubleshooting tool for the PIX firewall. It logs traffic both to and through the firewall. The level of detail provided by syslog is controlled by the level of detail at which PIX is configured for syslog. Seven syslog logging levels can be set on the PIX firewall as shown in Table 3-7.

Table 3-7. Syslog Messages

Log Level


# of Messages (Sum)






41 (41)



21 (62)



74 (136)



56 (192)



21 (213)



95 (308)



15 (323)

Work through the steps that follow to configure logging on the PIX firewall.


Define what you want to capture.

The first step is to enable syslog on the PIX firewall to define the amount of logging you want to capture. There are two ways to define what you want to capture: first with the syslog level, and second with the event_list. The general syntax for enabling logging is as follows:

PIX(config)# [no] logging console | buffered | monitor | trap | mail | asdm   event_list | level 

While defining a different level of syslog, you can direct the logging to a monitor, console, buffer, ASDM, syslog server, or e-mail. For example, to enable the logging level to debug and capture the information in the buffer, configure the following:

PIX(config)# logging buffer debug 

The No form of the command which follows will turn off debug level buffer logging.

PIX(config)# no logging buffer debug 

You can change the default buffer size with the following command:

PIX(config)# [no] logging buffer-size <bytes> 

For example, to set up the buffer size to be 8192 bytes, use the following command:

PIX(config)# logging buffer-size 8192 

If you connect to the PIX with Telnet or SSH and want to display the level 6 logging on the monitor, use the following command:

PIX(config)# logging monitor 6 

This can be written as follows:

PIX(config)# logging monitor informational 

To send debug level logging to a syslog server, use the following command:

PIX(config)# logging trap debug 

The following commands send the critical level information to e-mail recipient

PIX(config)# logging mail critical 

You can configure the "Modifiable syslog" feature on the PIX to reduce the amount of syslog. For example, to determine what commands are being executed on the PIX, message 111009 records this information, but by default it is at level 7 (Debug).

%PIX-7-111009: User 'xyz' executed cmd: show run 

So, to capture this syslog ID, the PIX must have the debug level enabled. With debug level logging, PIX generates a huge amount of logging. To cut this down, use the following command to bring the syslog ID down to some lower level, for example, level 1, which will reduce the number of messages substantially with the following command:

PIX(config)# logging message 111009 level 1 

You also can use the following command:

PIX(config)# logging message 111009 level alerts 

Now your syslog message should look like this:

%PIX-1-111009: User 'xyz' executed cmd: show run 

To disable the modifiable syslog, you can use the following command:

PIX(config)# no logging message 111009 level alerts 

Or, you can use the following command:

PIX(config)# logging message 111009 level 7 

With a modifiable syslog, you will still get some logs in different lower levels (for example level 0, 1, 2, and so on). If you just want to see a specific syslog message, use the event class configuration.

An event list can be configured to allow only the specific syslog ID to be logged. An event_list provides you the flexibility to track events by class, severity, or syslog message ID. If you just want to capture syslog for ID 101001 only, you can use the following commands:

PIX(config)# logging list mylist message 101001 PIX(config)# logging buffered mylist 


Define the syslog server.

You must define the external syslog server IP address to forward the syslog message to the external syslog server. If your syslog server resides on the inside network with an IP address of, use the following command:

PIX (config) # logging host inside 


Define the mail server.

If you decide to send out syslog information to e-mail addresses, you need to configure the mail server and the e-mail addresses to forward the syslog information.

PIX(config)# logging from-address pixbldg3@xyz.com PIX(config)# logging recipient-address admin@xyz.com level critical PIX(config)# smtp server pri-smtp-host sec-smtp-host PIX(config)# 


Turn on the time stamp for syslog.

You can configure the time stamp for logging with the following command:

PIX(config)# logging timestamp 

Use the following command to turn off the timestamp:

PIX(config)# no logging timestamp 


Redirect debug to syslog if needed.

To redirect the debug output to syslog, execute the following command:

PIX(config)# logging debug-trace 

To turn off the redirection, use the following command:

PIX(config)# no logging debug-trace 

The syslog message number used is 711011.


Turn logging on.

Finally, turn logging ON with the following command:

PIX(config)# logging enable 

The following command turns off logging:

PIX(config)# no logging enable 

Once logging is configured, you can verify the syslog configuration with the following command:

PIX# show running logging 

To remove the logging configuration, use the following command:

PIX(config)# clear config logging 

To display buffer logging syslog messages, use the following command:

PIX# show logging 

To display only the syslog configuration settings, use the following command:

PIX# show logging setting 

System syslog messages on PIX/ASA 5500 are found at the following link:


Syslog messages based on different severity levels on PIX/ASA 5500 can be found at the following link:


Get the syslog message ID from the syslog server, find the meaning, and perform the recommended action suggested by the syslog message ID in the previously listed links.


Traceback is a record of abnormal function calls that is usually shown on the console of the PIX firewall, when an abnormal situation occurs. Problems with PIX normal functionality may produce a console traceback message. Not every traceback is serious; some are cosmetic. But, every traceback should be decoded and analyzed. Because the traceback is in hexadecimal values, you will not be able to decode it. Therefore, you need to engage the Cisco Support team for decoding and analyzing it. The problematic function (routines) that causes the traceback might have severe effects, such as crashing the whole PIX and thereby requiring a reboot.

The method of traceback information collection depends on the version PIX is running. If your PIX is running a version earlier than 6.2, you need to connect the console to collect the traceback information for analysis. This is extremely inconvenient and poses security risks to the PIX, as you have to leave the console port connected for hours or even days to collect the traceback, as you do not know when the PIX will crash. Beginning with Version 6.2, the crash information is saved to Flash memoryby default. If saving the crash information to Flash is disabled manually, you can enable it with the following command:

PIX(config)# crashinfo save 

Other Tools

Several often overlooked tools can help minimize the implementation and downtime of network availability. In this section, we will go through these tools:

  • Conduit to Access-list Converter Cisco's recommendation is to convert all conduits into access-lists. Access lists are more flexible and more efficient in terms of processing packets. Because conduits work globally on the PIX, if you have multiple interfaces, the packet coming through one of those interfaces has to go through the sequential search to find a match, whereas with an access-list this is more specific to the interface. In PIX Version 7.0, this command is deprecated completely, so you must convert your existing conduit into access-list before proceeding with the upgrade. You can download occ-121.gz for UNIX and occ-121.zip for Windows for a conduit to access-list conversation from the following location: http://www.cisco.com/cgi-bin/tablebuild.pl/pix. The output interpreter can be used as well for conversion.

  • Output Interpreter This is a great tool for finding common configuration errors very quickly. Here is the link for the Output interpreter:


    Paste the write terminal or show running-config under the text box of Enter show command(s) output from your device for analysis.

  • Bugs Tracker Bugs Tracker allows you to look for a possible bug on a specific release. Search by using the string Bug Toolkit in the following link: http://www.cisco.com/kobayashi/support/tac/tools.shtml

  • Field Notices Field Notices contain information on whether you have severe hardware or software issues on any specific platform or version of the PIX firewall. The following link contains the field notices for the PIX firewall: http://www.cisco.com/en/US/customer/products/hw/vpndevc/ps2030/prod_field_notices_list.html

  • PSIRT Pages This security advisory contains Security Vulnerabilities and remedies for all Cisco products. The link for the PSIRT is: http://www.cisco.com/en/US/customer/products/products_security_advisories_listing.html

Cisco Network Security Troubleshooting Handbook
Cisco Network Security Troubleshooting Handbook
ISBN: 1587051893
EAN: 2147483647
Year: 2006
Pages: 190
Authors: Mynul Hoda

Similar book on Amazon

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net