DETECTING THE OPERATING SYSTEM

As we have demonstrated thus far, a wealth of tools and many different types of portscanning techniques are available for discovering open ports on a target system. If you recall, this was our first objectiveport scanning to identify listening TCP and UDP ports on the target system. And with this information, we can determine if the listening port has potential vulnerabilities, right? Well, not yet. We first need to discover more information about the target system. Now our objective is to determine the type of operating system running.

Active Operating System Detection

Popularity:	10
Simplicity:	8
Impact:	4
Risk Rating:	7

Specific operating system information will be useful during our vulnerability-mapping phase, discussed in subsequent chapters. It is important to remember that we are trying to be as accurate as possible in determining the associated vulnerabilities of our target system(s). We don't want to be crying wolf and telling the IT department to fix something that isn't actually vulnerable, or worse , not there. Therefore, we need to identify the target operating system, to as granular a level as possible.

There are a number of techniques for performing this work. We can perform simple banner- grabbing techniques, as discussed in Chapter 3, which will grab information from such services as FTP, telnet, SMTP, HTTP, POP, and others. This is the simplest way to detect an operating system and the associated version number of the service running. And then there is a much more accurate techniquethe stack fingerprinting technique. Today, we have available some good tools designed to help us with this task. Two of the most accurate tools we have at our disposal are the omnipowerful nmap and queso, which both provide stack fingerprinting capabilities.

Active Stack Fingerprinting

Before we jump into using nmap and queso, it is important to explain exactly what stack fingerprinting is. Stack fingerprinting is an extremely powerful technology that allows you to quickly ascertain each host's operating system with a high degree of probability. Essentially, there are many nuances that vary between one vendor's IP stack implementation and another's. Vendors often interpret specific RFC guidance differently when writing their TCP/IP stack. Therefore, by probing for these differences, we can begin to make an educated guess as to the exact operating system in use. For maximum reliability, stack fingerprinting generally requires at least one listening port. nmap will make an educated guess about the operating system in use if no ports are open. However, the accuracy of such a guess will be fairly low. The definitive paper on the subject was written by Fyodor, first published in Phrack Magazine, and can be found at http://www. insecure .org/nmap/nmap-fingerprinting-article.html.

Let's examine the types of probes that can be sent that help to distinguish one operating system from another:

• FIN probe A FIN packet is sent to an open port. As mentioned previously, RFC 793 states that the correct behavior is not to respond. However, many stack implementations (such as Windows NT/2000/2003) will respond with a FIN/ACK.

• Bogus Flag probe An undefined TCP flag is set in the TCP header of a SYN packet. Some operating systems, such as Linux, will respond with the flag set in their response packet.

• Initial Sequence Number (ISN) sampling The basic premise is to find a pattern in the initial sequence chosen by the TCP implementation when responding to a connection request.

• "Don't fragment bit" monitoring Some operating systems will set the "Don't fragment bit" to enhance performance. This bit can be monitored to determine what types of operating systems exhibit this behavior.

• TCP initial window size Initial window size on returned packets is tracked. For some stack implementations, this size is unique and can greatly add to the accuracy of the fingerprint mechanism.

• ACK value IP stacks differ in the sequence value they use for the ACK field, so some implementations will send back the sequence number you sent, and others will send back a sequence number + 1.

• ICMP error message quenching Operating systems may follow RFC 1812 (http://www.ietf.org/rfc/rfc1812.txt) and limit the rate at which error messages are sent. By sending UDP packets to some random high-numbered port, you can count the number of unreachable messages received within a given amount of time. This is also helpful in determining if UDP ports are open.

• ICMP message quoting Operating systems differ in the amount of information that is quoted when ICMP errors are encountered . By examining the quoted message, you may be able to make some assumptions about the target operating system.

• ICMP error message-echoing integrity Some stack implementations may alter the IP headers when sending back ICMP error messages. By examining the types of alterations that are made to the headers, you may be able to make some assumptions about the target operating system.

• Type of service (TOS) For "ICMP port unreachable" messages, the TOS is examined. Most stack implementations use 0, but this can vary.

• Fragmentation handling As pointed out by Thomas Ptacek and Tim Newsham in their landmark paper "Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection" (http://www.clark.net/<126>roesch/idspaper.html), different stacks handle overlapping fragments differently. Some stacks will overwrite the old data with the new data, and vice versa, when the fragments are reassembled. By noting how probe packets are reassembled, you can make some assumptions about the target operating system.

• TCP options TCP options are defined by RFC 793 and more recently by RFC 1323 (http://www.ietf.org/rfc/rfc1323.txt). The more advanced options provided by RFC 1323 tend to be implemented in the most current stack implementations.

By sending a packet with multiple options setsuch as no operation, maximum segment size, window scale factor, and timestampsit is possible to make some assumptions about the target operating system.

nmap employs the techniques mentioned earlier (except for the fragmentation handling and ICMP error message queuing) by using the -O option. Let's take a look at our target network:

 [root]  nmap -O 192.168.1.10  Starting nmap V. 3.70 by [e-mail] Interesting ports on shadow (192.168.1.10): Port    State       Protocol  Service 7       open        tcp        echo 9       open        tcp        discard 13      open        tcp        daytime 19      open        tcp        chargen 21      open        tcp        ftp 22      open        tcp        ssh 23      open        tcp        telnet 25      open        tcp        smtp 37      open        tcp        time 111     open        tcp        sunrpc 512     open        tcp        exec 513     open        tcp        login 514     open        tcp        shell 2049    open        tcp        nfs 4045    open        tcp        lockd TCP Sequence Prediction: Class=random positive increments                          Difficulty=26590 (Worthy challenge) Remote operating system guess: Solaris 2.5, 2.51 

By using nmap's stack fingerprint option, we can easily ascertain the target operating system with precision. The accuracy of the determination is largely dependent on at least one open port on the target. But even if no ports are open on the target system, nmap can still make an educated guess about its operating system:

 [root]#  nmap -p80 -O 10.10.10.10  Starting nmap V. 3.70 by [e-mail] Warning: No ports found open on this machine, OS detection will be MUCH less reliable No ports open for host (10.10.10.10) Remote OS guesses: Linux 2.0.27 2.0.30, Linux 2.0.32-34, Linux 2.0.35-36, Linux 2.1.24 PowerPC, Linux 2.1.76, Linux 2.1.91 2.1.103, Linux 2.1.122 2.1.132; 2.2.0-pre1 2.2.2, Linux 2.2.0-pre6 2.2.2-ac5 Nmap run completed -1 IP address (1 host up) scanned in 1 second 

So even with no ports open, nmap correctly guessed the target operating system as Linux (lucky guess).

One of the best features of nmap is that its signature listing is kept in a file called nmap-os- fingerprints . Each time a new version of nmap is released, this file is updated with additional signatures. At this writing, there are hundreds of signatures listed.

Although nmap's TCP detection seems to be the most accurate as of this writing, the technology is not flawless, and often provides only broad guesses that at times seem less than helpful. But despite the challenges, it was not the first program to implement such techniques. queso, which can be downloaded from http://packetstormsecurity.org/UNIX/ scanners /queso-980922.tar.gz, is an operating system-detection tool that was released before Fyodor incorporated his operating system detection into nmap. It is important to note that queso is not a port scanner and performs only operating system detection via a single open port (port 80 by default). If port 80 is not open on the target server, it is necessary to specify an open port, as demonstrated next . queso is used to determine the target operating system via port 25.

 [root] queso  10.10.10.20:25  10.10.10.20:25          * Windoze 95/98/NT 

Operating System Detection Countermeasures

The following detection and prevention steps can be taken to help mitigate the OS detection risk.

Detection Many of the aforementioned port scanning detection tools can be used to watch for operating system detection. Although they don't specifically indicate that an nmap or queso operating system detection scan is taking place, they can detect a scan with specific options set, such as the SYN flag.

Prevention We wish there were an easy fix to operating system detection, but it is not an easy problem to solve. It is possible to hack up the operating source code or alter an operating system parameter to change one of the unique stack fingerprint characteristics. However, this may adversely affect the functionality of the operating system. For example, FreeBSD 4. x supports the TCP_DROP_SYNFIN kernel option, which is used to ignore a SYN+FIN packet used by nmap when performing stack fingerprinting. Enabling this option may help in thwarting OS detection, but it will break support for RFC 1644, "TCP Extensions for Transactions."

We believe only robust, secure proxies or firewalls should be subject to Internet scans . As the old adage says, "security through obscurity" is not your first line of defense. Even if attackers were to know the operating system, they should have a difficult time obtaining access to the target system.

Passive Operating System Identification

Popularity:	5
Simplicity:	6
Impact:	4
Risk Rating:	5

We have demonstrated how effective active stack fingerprinting can be, using tools such as nmap and queso. It is important to remember that the aforementioned stack-detection techniques are active by their very nature. We sent packets to each system to determine specific idiosyncrasies of the network stack, which allowed us to guess the operating system in use. Because we had to send packets to the target system, it is relatively easy for a network-based IDS system to determine that an OS identification probe was launched. Therefore, it is not one of the most stealthy techniques an attacker will employ .

Passive Stack Fingerprinting

Passive stack fingerprinting is similar in concept to active stack fingerprinting. But instead of sending packets to the target system, however, an attacker passively monitors network traffic to determine the operating system in use. Thus, by monitoring network traffic between various systems, we can determine the operating systems on a network. This technique, however, is exclusively dependent on being in a central location on the network and on a port that allows packet capture (for example, on a mirrored port).

Lance Spitzner has performed a great deal of research in the area of passive stack fingerprinting and has written a white paper that describes his findings at http://project.honeynet.org. In addition, Marshall Beddoe and Chris Abad developed siphon, a passive port mapping, OS identification, and network topology tool. You can download the tool at http://packetstormsecurity.org/UNIX/utilities/siphon-v.666.tar.gz.

With that little background, let's look at how passive stack fingerprinting works.

Passive Signatures

Various characteristics of traffic can be used to identify an operating system. We will limit our discussion to several attributes associated with a TCP/IP session:

• TTL What does the operating system set as the time-to-live on the outbound packet?

• Window size What does the operating system set as the window size?

• DF Does the operating system set the "Don't fragment bit"?

By passively analyzing each attribute and comparing the results to a known database of attributes, you can determine the remote operating system. Although this method is not guaranteed to produce the correct answer every time, the attributes can be combined to generate fairly reliable results. This technique is exactly what siphon performs.

Let's look at an example of how this works. If we telnet from the system shadow (192.168.1.10) to quake (192.168.1.11), we can passively identify the operating system using siphon:

 [shadow]# telnet 192.168.1.11 

Using our favorite sniffer, Snort, we can review a partial packet trace of our telnet connection:

 06/04-11:23:48.297976 192.168.1.11:23 -> 192.168.1.10:2295 TCP TTL:255 TOS:0x0 ID:58934  DF **S***A* Seq: 0xD3B709A4   Ack: 0xBE09B2B7 Win: 0x2798 TCP Options => NOP NOP TS: 9688775 9682347 NOP WS: 0 MSS: 1460 

Looking at our three TCP/IP attributes, we can find the following:

• TTL = 255

• Window Size = 0x2798

• Don't fragment bit (DF) = Yes

Now, let's review the siphon fingerprint database file osprints.conf:

 [shadow]#  grep -i solaris osprints.conf  # Window:TTL:DF:Operating System DF = 1 for ON, 0 for OFF. 2328:255:1:Solaris 2.6 2.7 2238:255:1:Solaris 2.6 2.7 2400:255:1:Solaris 2.6 2.7 2798:255:1:Solaris 2.6 2.7 FE88:255:1:Solaris 2.6 2.7 87C0:255:1:Solaris 2.6 2.7 FAF0:255:0:Solaris 2.6 2.7 FFFF:255:1:Solaris 2.6 2.7 

We can see the fourth entry has the exact attributes of our Snort trace: a window size of 2798, a TTL of 255, and the DF bit set (equal to 1). Therefore, we should be able to accurately guess the target OS using siphon:

 [crush]#  siphon -v -i xl0 -o fingerprint.out  Running on: 'crush' running FreeBSD 4.0-RELEASE on a(n) i386 Using Device: xl0 Host                    Port    TTL     DF      Operating System 192.168.1.11            23      255     ON      Solaris 2.6 2.7 

As you can see, we were able to guess the target OS, which happens to be Solaris 2.6, with relative ease. It is important to remember that we were able to make an educated guess without sending a single packet to 192.168.1.11all this analysis was done by simply capturing packets on the network.

Passive fingerprinting can be used by an attacker to map out a potential victim just by surfing to their website and analyzing a network trace or by using a tool such as siphon. Although this is an effective technique, it does have some limitations. First, applications that build their own packets (for example, nmap) do not use the same signature as the operating system. Therefore, your results may not be accurate. Second, you must be in a position to capture these packets (which can be difficult on a switch without enabling port mirroring). Third, it is simple for a remote host to change the connection attributes. But this latter issue plagues even active detection techniques.

Passive Operating System Detection Countermeasure

See the prevention countermeasure under "Operating System Detection Countermeasures," earlier in the chapter.

The Whole Enchilada: Automated Discovery Tools

Popularity:	10
Simplicity:	9
Impact:	9
Risk Rating:	9

Many other tools are available, and more are written every day, that will aid in network discovery. Although we cannot list every conceivable tool, we want to highlight two additional utilities that will augment the tools already discussed.

cheops (pronounced KEE-ops) is available from http://www.marko.net/cheops and is depicted in Figure 2-7. It's a graphical utility designed to be the all-inclusive network-mapping tool. cheops integrates ping, traceroute, port-scanning capabilities, and operating system detection (via queso) into a single package. cheops provides a simple interface that visually depicts systems and related networks, making it easy to understand the terrain.

Figure 2-7: cheops provides many network-mapping utilities in one graphical package.

tkined is part of the Scotty package, found at http://wwwhome.cs.utwente.nl/schoenw/scotty.tkined is a network editor written in Tcl that integrates various network management tools, allowing you to discover IP networks. Tkined is quite extensible and enables you to perform network reconnaissance activities, graphically depicting the results. Although it does not perform operating system detection, it will perform many of the tasks mentioned earlier and in Chapter 1. In addition to tkined, several other discovery scripts provided with Scotty are worth exploring.

Automated Discovery Tools Countermeasures

Tools such as Scotty, tkined, and cheops use a combination of all the techniques already discussed. The same techniques for detecting those attacks apply to detecting automated tool discoveries.