Introduction

The majority of network-aware computer software leverages the functionality of the TCP/IP protocol stack through high-level interfaces, such as BSD sockets, or frameworks such as Distributed Component Object Model (DCOM). Some software, however, has to work with network data at a lower levela world populated by segments, frames, packets, fragments, and checksums. Looking for security vulnerabilities in lower-level network software is challenging and captivating work. Networking code is a vast topic that can't be covered adequately in one chapter. Therefore, this chapter covers the basics, and then offers the authors' thoughts and experiences, which should prove useful if you're charged with a related auditing project.

This chapter focuses on three of the core Internet protocols: IP, UDP, and TCP. Throughout the discussion, you learn about security issues that tend to plague software that implements these protocols. Chapter 15, "Firewalls," covers firewall technology, which works closely with these protocols. Finally, Chapter 16, "Network Application Protocols," discusses some popular application-layer protocols and security issues that tend to surface in the code that implements them. Note that the discussion in this chapter is specific to IP version 4the current standard for Internet communications. IP version 6, IPv4's successor, is not covered in this chapter.

In the course of reviewing certain software, an auditor might have to examine code that deals with low-level network traffic. This processing could include analyzing packets or frames taken directly from the network as well as modifying or fabricating packets and placing them directly on the network. This discussion focuses on software systems that implement the TCP/IP networking protocols and on systems that analyze and intercept network traffic, as they tend to be more security critical devices in a network. Your most common projects involving TCP/IP protocol implementations will most likely be one of the four following product types:

• TCP/IP stacks residing on end hosts The TCP/IP stack is the centerpiece of data exchange between two or more hosts on an IP network. Typically located in an OS kernel, the IP stack hides details of network state and data delivery from user applications. Applications are given a clean and simple interface so that they don't need code to deal with network problems, retransmissions, error message processing, and the like.

• Products that provide routing, Network Address Translation (NAT), or load-balancing services Multihomed hosts might be required to route data between their interfaces as dictated by a static set of simple routing rules, or a dynamic rule set that's continuously updated through the use of routing protocols. This routing functionality is really an extension of the basic IP stack, and most end hosts can be configured to act as a router. Naturally, dedicated routing products are often much more complicated. In addition to routers, load-balancing products are charged with dividing incoming data for a host between a number of end hosts, thus enabling requests to a single host to be served in parallel and speeding up access time to clients for high-volume servers.

• Security products: firewalls and intrusion detection/prevention systems A number of security products are required to analyze packets traversing networks that they are protecting. These products make decisions based on attributes of the packets or the data in them. Often attackers will attempt to exploit subtle flaws caused by differences between how the security product evaluates the packets and how the end host evaluates those same packets.

• Network-monitoring products Several tools passively listen on a network and interpret the contents of packets being transmitted. They are often used for diagnosing network issues or for administrators to get a better idea of the kind of data sent over a network. These tools provide not only packet interpretations, but also statistical data based on protocol analysis. They are often required to simply interpret packets and optionally log some sort of information, as opposed to acting on packets as other products do.

The codebases for performing packet analysis at this level are generally quite large, so auditors faced with reviewing these codebases might consider it an insurmountable task. This chapter has been included to give code reviewers a primer on some major protocols within a standard TCP/IP suite and to highlight some of the problem areas where mistakes are most likely to be made. You learn how to audit several major components of IP stacks and use the knowledge you gained in Part II, "Software Vulnerabilities," of this book. Although firewall technologies aren't covered in depth until Chapter 15, many of the concepts in this chapter are essential for understanding how firewalls make policy decisions and what possible evasion techniques exist for circumventing them.