Traffic control is a major technical problem on public networks such as the Internet. There is a severe lack of skilled resources, and service providers are often struggling to maintain equipment that is either underpowered or sub-optimal in its configuration. There have been many examples of misconfigured routers diverting traffic accidentally onto congested paths, and often these problems take hours to resolve due to poor diagnostics or lack of centralized management. It is also not unheard of for service providers to deploy routers that do not have enough processing power or memory to effectively manage their own routing tables (especially if these devices have been in situ for some years and have not been upgraded). Regardless of whether the network is public or private, many of the challenges we must face in traffic engineering are the same.
In this section we will look at ways in which we can improve bandwidth utilization and overall performance on an internetwork. These techniques focus primarily on ways to keep unwanted traffic off the backbone and any other critical network segments. Even though intelligent devices such as bridges, switches, and routers already perform traffic segmentation, there are many cases where you can improve matters through the use of packet filtering, route filtering, spoofing, or other protocol management tools. Routers are becoming ever more sophisticated at dealing with traffic intelligently and may offer advanced features such as priority queuing, combinatorial metrics, and proxy services. All of these features are described in this chapter.
Routing protocols and related addressing issues were described extensively in Chapters 2 through 4; this section summarizes techniques already discussed in those chapters and provides references where appropriate. There are a number of key points to consider in optimizing routing and switched networks, including the following:
Routing protocol selection—Select the best routing protocol for the environment. On anything other than very small internetworks (more than ten routers) choose a protocol that scales well and converges quickly. Protocols such as OSPF, Integrated IS-IS, and EIGRP are all strong candidates. If interoperability is important to you, choose only industry-standard protocols such as OSPF and IS-IS.
Tuning—Tune timers and configuration parameters to improve convergence times, according to the available bandwidth and resources (e.g., OSPF has a hello interval and dead interval, which can be changed to speed up convergence at the cost of additional bandwidth and processing).
Impose hierarchy—Use a hierarchical addressing model so that route summarization can be enabled, particularly in large networks. With route summarization, routers can reduce many routes to a single advertisement. This can significantly reduce load on the network, reduce router processing overheads, and reduce the perceived complexity of the network.
Load sharing—Configure load sharing down multiple physical and logical paths where possible to make the best use of the available bandwidth (an analogy would be to use parallel rather than serial communications to achieve greater throughput). Techniques include:
Point-to-point link sharing (e.g., Multilink PPP)
Multipath bridging (typically proprietary techniques such as Xyplex's Fully Distributed Redundant Bridging (FDRB) technique)
Multipath routing (OSPF Multipath, IGRP, and EIGRP load sharing)
Reduce the number of active protocols—Where possible, minimize the number of routing protocols running on your backbone and on the network as a whole. This frees up router CPU and memory resources as well as conserving valuable WAN and LAN bandwidth.
Configure route filters—If possible add routing filters to avoid router processing of unnecessary routes.
Tune path costs—Most designers will exert some control over routing metrics to improve control over the paths that traffic takes through the internetwork. For various reasons path costs selected automatically by routers may not be optimum.
In recent years new techniques have emerged to make more efficient use of wide area links, including QoS-Based Routing, Constraint-Based Routing (CBR), and MultiProtocol Label Switching (MPLS).
Networks can generate a surprising level of unwanted traffic—that is, traffic present on networks or subnetworks that has no place being there. Often this is symptomatic of legacy applications and routing protocols that rely heavily on broadcasts, or it may be due to suboptimal configuration of networking devices and protocol stacks. With bandwidth becoming an increasingly critical resource, it is important that the network designer eliminate unwanted traffic both during the design phase and after implementation.
The process of elimination is essentially a three-phase strategy, in the following order:
Phase 1—Disabling unwanted or unnecessary protocols
Phase 2—Tuning protocols or optimizing traffic flows by design
Phase 3—Filtering out unwanted traffic
In an ideal world we would approach this problem in the order presented, starting by eliminating problems at the source (by designing unwanted protocols out of the configuration altogether). In practice the biggest initial gains tend to be from Phase 2 and Phase 3, since the bulk of unwanted traffic is not due to Phase 1 issues (however, it makes little sense to install packet filters throughout the network to stop a protocol that can be discarded at the source).
Networked systems usually have default settings that are designed to assist the network administrator in deploying systems quickly (i.e., a network metaphor for plug and play). For example, when enabling a routing protocol via the command line, this may enable that protocol on all IP interfaces. While this is very useful at the commissioning phase, it can lead to significant levels of unwanted traffic later, especially if one imagines this problem multiplied by hundreds of nodes. Key things to look out for are as follows:
Routing or bridging protocols running on interfaces where they are not required. For example, a router interface may have RIP and OSPF enabled when only OSPF is required on that interface. RIP should be explicitly disabled at that interface to stop periodic broadcasts.
Boot protocols, commonly left enabled by default on devices such as terminal servers. These are typically broadcast solicitations (e.g., BOOTP, DHCP, DEC MOP, RARP). If the device is only booted locally or via one of these mechanisms, then all others should be disabled.
The standard configuration for network devices should be designed to disable all unnecessary protocols and protocol activities per interface. Unfortunately, for legacy installations there is no easy way to determine which settings are redundant after the event. In this case you may have no choice but to take detailed packet traces and search for anomalous events (such as protocols and messages appearing on networks where they are clearly not required). Broadcasts and multicasts are always worthy of close examination, since these are particularly significant in degrading network system performance. You will also need to review the configuration of each class of device.
Legacy routing protocols and LAN applications often communicate using broadcasts. There are also emerging applications and protocols that support multicast operation. As we saw earlier, broadcasts and multicasts may be forwarded to network segments where they are not required (often referred to as broadcast radiation and multicast leakage). Broadcast and multicast handling is also fundamentally different from unicast handling from the host perspective (with broadcast packets the network device driver must interrupt the processor for each broadcast packet received, even though the majority of packets received will be discarded). The level of background broadcast traffic, therefore, directly affects the performance of all network-attached hosts. Relatively modest broadcast rates (up to 100's per second) can cause serious system degradation, even though link utilization is not significant. This area of the design, therefore, requires careful examination.
At extreme levels of broadcasts (1,000's per second) networks can suffer what is colloquially referred to as a broadcast storm, and flat bridged/ switched networks are particularly vulnerable. Although rare, these events can completely disable a network, forcing a major reboot. The causes of broadcast storms are typically the result of misconfiguration (e.g., inconsistencies or errors in filter settings), major hardware or software malfunction (e.g., a PC card throwing out spurious frames), or a serious design flaw (e.g., a loop). It is important, therefore, that the network designer implements ways of containing the scope of these packet types. Broadcasts are especially difficult to block on flat networks, since filters may need to extend well into the user data part of the frame to differentiate applications. In switched networks multicasts may be dealt with using multicast-aware switches (using protocols such as GARP and GMRP), and broadcasts can be contained to some extent using VLANs ). We will now examine some of the common sources of broadcast and multicast traffic.
There are many sources of broadcasts and multicasts; the main culprits are LAN-based servers, workstations, routers, and LAN-originated applications, as follows:
IP routing announcements—RIP-enabled routers transmit entire routing table updates as broadcasts (or multicasts with RIPv2) every 30 seconds. RIP allows only 25 routes to be transmitted per packet (using 554-byte packets). On a 300-router network with a routing table of 1,000 entries this would equate to 40 packets per router update. This produces a broadcast rate of 400 per second (300 × 40/30). This consumes an average of 1.8 Mbps, assuming that the updates are spread evenly. If updates become synchronized (as they tend to do without some built-in randomization or scheduling), the traffic level will be 30 times worse. For a router pair on a relatively slow point-to-point serial link, this represents a traffic spike of 177 Kbps in each direction every 30 seconds. For this reason most designers would limit the number of RIP routers to about a dozen.
IP workstations—IP workstations cache on average between 5 and 50 addresses per hour and regularly transmit broadcast ARPs at an average rate of 25 per hour (or 0.0069 ARPs per second). Therefore, a network with 2,000 IP endstations would produce approximately 14 ARP broadcasts per second.
Multicast applications—Multicasting is an effective way to distribute some types of application data (e.g., video and audio conferencing), and these applications can be particularly bandwidth intensive (a typical packet video application can generate a 7-MB stream of multicasts). We saw earlier how multicasts in a bridged or switched enterprise LAN could be leaked to all segments, affecting all users (whether they wish to participate in the application or not) and degrading overall performance.
NetWare service announcements—Novell's NetWare Operating System is still quite widely used on LANs and for LAN servers. NetWare servers use Service Advertisement Protocol (SAP) broadcasts to regularly announce their presence.
NetWare routing announcements—NetWare routers use Router Information Protocol (RIP) broadcasts (these are IPX packets, not IP) to regularly announce their presence.
NetWare client solicitations—NetWare clients use broadcasts (e.g., Get-Nearest-Server) to locate NetWare servers.
NetWare version 4 management—SNMP-based network management applications (such as NetExplorer) regularly broadcast packets to discover changes in the network.
AppleTalk multicasts—AppleTalk uses multicasting extensively to advertise and solicit services and to resolve addresses. When AppleTalk stations boot up, they issue a series of mainly multicast packets (perhaps 20 or more) to resolve network address and local zone information.
AppleTalk Chooser—The AppleTalk Chooser is particularly broadcast intensive. The Chooser enables the user to select shared network services. It uses AppleTalk multicasts to find file servers, printers, and other services. When the user opens the Chooser and selects a type of service (e.g., a printer), the Chooser transmits 45 multicasts at a rate of one packet per second. If left open, the Chooser sends a five-packet burst with a progressively longer delay. If left open for several minutes, the Chooser reaches its maximum delay and transmits a five-packet burst every 270 seconds.
AppleTalk broadcasts—Several other AppleTalk protocols are broadcast intensive—for example, the Name Binding Protocol (used to bind a client to a server), the Router Discovery Protocol (an implementation of RIP), and AutoRemounter (the system initialization service, part of the Mac OS).
In order to constrain broadcast radiation, it is important that flat networks be broken up using routers (or VLAN-enabled switches). Where possible, limit the size of RIP networks to no more than a dozen routers, especially where low-speed WAN links are present. On bridges and switches, if possible use level-2 packet filters to discard multicast traffic from segments where this traffic is not required. Multicast-aware switches (e.g., those supporting GARP, GMRP, and GVRP) can also help control this traffic more efficiently. For further information about the IPX protocol suite, refer to ; for further information about the AppleTalk suite, refer to .
Packet filtering is a traffic management technique widely used in routers, bridges, and switches for reducing intersite or interdomain traffic. Filtering is often employed to keep unwanted traffic off the backbone and to preserve bandwidth on heavily utilized segments. Filters can also be used to implement basic security measures. Note that the term Access Control List (ACL) is effectively synonymous with packet filtering (although ACL is Cisco terminology). Even so, the implementation of packet filters differs widely across vendor platforms; you cannot assume that the same syntax or functionality is available throughout a multivendor network. This can lead to policy implementation issues and inconsistencies. Several smaller vendors have implemented a Cisco-like Command-Line Interface (CLI) to help simplify maintenance (based on the assumption that most internetwork engineers will already be familiar with this CLI).
A packet filter is a set of one or more logical rules that are examined whenever a new packet arrives at an interface. If an incoming packet meets one or more of these rules, then a set of actions are invoked, such as accepting the packet, discarding the packet, logging its occurrence, alerting an NMS, and so on. Depending upon the platform (bridge, switch, router, firewall, etc.), filters may be available at Layer 2, Layer 3, Layer 4, or even application layers. Typical capabilities include the following:
Interface control—The ability to configure individual filters on either all or named LAN and WAN interfaces, by direction (incoming, outgoing), possibly down to subinterface specifications for protocols such as ATM and Frame Relay (e.g., forward if-3:dlci 7).
Level 2 filtering criteria—Source Mac address, destination Mac address, EtherType.
Level 3 filtering criteria—IP protocol type (e.g., OSPF = 89), source or destination IP address/subnet mask/prefix.
Level 4 filtering criteria—Filter on the source or destination UDP or TCP port (e.g., Telnet = TCP port 23).
Special flags—The ability to filter on key protocol flags (e.g., the TCP SYN bit).
Prioritization—The ability to specify an order of precedence for a sequence of filters (in some products this may simply be the order in which filters are created or listed in the filter table).
Actions—Accept, forward [to all interfaces: to named interfaces], deny/discard, log, and so on.
Logical pattern matching—The ability to match specified fields by using content and offset, associated with logical AND, OR, and XOR operators.
Consult the vendor documentation to assess the facilities of the products on your network.
A modern multiprotocol router is likely to support filtering at several layers of the protocol stack, as follows:
Layer 2 type filter—The type filter is extremely useful for a broad-brush approach to filtering. Entire protocol stacks can be discarded with a simple type filter. For example, a filter could be configured to discard DEC LAVc (EtherType 0x6007) traffic on WAN router interfaces. This could be used to prevent VAX cluster broadcasts from being transmitted over wide area circuits. On a Xyplex bridge CLI we could type the following command to invoke this filter:
define filter type 6007 discard all
The following chart lists some other common protocol types (in hexadecimal) that you may wish to include in Layer 2 access lists.
The ability to discard frames by protocol type is a powerful tool but may lack sufficient granularity for some scenarios (e.g., it does not allow you to differentiate TCP services from UDP services).
Service type filter—Traffic filters based on service type are often used as part of a policy-based access control for backbone services. Service filters use access lists applied to particular protocols (e.g., IPX, UDP) or applications (e.g., SMTP, DHCP). For example, assume we have a wide area network with two Novell NetWare LANs, as illustrated in Figure 7.1. Policy mandates that you discard all unnecessary backbone traffic and that LANs of a certain size must offer file and print services locally. To meet these requirements we could filter Service Advertisement Protocol (SAP) messages at the router egress, since there is no reason for services to be advertised remotely (unless remote backup services are also mandated).
Figure 7.1: Policy implementation using SAP packet filtering.
Area and network filter—Area, or network-based, filters are used to police the selective transmission of traffic based on network addressing fields. They can be applied on either incoming or outgoing ports, or both.
Security filter—An example of a simple ACL on a Cisco router is shown in the following code segment. This rule is a basic security filter that denies any TCP connections from network 140.12.00.0 into network 220.127.116.11. Just to be awkward, Cisco uses an inverted mask format (i.e., mask bytes in reverse order) in its syntax:
access-list 90 deny tcp 18.104.22.168 0.0.255.255 22.214.171.124 0.0.0.255 established
The field called established refers to the SYN bit in the TCP header, which is set whenever a TCP station wishes to initiate a new connection as part of a three-way handshake.
There are several limitations of packet filtering, especially when used in large internetworks, including the following:
Statelessness—Packet filtering is a simple-to-use, crude but effective mechanism for controlling traffic; hence, it is widely used. Packet filters depend on static information. This lack of statefulness means that they cannot readily cope with more dynamic protocols such as HTTP
Processing overhead—ACLs tend to be CPU intensive (if not implemented in programmable hardware), since the router must parse this whole list of ACLs every time a new packet arrives. You should, therefore, aim to keep the number of ACLs to a minimum and prioritize ordering according to event frequency.
Lack of management—ACLs need to be implemented across the network in a controlled manner. Without central policy management they can quickly get out of control, become inconsistent, and become a real maintenance problem, especially in large multivendor enterprises.
Planning and transparency—A potentially serious problem with filters is that they need to be carefully planned and require that you have full knowledge of network operations to avoid errors. For example, you may install a filter that discards unnecessary BOOTP broadcasts without first checking where all devices on a segment are booted from. Even though the network could work fine for many months, a subsequent power outage may result in devices failing to reboot, by which time you will have forgotten about the modification. This could be both difficult and time consuming to debug, especially on remote sites.
Policy management systems enable multivendor ACLs to be centrally managed and distributed. In this way packet filters can be used as part of an overall security policy.
Data compression techniques, when applied to networking, are most effective when there are patterns of repetition in data flows. Fortunately, data networking offers two major areas for data compression techniques to attack, as follows:
Protocol repetition—Much of today's internetworking traffic is connectionless, and this leads to great inefficiency in protocols. Within application flows, all the way down the stack there are repetitions on a packet-by-packet basis to ensure successful delivery. For example, TCP headers (port IDs), IP headers (IP addresses), and MAC headers (MAC addresses and types). On a point-to-point link or virtual circuit much of this repetition is redundant since there is only one path for data to follow.
Data repetition—Much of the user data encapsulated in protocol traffic contains repeated patterns that can be compressed. Since text data are derived from a relatively small alphabet, the frequency in which characters occur in a data stream can be used to compress data using techniques such as Huffman encoding . Note that image data is often precompressed before transmission and is therefore hard to compress further (in fact it may result in some data expansion, as described later).
Telnet is quite inefficient; user keystrokes are typically transmitted as padded 64-byte packets. HTTP uses a large number of small packets, resulting in high overhead per unit of data transferred. All of this overhead greatly reduces the real data throughput of expensive wide area links and should be optimized if possible, especially on low-speed links.
The term compression ratio is a relative measure of the amount of data input versus the amount of data output, expressed as follows:
Compression Ratio = Size of Output Stream/Size of Input Stream
The term "compression factor" is the inverse of the compression ratio. Currently the efficiency of the modern compression techniques results in compression ratios of anywhere between 0.5 and 0.02.
A compression ratio greater than one indicates negative compression (i.e., expansion of the original data). This can occur where the original data are either random or have already been compressed (in which case these data will appear almost random, since, by definition, many of the patterns of repetition will have been removed). If you compress data at several levels in the protocol stack then this may actually result in more data being generated than intended (termed data expansion). For example, a JPEG image sent over a WAN link may expand if V.42bis compression is used, due to additional symbol data being produced. Fortunately, LZ-based algorithms are precise enough to allow determination of the worst-case maximum expansion size. For example, Stac LZS, described shortly, specifies a maximum expansion size of 12.5 percent over the original data size.
There are a number of standards-based and proprietary techniques that have historically been used to optimize wide area bandwidth; these include the following:
Lempel-Ziv (LZ) dictionary method derivatives (e.g., V.42bis and Stac)—These techniques compress entire packets by ratios of up to 4:1. Reference  provides details about the Lempel-Ziv compression algorithm. The use of Stac compression over PPP links is defined in .
Run-Length Encoding (RLE) is a widely used technique, sometimes called Data Frame Multiplexing (DFM) or Repetitive Pattern Suppression (RPS). RLE operates on the simple premise that if n occurrences of a data element, d, occur in a data stream, they can be replaced by the pair nd. The number of occurrences, n, is referred to as the run length. In practice an escape character is normally used to indicate that the next sequence is a run length pair (encoded as <escape><count><data>), and to avoid data expansion for short repetitions the minimum run length is 3. The compression factor for RLE can be expressed as:
Sl[S - R (N - 3)]
where S is the length of the string to be compressed, and the string contains R repetitions of average length, N. For example, for a piece of text 2,000 characters long, with 20 repetitions of average length 12, the compression factor is 1.099. RLE is also useful for image compression, since adjacent pixels often have the same characteristics (e.g., color). In general, the less complex the image the better the results achieved.
Microcom Network Protocol (MNP) is a commonly used set of data compression algorithms developed by Microcom, Inc. MNP defines a number of classes. MNP Class 5 is widely used in modems and uses a two-stage process comprising run length encoding followed by adaptive frequency encoding. MNP Class 5 (MNP5) uses a repetition count when three or more identical bytes appear in an input stream. This means that a run length of only three results in four characters being output (i.e., data expansion). The compression factor for MNP5 is similar to that of RLE:
S/[S - R (N - 4)]
MNP Class 7 (MNP7) is a more sophisticated variation of MNP5, which combines run length encoding with a two-dimensional variant of adaptive Huffman encoding. MNP7 can be very efficient, especially where the input data comprise natural language text.
ITU-T fax compression recommendations—Early data compression standards include T1 (Group 1) and T2 (Group 2). These have since been made obsolete by T4 (Group 3) and T6 (Group 4). T4 is designed to work on all fax machines that attach to the PSTN (currently operating at speeds up to 9.6 Kbps). T6 is designed to operate with fax machines connected to the ISDN (with speeds of 64 Kbps). Both methods can provide a compression ratio of 10:1 or higher .
TCP header compression—TCP header compression, described in , is based on a simple premise: Much of the data in the TCP header are repeated during a sustained transfer and thus are unnecessary. By squeezing all but the most essential data, compression can reduce the TCP header from 20 bytes down to 5 bytes (and even 3 bytes in some cases).
Proprietary header compression techniques—A variety of proprietary techniques have been used over the years, including zero fill padding suppression, a proprietary technique to reduce the minimum packet size by removing pad characters and indicating how much packing is required. MAC header compression is a proprietary technique where Ethernet frame headers are reduced from 14 bytes to approximately 2 bytes by setting up partial state information at each end of a WAN link. LAT header compression is a proprietary technique where DEC LAT headers can be reduced from 7 bytes to 2 bytes by setting up partial state information at each end of a WAN link. Many other techniques exist and these are typically vendor dependent.
Proprietary techniques require both routers to be from the same vendor and are largely dying out with the advancement of more effective standards-based technologies. The majority of WAN devices for use on low-speed links (i.e., modems, routers, FRADs, etc.) now support the PPP protocol. Virtually all PPP implementations support TCP header compression, and many support variations of LZ (such as Stac or V.42bis) as a negotiable option (via PPP's Compression Control Protocol (CCP) ). Reference  provides an excellent discussion of generic compression technology.
All dictionary-based compression methods are based on the work of J. Ziv and A. Lempel back in the late 1970s (referred to as LZ77 and LZ78). Stac Electronics developed an enhanced compression algorithm called Stac LZS, based on work published in . Stac LZS is commonly used on low-speed (under 2 Mbps) wide area links as a standard negotiable option with the PPP protocol. The LZS algorithm is designed to compress most file types as efficiently as possible (even string matches as short as two bytes are effectively compressed). It uses the previously examined input data stream as a dictionary, and the encoder uses a sliding window , which comprises a look-ahead buffer and the current dictionary. LZS supports single or multiple compression histories, and with circuit-based technologies such as Frame Relay the latter can significantly improve the compression ratio of a communications link by associating separate compression histories with separate virtual circuits. As indicated previously, the maximum expansion of Stac LZS is specified as 12.5 percent in . This has implications on a point-to-point link for the Maximum Receive Unit (MRU) size, as follows:
An MRU that is 12.5 percent larger than the size of a normal packet, should be negotiated ideally. In this way, packets can always be sent compressed regardless of expansion, since they will never exceed the MRU.
In the case where the MRU has not risen and the expansion plus compression header exceeds the size of the peer's MRU for the link, the PPP packet must be sent uncompressed, and the transmitter must reset the affected compression history.
For further information, the interested reader is referred to .
Compression may be software or hardware based and typically operates on a point-to-point basis, with both devices at either end of a link compressing and uncompressing data. The major router vendors all offer compression as part of their software. Software compression is very CPU intensive, and if you enable compression on several WAN links, you should monitor CPU utilization to ensure that sufficient resources are available (for this reason vendors often limit the number of interfaces and speeds they will support, particularly on lower-specification access routers). Several vendors now offer plug-in modules that perform hardware compression and offload this activity from the main processor. Standalone hardware devices are also available that sit between the router and the WAN link (behind the NTU or CSU/ DSU in the United States ). These devices generally offer good compression ratios (typically in the range of 2:1 to 4:1). Many of the emerging VPN devices also offer data compression, although these devices may be less efficient if data are already encrypted (since this tends to scramble any repetition).
There are a few caveats with all compression techniques when used in real network designs, as follows:
Ensure that the Maximum Receive Unit (MRU) on point-to-point links takes account of the maximum data expansion expected of the compression algorithm used.
Compression generally works best on relatively low-speed WAN links (typically less than 2 Mbps and possibly only up to 64 Kbps with some software implementations). In practice this means that as the link speed increases, the processing power required to keep up ultimately becomes a bottleneck. On very high speed links, compression may actually reduce throughput below the rate at which uncompressed traffic would be forwarded.
Compression is often employed on a point-to-point basis over wide area links (e.g., using PPP). Over a large internetwork several stages of compression and decompression may take place. This is inherently inefficient; for bulk data compression end-to-end solutions are preferable.
Note that some ISPs may be unwilling to enable compression on their CPE equipment, since it offers them no capability to compress that traffic further over more expensive backbone links (and you are effectively not given the opportunity to use more bandwidth than provided). Check with your provider.