Routing and the Internet

The Internet is a gigantic collection of IP-based networks that are linked to form a ubiquitous global internetwork. Getting past the dizzying scale of the Internet, you can appreciate that the mechanics of routing on the Internet do not vary substantially from the mechanics of routing you encounter in WANs. In fact, the concept of neighbors exchanging routing table information remains remarkably consistent. The routing protocols used are different (they need to be to achieve global scalability), but the fundamentals remain consistent. Neighboring routers continue to exchange routing information within a network. And, at the periphery of a network where two different networks connect to form an internetwork, the neighboring routers actually exchange routing information on behalf of all the routers in their respective networks.

Figure 11-6 builds on the previous example to show how the interconnection between two networks aggregates routing information for each of those networks for sharing externally. The iterative neighbor-to-neighbor communication process occurs both internally (in each of the two networks) and externally (between the two networks via the neighboring routers of each network that form the gateway).

Figure 11-6. Neighboring Routers in Different Networks Exchange Routing Information

NOTE

It is quite common for different routing protocols to be used for internal versus external communication. In fact, routing protocols tend to be classified as being either internal or external protocols. The acronyms IGP and EGP stand for internal gateway protocol and external gateway protocol, respectively. In very general terms, distance vector and link-state routing protocols tend to be IGPs, whereas policy-based routing protocols tend to be EGPs.

The notion of sharing routing information between networks takes on a whole new meaning in the context of the Internet. The Internet is intriguing because it is a collaborative yet highly competitive environment. If you are thinking that you can't collaborate with a competitor, you aren't far from the truth. In many instances, you wouldn't want to incur the business risk associated with that activity.

On the Internet, however, collaboration is imperative. No single ISP can operate alone, simply because not one contains all the Internet's end users and content-bearing hosts. Instead, the Internet's base of end users and destinations is scattered across thousands of ISPs around the world. In this environment, competitors must share routing information. Otherwise, the Internet's global reachability gets compromised. More pragmatically, an ISP that doesn't interconnect with other ISPs offers limited value to its customers and probably can't legitimately call itself an ISP.

Please don't get the impression that there is cheerful camaraderie between ISPs. In any commodity-based industry, competition is ruthless. The inter-ISP competition on the Internet, however, takes on an interesting and subtle form that is best described via the accepted tiered designations.

Tiers of Service Providers

Not all ISPs are created equal. Time and competitive market forces have helped the ISP industry coalesce into functional niches as each firm strives to differentiate itself from the competition. Today, there are three tiers of providers:

• Tier 1 Tier 1 ISPs are facilities-based. That is, they own their own operating infrastructure, including the telecommunications circuits that interconnect their routers. Typically, a Tier 1 ISP is large enough to be global, but some just span a continent.

  By virtue of its size and reach, a Tier 1 ISP enjoys a certain degree of leverage over its peers (other ISPs). Simply stated, it has a larger market share of end users and/or high-value destination sites than its smaller competitors.

• Tier 2 A second-tier service provider might or might not have a national backbone, but it is not facilities-based. Second-tier providers tend to develop specialized niches and enjoy a solid base of high-end customers. Such customers are usually businesses rather than individuals and require high-bandwidth connections.

• Tier 3 A third-tier service provider is a small, regional on-ramp to the Internet. Such an ISP might cover a specific geographic region or specialize in certain access technologies (such as any of the myriad flavors of Digital Subscriber Line [DSL]), dialup, or other telecommunications technologies.

How you, as an end user or customer, categorize an ISP is far less important than how ISPs categorize each other. ISPs must interconnect with each other in ways that make economic sense. How ISPs categorize each other can have a tremendous impact on how the costs of interconnectivity between two ISPs get divided. Thus, far from being a semantic discussion, the relationship between ISPs has some very real impacts in terms of economics and service levels to customers.

In very simple terms, no one service provider enjoys a monopoly on the Internet user community. End users and destination websites are scattered across thousands of ISPs around the world. Thus, the ubiquity of the Internet absolutely requires interconnectivity between all ISPs. But ISPs are not created equal, as you have seen in this section.

Tiers Translate into Hierarchy

Reading through the description of the various tiers of ISPs, you might have gotten a sense of the natural leverage that some ISPs enjoy relative to their peers by virtue of their size. Indeed, the more end users and/or destinations that are connected to an ISP's network, the greater the benefits of establishing a connection directly to that ISP. So, all ISPs are motivated to try and establish a direct link to a Tier 1 ISP.

The converse is also true. The smaller an ISP's customer base, the less reason other ISPs have to establish a direct connection with it. Thus, a Tier 1 ISP has almost no reason to invest in the hardware and facilities required to establish a connection with a Tier 3 ISP. The important concept here is that the negotiating leverage established by the relative sizes of ISPs translates directly into a topological hierarchy within the Internet. This hierarchy, in turn, affects how routing information gets propagated throughout the Internet. The terms used to describe one's relative position in this hierarchy are upstream and downstream. What flows up and down those virtual streams is not waterit's routing information and IP packets filled with data.

Figure 11-7 demonstrates the concept of upstream and downstream ISPs in the Internet. Although these are highly relative terms, they do make sense when viewed from the perspective of the tiered classification of ISPs. Thus, larger ISPs (such as Tier 1) form the backbone of the Internet. Tier 1 ISPs are, for all intents and purposes, the Internet's backbone. Peering connections between these large providers ensure ubiquitous connectivity for all their downstream customers, including smaller ISPs. The smaller ISPs, such as the Tiers 2 and 3, tend to form the periphery or edge of the Internet.

Figure 11-7. Upstream Versus Downstream Neighbors

NOTE

ISPs at the edge of the Internet are sometimes called on-ramp providers. This moniker implies that this is where you enter the Internet. Please don't misconstrue this to mean that you can buy access only from a Tier 3 service provider. Virtually any ISP would be willing to sell you access services. The differentiating factors are price and robustness of interconnectivity with other service provider networks (as indicated by the breadth of peering and transit relationships).

When viewed from the perspective of network address information, the notion of upstream versus downstream takes on greater significance. Essentially, a Tier 3 on-ramp advertises a limited number of network addresses to its upstream neighbors, yet it relies on those upstream neighbors for access to virtually every other network address. For the sake of example, assume that the Tier 3 ISP has a /16 address space (10.2.0.0/16, to be precise). This is the only network address it would advertise to other ISP networks it interconnects with. That ISP would need to be able to deliver its customers' packets to virtually any address on the Internet. Thus, it needs to obtain a full Internet routing table from a much larger ISP. This is illustrated in Figure 11-8.

Figure 11-8. Advertisement of Network Addresses Upstream Versus Downstream

It is important to recognize that a very real danger lurks in the interconnection of different networks. The very act of interconnection means that network address information is shared between networks operated by different entitiescompetitors. Each trusts the other to share only valid routing information and to operate a stable network that won't generate unnecessary routing updates. Either scenario can adversely affect neighboring networks. This creates the potential for snobbishness on the part of some ISPs when it comes to evaluating other ISPs for establishing interconnection. The result is an interesting trio of interconnection types on the Internet as dictated by the relative leverage of each tier of service provider. The three interconnection types are public peering, private peering, and purchased transit services.

Peering Versus Transit Services

The Internet is a bit of a misnomer in that it leads you to think it is a single network. As we have discussed throughout this chapter, it is really a patchwork of thousands of IP-based networks. IP and its addressing system let any endpoint connected to the Internet access any other endpoint. But that any-to-any connectivity assumes that physical connectivity exists between those two endpoints. Such connectivity between ISP networks can take one of two basic forms:

• Peering

• Transit

Cumulatively, peering and transit are known as egress among ISPs. Egress is a fancy word for exit. In the case of an ISP network, connectivity to other ISP networks extends the reach of an ISP network via the sharing of routing information. But, as you saw in the preceding section, there are great differences between ISPs. For instance, a regional Tier 3 ISP might have a few hundred end users but almost no destinations on its network. Such a provider would benefit far more than a Tier 1 ISP from the interconnection of their two networks. This concept of equality versus inequality forms the basis of differentiating between the two forms of egress.

Peering

Peering is the term applied to a voluntary interconnection between two ISP networks. The ISPs recognize that there are mutual benefits to the interconnection. In theory, peering agreements (both public and private) limit the traffic passing between two networks to just network-to-network traffic. Traffic destined for other networks shouldn't be dumped on a peer's network. Doing so can be a violation of trust and is contrary to the notion of being peers. This type of peered interconnectivity is achieved by sharing just the locally connected network addresses of each network. This limited exchange of network address information precludes packets addressed to other network addresses from using the peering connection. Figure 11-6 demonstrated how peers exchange only their own routing information.

Ostensibly, a peering connection is mutually beneficial to both parties. Consequently, peering partners typically agree to share the costs of their interconnection. Each must invest in the hardware necessary to support the physical interconnection (the router interface port and any equipment that might be required to support the transmission facility). If a leased telecommunications circuit is required, they might opt to share the costs of that circuit. Alternatively, an informal agreement might be made to simply "take turns." Company A buys the first circuit and pays for it entirely. If and when that circuit gets full, Company B springs for the cost of the next circuit. Regardless of how these mutual costs get apportioned, the connection benefits both companies. In other words, they are peers. Their private interconnection is known as a private peering connection.

Public peering is an interconnection between peers at a specialized facility that is specifically designed to enable large-scale peering among ISPs. Such facilities are known by various names, including telco hotels and exchanges. The facility's operator creates a data center-like environment complete with physical security, climate control, a "house" network, and electric power that is backed up by both Uninterruptible Power Supplies (UPSs) and a motor-driven generator to ensure that there is no disruption in electricity. Space is then leased to ISPs, and they install routers and other networking gear there. That networking equipment is used to extend the ISP's network into that peering facility and to connect to its operator's house network.

Public peering facilities offer smaller ISPs the opportunity to establish relatively limited-bandwidth connections to a large number of other service providers. The opportunity is borne of proximity to the other ISPs courtesy of the house network.

Although it should be obvious, I'll point out that peering is beneficial because it theoretically represents the shortest possible path between source and destination devices. As a result, peering creates the opportunity to improve network performance through the Internet by allowing ISPs to deliver packets directly to their destination networks.

Transit

Having discussed the importance, if not outright necessity, of interconnecting ISP networks, we need to acknowledge the inadequacies of peering. Public peering requires an ISP to extend its network to a public peering facility. That can require lengthyand expensivetelecommunications links. Small, regional ISPs that are not located near any public peering facilities might find themselves unable to afford a connection. Such ISPs are equally unlikely to be able to negotiate private peering agreements with any of the Tier 1 or Tier 2 ISPs. The good news is that this doesn't mean they are out of business. For them, there is always the option of purchasing transit services.

Virtually every ISP is willing to sell dedicated access to its network via leased telecommunications facilities (also known as private lines) to anyone pwho can afford it. If that customer happens to be another, albeit smaller, ISP, the service is called transit. The Tier 1 ISP charges the Tier 2 or Tier 3 ISP for the full cost of physically connecting to its network, as well as a variable cost for each megabit of traffic it has to carry for that customer ISP. Included in this traffic quantity might be some inbound traffic destined for the Tier 3 ISP's destinations.

In theory, the Tier 3 ISP should not be charged for these packets. Basis for that statement exists in the world of telecommunications. Telecom is an industry in which the owners of the networks must interconnect to ensure ubiquitous service. Costs are charged back to the customer per minute of use. But seldom does a call begin and end within just one carrier network. That means the call's costs are spread across at least two carriers, and the revenues must be shared as well. To facilitate this charge-back, minutes of use must be tracked between the carriers. The carriers then have to "settle up" their bills with each other.

The Internet isn't quite so mature a business environment. Costs are not as uniformly tracked or recovered. All ISPs must interconnect with other ISPs. In the Internet model, an ISP is either a peer (in which case it is exempt from settlement charges) or a transit customer (in which case it is exempt from settlement charges but must bear the entire cost of the connection and service).