Connecting to Two ISPs

Problem

You want to set up BGP to support two redundant Internet connections.

Solution

The following configuration shows how to make the basic BGP connections, but it has serious problems that we will show how to fix in other recipes in this chapter:

Router1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router1(config)#interface Serial0
Router1(config-if)#description connection to ISP #1, ASN 65510
Router1(config-if)#ip address 192.168.1.6 255.255.255.252
Router1(config-if)#exit
Router1(config)#interface Serial1
Router1(config-if)#description connection to ISP #2, ASN 65520
Router1(config-if)#ip address 192.168.2.6 255.255.255.252
Router1(config-if)#exit
Router1(config)#interface Ethernet0
Router1(config-if)#description connection to internal network, ASN 65500
Router1(config-if)#ip address 172.18.5.2 255.255.255.0
Router1(config-if)#exit
Router1(config)#router bgp 65500
Router1(config-router)#network 172.18.5.0 mask 255.255.255.0
Router1(config-router)#neighbor 192.168.1.5 remote-as 65510
Router1(config-router)#neighbor 192.168.2.5 remote-as 65520
Router1(config-router)#no synchronization
Router1(config-router)#exit
Router1(config)#end
Router1#

We do not recommend using this configuration as printed for a real Internet connection because it leaves out several key components. A more complete example is shown in Recipe 9.19.

Discussion

Perhaps the most common BGP application involves connecting a single router to two different ISPs to share information about a single /24 IP address range. A setup like this is the simplest way of building a redundant Internet connection. You can improve this redundancy by using two routers, one for each ISP connection, as shown in Recipe 9.5. Figure 9-1 shows the connections used in this recipe.

Figure 9-1. Using two ISPs

This example shows the configuration for the router at the customer site. The customer network uses ASN 65500, while the two ISPs use 65510 and 65520, respectively. Both of these connections are made through serial connections.

This configuration is a simple extension of the one shown in Recipe 9.1. The main difference is that we have set up two different peers, both in different ASs. This router is configured to distribute routing information for its 172.18.5.0/24 segment with both ISPs, and to receive their routing tables.

There are two critical problems with this simple configuration. First, the full Internet routing table is extremely large and consumes a vast amount of memory, so we will probably want to do some filtering. The second problem is that this configuration allows your network to act as a transit path between the two ISPs, which you probably don't want in practice.

The full Internet routing table has roughly 187,000 prefixes, a number that grows over time. Each BGP route entry consumes somewhere between 100 and 200 bytes of memory on the router, and you wouldn't use BGP unless there were at least two ISPs, each of which will likely supply a similar sized routing table, doubling the memory requirement. Then, if the router puts all of these prefixes into its main routing table, as well as the CEF table, you can wind up consuming as much as 1KB of router memory per route prefix. So we don't recommend using a router with less than 100 MB of memory when connecting to the Internet without significant filtering. In fact, Internet backbone routers frequently have hundreds of megabytes of memory.

Here is a typical routing summary taken from a BGP route server:

route-server>show ip route summary
IP routing table name is Default-IP-Routing-Table(0)
Route Source Networks Subnets Overhead Memory (bytes)
connected 0 3 272 480
static 2 9 704 1760
ospf 2 0 2 128 320
 Intra-area: 2 Inter-area: 0 External-1: 0 External-2: 0
 NSSA External-1: 0 NSSA External-2: 0
bgp 65000 115207 70435 11881088 29713940
 External: 185642 Internal: 0 Local: 0
internal 1808 2133440
Total 117017 70449 11882192 31849940
route-server>

As you can see here, this router's routing table consumes most of the over 31 MB of system memory. The same device uses roughly 29 MB of memory just for its BGP table, as you can see from the following output:

route-server>show ip bgp summary
BGP router identifier 10.1.2.5, local AS number 65000
BGP table version is 283729, main routing table version 283729
185761 network entries using 18761861 bytes of memory
3529036 path entries using 169393728 bytes of memory
44183 BGP path attribute entries using 2474248 bytes of memory
40017 BGP AS-PATH entries using 1041908 bytes of memory
4 BGP community entries using 96 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 191671841 total bytes of memory
Dampening enabled. 1637 history paths, 559 dampened paths
185644 received paths for inbound soft reconfiguration
BGP activity 186953/1192 prefixes, 3559638/30597 paths, scan interval 60 secs



route-server>

This represents a 45 percent increase in memory requirements since we wrote the first edition of this book, just a few years ago!

We will discuss BGP route servers in more detail in Recipe 9.19.

Fixing the transit problem is somewhat easier than the route filtering that is necessary to reduce the size of the Internet route tables. To prevent the external networks from using your network for transit, you simply have to ensure that you never pass BGP routing information that you learn from one ISP over to the other ISP. This way neither ISP will know that it can reach the other through your network, so they won't send their traffic this way.

The easiest way to accomplish this is to put a filter on the AS Path. In the following example, we will apply the same filter to both BGP peers. This filter will force our router to advertise only local routes. Any route that already has an entry in its AS Path must have come from somewhere else, so we prevent the router from forwarding these routes. The router will add its own ASN to the AS Path only after doing this filter processing, so the local routes will still be sent out:

Router1#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router1(config)#ip as-path access-list 15 permit ^$
Router1(config)#router bgp 65500
Router1(config-router)#network 172.18.5.0 mask 255.255.255.0
Router1(config-router)#neighbor 192.168.1.5 remote-as 65510
Router1(config-router)#neighbor 192.168.1.5 filter-list 15 out
Router1(config-router)#neighbor 192.168.2.5 remote-as 65520
Router1(config-router)#neighbor 192.168.2.5 filter-list 15 out
Router1(config-router)#exit
Router1(config)#end
Router1#

Please refer to Recipe 9.10 for more information about how to use AS filters.

Before you can solve the problem of the large size of the Internet routing tables, you have to make some decisions about how you want your Internet connections to work. Specifically, you might want one of these ISPs to be the primary and the other the backup for all traffic. Alternatively, you might want to just use the first ISP to handle traffic for its directly connected customers, while the second ISP handles everything else. Or you could opt to have load sharing between the two ISPs. These options are discussed in Recipes 9.7, 9.8, and 9.19.

You should also think about whether you want to control which path inbound traffic uses to reach you. If one of your ISP links has a large usage charge, you might prefer to force all of the inbound traffic through the other link. This can be slightly tricky because you don't directly control the ISP routers. But you can control how your routing information looks to the ISP. Techniques for doing this are discussed in Recipes 9.13 and 9.19.