Load-Balancing BGP Traffic

Problem

A customer is multihomed to two different routers in your point of presence (POP). Instead of having BGP send all traffic across one of the links, which is the default behavior, you want to load-balance the EBGP traffic from the customer across the two links.

Solution

To enable load balancing across multiple EBGP peerings, configure the BGP group to use multipath:

	[edit protocols bgp group external-group ]
	aviva@Router1# set type external 
	aviva@Router1# set peer-as 65505 
	aviva@Router1# set multipath 
	aviva@Router1# set neighbor 192.168.1.1 
	aviva@Router1# set neighbor 192.168.1.3 

For the load balancing to happen, configure a load-balancing policy:

	[edit policy-options]
	aviva@router1# set policy-statement LoadBalance from route-filter 192.168.1.0/24
	orlonger
	aviva@Router1# set policy-statement LoadBalance then load-balance per-packet

	[edit routing-options]
	aviva@Router1# set forwarding-table export LoadBalance

Discussion

Multihomed connections from a customers network to the ISPs POP provide redundant Internet connectivity. If one of the links goes downfor example, because a fiber was cutthe second connection provides backup. If the paths from both connections are equal-cost, the default BGP behavior is to select the single best route to a destination. The result is that BGP uses only one of the links to forward traffic. As long as both links are up, the customer wants to use both, spreading the traffic between them to increase the bandwidth available for sending traffic to the Internet. Multipath BGP allows the traffic to be load-balanced across the two links.

Figure 13-5 shows a network topology with redundant peerings. Router1 in AS 65500 has EBGP peerings with two routers in AS 65505. If Router1 receives two paths from Router3 and Router4 for a prefix in the remote AS, and if all route parameters are the same except for the router ID, BGP will pick the path with the lower router ID. The result is that only one of the paths (either through Router3 or Router4) will be used for traffic from Router1 to a customer in AS 65505.

Figure 13-5. Multiple BGP peerings to another AS

Multipath BGP overrides the default BGP behavior and allows both links to be used for forwarding traffic. The result is equal-cost load balancing between the two links, which allows Router1 to send traffic both to Router3 and Router4.

Multipath BGP works by allowing a path other than the best one to be placed into the forwarding table to be used as an alternative. When BGP evaluates multiple routes to determine the best path to a prefix, if all the conditions are the same except for the router ID, which is the last route property that BGP considers as a tie-breaker, BGP installs all the equal-cost routes into the forwarding table (see the Introduction to Chapter 8).

Another restriction for multipath BGP is that both neighboring routers must be in the same AS. This is necessary because BGP can advertise only a single path, and it makes assumptions regarding the forwarding path matching the routing information.

This recipe creates an EBGP group external-group on Router1, in AS 65500, with two neighbors in AS 65505 (configured with the set peer-as 65505 command) (see Figure 13-5). The set multipath command configures BGP multipath. However, for BGP multipath to work, you must also set up a load-balancing routing policy that is applied to the routers forwarding table.

Looking in the routing table, you see the routes learned from both routers:

	aviva@Router1> show route 172.16.14.0/24
	inet.0: 176580 destinations, 1421698 routes (176272 active, 1 holddown, 79688 hidden)
	+ = Active Route, - = Last Active, * = Both
	172.16.14.0/24 *[BGP/170] 3w4d 05:01:06, MED 0, localpref 100
	 AS path: (65505) 65510 65520 ?
	 > to 192.168.1.1 via ge-0/1/0.0
	 to 192.168.1.3 via ge-1/1/0.0
	 [BGP/170] 3w4d 05:01:15, MED 0, localpref 100
	 AS path: (65505) 65510 65520 ?
	 > to 192.168.1.3 via ge-1/1/0.0

The routing table shows two paths to this destination, the first goes through Router3 (192.168.1.1) and the ge-0/1/0 interface, and the second goes through Router4 (192.168.1.3) and the ge-1/1/0 interface. The first path is active (marked with an asterisk) because it was learned from the router with the lower router ID (192.168.1.1 is lower than 192.168.1.3). The second path is the alternate next hop. The first path lists two next hops:

	> to 192.168.1.1 via ge-0/1/0.0
	 to 192.168.1.3 via ge-1/1/0.0

The first hop is learned from 192.168.1.1, and the second is learned from 192.168.1.3. The second next hop has been copied from the second path up to the first one.

The detail version of the show route command gives more information about the two paths:

	aviva@Router1> show route 172.16.14.0/24 detail
	inet.0: 176576 destinations, 1426247 routes (176265 active, 4 holddown, 79690 hidden)
	172.16.14.0/24 (12 entries, 1 announced)
	 *BGP Preference: 170/-101
	 Next-hop reference count: 14184
	 Source: 192.168.1.1
	 Next hop: 192.168.1.1 via ge-0/1/0.0, selected
	 Next hop: 192.168.1.3 via ge-1/1/0.0
	 Protocol next hop: 192.168.1.1
	 Indirect next hop: ac0a200 524714
	 Protocol next hop: 192.168.1.3
	 Indirect next hop: 24479700 524586
	 State: <Active Int Ext>
	 Local AS: 65000 Peer AS: 65505
	 Age: 3w4d 5:08:45 Metric: 0 Metric2: 0
	 Task: 
BGP_65505.192.168.1.1+3718
	 Announcement bits (6): 0-KRT 1-RT 5-KRT 6-BGP.0.0.0.0+179 7-Resolve
	tree 3 8-Resolve tree 4
	 AS path: (65505) 65510 65520 ? (Atomic) Aggregator: 2468 192.168.246.3
	 Communities: 65500:390 65500:2400 65505:3400
	 Localpref: 100
	 Router ID: 192.168.16.5
	 BGP Preference: 170/-101
	 Next-hop reference count: 12709
	 Source: 192.168.1.3
	 Next hop: 192.168.1.3 via ge-1/1/0.0, selected
	 Protocol next hop: 192.168.1.3
	 Indirect next hop: 24479700 524586
	 State: 
	 Inactive reason: Router ID
	 Local AS: 65000 Peer AS: 65505
	 Age: 3w4d 5:08:54 Metric: 0 Metric2: 0
	 Task: BGP_65505.192.168.1.3+179
	 AS path: (65505) 65510 65520 ? (Atomic) Aggregator: 2468 192.168.246.3
	 Communities: 65500:390 65500:2400 65505:3400
	 Localpref: 100
	 Router ID: 192.168.1.6

The Source field shows the router from which the path was learned. The first path is learned from Router3 (192.168.1.1) and the second is from Router4 (192.168.1.3). The Next hop field repeats the information in the standard show route output, showing the next-hop IP address and the interface toward the destination.

The Protocol next hop field is the next-hop information that BGP learned from its peers. Because BGP learns only about routers in the network, not in the interface, this IP address is listed without a corresponding interface on the local router. The indirect next hop is an internal RPD address of the next hop. The second number in this field is the kernels index of the indirect next hop. You will see this index when you look at the contents of the forwarding table.

The State field gives additional information about the path. For the first path to 172.168.14.0/24, this field shows that the path is active. For the second path, this field lists the path as an alternate (NotBest) and indicates that it was not chosen as the active path because it does not have the lowest router ID:

	State: 
	Inactive reason: Router ID

Both next hops are installed in the Routing Engines forwarding table:

	aviva@Router1> show route forwarding-table destination 172.16.14.0/24
	Routing table: inet
	Internet:
	Destination Type RtRef Next hop Type Index NhRef Netif
	172.16.14.0/24 user 0 ulst 526570 3545
	 indr 524714 523
	 192.168.1.1 ucst 336 17 ge-0/1/0.0
	 indr 524586 1864
	 192.168.1.3 ucst 340 13 ge-1/1/0.0

This forwarding table is then copied to the PFEs forwarding table:

	aviva@Router1> show pfe route ip prefix 172.16.14.0/24
	Slot 0
	IPv4 Route Table 0, default.0, 0x0:
	Destination NH IP Addr Type NH ID Interface
	--------------------------------- --------------- -------- ----- ---------
	172.16.14/24 Unilist 526570 ge-0/1/0.0
	Slot 1
	IPv4 Route Table 0, default.0, 0x0:
	Destination NH IP Addr Type NH ID Interface
	--------------------------------- --------------- -------- ----- ---------
	172.16.14/24 Unilist 526570 ge-0/1/0.0

Both the routes are of type Unilist, which means that they are in the list of unicast next hops maintained by the Packet Forwarding Engine. The NH ID field shows the kernels index of the indirect next hop, which matches what you saw in the show route detail output.

Even though multipath BGP selects multiple paths for forwarding and installs two paths in the forwarding table, BGP advertises only one path to its peers, which is the best path toward the destination. This is the same path that BGP would advertise if BGP load balancing were not configured.

You can also use multipath BGP across IBGP peerings. An additional restriction is that the IGP metric distance to the two IBGP peers must be identical. Ascenario for doing this might be to load-balance traffic across redundant paths within a POP.

See Also

Recipes 8.9 and 9.1