Technology Overview | MPLS and Next-Generation Networks: Foundations for NGN and Enterprise Virtualization

To better understand the process of rerouting traffic on nonshortest paths and setting up of MPLS TE, let us now investigate how the technology works. The previous section described elements of MPLS TE as follows:

Network devices must be capable of routing traffic on nonshortest paths.
Network devices must be capable of routing traffic based on delay and bandwidth constraints.
Network devices must be capable of supporting mechanisms that establish the traffic engineering tunnels. Traffic must now be mapped on these TE tunnels.

Based on these requirements, you can see that routers in a given network must have available link bandwidth information for the entire network. Sufficient bandwidth information can be established by sending available link bandwidth information in the routing protocol updates.

IGP Extensions and Distribution of Constraints

Interior Gateway Protocol (IGP) extensions are needed to flood available link bandwidth information. This flooding ensures that all routers participating in the traffic engineering setup understand which links to use to satisfy the constraint criteria. For example, in our topology all links that do not have 4 Gbps of available bandwidth can be removed from the topology to compute the best available path between Seattle and New York. Because the directly connected links via Chicago have only 2 Gbps of available bandwidth, they are removed from the topology and a new path is computed that satisfies the 4 Gbps bandwidth requirement.

The flooding of available bandwidth information happens via IGP (OSPF or IS-IS) link state advertisements (LSA). This implies that IGPs must be extended to use the new LSA type (Opaque LSA) to flood the information. Other routers in the same IGP learn about the links and available bandwidth on each one of them. The flooding of information can be periodic or it can occur when a significant change occurs in the available bandwidth of a link. The significant change is determined by a defined thresholdfor example, when the available bandwidth on a link drops below a threshold value. Thresholds provide a reasonable compromise between information flooded less often versus information flooded every time some bandwidth is taken away from the link.

Signaling of TE Tunnels

After the available bandwidth information is learned by the router, the router computes the set of paths that meet the bandwidth or delay constraint. This computation is often referred to as a constrained-based SPF (CSPF) calculation. After a path is obtained through the constraint-based SPF, a traffic engineering tunnel must be established along that path. To establish the traffic engineering tunnel, a signaling mechanism is required to set up an LSP along the constraint-based path.

During the standardization days of traffic engineering, there was a hot industry debate about which protocol had to be used for traffic engineering. Should LDP be extended to constraint-based routing and admission control or should RSVPa protocol that has admission control capabilitybe extended to do label distribution?

After much debate, the industry settled on RSVP as the de-facto standard for the setup of traffic engineering tunnels. RSVP was previously used for per-flow bandwidth reservation in the IntServ QoS model. Because it had the capability to perform admission control and traversed hop by hop for reservation establishment, it could be easily extended to perform label distribution and explicit routing. Other attributes of RSVP, such as shared-explicit (the capability to share the reservation on common links without double counting bandwidth), allow the setup of make-before-break operations without any double counting of bandwidth.

RSVP uses PATH messages to establish state hop-by-hop and reservation messages to perform admission control and reserve bandwidth along the path. In the case of MPLS TE, the same PATH messages traverse hop by hop along the path computed by the CSPF calculation done by the router. For instance, if the CSPF determines that the path going through San Francisco, Denver, and Dallas meets the 4 Gbps bandwidth requirement from Seattle to New York, the RSVP PATH messages from the Seattle routers travel hop by hop to Denver, Dallas, Atlanta, and to New York, as shown in Figure 8-3.

Figure 8-3. RSVP PATH and MESSAGE Flows

At each hop along the path, a state associated with the TE tunnel is stored. After the PATH messages reach the destination, New York, admission control is performed to see whether the link has the available bandwidth that was requested in the PATH message. If the link has available bandwidth, this bandwidth chunk is removed from the available bandwidth pool, a label is allocated for the TE tunnel, and an RESV message is sent back with the label along the same route on which the PATH message was received. At each hop, the same operation is performed in which the admission control process is performed and a label is allocated for the TE tunnel and chunk of the bandwidth that was removed from the available pool of bandwidth. The RESV message finally reaches the source, or the head end of the TE tunnel. The TE tunnel is not established and labels are programmed into the forwarding hardware that correspond to this tunnel.

If the admission control fails at any hop along the path, an error message is sent back to the head end of the tunnel. The head end router can then pick up a new path and resignal the TE tunnel.

A network operator can decide to bypass the CSPF process, ignore the available bandwidth information on the links, and manually specify an explicit route along which a TE tunnel is placed in the network. This is called explicit routing and is analogous to PVC placement of the ATM world.

Forwarding Packets Through the Network Core

After the TE tunnel is established, traffic must be mapped to this TE tunnel. There are multiple ways of mapping traffic onto TE tunnels. The simplest form of mapping is by the static routing of traffic onto a TE tunnel. The TE tunnel is a special interface onto which traffic can be routed. By static mapping the traffic, specific prefixes can be routed onto a TE tunnel.

Another important method of mapping traffic onto a TE tunnel is by using a policy routing mechanism. With policy routing, the operator can specify a criterion that, when matched, results in packets being mapped to a TE tunnel. For example, voice traffic destined to New York can be mapped to a specific TE tunnel using policy routing while data traffic uses the longer route. Policy routing enables the creation of flexible maps for routing traffic in the network.

Another method of mapping traffic is using a feature called AutoRoute. In AutoRoute, the TE tunnel is treated like a link connecting the head end and tail end of the tunnel and is fed into the IGP database with a metric. The IGP then thinks it has direct connectivity to the end node and routes traffic over that link (TE tunnel). However, the IGP does not advertise LSAs over this link (TE tunnel).

The last most important method of mapping traffic onto TE tunnels is using EXP bits in the MPLS label or DSCP bit in the IP header in addition to the destination address. This is also referred to as class-of-servicebased tunnel selection. In this procedure, at the head end, the traffic is mapped based on a class-of-service (CoS) value toward a given destination. In Figure 8-2, for instance, there could be two TE tunnels: one for carrying voice traffic and another for carrying data traffic. A packet marked with an EXP value 5 can be mapped to TE tunnel 1, and all other packets can be sent on TE tunnel 2. In this manner, traffic engineering can be used to build low-delay and available bandwidth paths for the efficient routing of traffic in the network core. By placing TE tunnels around network choke points (they can be links or nodes, such as Chicago in our example), more traffic can be routed in the network.

Referring to our example, the only reason an extra 2 Gbps of traffic was accepted into the network is because more traffic could be sent along the nonshortest path. One of the providers terms this capability additional bandwidth inventory.

Sequence of Operation

The sequence of steps in establishing a TE tunnel are as follows:

IGP reachability is established using a link state routing protocol such as OSPF or IS-IS.
The operator enables MPLS traffic engineering on the network elements and configures link bandwidth available for traffic engineering. The operator also configures other attributes, such as SRLG (Shared Risk Link Group is described in more detail later), and link affinity.
The MPLS TE configuration triggers the flooding of available link bandwidth information in the IGP. All routers and network elements learn the information and store this in their constraint databases.
The operator configures a TE tunnel only at the head end, specifying a destination or tail end of the tunnel, bandwidth, and a path-option (explicit or dynamic).

- Explicit path-option implies which explicit path a TE tunnel must take from the head end to the tail end. This includes a list of router IDs or hops the tunnel must travel.

- A dynamic path option implies that the path and the list of hops are computed based on available link bandwidth using CSPF, as discussed earlier.

The configuration of the TE tunnel triggers RSVP signaling, and the head end sends a PATH message to the next hop as specified in the path option (it is the explicit node if the path-option is explicit or the computed next hop across the link that had available bandwidth for a dynamic path option).
The midpoints store the tunnel ID and bandwidth information and forward the PATH message to the next downstream hop.
After the PATH message reaches the destination or tail end, the tail end accepts the TE tunnel if it has the bandwidth capacity. Then it allocates a label and sends a RESV message back toward the head end of the TE tunnel to the previous hop.
The RESV message now retraces the same path on which the PATH message was sent. At each hop, admission control is performed to check whether the link has available bandwidth.

- If the admission control process fails on any of the hops, a reservation error is sent back toward the tail, a path error is sent toward the head end, and the tunnel setup process fails.

- If the admission control passes, a local label is allocated and RESV is sent upstream toward the head end.

After the RESV reaches the head end, the head end programs the labels, and the TE tunnel setup is now complete.
The operator must now map different types of traffic onto this TE tunnel via the specified criteria as stated in the previous section.
The traffic can now be forwarded onto the TE tunnel. The label is imposed onto the incoming IP traffic, and the packet is forwarded onto the TE tunnel.
The traffic is label switched in the network core at the midpoints of the TE tunnel.
At the tail end of the TE tunnel, the MPLS label is removed or disposed; the traffic is then natively forwarded to the destination.

TE Tunnel Maintenance

TE tunnels are point-to-point and are analogous to Frame Relay or ATM PVCs. However, they scale far better than PVCs because they carry aggregate traffic from PE to PE rather than individual flows.

Unlike the hard state of ATM PVCs, TE tunnels are soft state and require periodic refresh messages that refresh the state of the tunnel along its path. If the refresh message is not received in a specified time period, the tunnel can be declared inactive and the state cleared and removed. When too many tunnels exist, many refresh messages might need to be processed. A technique called refresh reduction is used to reduce the number of refresh messages, thereby increasing scalability to a large number of tunnels.

The soft-state nature of the RSVP protocol is also helpful when resizing TE tunnels to increase or decrease in bandwidth. While the tunnel is operational, a new path message can be sent with the new bandwidth values toward the tail end. The process of signaling and admission control is the same as the establishment of a new tunnel. After the new bandwidth is accepted, the tunnel attributes are updated and the tunnel can now carry more traffic. If the admission control process fails during the resize operation, the resize request is rejected and the TE tunnel continues to operate at the old bandwidth value. This is done in a manner that results in zero traffic loss.

Inefficiencies can occur over a period of time if the tunnels are frequently set up and torn down with various bandwidth values. Each node might look to reoptimize tunnels independently. Periodic reoptimization can be done at a specified interval, or it can also be triggered by the operator or by a significant change in link or bandwidth parameters. Should a provider require global reoptimization of tunnel placement, an offline tool with sufficient intelligence can provide the tunnel placement information to the network elements by specifying the explicit route to which a TE tunnel is placed.

Many more complicated features are available that allow resizing or reoptimization of TE tunnels. For further reference, they are all discussed at length in the book Traffic Engineering with MPLS, by Eric Osborne and Ajay Simha (Cisco Press, 2002).

Managing MPLS TE tunnels can be done via the command-line interface (CLI) or via sophisticated network management tools such as Cisco's IP Solution CenterTraffic Management (ISC-TM). ISC-TM enables the configuration and monitoring of TE LSPs via a GUI with sophisticated mechanisms to compute paths under additional constraints and provide the explicit placement of both primary and backup tunnels. Additionally, online diagnostics and troubleshooting capabilities exist that provide the capability to check the liveliness of the TE tunnels. This is discussed at more length in Chapter 12.