7.3 SONETSDH Interfaces

7.3 SONET/SDH Interfaces

Although SONET has actually been around for some time in the nonconcatenated form, POS is a relatively new implementation. Due to its extremely high bandwidth capabilities, coupled with its very low overhead, POS has been making very fast in- roads in the Datacom world.

Juniper Networks supports the full range of current POS speeds, from OC-3 to OC-192. OC-192, at approximately 10Gbps, is at the time of writing one of the fastest commercially available interfaces, and Juniper Networks is one of the few companies that offers it. These high-bandwidth interfaces allow Internet providers to drastically increase the capability of their core infrastructure to meet the high use demands of Internet users.

SONET and SDH are basically two sides of the same coin. SONET is primarily a U.S. standard, while most of the rest of the world uses SDH. For configuration purposes, and unless otherwise specified, the term SONET in this section will represent both SONET and SDH.

SONET is an optical timing protocol that works at the physical layer of the OSI model. An encapsulation type for the data link layer needs to be defined and configured to enable the upper layers to communicate. Juniper Networks offers three encapsulation types for physical SONET interfaces: PPP, Cisco-HDLC, and Frame Relay. PPP is the Juniper Networks SONET default and does not need to be configured explicitly. Because Cisco-HDLC and PPP are both point-to-point protocols, if they are configured on the physical interface, there can be only one logical unit configured. If the encapsulation is Frame Relay, a multiaccess protocol, more than one unit can be configured on a physical interface.

7.3.1 Configuring Framing on Optical Interfaces

The Juniper Networks optical interfaces are framed in either the SONET or SDH mode. Framing is the process of sequencing the bits of data into standard patterns. SONET framing is described in more detail in Chapter 2. The commands to change the framing type are entered at the [edit chassis] hierarchy level. In JUNOS software, you configure SONET to turn on the SDH features. The PIC is called SONET, but is not limited to SONET framing. In the following sample output, the choices for framing SONET or SDH are shown:

 [edit]  lab@Boston# edit chassis fpc 2 pic 2 [edit chassis fpc 2 pic 2] lab@Boston# set ? Possible completions: + apply-groups         Groups from which to inherit configuration data  framing              Framing mode  no-concatenate       Don't concatenate channels   vtmapping            VT mapping mode [edit chassis fpc 2 pic 2] lab@Boston# set framing ? Possible completions:   sdh                  SONET/SDH mode   sonet                SONET mode [edit chassis fpc 2 pic 2] lab@Boston# 

7.3.2 PPP Configuration

SONET characteristics for the interface are configured at the physical interface hierarchy level. This is where such parameters as clocking, keepalive intervals, and physical interface encapsulation are configured. In addition to those parameters listed above, there is a subhierarchy called sonet-options , where SONET parameters, such as FCS, payload scrambling , and header byte configuration options, can be configured.

POS interfaces on Juniper Networks routers default to the following settings: payload scrambling enabled, 16-bit FCS, 10-second interval keepalives , PPP encapsulation, internal clocking, and 4,474-byte MTU. A default physical SONET interface looks as follows :

 lab@newyork> show interfaces so-0/3/0  Physical interface: so-0/3/0, Enabled, Physical link is Up   Interface index: 16, SNMP ifIndex: 25   Link-level type:  PPP  , MTU:  4474  , Clocking:  Internal, SONET mode  , Speed: OC12,   Loopback: None, FCS: 16, Payload scrambler:  Enabled  Device flags   : Present Running   Interface flags: Point-To-Point SNMP-Traps   Link flags     :  Keepalives   Keepalive settings: Interval 10 seconds, Up-count 1, Down-count 3  Keepalive: Input: 27 (00:00:03 ago), Output: 27 (00:00:08 ago)   LCP state: Opened   NCP state: inet: Opened, iso: Not-configured, mpls: Not-configured   Input rate     : 0 bps (0 pps)   Output rate    : 0 bps (0 pps)   SONET alarms   : None   SONET defects  : None 

Notice that some of the important physical parameters are highlighted. These include some of the items that must be checked when connecting a Juniper Networks POS interface to that of another vendor. FCS is also referred to as a CRC. The FCS value is either 16- or 32-bit. The following shows where you can configure SONET physical characteristics:

 [edit interfaces so-0/3/0]  lab@chicago# set ? Possible completions:   accounting-profile   Accounting profile name + apply-groups         Groups from which to inherit configuration data  clocking             Interface clock source  dce                  Respond to Frame Relay status enquiry messages   description          Text description of interface   disable              Disable this interface   encapsulation        Physical link-layer encapsulation > hold-time            Hold time for link up and link down  > keepalives           Send/demand keepalive messages  > lmi                  Local Management Interface settings  mtu                  Maximum transmit packet size (256..9192)   no-keepalives        Don't send/demand keepalive messages  no-traps             Don't enable SNMP notifications on state changes > receive-bucket       Set receive bucket parameters >  sonet-options        SONET interface-specific options  > traceoptions         Interface trace options > transmit-bucket      Set transmit bucket parameters   traps                Enable SNMP notifications on state changes > unit                 Logical interface [edit interfaces so-0/3/0] lab@chicago# edit sonet-options [edit interfaces so-0/3/0 sonet-options] lab@chicago# set ? Possible completions:   aggregate            Join a sonet aggregate + apply-groups         Groups from which to inherit configuration data > aps                  Automatic Protect Switching >  bytes                Set SONET header bytes   fcs                  Frame checksum  loopback             Loopback mode   no-payload-scrambler  Don't enable payload scrambling   no-z0-increment      Don't increment Z0 in SDH mode   path-trace           Path trace string > payload-scrambler    Enable payload scrambling   rfc-2615             RFC 2615 compliance   z0-increment         Increment Z0 in SDH mode [edit interfaces so-0/3/0 sonet-options] lab@chicago# 

Now we will look at the three encapsulation types (with their CCC counterparts) possible under the SONET interface encapsulation. The following output is an example of the options for the set encapsulation command under a SONET interface:

 [edit interfaces so-0/3/0]  lab@chicago# set encapsulation ? Possible completions:   cisco-hdlc           Cisco-compatible HDLC framing   cisco-hdlc-ccc       Cisco-compatible HDLC framing for a cross connection   frame-relay          Frame Relay encapsulation   frame-relay-ccc      Frame Relay encapsulation for cross connection   ppp                  Serial PPP device   ppp-ccc              Serial PPP device for a cross connection [edit interfaces so-0/3/0] lab@chicago# 

In Figure 7-9, router Chicago and router New York are connected by an OC-12 POS link. The FCS is set to a value of 32 and the New York interface will be used for clocking, which means that router Chicago has to be configured to use an external clocking source on the link (i.e., router New York) to ensure that their timing is synchronized. Since POS provides internal clocking by default, we only have to alter clocking on Chicago to recognize the external clocking source of New York.

Figure 7-9. SONET PPP Configuration

In the example below, router Chicago will have clocking configured to external to use the timing from router New York, the FCS will be changed to 32, and an IP address will be applied. Since Juniper Networks routers default to PPP for SONET, encapsulation does not need to be set.

 [edit interfaces so-0/3/0]  lab@chicago# set clocking external [edit interfaces so-0/3/0] lab@chicago# set sonet-options fcs 32 [edit interfaces so-0/3/0] lab@chicago# set unit 0 family inet address 192.168.40.1/30 

Router New York is configured next with the same FCS options and the IP address of the link.

 [edit interfaces]  lab@newyork# edit so-0/3/0 [edit interfaces so-0/3/0] lab@newyork# set sonet-options fcs 32 [edit interfaces so-0/3/0] lab@newyork# set unit 0 family inet address 192.168.40.2/30 

The configuration is checked by successfully pinging the other interface.

 lab@newyork> ping 192.168.40.1  PING 192.168.40.1 (192.168.40.1): 56 data bytes 64 bytes from 192.168.40.1: icmp_seq=0 ttl=255 time=0.860 ms 64 bytes from 192.168.40.1: icmp_seq=1 ttl=255 time=0.748 ms 64 bytes from 192.168.40.1: icmp_seq=2 ttl=255 time=0.765 ms ^C --- 192.168.40.1 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.748/0.791/0.860/0.049 ms lab@newyork> 

Now, look at the so-0/3/0 interface for router Chicago. The Link Control Protocol and Network Control Protocol (family inet) of PPP are in the Opened state.

 lab@chicago> show interfaces so-0/3/0  Physical interface: so-0/3/0, Enabled, Physical link is Up   Interface index: 18, SNMP ifIndex: 31   Link-level type: PPP, MTU: 4474, Clocking:  External  , SONET mode, Speed: OC12,   Loopback: None, FCS: 32, Payload scrambler: Enabled   Device flags   : Present Running   Interface flags: Point-To-Point SNMP-Traps   Link flags     : Keepalives   Keepalive settings: Interval 10 seconds, Up-count 1, Down-count 3   Keepalive: Input: 82 (00:00:09 ago), Output: 82 (00:00:04 ago)  LCP state: Opened   NCP state: inet: Opened  , iso: Not-configured, mpls: Not-configured   Input rate     : 0 bps (0 pps)   Output rate    : 0 bps (0 pps)   SONET alarms   : None   SONET defects  : None   Logical interface so-0/3/0.0 (Index 12) (SNMP ifIndex 26)     Flags: Point-To-Point SNMP-Traps Encapsulation: PPP     Protocol inet, MTU: 4470, Flags: None       Addresses, Flags: Is-Preferred Is-Primary         Destination: 192.168.40.0/30, Local: 192.168.40.1 lab@chicago> 

The example above shows a basic single SONET PPP connection configuration. Since PPP is the default encapsulation, it did not have to be specified. One side of the link (New York) provided the timing by designating router Chicago's interface timing as external.

7.3.3 Frame Relay Configuration

SONET interfaces can also use Frame Relay encapsulation. There are a few differences when configuring Frame Relay on an interface. Multiple logical units can be configured either point-to-point or multipoint. Keepalives are disabled on both interfaces so that the local management interface (LMI) packets can be sent instead. LMI packets are used to maintain the Frame Relay PVCs. Juniper Networks routers support the two common LMI types: ANSI and ITU. A DLCI is configured on the logical units to differentiate the origin of the data.

In Figure 7-10, routers Chicago and New York have a point-to-point Frame Relay connection between them using DLCI 103. First New York is configured. Under the physical SONET interface, the encapsulation is set to Frame Relay. Afterwards, the unit parameters and the IP address are configured. For back-to-back Frame Relay connections, either disable the sending of keepalives on both sides of the connection, or configure one side of the connection as a DTE (the default JUNOS configuration) and the other as a DCE.

Figure 7-10. SONET Frame Relay Configuration

 [edit interfaces so-0/3/0]  lab@newyork# set encapsulation frame-relay [edit interfaces so-0/3/0] lab@newyork# set no-keepalives [edit interfaces so-0/3/0] lab@newyork# edit unit 103 [edit interfaces so-0/3/0 unit 103] lab@newyork# set point-to-point [edit interfaces so-0/3/0 unit 103] lab@newyork# set dlci 103 [edit interfaces so-0/3/0 unit 103] lab@newyork# set family inet address 192.168.50.10/30 

The configuration looks as follows in a show command for the entire SONET interface:

 [edit interfaces so-0/3/0]  lab@newyork# show no-keepalives; encapsulation frame-relay; unit 103 {     point-to-point;     dlci 103;     family inet {         address 192.168.50.10/30;     } } [edit interfaces so-0/3/0] lab@newyork# 

Next, router Chicago is configured with the same SONET parameters as New York, except that the clocking is assigned as external for Chicago.

 [edit interfaces so-0/3/0]  lab@chicago# set encapsulation frame-relay [edit interfaces so-0/3/0] lab@chicago# set no-keepalives [edit interfaces so-0/3/0] lab@chicago# set clocking external [edit interfaces so-0/3/0] lab@chicago# edit unit 103 [edit interfaces so-0/3/0 unit 103] lab@chicago# set point-to-point [edit interfaces so-0/3/0 unit 103] lab@chicago# set dlci 103 [edit interfaces so-0/3/0 unit 103] lab@chicago# set family inet address 192.168.50.9/30 

New York's SONET interface configuration is as follows:

 [edit interfaces so-0/3/0]  lab@chicago# show no-keepalives; clocking external; encapsulation frame-relay; unit 103 {     point-to-point;     dlci 103;     family inet {         address 192.168.50.9/30;     } } 

To test the configuration of the link between the two routers, ping the IP address of the directly connected router, in this case New York's IP address.

 [edit interfaces so-0/3/0]  lab@chicago# run ping 192.168.50.10 PING 192.168.50.10 (192.168.50.10): 56 data bytes 64 bytes from 192.168.50.10: icmp_seq=0 ttl=255 time=0.953 ms 64 bytes from 192.168.50.10: icmp_seq=1 ttl=255 time=0.727 ms 64 bytes from 192.168.50.10: icmp_seq=2 ttl=255 time=0.768 ms 64 bytes from 192.168.50.10: icmp_seq=3 ttl=255 time=0.745 ms ^C --- 192.168.50.10 ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.727/0.798/0.953/0.091 ms [edit interfaces so-0/3/0] lab@chicago# 

Looking at the show interfaces command, notice there are some differences from the earlier PPP encapsulation configuration. To start with, the link-level type is now shown as Frame Relay. Link flags shows that the interface is DTE, or receiving the timing from the far end of the link. Also, the ANSI LMI settings have been added for the Frame Relay parameters.

 lab@chicago> show interfaces so-0/3/0  Physical interface: so-0/3/0, Enabled, Physical link is Up   Interface index: 18, SNMP ifIndex: 31  Link-level type: Frame-Relay  , MTU: 4474, Clocking: External, SONET mode,   Speed: OC12, Loopback: None, FCS: 16, Payload scrambler: Enabled   Device flags   : Present Running   Interface flags: Point-To-Point SNMP-Traps  Link flags     : No-Keepalives DTE   ANSI LMI settings: n391dte 6, n392dte 3, n393dte 4, t391dte 10 seconds  LMI: Input: 0 (never), Output: 0 (never)   Input rate     : 0 bps (0 pps)   Output rate    : 0 bps (0 pps)   SONET alarms   : None   SONET defects  : None Logical interface so-0/3/0.103 (Index 12) (SNMP ifIndex 36)     Flags: Point-To-Point SNMP-Traps Encapsulation: FR-NLPID   Input packets : 10   Output packets: 10     Protocol inet, MTU: 4470, Flags: None       Addresses, Flags: Is-Preferred Is-Primary         Destination: 192.168.50.8/30, Local: 192.168.50.9  DLCI 103  Flags: Active       Total down time: 0 sec, Last down: Never       Traffic statistics:        Input  packets:                   10        Output packets:                   10 lab@chicago> 

In the above output, the ANSI-specified LMI settings can be changed manually as long as both devices are properly configured. These numbers represent the management functions, such as error thresholds, LMI keepalive timers, and so on. In addition, the DLCI 103 is shown under the logical unit and the statistics for that PVC.

7.3.4 APS

APS, or automatic protection switching, is a system used to backup SONET connections by fail-over . With APS, devices can have redundant SONET links between them, one active and one on standby. The active connection is called the working circuit; the backup link is called the protect circuit. These standby circuits can be connected on another interface on the same PIC, on another interface on a different FPC, or even on a different router.

APS can be configured between routers, or between ADMs and routers. Figure 7-11 illustrates router-to-router and ADM-to-router APS configurations. In addition, APS can be configured for concatenated or nonconcatenated SONET interfaces. SONET is in the nonconcatenated mode for use with channelized TDM circuits. SONET is in the concatenated mode for using clear channel between devices. SONET concatenation was discussed in Chapter 2.

Figure 7-11. APS Configuration Models

APS has several configurable parameters, such as keepalive intervals, authentication, and load sharing. Here we show the possible commands in the [edit interfaces (interface) sonet-options aps] hierarchy.

 [edit interfaces so-0/2/0]  lab@chicago# set sonet-options aps ? Possible completions:   advertise-interval   Advertise interval (milliseconds) + apply-groups         Groups from which to inherit configuration data > authentication-key   Authentication parameters   force                Force circuit state   hold-time            Hold time (milliseconds)   lockout              Lockout protection   neighbor             Neighbor address   paired-group         Name of paired APS group   protect-circuit      Protect circuit group name   request              Request circuit state   revert-time          Circuit revert time (seconds)   working-circuit      Working circuit group name [edit interfaces so-0/2/0] lab@chicago# 

APS sends keepalives at designated intervals to inform devices that the working circuit is still operational. If a keepalive is not received within three times the designated interval, the protect circuit will take over. The default interval for APS keepalives on a Juniper Networks SONET interface is 1,000 ms (1 sec). This is called the advertise-interval , which means that the default time for a switchover on a failed SONET connection is three seconds. If two routers are being used, authentication can be enabled using the authentication-key parameter for security.

APS is configured under the aps subhierarchy of sonet-options . The first step is to determine which of the links will be the primary (working) and which will be the secondary (protect). A group name is then defined after the circuit type to designate the relationship between protecting interfaces and working interfaces. If a binding factor was not used, and a router had several working and protect circuits, how would it know which circuits they were protecting? The same name is used on both the working and protect interfaces to bind them in a group. Figure 7-12 shows a router-to-router APS scenario with the APS ports in different PICs.

Figure 7-12. Router-to-Router APS Configuration

In this instance, both interfaces have the same IP address. Routers from other vendors may not have an APS implementation as fully featured as Juniper Networks. In some cases, the router may have just primary and secondary interfaces. If this is the case, the APS interfaces would have to have the same IP address. Here we see that router Chicago has been configured for APS with the group name of juniper . Router New York has been configured in the same way with IP address 192.168.50.2 .

 [edit interfaces]  lab@chicago# show fe-0/0/0 { <<<output omitted for brevity>>> } so-0/2/0 {     clocking external;     encapsulation cisco-hdlc;     sonet-options {         aps {  working-circuit juniper;  }     }     unit 0 {         family inet {             address 192.168.50.1/30;         }     } } so-0/3/0 {     clocking external;     encapsulation cisco-hdlc;     sonet-options {         aps {  protect-circuit juniper;  }     }     unit 0 {         family inet {             address 192.168.50.1/30;         }     } } <<<output omitted for brevity>>> [edit interfaces] lab@chicago# 

To show the status of APS on a router, use the show aps command as shown below. This will show the groups to which the interfaces are assigned, circuit types configured, and the interface state.

 lab@chicago> show aps  Interface  Group                           Circuit  Intf State so-0/2/0   juniper                         Working  enabled, up so-0/3/0   juniper                         Protect  disabled, up lab@chicago> 

The first field of the interface state is the administrative state; the second is the physical state. When the protect circuit will not be used for forwarding traffic while the working circuit is up, the protect circuit will be administratively disabled by the router. However, the physical state of the interface is Up , as is shown below. Interface so-0/2/0 is Enabled and Up , while interface so-0/3/0 is Administratively Down and Up .

 lab@chicago> show interfaces so-0/2/0  Physical interface: so-0/2/0,  Enabled, Physical link is Up  Interface index: 15, SNMP ifIndex: 17   Link-level type: Cisco-HDLC, MTU: 4474, Clocking: External, SONET mode,   Speed: OC12, Loopback: None, FCS: 16, Payload scrambler: Enabled   Device flags   : Present Running   Interface flags: Point-To-Point ... lab@chicago show interfaces so-0/3/0 Physical interface: so-0/3/0,  Administratively down, Physical link is Up  Interface index: 16, SNMP ifIndex: 31   Link-level type: Cisco-HDLC, MTU: 4474, Clocking: External, SONET mode,   Speed: OC12, Loopback: None, FCS: 16, Payload scrambler: Enabled   Device flags   : Present Running   Interface flags: Down Link-Layer-Down Point-To-Point SNMP-Traps 

Removing the fiber connection from so-0/2/0 will result in the connection failing over to so-0/3/0 , because so-0/2/0 will go into a disabled and down state. Using the show aps command again, you can see that so-0/3/0 is now enabled , while so-0/2/0 is now disabled . It is also, of course, physically down because the fiber is removed from the interface.

 lab@chicago> show aps  Interface  Group                           Circuit  Intf State so-0/2/0   juniper                         Working  disabled, down so-0/3/0   juniper                         Protect  enabled, up lab@chicago> 

The most important point of all is whether or not router Chicago can still communicate with router New York on the newly enabled link. It can, as we see below:

 lab@chicago> ping 192.168.50.2  PING 192.168.50.2 (192.168.50.2): 56 data bytes 64 bytes from 192.168.50.2: icmp_seq=0 ttl=255 time=0.795 ms 64 bytes from 192.168.50.2: icmp_seq=1 ttl=255 time=0.735 ms ^C --- 192.168.50.2 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.735/0.765/0.795/0.030 ms lab@chicago> 

Of all the types of interfaces Juniper Networks has, SONET is the fastest and is generally preferred for WAN data communications. Even though ATM is more widely used because of its installation base and acceptance in the Telecom industry, SONET has much less overhead and can transfer data at much higher rates. Juniper Networks makes a strong case for migrating core networks to SONET from ATM. This case gets stronger as IP-enhancing features are implemented that allow policy and service guarantees that are not inherently found in the standard IP protocol.