Ethernet in the Local Loop

Ethernet has reigned as the market leader in the LAN for longer than most dot-commers have been high-school graduates. However, the once all-LAN technology is now encroaching on Metropolitan Area Network (MAN) turf. Ethernet's local loop appeal whether Fast Ethernet, Gigabit Ethernet, or in-the-planning-stages 10 Gigabit Ethernet (10GbE)comes from the economic incentive of using inexpensive Ethernet hardware and phasing out costlier WAN technologies, such as ATM and SONET, in the metropolitan area. Ethernet hardware costs a tenth of that required by the SONET and ATM competition, says Yipes (www.yipes.com), a Competitive Local Exchange Carrier (CLEC).

As compelling as the near-ubiquity and affordability of Ethernet are, limited fiber availability in the local loop has thus far kept the technology on the outskirts of the enterprise. Ethernet in the local loop is, however, a reality for a cherry-picked few.

Great Deal If You Can Get It

Optical Ethernet outside of the LAN has four primary benefits: less-expensive hardware, unification of multiple LANs in a MAN under one protocol, flexibility of bandwidth provisioning, and a familiarity that requires no new training. Trumpeting the various selling points of Ethernet, CLECs such as Yipes, Cogent Communications (www.cogentco.com), IntelliSpace (www.intellispace.net), and Telseon (www. telseon.com) offer bandwidth-hungry businesses the access they covet . These start-up providers are generous with bandwidth and conjure visions of native Gigabit Ethernet connections between LANs in a metropolitan area. However, availability is limited to just a few urban markets.

In what is referred to as a transparent LAN, an Ethernet-based MAN can be used to connect multiple LANs within a 200-kilometer diameter. With WAN lines as fast as LAN speeds, for all practical purposes, what once was a collection of disparate LANs connected over expensive leased lines becomes one big LAN. The MAN sustains the traffic in its native formatneedless conversion is unwieldy and inefficient. Since Ethernet carries IP and legacy LAN traffic such as SNA, IPX, and others, all traffic stays just as it would on the LAN, but it's no longer restricted to one street address. VLAN tagging is the principle method for constructing VPNs between locations.

Businesses receive generous doles of bandwidth and experience rapid provisioning times when they want to boost bandwidth capacity. Critics of ATM and SONET contrast this flexibility with the relative difficulty of performing the same tasks with ATM switches or SONET gear. Over half the demand for Ethernet in the local loop is for high-speed Internet access.

Show Me The Fiber

Early subscribers to the Ethernet local loop fit a certain profile, largely because pro-viders have strategically dug fiber in direct proximity to the companies with the biggest wallets. According to International Data Corp. (IDC, www.idc.com), fiber availability is primarily confined to downtown areas of large cities, where approximately 8 percent to 10 percent of buildings have existing fiber connectivity. Customers of the metropolitan-area Ethernet service providers are not solely enterprises ; in fact, many are themselves service providers, whether Web-hosting providers, Application Service Providers (ASPs), or others.

The metropolitan-area service provider start-ups that currently offer Ethernet in the local loop are relatively new to the game, so they generally lease the fiber from multiple sources, such as utility companies and firms like Metro Media Fiber Network (www.mmfn.com). Given the volatility of the CLEC space and their difficulty in turning any kind of a profit, ownership of one's fiber could turn out to be a huge liability for a CLEC; instead, they ink long- term leases. It's impossible to get a comprehensive footprint from just one leased fiber source, thoughproviders cobble together fiber lines from multiple sources to achieve adequate reach.

The enormous expense of land rights, construction labor, and equipment for digging trenches freights every inch of a proposed fiber extension with cost. Providers will break out the shovels if the expense can be cost-justified, but they focus much more on signing up customers directly on the path of existing fiber rings. Most service providers are reluctant to reveal specific fiber-dig prices (and charges vary wildly depending on logistical barriers and topography ), but $90,000 per tenth of a mile seems to be a ballpark figure; $400,000 for the same distance is not unheard of.

Ethernet Outside Of The LAN

When Ethernet ventures outside of the LAN, it does so on fiber optic cable. Gigabit Ethernet can run over fiber in one of three ways: in its native fiber-based format, via Dense Wavelength Division Multiplexing (DWDM), or packed into SONET frames . Depending on the service provider's network, Ethernet data may utilize all three methods . Until the adoption of 10GbE, projected for IEEE ratification in March 2002, Gigabit Ethernet and Fast Ethernet will serve as the Ethernet ambassadors to the outside world. Both standards can support Carrier Sense Multiple Access with Collision Detection (CSMA/CD) in half-duplex mode, but due to distance constraints, Ethernet outside of the LAN operates in full-duplex mode. 10GbE will support only full-duplex transmission.

CSMA/CD-based Ethernet acts as if it had a strict upbringing as a child. A node listens to the line before transmitting data. In the event that two or more nodes transmit at the same time, they each back off a random amount of time before again peeking out and listening to the line, waiting their turn to be heard in an orderly fashion. On high-speed point-to-point links, however, the method is out of place.

Ethernet networks must conform to inevitable distance limitations. If the network is too large or convoluted, delays will adversely affect error detection capabilities. When collision notifications are received late, packets aren't re-sent, and data comes across as garbled. To counteract round-trip propagation delay, the IEEE increased the minimum frame size for Gigabit Ethernet from 64 bytes to 512 bytes and the maximum frame size from 1,514 bytes to 9,000 bytes. Larger frame sizes effectively buy transmitting devices more time to receive collision notifications and lessen the processing time of header information and so forth.

Regarding distance capabilities, 100Base-FX maintains data integrity up to distances of 2 kilometers on multimode fiber. Per the IEEE 802.3z specification, Gigabit Ethernet reaches up to 5 kilometers on single-mode fiber and 550 meters on multimode fiber. 10GbE should pass the 40-kilometer marker on single-mode fiber.

Switches in the MAN's backbone portion contain Gigabit Interface Converters (GBICs) that perform optical-electronic conversion and regenerate optical signals. Fiber runs between switches generally don't exceed a maximum of 70 kilometers, but some vendors such as Cisco Systems say that Gigabit Ethernet transmission over their switches can reach distances between 100 kilometers and 150 kilometers with appropriate lasers and GBICs.

Vendors that provide equipment to this metropolitan Ethernet space include Cisco Systems, Juniper Networks, Extreme Networks, Nortel Networks, Foundry Networks, and Sycamore Networks.

Imagine Yourself In The Driver's Seat

Let's assume you have the good fortune to work in a building right off the path of a fiber ring. No less, you also use Ethernet throughout your business. You know the technology, you trust the technology, and you already have Ethernet equipment throughout your company's LAN or LANs.

Most start-up providers that offer Ethernet in the local loop start the bidding at 1Mbit/sec and don't flinch up to 1Gbit/sec in fact, the higher the data rate, the broader they smile. The provider brings fiber to the building and connects it directly to a 1Gbit/sec or 100Mbit/sec port of an onsite Ethernet switch. (It is also possible to locate the switch in a rack at a service provider POP or at a colocation facility.) The customer then distributes the bandwidth resources through the different areas of his or her business via the switch. For those accustomed to choosing between 1.54Mbits/sec of a monthly $1,000 T1 line or 45Mbits/sec of a $5,000-a-month T3, the name -your-fancy bandwidth increments of an Ethernet provider such as Yipes should be mighty appealing. Not to be outdone, Cogent has garnered countless headlines for its monthly flat rate of 100Mbits/sec for $1,000.

Within the fiber optic network the provider has assembled for the MAN, routers aggregate streams of Gigabit Ethernet and shuttle the data traffic throughout switches in the network. Because the routers' reach extends to the Ethernet switch at the customer premises, providers can leverage Ethernet rate-limiting capabilities that lend themselves to flexible service adjustment. Ramping up bandwidth capacity for a particular time frame of anticipated increased demand is easily accommodatedprovisioning is processed in near real time.

Retain Baby, Drain Bathwater

Don't throw out the baby with the bathwater, as the saying goes. The fact is that no provider will waste equipment that works. Ethernet in the local loop does not eliminate the need for ATM and SONET. Despite the appeal of all-Ethernet, no one's about to forklift out SONET equipment just to have one common protocol throughout the metropolitan area. Many service provider networks have embedded investment in packet-over-SONET technology. For these reasons, it is important that 10GbE incorporates OC-192 interfaces for easier integration of Ethernet with SONET.

SONET is considered inefficient because one of its two fiber rings is reserved purely for failover, but when it comes to restoration capabilities and long-haul transmission, Ethernet zealots eat their words. Likewise, ATM is adept at delivering QoS, a claim that Ethernet cannot make. It's not an all-or-none proposition: Ethernet as the sole transport method offers certain advantages, but the technology can leverage SONET and DWDM when the situation requires. Furthermore, Ethernet must concede traffic with strict QoS requirements to a separate ATM WAN connection.

The very ring-based redundancy that earns SONET criticism for inefficiency is the same feature that provides rapid traffic restoration in the event of a fiber cut. (Incidentally, some providers put Ethernet traffic over both rings, which doubles the ring's capacity but sacrifices the medium's restoration capabilities.) For the long haul, SONET encapsulates Gigabit Ethernet traffic within its payloadfor example, on a crosscontinental backbone. This scenario delegates the traffic to a larger carrier, but the relationship is rarely one of peering, due to the disparity in what the two service providers can offer one another.

Cracks In The Ethernet Vase

Ethernet in the local loop can furnish high-capacity access for a low cost per bit, but the technology is not without its flaws, namely in the areas of QoS and restoration capabilities.

When a pipe is at capacity, and important traffic isn't getting through, there will be QoS issues, something Ethernet is ill-prepared to handle. Similarly, congestion in the service provider's network resulting from the aggregation of multiple gigabit streams in the core can cause problems. Providers using Ethernet will usually pile on the bandwidth to maintain acceptable QoS, but this does not scale well. Once capacity is reached, delay-sensitive applications suffer without ATM's QoS levels.

VLAN tagging based on the IEEE 802.1q and 802.1p protocols is the most promising method of assigning priority to Ethernet traffic. DiffServ marking can also be used on IP packets to support different Class of Service (CoS) levels.

In January 2001, the IEEE 802.17 Resilient Packet Ring working group met to develop an approach analogous to the ring formations in SONET. More attuned to the needs of Ethernet, the dual counter-rotating rings should mimic the quick restoration times of SONET, which can respond to a fiber cut by fully rerouting in under 50 milliseconds .

Ethernet in the local loop is gaining momentumlargely because it gives customers the high-speed access they want in the way they want it. But keep your fingers crossed regarding your proximity to existing fiber.

Resources

The IEEE Web site at www.ieee.org contains up-to-date information on recent Ethernet standards, as well as information on interesting projects such as the IEEE 802.17 Resilient Packet Ring working group and the IEEE 802.3 EFMEthernet to the First Mile study group.

An online section by Sycamore Networks hosted at www.lightreading.com/opticalintellect/ has several in-depth white papers that cover optical Ethernet, including the providers and equipment manufacturers in the space.

A useful resource on Ethernet in the Metropolitan Area Network (MAN) can be found at the aptly titled Optical Ethernet Web site, located at www.optical-ethernet.com. The site contains links, news on standards activity, and various white papers.

This tutorial, number 152, by Rob Kirby, was originally published in the March 2001 issue of Network Magazine.