Troubleshooting Ethernet Problems

The most common medium for local networks, 10BaseT-compliant Ethernet, has proven itself to be reliable, forgiving , and relatively trouble-free. As 100BaseT becomes more prevalent , you can expect a few additional problems, but nothing resembling the days of coaxial-cable-based Ethernet. Nevertheless, it's important to be prepared to cope with any difficulties that arise, and this tutorial gathers together information about the tools you can use to solve Ethernet problems.

Most 10BaseT and 100BaseT links have three hardware components: a NIC in a client or server computer; a port in a hub, switch, router, or other device; and a cable connecting the first two components .

The parts of the network interface that can cause trouble include the driver software that talks to the computer's OS, the configuration settings in the OS, and perhaps some configuration software or hardware-based jumpers or switches that configure the interface card.

The port at the other end of the cable from the computer is quite straightforward on a hub, though on a switch or a router there could be complex configuration issues, such as VLAN definitions or policy settings.

The cable will often consist of a patch cord with eight-pin modular plugs (often called RJ-45 connectors) on each end; a wall plate with an eight-pin modular jack; a section of cable running inside the walls or ceilings to a punchdown block in a wiring closet; perhaps another jumper cable connecting the punchdown block to a patch panel; and finally a patch cord that runs to the hub, switch, or router port (see figure). The good news about the cable segment is that once it is correctly installed, it will be reliable indefinitely-as long as no one reconfigures it incorrectly.

Connections: Most 10BaseT and 100BaseT Ethernet installations connect a network interface adapter to a hub or switch via a series of copper cables that total less than 100 meters in length.

Everyday Troubleshooting

The first thing to check when there is an apparent Ethernet link problem is the link light. Unfortunately, 10Base2 and 10Base5 (coax-based Ethernet) don't typically have link indicators. But practically every NIC you'll encounter nowadays has a link light that lights up whenever the interface receives link pulses from the other end. (10BaseT interfaces generate link pulses about 60 times each second when there is no data on the cable.) Practically every hub, switch, or router also has a link LED that stays on steadily when it receives link pulses . (For the rest of this tutorial, I'll save space by calling the devices at the other end of the cable from the computer "hubs." Unless it's spelled out otherwise , when I refer to hubs, you can substitute "switch" or "router" and so on.)

Thus, if the link lights at both ends of a questionable link are lit (and if they go off when you unplug the cable), you can be confident that the cable is wired and connected correctly, that the NIC and the hub are powered up and capable of receiving data, and that the most likely source of the problem is the configuration of the computer or the network interface.

If only one link light is on, the cable segment is almost certainly bad; the interface at the lit-up end is successfully receiving link pulses, while the interface at the other end is sending them but not receiving them. If possible, plug the computer into another cable that supports a working system. If the computer works on the other cable, you know the computer is configured properly. If a computer known to be working properly shows the one-link-light-out symptom on the original cable, you have added reason to suspect the cable segment.

At this point, a cable tester will quickly narrow down the problem to a particular length of cable or to a connector along the path . If you don't have cable testing equipment, your best bet is to simply substitute in known good cable lengths until the link lights come on at both ends. In this situation and many others, having a laptop computer known to be configured correctly for Ethernet connectivity, along with a good patch cord, will allow you to rapidly substitute the computer (the most easily misconfigured component) and the computer patch cord (the most exposed and vulnerable part of the cable link)-thus speeding confirmed diagnoses for the most common problems.

If both link lights are off, don't blame the cable right away. Hubs often have other LEDs besides the link indicators, such as power and traffic indicators. If the traffic indicator shows data, then at least some ports of the hub are working correctly. Hubs are designed to automatically "partition" or isolate misbehaving ports, so a malfunctioning port might cause the link lights to be dark.

To check this, plug the cable into a different port on the hub and see if the link lights come on. (If the device is a router or a VLAN-configured switch, it may be necessary to reconfigure the port before you connect the cable.) If a known good hub port doesn't light up the link lights at both ends, you should connect a known good computer, such as your laptop, to the computer end of the cable segment. If the lights stay out at this point, then something along the cable path is definitely the culprit.

Remember that 10BaseT cables connecting one hub to another must "cross over." (The cable must also cross over when you connect one computer to another to create a minimal two-computer network.) That is, pins one and two on one end must be connected to pins three and six, respectively, on the other end, and vice versa. Most hubs have one or more crossover ports with a switch, so you can use a standard cable and let the switch do the crossing over. Really sophisticated hubs detect the need for crossing over and do the job automatically, so you don't have to worry about whether your cable is crossed over or not.

The other indicator light you'll commonly find on hubs is the "collisions" light. A flicker of the collisions light every few seconds is to be expected on a normal network, but anything resembling a constantly lit collision indicator likely indicates serious problems. The most common reasons for excessive collisions are violations of the 10BaseT rules: no links longer than 100 meters and no more than five repeated segments, with no more than three of the segments populated and no more than four repeaters (hubs) between any two nodes.

Ethernet normally becomes aware of collisions in the first 64 bytes of the frame, and networks with too many repeaters or overly long segments can experience collisions after the 64th byte-time. Since the shortest allowable frame is 64 bytes, whole frames could be accepted by nodes on the network, despite their being corrupted by a collision, if late collisions occur. (For 100BaseT networks, there can be at most two repeaters linked by a five-meter cable, and late collisions occur after 512 bytes. Frames are always padded to a minimum length of 512 bytes for this reason.)

Advanced Tools

With built-in LEDs, a network-configured laptop, and substitute cables, you can consistently weed out bad configurations, as well as shorted, open , or miswired cables and connectors. A laptop or other correctly configured computer will generally be able to log on to a NetWare or Windows NT network or ping an IP address if the link lights come on. (Windows 95/98 clients generally have ping.exe in the Windows directory, and ping programs are also standard with Mac OS and Unix implementations .)

If you can't log on to a server when others can, or if you can't ping a destination that responds to other pings , then the cable comes under suspicion again, even if the link lights are on. Particularly at 100Mbits/sec, Category 5 cable can be sufficiently out of compliance to prevent a link from working, without obvious electrical or mechanical faults. Inadequate connectors, electromagnetic interference, or unsound wiring techniques are three of the most common ways the cable link might fail while still letting the link lights come on.

A cable tester that can measure near-end crosstalk and attenuation across the frequency band from 0Hz to 100MHz is the only way to be sure of a good cable link for 100BaseT. (10BaseT is more forgiving, but not infinitely so.) If you have a lot of coaxial cable-based Ethernet to troubleshoot, cable testers that estimate the distance of problems along the cable using time domain reflectometry can save a lot of trial-and-error operations. The big names in cable testing are Fluke (www.fluke.com), Microtest (www.microtest.com), Wavetek Wandel Goltermann (www.wavetek.com), Datacom Textron (www.datacomtextron.com), and Hewlett-Packard/Agilent Technologies. Of course, a cable plant certified by qualified cable installers is likely to hold up for many years and keep cable problems to a minimum.

One useful gizmo is a 10BaseT/100BaseT indicator. Psiber Data Systems (www.psiber.com) is one manufacturer of what the company calls "link testers." Cards that can run at either 10Mbits/sec or 100Mbits/sec are widespread, and while they are generally designed to autonegotiate the data rate based on what they're connected to, the autonegotiation protocols were not fully baked and interoperable when early products shipped. This means an interface can get stuck at 100Mbits/sec and freak out the poor 10Mbit/sec-only interface at the other end. The 10BaseT/100BaseT indicator is inexpensive and about the size of an electric toothbrush, with an eight-pin modular plug on the end and LEDs that indicate whether the connection is 10Mbits/sec or 100Mbits/sec, and whether it's full duplex or half duplex.

The next step up from a cable tester is a handheld troubleshooting tool. (Fluke has the broadest line of these.) These devices measure broadcast traffic, network utilization, and errors. In most cases they will list all the nodes on the network and indicate who the top senders and receivers of traffic are. These statistics will often quickly narrow down the possible causes of a problem, and then the troubleshooting tool can test hub, NIC, and cable performance individually. For example, overly long frames, known as "jabbers," almost always indicate a defective network interface or corrupted driver software. Also, if utilization consistently remains at levels higher than 60 percent, while collision rates remain low (let's say less than 5 percent), the likely problem is overall congestion. The solution is to segment the network, perhaps with a switch that gives the most congested nodes a dedicated path.

The most comprehensive handheld troubleshooting tools analyze traffic at the Network layer. These tools distinguish IP, IPX, AppleTalk, and other protocols and present statistics about the top talkers for each. Some of these devices even act as SNMP consoles, collecting, for example, switch and router port statistics and capturing RMON data. Many of the problems these devices can identify, such as duplicate IP addresses and addresses for nonexistent subnets, are not Ethernet problems, so I'll say no more about them here.

The ultimate Ethernet troubleshooting tool is the protocol analyzer (see "Lesson 131: Protocol Analyzers," June 1999, page 26). At the high end of the market, there are specially designed network interfaces on some protocol analyzers that can isolate and identify Physical-layer problems that might not be picked up by an ordinary NIC in a laptop computer running protocol analysis software. For 100BaseT networks, most protocol analyzers have add-on buffers that permit full wire-speed captures-comprehensive decoding software can't keep up with continuous levels of 100Mbits/sec, much less full-duplex traffic. Of course, like the handheld tools, protocol analyzers can look at all seven layers, not just Ethernet's Data-link and Physical layers . Cable testing functions, generally built into handheld testers, are not an option with protocol analyzers.

The main drawback of a protocol analyzer as an Ethernet troubleshooting tool, however, is the training and sophistication required for a user to become effective. Nevertheless, the most complex and thorny fault conditions will continue to require protocol analysis technology for the indefinite future.

This tutorial, number 135, by Steve Steinke, was first published in the October 1999 issue of Network Magazine.