In November 2002, a system administrator named Trey Harris posted the tale of a remarkable bug that he had diagnosed and fixed. He later posted a clarification of the bug's details and a FAQ about the bug, both of which you can find at http://www. ibiblio .org/harris/500milemail.html.
While working as a system administrator for a campus e-mail system at the University of North Carolina, Trey received a phone call from the chairman of the statistics department saying that nobody in the statistics department could send e-mail farther than 520 miles from the campus! After verifying that the call wasn't a practical joke, Trey ran some tests of his own on the e-mail system. Sure enough, when he sent test e-mails to Richmond, Virginia; Atlanta, Georgia; Washington D. C.; Princeton, New Jersey; and New York City, all of which are destinations within 520 miles, the e- mails were sent successfully. When he sent a test e-mail to Memphis, Tennessee, about 600 miles away, it failed to deliver. Likewise for Boston, Massachusetts; Detroit, Michigan; and Providence, Rhode Island, the latter being 580 miles away. A minor comfort was that when he sent an e-mail to a friend in North Carolina whose ISP was in Seattle, Washington, it also failed. If the problem had been related to the geographic location of the e-mail recipient rather than the mail server, there would have been some real explaining to do.
Having duplicated the problem, he now had to figure out what was causing it. After all, it's not every day that you find such an unusual bug. He knew that a consultant had recently patched the mail server to upgrade its SunOS operating system, but the consultant hadn't touched the mail system itself. The first obvious place to look for problems was the configuration file for the sendmail utility. But the sendmail.cf file on the offending mail server looked perfectly normal.
To investigate further, Trey telnetted into the SMTP port on the mail server, and was greeted with a SunOS sendmail banner for sendmail version 5. At this point in time, Sun shipped the tried and trusted sendmail version 5 with its operating system, even though the version of sendmail used by the university had been standardized at version 8. So when the consultant patched the OS, the sendmail utility had been downgraded from version 8 to version 5, but the sendmail.cf configuration file had not been downgraded'this was the first clue.
Although the sendmail version 5 shipped by Sun had been tweaked to cope with a version 8 configuration file, it only did so by ignoring the configuration options that it couldn't understand. One of these options was the time-out to connect to the remote SMTP server, which sendmail set to zero because it couldn't understand the version 8 setting. Some experimentation revealed that under the typical load experienced by this particular mail server, and after accounting for router delays and transmission speeds across optic fiber, a zero time-out would abort a connect call to a remote mail server in approximately 3 milliseconds .
How far does light travel in 3 milliseconds? Slightly over 500 miles.