Section 23.5. Gigabit Ethernet

23.5. Gigabit Ethernet

As the amount of data that needed to be backed up grew exponentially, backup software became more and more efficient. Advanced features like dynamic parallelism and software compression made backing up such large amounts of data possible. However, the amount of data on a single server became so large that it could not be backed up over a normal LAN connection. Even if the LAN were really fast, only so many bits can be sent over such a wire.

Gigabit Ethernet was supposed to save the backup world. Ten times faster than its closest cousin (Fast Ethernet), surely it would solve the bandwidth problem. Many people, including me, designed large backup systems with gigabit Ethernet in mind. Unfortunately, we were often disappointed. While a gigabit Ethernet connection could support 1,000 Mbps between switches, maintaining such a speed between a backup client and backup server was impossible. The number of interrupts required to support gigabit Ethernet consumed all available resources on the servers involved. Even after all available CPU and memory had been exhausted, the best you could hope for was 500 Mbps. While transferring data at this speed, the systems could do nothing else. This meant that under normal conditions, the best you would get was around 200400 Mbps.

As of this writing, 10 Gbps NICs are becoming generally available, and they're going to solve the world's problemsat a cost of several thousand dollars per NIC. Believe it or not, I've talked to at least one person that was able to achieve 4,0005,000 Mbps using such a NIC on a high-end Solaris server and Solaris 10's IP stack. If those tests hold true in other shops, it will help. In my heart, however, I think we're fighting a losing battle.