5.16 Sample Plan B: A Not so Wide Area Network

< Day Day Up >

We built a corporate campus that consisted of two buildings side by side with an access road between them. The design called for each building to be a separate node on the corporate backbone. Each site, in other words, was intended to have its own connection to the high-speed fiber optic corporate network, or cloud, to which the other corporate sites were attached. This is best practice for network design - the strategy being that each building has an independent WAN connection and thus is not dependent on the other building, in case its WAN connection dies for whatever reason.

The plan called for turning up the first building in late spring, with the "B" building coming online somewhere near Halloween. It was a good plan that suited the move-in schedules of beneficiaries quite nicely and gave the telecom teams adequate time to do all their work in both buildings. Building A came up quite nicely, in fact, a week ahead of schedule. Not long after that, we learned that there was absolutely no way that the B building would come up at Halloween due to issues beyond our control. In fact, our telecom team lead learned, after considerable investigation, that instead of coming up in early November, January was a far more realistic date for Building B. That slippage was totally unacceptable from the beneficiary relocation standpoint, because we needed a minimum of 45 days for network element burn-in between the day WAN connectivity was achieved and the first wave of new beneficiaries showed up in the lobby of Building B. Because the move-in dates were nonnegotiable, we needed to come up with a snappy Plan B, and pretty quickly at that, given the long lead times of this sort of work.

What to do? Given market conditions and existing contracts, we were precluded from going to an alternate vendor. So, it did not take us long to make the only sane decision available to us: to connect the two buildings by running some mighty expensive fiber underneath the street separating buildings A and B. We also had to add conduits in the second building so as not to hog the space required in existing pipes once the "real" network connection was ready to be made. Exhibit 8 is a very simple look at the story in pictures.

Exhibit 8: Wide Area Network (WAN) Connectivity

click to expand

The network guys did a great job of coming up with a design that would support this "kluge" that had to transport LAN and mainframe traffic, IP Telephone, satellite video, and security data for the turnstiles and end point devices between the two buildings.

We made these decisions in May and started in July because we had to jump through a few other hoops: for instance, street trenching to support the additional cabling required permits, and it had to pass through two other utilities other than the carrier because of right of way issues. Only so many feet of trenching could be dug each day for reasons I still do not comprehend. And, of course, the network and phone system had been designed and all components already bought, so we had to buy more switches and routers. Needless to say, we accrued significant incremental costs to implement Plan B.

This is a great example of the degree of difficulty associated with high-risk contingency plans. I would like to tell you that this Plan B had been cooked up a year before we needed it, when the project was in the planning phase. That would not be truthful. We did discuss this as a potential risk months before we had to react, however, so we can claim adequate prescience on this one.

Our Plan B worked. The move deadlines were met, so the outside world remained in the dark as to the effort and expense required to keep the plan whole. Eventually, we were able to implement the original design. We recovered most of the additional expense by "selling" the additional routers and switches to another business unit. That was a beautiful thing!

< Day Day Up >