What s Needed and Why

When they got to the conference room, Jim wanted answers. "Now tell me what happened. Why is that server down?"

Dan explained the situation in a few sentences. Jim turned on Bill. "Your people already have the most expensive machines in the company! Isn't that enough?"

"No, it's not!" Bill retorted. He had been getting more and more upset throughout Jim's tirade and was determined to have his say.

"It's not like it used to be, Jim. Once upon a time, we wrote stand-alone applications that we would compile and test at our own machines. We needed the big boxes to speed up the compiling time, so we wouldn't waste a morning watching a computer churn through Clipper code, for instance. But once the compile was done, we could test the app locally using database files on our own computers. If it worked there, we simply put it on the network and told everyone how to get to it and run it. For the really incompetent ones, we might write a batch file and put the app in their login scripts.

"But we don't write that way anymore. We break our applications into components, and some bits go on the user's computer and some go on a server. We use back-end databases that servers have to run, and we tie those servers to other servers running Web hosting applications, and security packages, and electronic commerce, and data warehouses, and who knows what else."

Bill went to the whiteboard, grabbed a marker, and drew some boxes on the board. Then he connected them with lines and turned to Dan. "And now, we've added another piece to the puzzle, or rather a bunch of new pieces. We're writing N-tier, which means we've got a user layer, and a business services layer, and a data access layer. We need MTS, and SQL Server, and IIS, and each of them should really be running on its own server. But in the development area, we don't have these apps set up on their own boxes, so what can we do? Either we don't test until we roll the app out, or we test on the production systems because they are the only systems running the services we need."

When Bill paused for a moment, Dan stepped in. "Bill, you come from a mainframe environment, so you know the value of the four-step development environment. Why haven't you implemented it at Ferguson and Bardell?"

"One simple reason—dinero," said Bill, rubbing his thumb and fingers together and looking at Jim. "Every year we submit a budget request for a stand-alone testing lab, but the executive committee always cuts it from the final budget." Bill shrugged. "The old IT Director reported to the CFO and had to rely on the CFO to argue his case to the executive committee, so there was never anyone on that committee who understood the need."

"Point made, and point taken," said Jim, nodding. "You are absolutely right. I never understood the need until now. Of course," he said, looking at both Bill and Dan, "Until now, I never was invited to IT meetings, either."

"Point made, and point taken," said Bill. He turned to Dan. "So where do we go from here? I don't want to be responsible for any more lost revenue, and I suspect you don't either."

"First of all," Dan said, "I want you to send out a memo—and I want it sent today—that no one in development is to touch a production machine for testing purposes, no matter how urgent the test is or how trivial it seems. I'll follow up with a memo to the entire company, explaining why some machines are designated as production machines and listing all the machines we are giving that designation. I'll also have Tim double-check our security settings and procedures on those machines to make sure that well-meaning people can't take a server down accidentally."

After a moment's thought, he continued. "When we've shut the production barn doors, we need to figure out what we need for a test barn. We want to balance need with cost, though, so I'm not sure just what direction to go."

"Dan, what's the four-step development environment you mentioned a few minutes ago?" Jim asked.

"Basically, it consists of four different levels of isolation and criticality," said Dan. He erased Bill's drawing from the whiteboard and drew four new boxes, which he labeled Dev, Test, Cert, and Prod. "The developer builds a new app, or a part of one, or a new version of an old one, on his or her own computer. That's the Dev box, for development. When the developer thinks the app is ready, he installs in on a test computer that mimics the normal operating environment of the company. That's the Test box. In some companies, a testing group within the development department takes over at this point and walks the application through the testing process.

"If the app runs OK in the simple test environment, it is moved to another computer where it is thoroughly tested, including testing for load issues, response times, and interaction with other production-level applications. That's the Cert box, for certification. And when it passes all the tests thrown at it in the certification stage, it is put on a production machine. That's the Prod box." Dan put down his marker. "I've simplified the process a great deal. Ideally, it should include security settings at each level so that the people from the level before and after can't bypass the process, as well as a clearly defined documentation and change control process. The bottom line, though, is two-fold: Testing is never done on a production machine, and nothing is ever installed that hasn't been tested beyond the developer's machine."

"The problem is," added Bill, "that to do testing the way it should be done, you need a testing environment that mimics your production environment at each level. Whatever services you are running in Production should be present in Certification, and, if you're testing N-tier applications, you'll need them all in Testing as well. It can be expensive to set it all up, and you need one or more people to run and maintain it."

Jim looked at the board and rubbed his chin. "Bill, I used to think IT was always looking for new toys to spend the company's money on. I still think some IT people are like that. But since I've been working with the RMS project, I've come to realize that you and Dan just want to build the most professional IT organization you can. And when you can get his mind off the doughnuts, I think Tim wants the same thing, too."

He walked back and forth a few times. "It sounds to me as if a top-notch testing lab would take some space. Am I right?" Bill nodded. "Have you ever looked at that space on 34?" Jim asked Dan.

"You mean the old executive dining room? I thought the telemarketing group was taking that."

"They are," said Jim. "In fact, they're moving next week. But that is going to free up their space…"

"…which is right above us!" Bill exclaimed. "Wow, that would be perfect: near, but still isolated."

"Now, you can't have all of it," cautioned Jim. "I've promised some of it to the sales folks. But I think you could take half of it without a problem. Would that work?"

"We'll make it work, Jim," said Dan. "Not to seem ungrateful, but space is only part of the equation. What are we going to put in that space?"

"Can you get me a realistic proposal for a top-quality testing lab by the end of the week?" asked Jim. "I can't promise we can do it all, but I think the business case is certainly there." He grinned. "Especially if I recite some revenue numbers."

"How about if we make it a phased proposal," suggested Dan. "The first phase what we need to test RMS, the second phase later this year, and the final phase next spring? That way we spread out the pain some, with the bulk of it coming after everyone has seen the success of RMS."

Jim looked at Dan and smiled. "You know, you've got a streak of politician in you. That's an excellent suggestion. Write it that way, and you and I will present it to the Old Man together. He can be crusty, but he also understands that you have to spend money now to make—or save—money later. You'll get your lab."