Sample Machine Configuration

The following information will give you an idea of a typical SNAP hardware configuration that you need to budget for if you plan to create or purchase this system.

Following is the SNAP build machine configuration:

• The fastest processor and memory available to optimize Windows 2003 performance is recommended.

• The C drive is a minimal 20GB, the D drive is 100 to more than 200GB, and RAID Level 5 is used on the drive subsystem. The lab team maintains the hardware and performs the initial installation.

• The operating system is Windows 2003 Server Enterprise Edition with all the latest service pack/patches.

• Anti-virus real-time monitoring is disabled. This kills your build times if you leave it on.

• The system indexing service is turned off, which is another performance killer.

And here is the test machine configuration:

• The fastest processor and memory available to optimize Windows XP Professional performance is recommended.

• The drive is split so that you have a D drive with around 5GB. The remaining approximately 105GB goes to the C drive.

• The D drive is formatted and is boot enabled.

• The SNAP daemon is located on this drive. A shortcut for it is placed in the startup group of the Windows installation.

• The C drive has Windows XP Professional with the latest service pack/patches.

After a SNAP system is set up, you should not need to configure one of these machines from scratch unless you have a catastrophic failure that requires rebuilding the unit.

The SNAP administrators are responsible for keeping all the SNAP machines current with all hotfixes, patches, and critical updates. When indications of such updates are available, you need to check each machine for new updates via the Windows Update site. The biggest administration required is monitoring the test machines periodically to ensure that they are all being reset and updated correctly.

SNAP is a queue-based check-in system. A developer makes a submission to the SNAP system either via the command line or through a Web UI instead of checking his files directly into the mainline source code tree. The changes that the developer submits are then copied to a SNAP server, and a check-in is added to a queue in the lab. Inside the lab, a control process pulls check-ins off the queue and builds and tests them one at a time. The test team provides the tests. The system is designed to rerun parts of a check-in, ignore failures, abort a check-in, or allow modification of files in a check-in if needed.

Warning

If you commit to a testing process in your check-in queue, expect to spend significant time maintaining it. If your test harness does a poor job of catching and reporting errors, you will be wasting your time. Your test harness needs to be able to reproduce lab failures on developers' desktops. The test harness needs to have few bugs of its own and needs to clearly log stack traces and assert text.

If a developer's check-in breaks the build or something else goes wrong with the check-in, the system sends an e-mail to the developer and the entire team letting them know where things have failed. The developer then must figure out why his job was aborted and resubmit it.

Microsoft Sidenote: Lessons Learned Part 1 (from the NetDocs Team)

So many different bugs and problems came and went in the course of running the check-in process that it was impossible to keep our attention focused on the lab. By watching the trend lines on our lab status reports, we were able to recognize when a tolerable level of degraded service became intolerable. Because the back end of the SNAP system used a Microsoft SQL 2000 database to store detailed logs about the progress of the check-ins, we could analyze the check-in data easily, including build failure rates, number of check-ins, and number of aborted check-ins.

Occasionally, we (the NetDocs team) deliberately blocked the queue to all but high-priority jobs and only allowed through check-ins that fixed these merging or integrating problems. Inevitably, these exercises were efforts to stabilize the core functionality of the product. Time and again, we learned that by making the check-in process go smoothly, we reaped dividends in having many important bugs fixed in the product.