What does this dog food philosophy mean? As explained in the earlier chapters, we at Microsoft test our own software on our developer, IT, and in some cases home systems before we ship a product out the door. In other words, we eat our own dog food. Why is this relevant? The following sidenote illustrates why.
It's possible to work on a Web application or a small standalone application that doesn't take your whole company down if there's a critical bug, but don't let that stop you from implementing BVTs and smoke tests. Every software application needs them, and they should always be the first round of tests that are run on your built binaries.
Some groups have teams dedicated to running and reporting the results of BVTs, whereas others assign this task to the build team. This chapter covers how to set up BVTs and smoke tests and what the difference is between the two. One of the questions I am often asked is, "What should we use for BVTs or smoke tests?" The correct answer? It depends, and I am really not the person to answer that question. The correct people to ask are your testing personnel, because they know which critical pieces of the product need the most testing.
The way that we gathered BVTs to run in the build lab was to ask the testers to give us a certain number of tests that ran for a fixed amount of time, say three hours. We did this for every component of the product.
Every company should run some standard tests that not only try out the application, but also try out the integrity of the build. I mention those tests later. I also mention a good "Testing Guide" (complete guide in Appendix C) compiled by Josh Ledgard, a program manager in the developer division at Microsoft. It is a great starting point for building a test plan.
I would like to define two terms that are used loosely to label a build once tests have been run on it. Feel free to customize the definitions to what you and your development/test team would like:
Another term I would like to define is unit test. A unit test is a test that checks the functionality of a particular code module or component. Usually, unit tests make good BVTs or smoke tests.
Let's look further into smoke tests and BVTs, which are really a subset of smoke tests.