| The test sharing ideas presented so far have been internal methodsthat is, the people that would help you share the testing are either from your test team or the project development team. Another common method for having others verify and validate the software is a process called beta testing. Beta testing is the term used to describe the external testing process in which the software is sent out to a select group of potential customers who use it in a real-world environment. Beta testing usually occurs toward the end of the product development cycle and ideally should just be a validation that the software is ready to release to real customers. The goals of a beta test can vary considerably from getting the press to write early reviews of the software to user interface validation to a last-ditch effort in finding bugs. As a tester, you need to make it known to the person managing the beta testing what, if anything, you want to achieve from it. From a test standpoint, there are several things to think about when planning for or relying on a beta test: Who are the beta testers? Since a beta test can have different goals, it's important to understand who the beta participants are. For example, you may be interested in identifying any remaining usability bugs left in the software, but the beta testers may all be experienced techies who are more concerned with the low-level operation and not usability. If your area of the software is to be beta tested, make sure that you define what types of beta testers you need in the program so that you can receive the most benefit out of it. Similarly, how will you know if the beta testers even use the software? If 1,000 beta testers had the software for a month and reported no problems, would that mean there were no bugs, that they saw bugs but didn't report them, or that the disks were lost in the mail? It's not uncommon for beta testers to let the software sit for days or weeks before they try to use it, and when they do, only use it for a limited time and a limited set of features. Make sure that you or someone running the beta program follows up with the participants to make sure that they're using the software and meeting the plan's goals. Beta tests can be a good way to find compatibility and configuration bugs. As you learned in Chapters 8, "Configuration Testing," and 9, "Compatibility Testing," it's difficult to identify and test a representative sample of all the real-world hardware and software setups. If your beta test participants have been wisely chosen to represent your target customers, they should do a good job finding configuration and compatibility problems for you. Usability testing is another area that beta testing can contribute to if the participants are well chosena good mix of experienced and inexperienced users. They'll be seeing the software for the first time and will readily find anything that's confusing or difficult to use. Besides configuration, compatibility, and usability, beta tests are surprisingly poor ways to find bugs. The participants often don't have a lot of time to use the software, so they won't find much more than superficial, obvious problemsones that you likely already know about. And, because beta testing usually occurs near the end of the development cycle, there's not much time to fix bugs that are found. NOTE Trying to rely on beta testing as a substitute for real testing is one of the major pitfalls of software product development. Don't do it! If such a process would work, why not do the same with the software design and programming?
A beta test program can take up a lot of a tester's time. A common job for a new tester is to work with the beta customers to help solve their problems, answer their questions, and confirm the bugs they find. If you're assigned this task, you'll also need to work with your fellow testers to understand how the bugs slipped through to the beta testers and how to improve the test cases so that the bugs are found internally in the future. All this can be a full-time job, leaving little room to do any real testing yourself.
If you and your team plan on holding a beta test program, make arrangements in advance, preferably when the product's schedule is being defined. Make sure that the beta test's goals mesh with goals that you and your testing team want out of it and work closely with the person (or team) managing the beta program to keep testing's voice heard. Beta testing can prove to be a valuable method for getting solid, independent test data back on your software, but to be effective it must be properly defined and managedyou could almost say it needs to be tested. |
