5.2 Managing a Beta Site

The purpose of beta is to ensure that the project works in the user s environment. Users are doing the team a favor by testing this project. Good beta sites are hard to come by. Teams should take special care of their beta sites. An effective flow for getting the project to a beta site ( specifically closed beta sites) follows . This flow may need to be modified based on the scope of the project, since a project that adds a simple Web tool will need to be managed differently from a project that includes shipping hardware and software, and differently from a project that will be accessed by users outside of the company. For a complex project, it may be necessary to coordinate the install date with the user and have a technical employee be present. Again, depending on the complexity of the project, the technical employee may need to give a short project training class and be present when the user first installs the project. The technical employee can review with the user what the team would like him to review and can outline how questions, bugs, and comments are to be handled. If applicable, at mid-beta the team may update the project with a new release of software. If paper documents were provided, ask the user to write questions or comments in the actual document. At mid-beta, give the annotated documents to the person responsible for documentation. Documentation can use the annotated documents to make changes where applicable . It is a good policy to have users filter all questions through the Help Desk organization. The Help Desk organization has the facilities to log information and track responses. This policy allows the Help Desk organization to hear user questions regarding the project. Quality Assurance and IT are responsible for supporting the Help Desk people with any questions they cannot answer. All questions and problems should be funneled through Quality Assurance, since they will need to reproduce the bugs .

Closed Beta

After the project is shipped and installed, the beta coordinator (typically a QA representative) periodically calls each beta site and updates an internal document outlining what the user has tested , what comments and recommendations were given, what bugs were found, and what the overall acceptance of the project was. For a public beta, it is recommended a beta chat line be created, with the QA representative chairing this chat line. It is also useful to have a bulletin board noting the known problems, expected fix dates, and frequently asked questions (FAQ). The QA person is not responsible for answering user questions. Help Desk should be involved with the chat line to support any user questions. Both QA and Help Desk should make it clear to users that the code is beta quality and major bugs will be noted and fixed in future releases. QA and Help Desk bring their findings to the weekly beta committee meeting. These findings are reviewed at the meeting so all departments involved in the beta testing program are aware of what the user is doing, understand the common problems, and have an idea of user comments. The QA representative is responsible for updating the project team with the beta status. Halfway through beta, the project should be updated. These updates are sent by QA to the beta site, or in the case of a public beta, the Web site is updated and an e-mail notification goes out to all registered beta users. The existing annotated document is given to Documentation. Documentation updates the manuals and help files with the users suggestions. If the project will include a paper manual, changes to the documentation will freeze by the middle of beta.

Open Beta

Open beta refers to a beta that is open to everyone. Open betas tend to be the easiest betas on the team. Projects that are candidates for open betas are applications that are easy to access and generally do not interfere with the workings of applications sitting on a user s computer. Most new Web services are good candidates for an open beta. It is worthwhile having a testing period for any new Web application. A team can have the corporate Web editor highlight the new application. Users interested in accessing the new application can fill in an access form that provides the team with name , e-mail address, and phone number. The user is informed this is a new application. Since the beta is open it is easy to get users to test the new application. To receive beta feedback the team should create a small survey for the beta users.

Hybrid Beta

For a hybrid beta the team will need to set up the procedures for both a closed beta and a public beta. At a predefined time, the team will need to segue the beta from closed to public. The team may decide to run two different beta tracks. For a three-month beta period, the team will run a three-month closed beta and a one-month public beta. For instance, if closed beta starts in January, it will run through March and the public beta will run only for the month of March. The project will be announced in March.

At the End of Beta

After all severity 1 and 2 bugs have been fixed, beta sites are asked if they consider the project stable for release. Once the beta sites agree that this project is stable, they are asked to sign a beta sign-off letter. To maintain goodwill with the beta users, it is recommended a gift be sent to them, since they helped certify the project. Before you send a gift, confirm that the company doesn t have a policy against gifts. For public beta sites you may want to send an e-mail announcing the end of the beta, asking for comments, and asking the user to try the released project. You might give public beta users a limited discount on the project or make them eligible for a raffle.

5.2 in a Nutshell

A good beta site is hard to come by, and users are doing the team a favor.

• For complicated projects a technical employee should be assigned to help the early beta sites.

• If applicable, at mid-beta the team updates the project with a new release of the software.

• At periodic intervals (defined by the team), a QA representative should call the site to ensure that everything is working as expected.