THE QA PLAN

To assure that visitors to your website have a pleasant and problem-free experience, develop and implement testing protocols to improve production specifications, visual and HTML style guidelines, and process flow — this is your Quality Assurance Plan.

A properly implemented QA Plan can:

• Solve browser incompatibility problems.
• Help you to stay current with HTML standards.
• Help you with the use of browser-specific HTML.
• Aid in the review of your site for bugs.
• Set up usability testing to insure that website user interfaces meet user needs.

Test every aspect — from Microsoft’s Internet Explorer and Netscape to Opera and WebTV and other web appliances — for validation that your website provides a problem-free experience for your customers, no matter how they surf the Web.

After following the design guidelines set out in Chapter 2, and before your new website is ready for launch, “freeze” your prototype and intensively test it prior to giving the public access. Structure the tests per your QA Plan. Be prepared for fine-tuning and testing to go on for a month. You must stop all development and changes of the site during this testing period.

Consider investing in software tools that specifically test content accessibility, basic functionality, and behavior under controlled access loads. As your last test, perform stress testing to see how the entire system reacts under really heavy or “bursty” traffic.

Website testing tools can help illuminate what happens as traffic and load increase. However, some of the software that can help you implement your QA Plan can be hard to use. But persevere. Here are some products that might help you in your search for a trouble-free website (note that the cost of these products will depend upon your specific needs).

Watchfire’s WebQA 2.0 Web Content Quality Testing tool (www.tetranetsoftware.com/products) helps put you in control of your website by supplying intelligence on simulated visitor interactions with site content and website transactions. Reporting, analysis, and measurement solutions provide you with a real-time view of your website. This vendor also offers, WebXACT, a free online service that lets you test single pages of web content for quality, accessibility, and privacy issues. You can access WebXACT at http://webxact.watchfire.com.

Webtrend’s Enterprise Suite (www.webtrends.com) assists you in improving the quality, performance, and integrity of your website. It can illustrate broken links, chart biggest and slowest pages, document the loading time of connections, check the syntax of various HTML components, find the availability of external servers linked to your web server, and more, even crawling your website as a user would.

Segue’s Silk Product family (www.segue.com) lets you test the performance of your website rigorously using as many simulated concurrent users as your site and network will support. Its ability to stress test web applications under heavy loads and simulate bursts of activity make it ideal for use by virtually any web-based business. In addition, its reporting features enable you to chart and to correlate response time results with server statistics to identify bottlenecks and problems quickly.

Site Mapper (http://trellian.com/mapper/) is perhaps the best of the lot. This product will analyze the content of your website and create a detailed map of all resources with an indexed listing by page and category. It also validates all links, so that visitors need never come across a “File not found” error. Some of the quality assurance features Site Mapper offers include: spellcheck of documents as they are mapped, built-in document preview functionality, the ability to integrate with your favorite document editor, it can map javascript files, stylesheets, and media files, and it can list and map external links and broken links.

Another quality service option is a full-service QA consulting firm like OTIVO (www.otivo.com). Services include browser compatibility testing, functional testing, and usability testing.

Testing the Prototype

The majority of your QA Plan should cover testing your website not only during development but also during your prototype stage before “going live.” The optimal situation would be for your website to be developed on a development server and to be launched to a staging server for QA. We realize the smaller web-based businesses will not have the resources for these additional servers. Nonetheless, your QA Plan should set out a full testing plan to ensure you tested as much of the expected customer interaction and site functionality as possible.

Since your website’s design should already meet the “browser neutral” criteria we specified in Chapter 2, you should test your design to see how it looks when viewed with:

• Different screen sizes.
• Different browser window settings.
• Different color resolutions.
• High security settings.
• Minimal feature settings.
• Different browsers and versions.
• Different client platforms.

When testing, manipulate the browser’s options settings. Change them so that the page has a white background and the links are presented in the default color. Turn on the “don’t load images” menu item, since some of your customers will have their browsers set this way. Note the results to each of the following questions.

• How does your website look with these settings?
• Can you still navigate your site with ease?
• Does your website use a text font that isn’t one of the defaults? If so, does your site look okay?
• If you used Java on your website, test it with a browser that doesn’t support Java as well as with a Java-enabled browser with that feature turned off.
• If your website requires a special plug-in, a special helper application, or special file type, test the website both with the special “whatever” installed and then without the special “whatever” installed.

If you can, test your site with WebTV and other web appliances. Some of these products are relatively primitive, generally having limited capabilities and lower resolutions. You’ll find that web pages must be reformatted so they can be read on a TV screen.

Your QA Plan should take into account each static page and each of the unique templates so they can be tested at least once in each browser configuration to ensure cross-browser layout and functionality. All other non-browser-specific functionality should be tested in at least one to three browser configurations.

Next, your QA Plan should have “no open bugs” defined as its “go live” criteria. In other words, before your website is completely available to the public-at-large, fix all problems and errors found on your site. The best way to accomplish this task is to create and maintain a “bugbase” file (see Fig. 24).

Figure 24: Example of a bugbase file.

Bugbase File
Bug	Date	Pending	Date	Deferred
Description	Fixed	(y/n)	Closed	W/Explanation

Place all reported bugs in the bugbase as they are found. As each bug is fixed on the development server, launch the fixed elements and pages to the staging server as a new version of your site. Once that is completed, change the bugs’ status in the bugbase file to “pending.” Finally, verify the “fixed” bugs and either re-open them (if they weren’t fixed) or close them (if the fix was verified). The final step is to document your testing procedure detailing the testing activity, remaining bugs, and their status (deferred for later fix, feature requests, etc.).