Introduction to QA

Somewhat understandably, expectations of the reliability of the Web and the Internet at large are still rock-bottom among its everyday users. Chances are you've had a client call you recently to say that her e-mail was down and so she might not have received anything you've sent her. Maybe you've tried to order something online recently, only to get kicked off while your credit card was being processed. Maybe your friend's ISP has been down for a couple of days and nobody in tech support seems to be able to tell him when he'll be back online again.

Scenarios like this are all too common. What is truly terrifying is that they accepted. Your client didn't sound surprised when she told you her e-mail was down. You probably didn't even curse under your breath when your online order went wrong. And your friend shrugged his shoulders and found an Internet caf downtown.

This is unique to the Internet. If you bought a new car, and it stalled on the highway just weeks later, you'd be pretty mad at your dealer. If you ordered a mocha latte before work and your barista gave you a cappuccino, you'd get mad at her and make her fix it. And if your DVD player went wrong, you'd take it back.

There are two reasons people don't get mad when the Internet doesn't work right. The first is that they reason that there's something innately complicated about the Internet, and that it must go wrong as a result. This is a non sequitur, of course; yes, the Internet is complicated, but so is your DVD player. In practice, too, the underlying infrastructure (that is, the servers, routers and so forth) of the Internet is pretty much rock solid; it's bad programmers who make things break.

The second and more crucial reason applies to those who have had previous experience with Internet application developers. Frequently, their past experiences with these agencies and individuals have been such that their expectations with present vendors have hit rock bottom. The fact of the matter is that there are few other industries in which such an explosion of dangerously underskilled labor has occurred in the past few years. This has manifested itself in both appalling freelance Web "professionals'' and some truly appalling Web agencies. Excellence is the exception rather than the rule. In effect, there are literally thousands of vendors who talk the talk but don't walk the walk.

The more cynical inference from this observation is that because your client's expectations will be low, you should aim low, too, because they're not likely to get mad at you if things go wrong. This approach, however, is not only cynical but also not commercially sensible.

Why You Should Aim High

Aiming to have a high-quality product before handing over to the client is not just about keeping the client happy; it's about maintaining your own sanity, too.

It is inevitable that your assembled system will, initially at least, be functionally defective in some way. This does not necessarily mean that it will malfunction in the traditional sense; it could be slow when real-world data is applied, or have some unforeseen business logic error. A systematic approach to quality assurance prior to handover ensures that:

  • Developer time and, hence, cost spent on resolving defects is minimized through a thorough, structured approach

  • The team's timely transition onto other projects can be closely adhered to

  • Problems encountered during deployment can be isolated as being unrelated to the application itself

Quite apart from that, it is perhaps a question of attitude. Compare the following with one another:

  • "The client's expectations are low, therefore I can underdeliver without fear of reprisal.''

  • "The client's expectations are low, therefore when I overdeliver I will be seen to excel.''

It goes without saying that being seen to excel will quickly set you apart from the competition. The client's previous experience with other suppliers may have been lackluster and their expectations low as a result, but have you noticed how they have not used those other suppliers again?

Aiming high means striving for high quality. The difference between striving and achieving starts with knowing what quality means in the first place.

What is Quality?

There is some confusion over what constitutes quality in a software application. Let us return to our analogy of a family sedan. When a motoring journalist speaks of a high-quality car, he or she is probably not referring explicitly to reliability or performance per se but rather to less tangible and quantifiable factors, such as:

  • The durability of the interior

  • The use of expensive, solid materials such as brushed chrome

  • Purely aesthetic touches, such as blue-illuminated readouts

  • The firm clunk noise made when the driver slams the door shut

Quality here is not so much a single deciding factor or even anything positive; rather, it is a function of all the areas in which something could go wrong, but doesn't.

This view is supported in the approach often taken by motoring journalists when reviewing a car or inspecting it for the first time at a trade show. The repeated activation of the cup-holder mechanism, the pressing of switches, the caressing of upholstery these are the quick-and-dirty litmus tests used to assess quality. Yet none of these tests tells us anything about the car's handling, gas mileage, reliability, or annual depreciation. So why are they important?

The truth of the matter is that although you can't judge a book by its cover, you can more often than not judge a car by its interior. A car with a smoothly ejecting cup holder, tactile, chrome-tipped switches, and seamless mechanisms is almost certainly also going to enjoy superb reliability going forward. The highest-quality brands in these exterior facets BMW, Audi, Mercedes are all consistently rated among the most reliable on the road, too.

This same rule can be applied to your Web applications. Quality is not just about whether the site stays up and does what the functional specification said it would do. It's more about the little things and, inevitably, when the little things on the surface are right, the big things underneath are right, too.

Enough about cars! What makes a Web application high quality, and how is it measured?

Measurable and Quantifiable Quality

There are a number of core areas of quality that need to be assessed when conducting quality assurance on your application. Inevitably, the importance you apportion to each one depends very much on the nature of your product and its target audience.

Functional Compliance

At the most simple and basic level, your application should provide functionality exactly as detailed in your project's original functional specification.

This goes beyond simply ensuring that no PHP warnings or errors appear, of course. The behavior of your application must be correct, under an exhaustive set of test conditions. This includes:

  • When an update button is pressed, the database is actually updated.

  • When a welcome e-mail should be sent to a new user, the e-mail is sent and is correct.

  • When a user conducts a search, the data returned is accurate against the database and the search criteria specified.

Such compliance may seem obvious, but often the most obvious points are overlooked in quality assurance programs.

Real-World Tolerance

Real World Tolerance (or RWT) means that your application should be able to cope with environments and conditions closer to its real-world, live operation than those it may run under during development.

This does not just refer to quantities of users or data on the system. It also refers to the requirement that the system be able to cope properly when a user, for example, does not enter data in an expected format. The correct behavior in such a situation should be, of course, to provide the user with an error message. It is hugely important to simulate such conditions because it is all but inevitable that users will not only themselves do bad things from time to time, but that the data they supply shall from time to time be bad, too.

Grace During Failure

There is a temptation when developing to not account for every possible scenario; certain scenarios might seem unlikely.

However, this temptation should be resisted. The only thing worse than the failure of a user's credit card to be processed successfully, for example, is failing to inform the user that his or her credit card was not processed successfully.

At any stage at which a failure could occur, however unlikely it may be, your application must cope gracefully. Consider, for example, an unreachable database server. Clearly, this is an undesirable situation and the application will not function correctly when this occurs; with luck, it is a problem that will be resolved correctly by your systems administrators. However, if not accounted for, your application may simply return zero search results on searches or tell people their login is incorrect. Both these pieces of information arise because your application does not account for the condition of the database being completely inaccessible, and your user will be severely misled as a result. Providing a simple "Sorry, the site is unavailable at present'' page offers an obvious and immediate workaround.

Load Tolerance

During functional testing, your application will have been hit by a combined audience probably not much bigger than your development team; ten, maybe eleven people simultaneously using the system.

But in practice, of course, your audience may be much bigger than this, with a requirement for a hundred, a thousand, maybe even ten thousand simultaneous users to be supported.

You will know this figure from your technical specification and will have drafted an architecture to suit. What may have worked on paper will not necessary translate to the real world, however. It is vital, of course, that it does.

If the application's response under heavy load is an unacceptable response time or, worse still, outright failure, then clearly the client will not be in a position to accept the product as satisfactory.


The latest buzzword in interface architecture these days is that of usability. Championed by leading experts such as Jakob Nielsen (see his excellent Web site at, it refers to the inherent intuitiveness and logic of a user interface. It can apply to not only the Web but also traditional client-side software applications, and even the control panel of a washing machine.

It is typically measured by the ease and speed with which a user new to the application can perform certain tasks. These tasks are expressed as objectives, and no specific instructions on their completion are issued. For example, users of a Web mail application might be asked to compose a new message to . They would not be asked to click the Compose button in the top-left corner of the screen, or enter in the recipient box, and so forth.

The observation of the user's effectiveness in carrying out these tasks is the cornerstone of usability testing, discussed shortly.


Just one step removed from usability is the veneer or finish of the product. Remember the earlier analogy involving chrome switches and cup holders in the sedan? The veneer of your application is the direct equivalent, and, because it is the first real experience users will have of your application, it provides a telling insight into the underlying quality of the application beneath.

What do we mean by veneer, however? Consider the quick-and-dirty examples throughout this book, outside the realm of our larger Sales Force Automation project. We have constructed them to demonstrate a concept, with no other purpose in mind. As a result, they've never looked or felt that great a bit rough around the edges, perhaps. That's never really mattered, however, because we've merely been demonstrating a technical concept. A finished product should never adapt this approach, of course.

The chrome switches in the driver's cabin are the properly rendered Previous and Next buttons on your pagination. The cup holders' smooth eject mechanism is the manner in which the navigation refreshes updates with each section of your application selected. This is the veneer or finish of your application and, although not a showstopper if poorly implemented, it can speak volumes about the overall quality of your application.

Many software architects and engineers are less gifted with designing the aesthetics of a user interface than you might think. As part of your own project management methodology, it is important to ensure that an experienced senior designer with interface architecture skills is on hand to ensure that the polish on your finished product stays well and truly intact.

When a small change is requested late in the project, the temptation is to cut corners and allow a software engineer to take ownership of the new feature. Do this only if you have a rock-solid style guide in place, as discussed in the chapter on project management methodologies earlier in this book. Otherwise, you may find your beautiful application blighted by the appearance of a grey rectangular Submit button just where you least expect it.

Professional PHP5 (Programmer to Programmer Series)
Professional PHP5 (Programmer to Programmer Series)
Year: 2003
Pages: 182
BUY ON AMAZON © 2008-2017.
If you may any questions please contact us: