The focus of the tester is primarily about Objective Assessment. Testers offer their services to other parts of the development organization to help assess the software product based on appropriate criteria, such as perceived quality, conformance to standards, and defect discovery. The evaluations provided by this service help other team players (developers, managers, even customers) to make the right decisions about their next actions, based on demonstrable and objective information. Anyone who is serious about producing an excellent product faces two problems in particular:
The answer to the first question determines when an organization releases a product. The answer to the second question prevents the organization from releasing a bad product. The Concept of Product Quality in the RUPYou may be thinking, "I don't want to ship a merely satisfactory product; I want to ship a great product." Let's explore that. What happens when you tell your coworkers, your management, or your investors that your quality standards are high, and that you intend to ship a great product? If it's early in the project cycle, they probably nod and smile. Everyone likes quality. However, if it's late in the project cycle, you're probably under a lot of pressure to complete the project, and creating a great product may require that you engage in extensive testing, fix many problems (even small ones), add features, or even scrap and rewrite a large part of the code. You will also have to resolve disputes over different visions of "good" quality. Greatness is hard work. Perfection is even harder. Eventually, the people who control the project will come to you and say something like, " Perfection would be nice, but we have to be practical. We're running a business. Quality is good, but not quality at any cost. As you know, all software has bugs ." Greatness can be a motivating goal. It appeals to the pride you have in your work. But there are problems with using what amounts to "if quality is good, more quality must be better" to justify the pursuit of excellence. For one thing, to make such an argument can make you seem like a quality fanatic, rather than a balanced thinker. For another thing, it ignores the cost factor. In leaving cost out of the picture, the "more is better" argument also ignores diminishing returns. The better your product, the harder it is to justify further improvement. While you labor to gold plate one aspect of a product, you must ignore other aspects of the product, or the potential opportunities presented by other projects. Every day, the business has to make choices about the best use of resources, and it must consider factors other than quality. The RUP product has incorporated and extended the concept of Good Enough Quality (GEQ) from the work of James Bach. [2] This GEQ concept provides, paradoxically, a more effective argument than "more is better," because it provides a target that is either achievable or not achievable, in which case it becomes a de facto argument for canceling or rechartering the project.
Paradigms of "Good Enough"Most businesses practice some form of "good enough reasoning" about their products. The only ones that don't are those that believe they have achieved perfection, because they lack the imagination and skill to see how their products might be improved. Here are some models of the "good enough" approach. Some of them are more effective than others, depending on the situation, but all have their weaknesses:
The Cost of QualityIs a high-quality product necessarily more expensive? Depending on a lot of factors, such as process, skill, technology, tools, environment, and culture, you may be able to produce a much higher-quality product for the same cost than would otherwise be possible. A more testable and maintainable product will cost less to improve over the long run. Conversely, there are costs associated with poor quality, such as support costs and costs to the customer. The cost of quality is a complex issue, and it is difficult to make broad generalizations . However, we can say with certainty that we can always spend more time on much better tests, much more error handling, and fixing or rewriting every part of the product. No matter how good you are, quality does cost something. And if you can't think of more improvements to make, it's more likely that you've reached the upper limit of your imagination, not of quality. There must be a point of "diminishing returns," a point where your "test ROI" becomes negative. In the software industry, GEQ is viewed more as a response to one particular cost over any other ”the cost of not releasing the product soon enough. The specter of a market window, or an external deadline, imposes tangible penalties upon us if we can't meet the challenge to deliver on time. That's why the project endings are so often characterized by frenzied triage. If you want to know what an organization really believes is good enough and how well prepared they are for it, witness the last three days of any six-month software project; see what happens when a new problem is reported on the last day. Wouldn't Quantification Help?It can be tempting to reduce quality to a number and then set a numerical threshold that represents good-enough quality. The problem with that is you can measure only factors that relate to quality. But you can't measure quality itself. This is partly because the word "quality" is just a label for a relationship between a person and a thing. The statement "this product is high in quality" is just another way of saying, "Somebody values this product." It's a statement not only about the product, but also about people and the surrounding context. Even if the product stays the same, people and situations change, so there can be no single, static, true measure of quality. Read (or reread) Robert Pirsig's Zen and the Art of Motorcycle Maintenance , which states, "At the leading edge there are no subjects, no objects, only the track of Quality ahead, and if you have no formal way of evaluating, no way of acknowledging this Quality, then the train has no way of knowing where to go." [3]
There are many measures you might use to get a sense of quality, even if you can't measure it completely and objectively. Even so, the question of what quality is good enough requires sophisticated judgment. You can't escape from the fact that, in the end, people have to think it through and make a judgment. For a simple product, that judgment might be easy. For a complex, high-stakes product, it's very hard. Making this call is not the responsibility of the tester, but the tester does provide the information for this decision. Finally, the definition of quality also depends on the problem space; quality for NASA or for surgical automation has a very different meaning from quality for a customer service rep's workstation and a different meaning from quality for a stock exchange. Conformance to StandardsAssessment implies the comparison of the product to some standard. One standard of reference that often comes to mind is the requirements ”the " specs " for the system. There may be other standards referred to, either formal standards, such as a Usability or Accessibility standard, or implicit standards, such as those related to look and feel (for example, we want it to match Windows 2000 conventions). There are also standards that are industry-specific : standards for the biomedical industry, the pharmaceutical industry, the aeronautical industry, and so on. In practice, assessment is not just about comparing the software product against the requirements and other specification artifacts. We use ”implicitly or explicitly ”other standards for determining whether quality is acceptable. There are two important concerns that often require a broader consideration than the documented specifications:
A diligent tester will consider tests that help to expose issues that arise from implied and omitted requirements. The Quality Assurance Plan, which is part of the Software Development Plan and under responsibility of the Project Manager, defines the overarching approach to GEQ on the project and the techniques that will be used to assess whether an acceptable level of quality is being achieved (see also Chapter 12). |