What s Not Working: The Traditional Tools for Controlling Quality

What's Not Working: The Traditional Tools for Controlling Quality

So, now I have pointed out some of the problems associated with the ideas underlying what quality assurance is. But how do we control quality and bring about effective testing to ensure effective products and services? To determine that answer, we need to examine some of the tools used by traditional quality assurance people and see where there is room for improvement.

What, then, is quality control? The British Standard 4778 defines quality control as "the operational techniques and activities that are used to fulfill requirements for quality."

We try to control quality and defects in software by instituting processes to control design, coding, testing, delivery, and so forth. The idea is that if we standardize the process by which something is done, we can establish repeatability in the process and reliable uniformity in the product of the process. Further, we can automate a standardized process, eliminating the need to have people participate in it at all. People are nonuniform by nature. Process is a good tool for minimizing the impact of (nonuniform) people on a project. But, when all is said and done, software is written by people to be used by people. It is this fact that may ultimately be our salvation even though it is proving to be a challenge during this time of transition.

Software development is a creative, innovative process. Quality assurance principles were developed for manufacturing, which is based on repeatability and is rarely innovative. Traditional quality assurance principles are a poor fit for the needs of today's fast-paced software development environment.

Traditional quality assurance principles equate quality primarily with reliability. This definition is too simplistic for today's market, which means that the priorities of traditional quality assurance are out of synch with the needs of the software development community.

Manufacturing has not been excessively concerned with human factors. Manufacturing was founded in an environment where the largest asset was machinery, where repetitive mechanical processes could be institutionalized by managerial decree, and where the role of people was to service the process.

In entrepreneurial software shops, the largest asset is the people and the intellectual property that they generate. Only a process that is accepted by the people doing the work will succeed. And only a process that treats people as assets will be accepted.

Traditional QA and Testing Tools Can't Keep Up

The following are traditional tools used by quality assurance and software testers:

• Records. Documentation that keeps track of events, answering the questions when, where, who, how, and why.

• Documents. Standards, quality plan, test plan, process statements, policy.

• Activities. Reviews, change management, version control, testing.

These are primarily paper-dependent tools. Most traditional quality assurance tools rely on paper. It is virtually impossible to perform work that is more precise than the tools used to create the work. How can quality assurance and test groups be expected to be more effective than their best tools?

The Paper Problem

Paper is the biggest single impediment to software quality today. It is no coincidence that document inspection and reviews are the most effective ways to take bugs out of our software. Inspections and reviews test the paper documentation. Paper documentation is the single biggest bug repository in software development. In addition to the number of design errors, mis-communications, ambiguities, and fallacies we introduce and entrench in our products, the number of errors introduced by outdated or discrepant paper documentation is a major quality problem. Furthermore, the creation and compilation of paper documents is expensive and slow.

The main problem with traditional quality control in RAD is that the productivity-enhancing tools used by software developers have far outdistanced the paper-producing tools used by quality assurance groups, testers, and documentation groups. The development of software proceeds at a pace that is faster by several orders of magnitude than the knowledge transfer, composition, layout, and review of paper documentation. The result is paper-producing groups not keeping up with the pace of development.

The distribution of information through paper documents is expensive and slow. Paper documentation is typically at least somewhat out-of-date by the time it is printed. When we need to distribute information to more people, we make more paper copies. When we need to update information, we must make enough copies to replace all existing earlier versions and try to distribute these new paper copies to everyone who had a copy of the earlier version. This is a manual version control process. It cannot hope to keep all distributed information fresh.

In the time it takes to explain the paper problem, I can make major changes in the functionality of a software application, recompile it, and have it ready for testing. The paper documentation is now out-of-date.

In general, development takes hours to build a release, but change management needs days to approve the release and the testers need weeks to test it. Meanwhile, the design changes daily, and documentation simply cannot keep up.