Quality Control

Although quality is one of the common constraints all projects share, it is an area that does not always receive the same amount of focus as the scope, budget, or schedule. However, lack of quality management can have severe negative impact on the project. Quality control is the process of reviewing project results to confirm compliance with defined standards and making appropriate changes to remove causes of unacceptable quality. The quality management plan discussed in Chapter 6 is the foundation for the specific activities carried out during quality control. The quality activities, the procedures used to complete the quality activities, and the resources required are documented in the quality management plan.

Quality control is done throughout the project. As we have mentioned in earlier chapters, milestones are often included in a project schedule to mark the completion of a project phase or major deliverable . Quality tools and techniques are used to determine compliance with a minimum sufficiency standard, and quality activities are often a key part of the formal process of approving the completion of a phase.

We will focus on the use of testing to monitor the project work results, as well as mention some other tools and techniques that are used in quality control. The results of your quality control activities may result in rework , process changes, or acceptance of any defects that are found.

Inspection (Testing)

Inspection is a broad category that includes examining, measuring, or testing work results. The most common use of inspection on IT projects is testing.

Testing is something you should always try to devote extra time to. Testing is a boring job- someone has to run the code over and over again, testing different things in different modules, taking notes about its performance. But it's got to be done. You should have at your disposal testers who are ready and willing to thoroughly run your new code through its paces.

Testing is done throughout the project, and several kinds of testing are common on IT projects:

Module Testing A programmer completes a module of code and needs to test it. Some modules don't lend themselves to module testing, because they're designed to interface with other modules. Still, the programmer can check that variables load correctly, that the code goes to the places it's supposed to go to depending on the input, and that memory gets cleaned up and the program exits correctly. Often developers can run code through a checker that steps the code line by line to see how it behaves and how it loads memory variables . The process can be at once very interesting and yet extremely frustrating, because the developer has something going wrong but can't figure out where-all of the code appears to be doing what he or she wants it to do.

At this stage of the game, dynamic link libraries (DLLs) and other informational files are also tested for complete and accurate content.

Unit Testing Once several modules that go together have been satisfactorily tested, the developer can test them all at once in unit testing. They might test an entire printing system or a set of algorithms that calculates something. They're testing the functionality of pieces that have been put together to form a cohesive group .

System Testing Next, the developers test the entire system as a whole. They make sure the system flows as expected, that the user doesn't encounter unexpected loops or gotchas (such as 'deer in the headlights' frozen screens), that the system is fast and functional, and that it delivers what the customer is expecting. System testing should take a long time, because they thoroughly test each component and the whole.

User Acceptance Testing (UAT) This is the time when they actually bring in a small set of users to begin testing the deliverable. By the time developers get to UAT, they should've worked out all bugs , speed or logic issues, and flow problems. The system should be at its best, most pristine state. This is the system they currently expect the user to utilize in production-until the UAT testers find the problems others missed.

Factory Acceptance Testing (FAT) Sometimes a system is so large it's better if testing by the users is done on site at the place where the initial development was done. Think of the development of a new radar system for weather forecasting in which there is new gear being developed plus new software. All of this is done at the vendor's site in preparation for delivery to the new location once the customer has accepted the new system. FAT is part and parcel of big government contractors such as Raytheon and other companies that specialize in big development work.

Site Acceptance Testing Site acceptance testing involves customer testing at the customer premises. Consider the radar system example above. A radar specialty contractor has developed the new radar system and all of its elements. The customer has come out to the factory for FAT and finds the system acceptable. Now the company delivers the new system to the customer and they go through another round of testing to make sure that all things are operational and that the system operates as expected.

Testing is always an important step in confirming the quality of an IT project, and in a couple of scenarios you may need to conduct even more testing.

Geographically Dispersed Team Project teams are often dispersed across multiple locations. A major software deliverable may be the result of code compiled from modules written by several programming teams. Even though you have established communications and work standards for the teams , the lack of daily face-to-face contact can increase the risk of misinterpretations, which could lead to differences in the modules that may impact the compiled package. In this situation, you may want to have more detailed testing on the individual modules to catch problems early on.

Vendor Deliverables Another scenario where testing takes on significant importance is when a vendor produces deliverables. The specifics of how and when you can test during the development process should be included in the contract. Integration testing of the vendor deliverable with the rest of the system is commonly a requirement for final acceptance of the vendor work. Acceptance of a vendor deliverable equates to vendor payment, so these tests need to cover all requirements associated with the work the vendor was contracted to produce.

Although testing is the most frequently used for quality control in IT projects, it is not the only method available.

Other Quality Control Tools and Techniques

Anyone who has been on an IT project is probably familiar to some degree with the amount of testing that is conducted over the course of a project. Other tools and techniques can be used in combination with testing to address quality defects.

Pareto Diagram A Pareto diagram is used to rank the importance of a problem based on its frequency of occurrence over time. This diagram is based on the Pareto principle, which is more commonly referred to as the 80/20 rule. The Pareto principle is named after Vilfredo Pareto, an Italian sociologist and economist , who observed that 80 percent of the wealth in Italy was held by 20 percent of the population. This principle has been applied to many disciplines since Pareto first discovered it. Applying the principle to quality control, it says that the majority of the project defects are caused by a small set of problems. A Pareto diagram helps to isolate what the major problems are, so that you can take action that will have the greatest impact. A bar graph is used to display problems in decreasing order of occurrence so that priorities for improvement can be established.

The purpose of a Pareto diagram is twofold:

It displays the relative importance of the defects.
It directs the improvement efforts to those areas that will have the biggest impact.

Let's take a look at how this works. A Pareto diagram typically starts with a table that lists information regarding the frequency of the defects or failures uncovered during testing. Table 9.1 shows the following frequency of failure for items A-E: the number of occurrences, the percent of defects that this item represents, and a cumulative percent.

TABLE 9.1 : Frequency of Failures
Item	Defect Frequency	Percent of Defects	Cumulative Percent
A	800	.33	.33
B	700	.29	.62
C	400	.17	.79
D	300	.13	.92
E	200	.08	1.0

With this data in hand, you can create a Pareto diagram, as shown in Figure 9.1 The bars are ordered from left to right based on the frequency. The bars depict the defect numbers , and the cumulative percentages are plotted using the circles. By looking at the data in Figure 9.1, you can see that the most significant problems you want to focus on are A and B. Fixing these two items will resolve over half of the defects.

FIGURE 9.1 Pareto diagram

Control Charts A control chart is a picture of the variance of several samples of the same process over time. It is most commonly used in manufacturing. A control chart is based on a mean, an upper control limit, and a lower control limit. The upper control limit is the point beyond which preventing additional defects becomes cost prohibitive. The lower level is the limit at which the client or end user will reject the product because of the defects. The goal is to stay in the middle area (the mean), where the best product for the lowest cost is obtained. An example of a control chart is shown in Figure 9.2.

FIGURE 9.2 Control chart

Statistical Sampling If you have numerous work results that require inspection or testing, you may decide to use statistical sampling, which gathers a subset of all the applicable work results and randomly selects a small number for testing or examination. Statistical sampling can be very cost effective, especially in projects where multiple versions of the same product are produced.

Flowcharting Flowcharting was discussed in the quality planning section of Chapter 6 as a means to create the process that produces the product. Flowcharting can also be an effective tool during quality control to help determine the how the problem occurred.

Trend Analysis Trend analysis is a mathematical technique that can be used to predict future defects based on historical results. We will discuss trend analysis in more detail later in this chapter.

The results you obtain from testing or other quality control tools and techniques are used to determine whether any action should be taken to correct poor quality.

Quality Control Actions

As you implement your quality control activities, you need to make decisions on the appropriate course of action based on the results received. Any action taken to resolve quality problems has trade-offs, so you will need to involve other stakeholders in the decision process. The most common actions taken as a result of quality activities are rework, process adjustments, and acceptance.

Rework

Rework is any action that is taken as a result of quality activities to correct a defect. A module test may result in the rewrite of a section of code from the module tested.

Rework sounds like the ideal solution to any quality problem that is found. If you discover a problem, you should fix it-right? In an ideal world with no time and budget constraints that would be true, but rework often impacts both the project schedule and the budget. The time to complete the deliverable will be longer than estimated to account for the time it takes to fix the problem, and the people doing the rework will be billing additional hours to your project.

Note	There may be exceptions to the increase in billable hours, if you are working with a vendor and have quality standards written into the contract. However, you should always consider the possible financial impacts.

A decision on rework is often tied to the severity of the defect and its impact on the ability of the end user to use the product. Your client, sponsor, and other impacted stakeholders need to be involved in rework decisions.

Process Adjustments

Changing a process can ripple throughout the rest of the project. Unless it is very clear that a process change is contained to a small work group or a few team members with no downstream effects, it is best to use the change control process to analyze the impacts of a process change and obtain formal approval before making any changes.

Acceptance

Acceptance is the decision to accept any defects that are found as a result of the quality testing. Acceptance is a common action based on analysis of the severity and frequency of the defects uncovered during testing. For example, some commercially available software products are released for sale to the public with known defects to be fixed with an upgrade later. Meeting the publicized release date for the product is more important than fixing defects; in other words, the schedule takes priority over quality. The overall impact of accepting a defect should be analyzed and communicated to project stakeholders. You need to get stakeholder sign-off to accept defects.

Another important aspect of quality control is the documentation quality.

Documentation Quality

IT projects often produce documentation, both technical and nontechnical. Your quality control activities include ensuring the quality of all project- related documentation and other material turned over to groups who use or manage your deliverable. This includes user documentation, user training, help-desk training, and documentation for other support groups.

User Documentation

Any documentation-whether online or paper copy-that the user requires in order to use the system should be proofread for spelling, grammar, and content. All instructions regarding the use of the system should be tested to confirm accuracy. Paper copies should be prepared and bound for distribution to the users. Help-screen documentation should be complete and thorough. There is nothing more annoying than hitting F1 for help only to find 'No help is available for this topic' or extremely minimal information.

User Training

User training may be instructor led, a self-paced workbook, or online. Regardless of the delivery medium, the curricula should be checked for content and completeness. A pilot class may be held to confirm that the training is thorough and understandable and that any online practice features of a training database work correctly. The trainers who will provide or support the training should be educated on the new system and ready to go, and any training documentation should be printed and bound, ready for class. Training materials such as demonstrations and visuals should be ready. The scheduling of training classes should be part of the project schedule to ensure that users will be ready for the deployment.

Help-Desk Training

IT project managers are often good about considering end-user training and getting classes ready for users, but they may overlook the person in the trenches, the help-desk technician, who will wind up getting calls from users about how the new system works. Take into consideration where the help desk is going to fit in terms of supporting the new system, and prepare help-desk technicians in advance with the training they will need to support users. Help-desk training should include all of the aspects of end-user training as well as the more detailed aspects of resolving problems and troubleshooting. All of the proofreading and testing discussed in reference to user training quality also applies to help-desk training. Especially during the early phases of new system release, users are going to have questions about how the system works and may turn to the help desk for answers. The help desk needs to be ready to go at deployment time.