13.4 Certification Audits

13.4.1 You Cannot Fail

It is impossible to fail certification (unless you quit). The worst thing that can happen is that it might take a little longer and cost a little more.

The final point that we wish to make in our discussion of the direct sequence manual is that you cannot fail an initial assessment, unless you simply quit. The worst thing that can happen is that is might take longer and cost more. This is an established fact for the initial systems assessment (certification assessment). One does not fail a third-party assessment; it is a part of the ISO mythology. One does get nonconformances that need to be corrected. The worst case is a major finding that could delay the certification process by up to three months and cost some more to pay the registrar's lead assessor to come back and clear the nonconformance. But that is it. This is the primary reason that so many consulting groups will agree to guarantee certification/registration ^[5].

The steward's task is to make sure that there are no major findings possible. This is accomplished via in-depth internal audits by well-trained auditors. The audits should be evenly distributed throughout the creation process and not left to the last moment prior to the document review. The audits not only increase the probability of a major nonconformance-free certification assessment, but they form the base of a dynamic corrective and preventive action program.

Inevitably there will be minor findings at the initial systems assessment, the first surveillance, the second surveillance, the recertification assessment, and the re-recertification assessment. That is what continuous improvement is all about. I still come up with nonconformances with clients that I have audited for over 8 years.

Organizations undergo all manner of change over 3 years (e.g., top management changes; mergers; acquisitions; moves to new facilities; market ups and downs; national and international tragedies, including war, floods, and fires). Without sufficient audits, the documentation falls behind reality and even the act of auditing begins to evaporate. It is equivalent to firing the sales staff because sales are down. Find the root causes, make the necessary changes to match the changed scenario, and move forward.

There, of course, can be major findings. By major findings we mean, for example, an ineffectual management review, a poorly managed training program, a lack of internal quality audits, a corrective and preventive action program that is uncertain and loosely managed. The stewards must pay close attention to these areas. One of the traps in the management review process is for the top manager to use the management review as a "rah rah" session instead of focusing on the enterprise's deviations from its planned goals based on firm and quantitative metrics. You say, "Never happens"? It does.

Another danger area is the loss of internal auditors due to downsizing, burnout, disinterest, and promotion. It is important to maintain a constantly trained group of auditors to cover such contingencies. A safe level of auditors depends on the organization's size in both people and square footage and the degree of outsourcing. Today, we have situations where the organization consists of one person in the site and everything else is outsourced. Your registrar will work with you to cover this event. It does happen and people get certified.

13.4.2 Audit Focus

An experienced assessor pays special attention to the requirements in the following:

Section 4: Quality Management System—In this set lies the superstructure of the QMS and where change is controlled, especially with regard to processes and continual improvement.
Section 5.4: Planning—This determines how closely quality objectives are planned and measured.
Section 5.6: Management Review—This somewhat prescriptive set of paragraphs contains the review of continual improvement drivers of internal audits, customer feedback, process performance, product conformity, preventive and corrective actions taken, and the manner in which top management responds to required change and opportunities for improvement.
Section 7.3: Design and Development—Special attention is to be directed to the design review, verification, and validation functions.
Paragraph 8.2.2: Internal Audit—This looks especially at whether all areas of the organization have been audited against all appropriate paragraphs and the audits have included all pertinent regulatory requirements.
Paragraph 8.5.2: Corrective Action—This applies especially the management of customer complaints.
Paragraph 8.5.3: Preventive Action—This requirement indicates clearly the degree to which the organization is either reactive to nonconformances (e.g., performs root-cause analysis on a set of nonconformances reported during corrective action) or takes a proactive perspective (e.g., performs risk analysis and designs in safety and introduces best practices to all operating groups based on improvements in one group to prevent nonconformities ^[6].) not only during the initial assessment but at every subsequent surveillance assessment. It is customary for registrars to require management review, design and development, internal audits, review of customer complaints, and review of QMS document changes to be mandatory for some percentage of the surveillance audits (e.g., every 6 months for internal audits and every 12 months for the design and development).

Special attention to these requirements ensures that the continuous improvement cycle is maintained throughout the life of the ISO 9000 program. When the Shewhart cycle is enforced, the odds are very high that the supplier will derive the benefits inherent from an effective QMS ^[7].

13.4.3 Assessor Role

Indeed, the role of the assessor is to teach and clarify. If this goal is met, the assessor feels fulfilled at the end of a long and intense audit, and the client feels that the effort was worth it. Alternately, if the assessor feels that the goal is to catch the client, both parties will end up with a feeling of uselessness, and the client will begin to seek out other registrars ^[8]. That the audit findings must be substantive, and of value to the client, is the foundation upon which the ISO third-party schema will either continue to expand or eventually decline.

In the search for added value, my most effective rule is to ask the gutoriented question: does the method sound stupid? If it sounds stupid, it is—try another approach. This works every time. I always consider whether my finding will be of economic value to the enterprise. There is a fine line between conformance to the Standard and worth to the client. No system is perfect to start with, and no system becomes perfect in the process. Organizations are in constant change through new products, new technologies, acquisitions, mergers, the vagaries of markets, and the potential horrors of nationalistic power mania.

It is vital that the organization continually stretch its processes for improvement but not stretch beyond its economic boundaries. The auditor can play an important role in this scenario. It is best to try to get inside the mind of the top executive and see what makes sense within the strategic parameters of the operation. Auditors with this perspective will find themselves welcomed back more times than not.

13.4.4 Structure of the Audit

To carry out an effective audit of the Standard requires that we apply the pertinent clauses of the Standard against every enterprise process. This also means that we also ensure that each subprocess is covered in detail. Table 13.5 uses the same core competencies as shown in Figure 1.2.

Table 13.5: Audit Plan for a Typical Manufacturing Enterprise
Processes	1. Executive	2. Marketing nd Sales	3. RDTE	4. Operations	5. QARA	6. Finance	7. Human Resources	8. Service
ISO Clauses
4.0: Quality Management System
4.1:					*
4.2.1:					*
4.2.2:					*
4.2.3:			*		*
4.2.4:			*		*
5.0: Management Responsibility
5.1	*
5.2:	*
5.3:	*
5.4.1:	*
5.4.2:	*
5.5.1:	*
5.5.2:	*
5.5.3:	*
5.6:	*
6.0: Resource Management
6.1:	*
6.2:							*	*
6.3:				*	*			*
6.4:				*	*			*
7.0: Product Realization					*
7.1:				*	*
7.2.1:		*	*					*
7.2.2:		*						*
7.2.3:		*			*			*
7.3:			*
7.4:				*	*			*
7.5.1:				*	*
7.5.2:				*	*			*
7.5.3:				*	*			*
7.5.4:		*		*	*			*
7.5.5:				*	*			*
7.6:			*		*			*
8.0: Measurement, Analysis, and Improvement
8.1:		*		*	*			*
8.2.1:	*	*			*			*
8.2.2:					*
8.2.3:	*	*	*	*	*	*	*	*
8.2.4:				*	*			*
8.3:				*	*			*
8.4:	*				*	*
8.5.1:	*				*	*
8.5.2:	*				*			*
8.5.3:	*				*			*

Our example, shown in Table 13.5, is based on a small organization hierarchy. We have assumed that the departmental processes contain the following subprocesses:

Executive: business plan, management review, and steering committee;
Marketing and sales: servicing, product managers, marketing, sales, and distributors;
RDT&E: research and development, design, product support, engineering change, and document and engineering records control.
Operations: QA&RA, manufacturing, production control, purchasing, inventory control, and shipping and receiving;
QA&RA: ISO management representative, document and record control, metrology, corrective and preventive action, audits, quality control inspection, reliability, and data analysis and trending;
Finance: human resources, management information systems, financial control and analysis, and cost of quality support;
Human resources: hiring, training, and employee development;
Servicing: customer service, repair, and installation.

The chart suggests which clauses to apply to which process and thereby suggests which employees are to be interviewed. The planned date of the audit and auditors could also be placed in the box instead the star. Other usual audit activities are also implied, such as auditing the distribution of documents throughout the facility, auditing records in various file cabinets, asking employees what they believe the quality policy means and who they think is the ISO 9000 management representative, and examining the status of training.

Unfortunately, there is no end of concern with regard to the manner in which we are to audit either (1) the requirement that no procedure is required for many clauses, or (2) the sometimes extremely descriptive language of some clauses (e.g., Clause 7.5.5: Preservation of Product). This clause is about as short and sweet as you can get with regard to a most complex and extensive issue that includes electrostatic discharge protection, shelf-life control, and a number of different types of preservation coatings as well as packaging and delivery. Fortunately, the topic of audit management has received wide recognition and many authors offer sensible ideas on how to approach the subject ^[9].

To formulate such an audit structure, it is important to realize that this process-oriented scenario has an intrinsic hierarchal structure of the type shown in Table 13.6.

Table 13.6: Possible Hierarchial Organizational Structures
Small Organization	Large Organization
I Total process	I Total process
II Departmental processes	II Divisional processes
III Functional processes (subprocesses)	III Departmental processes
	IV Functional processes (subprocesses)

13.4.5 Audit Plan for Sector-Specific Requirements

We can demonstrate the impact of a sector-specific requirement on the certification audit by means of the audit plan for sections of 4.0: Quality Management System and 5.0: Management Responsibility, as illustrated in Table 13.7. Notice that the although the assessor seeks answers to additional questions above and beyond the basic issues in ISO 9001, the questions are quite similar. The additional topics are highlighted in italics.

Table 13.7: Sector-Specific Impact on ISO 9001 Audits—Example 1
ISO 9001:2000 Element	Base ISO 9001 Assessment	Sector-Specific QS-9001 Assessment	Sector-Specific CGMP 820 Assessment	Sector-Specific ISO 9000-3 S/W Assessment
5.0: Management Responsibility	9:30 Scope Management commitment Customer focus Quality policy Quality objectives QMS planning Responsibility, authority, and communication Management representative Management review	9:30 Scope Management commitment Customer focus Quality policy Quality objectives QMS planning Responsibility, authority, and communication Management representative Management review Business plan Analysis and use of company-level data Customer satisfaction	9:30 Scope Medical class Quality objectives Customer complaints Management commitment Customer focus Quality policy QMS planning Responsibility, authority, and communication Management representative Management review	9:30 Scope Management commitment Customer focus Quality policy Quality objectives QMS planning Responsibility, authority, and communication Management representative Management review Customer's management responsibility Organization and customer joint reviews
4.0: Quality Management System	10:30 General requirements Documentation requirements Quality manual Control of documents Factored items Interface issues Currency of Standards and codes/statutory/ regulatory	10:45 General requirements Documentation requirements Quality manual Control ofdocuments Control of records Control plans Special characteristics	10:45 General requirements Documentation requirements Quality manual Control of documents Quality plans Control of records with: design history file (DHF)	10:45 General requirements Documentation requirements Quality manual Control of Documents Control of records Life-cycle planning Factored items
		Use of cross-functional teams Feasibility reviews FMEAs Factored items Interface issues Currency of Standards and codes/statutory/regulatory	Device master record (DMR) Quality system records (QSR) Factored items Interface issues Currency of Standards and codes/statutory/regulatory	Interface issues Currency of Standardsand codes/statutory/regulatory

As indicated, more time is needed in the sector-specific cases because there are more SHALLS to cover and there is an increase in concomitance (e.g., there are additional sections in QS-9000 compared to the five in the Standard ^[10]).

The manner in which the organization provides answers to the additional questions is in exactly the same way that quality policy statements are used to respond to each SHALL of the Standard. In a previous book we demonstrated this technique and took an example from each of the three specific sectors shown in Table 13.7 ^[11]. We have repeated this work because the technique is invariant under the many changes that standards are scheduled to undergo. As a result, the exact language of the quoted standard may change but the method remains valid.

This discussion includes a more recent set of requirements in the medical device industry (i.e., we will examine the specific impact of the FDA CGMP 820, EN46001:1996, and ISO 13485:1996 on a manual:2000). Table 13.8 illustrates how this second set of medical device requirements are inter-twined for two typical ISO 9001:2000 sections. Note that at the time of this writing, both EN46001 and ISO 13485 were still in the ISO 9001:1994 format. This situation has already caused some confusion in manual:2000 creation. However, as we have seen, cross-reference charts provide a quick way to harmonize the requirements and do not invalidate the suggested techniques ^[12].

Table 13.8: Sector-Specific Impact on ISO 9001 Audits—Example 2
ISO 9001:2000 Element	Base ISO 9001 Assessment	Sector-Specific EN46001 Assessment	Sector-Specific CGMP 820 Assessment	Sector-Specific ISO 13485 Assessment
4.0: Quality Management System	10:30 General requirements Documentation requirements Quality manual Control of documents Factored items Interface issues Currency of Standards and codes/statutory/regulatory	10:45 General requirements Regulatory requirements according to class Documentation requirements Technical files Quality manual Control of documents Control of records Factored items Interface issues Currency of Standards and Codes/statutory	10:45 General requirements Documentation requirements Quality manual Control of documents Control of records with: DHF DMR DHR QSR Factored items Interface issues Currency of Standards and codes/statutory/regulatory	10:30 General requirements Regulatory requirements according to class Documentation requirements Technical files Quality manual Control of documents Control of records Factored items Interface issues Currency of Standards and codes/statutory/regulatory
7.3: Design and Development +7.2.1(c): Statutory and Regulatory Requirements (S&R)	1:00 Design and development Planning Inputs with S&R Outputs Review Verification Validation Change control	1:00 Design and development Procedure Planning Inputs with S&R and with safety Outputs Review Verification with clinical investigation Validation Change control	1:00 Design and development Procedure by class Planning with review and approval Inputs with S&R and with intended use Outputs with signatures Review with DHF Verification with DHF Validation with pilot runs, S/W risk and validation, production specification Change control with DHF	1:00 Design and development Procedure Risk analysis Planning Inputs with S&R Outputs Review Verification Validation with clinical investigation Change control

For completeness, the sector-specific requirements for software are also shown in Table 13.7 based on ISO 9000-3, the guidelines for the application of ISO 9001 to the development, supply, and maintenance of software ^[13].

13.4.6 Tip of the Iceberg

When the day of the initial assessment arrives, it is important to realize that the assessors' observations represent the tip of the iceberg (see Figure 13.3). They only see what they need to see in order to assure themselves that the supplier has a workable QMS that will most likely produce a reasonable payback in a reasonable time. At least 90% of the nonconformances lie below the surface.

click to expand
Figure 13.3: The tip-of-the-iceberg effect.

You, of course, know exactly what they are, and the assessors rely on you to make those corrections as part of an effective QMS program—especially by means of the internal audit process and, indeed, where applicable, audits of your suppliers.

It is not uncommon to feel that you have fooled the assessors once they leave. On the contrary, if you have, it is really a case of biting your nose to spite your face. They saw it, but did not have the time to investigate. On the other hand, you know it is there. So you need to fix it.

Otherwise, you can bet it will be found in a surveillance audit. Worst yet, it is a hole in the system through which profit dollars fall—and that is the whole point of an effective QMS—to fill those holes.

13.4.7 Dynamics of the Initial Assessment

At the close of the initial assessment, the lead assessor recommends certification, either with or without condition. The registrar's executive board approves and issues the registration numbers and certificates. The several possible conditions for approval include the following (these vary considerably from registrar to registrar):

All NCRs cleared during initial assessment—recommend certification without condition;
Minors left to be cleared after initial assessment, but plans accepted—recommend, certification but hold issuance until all are cleared or hold clearance for first surveillance;
Make sure there is a clear plan to be followed up at first surveillance;
Some minors can be declared concerns to be monitored at the first surveillance;
Opportunities for improvement—potential economic savings; these are to be acted upon at the discretion of the auditee.

The exception is in regard to major nonconformances. They are usually treated as follows:

Majors left to be cleared during initial assessment require a return audit of those areas within usually 90 days, then recommendation to certify ^[14].
Majors can be downgraded during the initial assessment to avoid this problem. The resulting minor can then be treated as discussed in the recommended-for-approval protocols. Downgrades are highly discretionary on the part of the lead assessor and must be examined in the context of the observed overall effectiveness of the audited QMS. Some registrars have strict protocols for downgrades.

What is abundantly clear during the initial assessment is that the essence of the Standard is to state with great clarity who manages, performs, verifies, and validates the processes and subprocesses for documentation, implementation, and demonstration of effectiveness.

^[5]"ISO 9000 Consultants Guide," Quality Digest, May 2001, p. 69, at http://www.qualitydigest.com.

^[6]See, for example: Hiebler, Robert D., Thomas B. Kelly, and Charles Ketteman, Best Practices: Building Your Business with Customer-Focused Solutions, New York: Simon & Schuster, 1998, and Camp, Robert C., Business Process Benchmarking: Finding and Implementing Best Practices, Milwaukee, WI: ASQ Quality Press, 1995.

^[7]See, for example: Hendricks, Kevin B., and Vinod R. Singhai, "Don't Count TQM Out," Quality Progress, April 1999, p. 35, and Tai, Lawrence S., and Zbigniew H. Przasnyski, "Baldrige Award Winners Beat the S&P 500," Quality Progress, April 1999, p. 45.

^[8]The selection of third-party assessors is integral to the selection of a registrar. For a complete exposition on this topic, refer to Weightman, R.T., "How to Select a Registrar," Quality Systems Update, August 1996. Mr. Weightman is the president of Qualified Specialists, Inc., Houston, TX. Also see, Russell, J.P., The Quality Audit Handbook, Second Edition, Milwaukee, WI: ASQ Quality Press, 2000.

^[9]See, for example: Russell, J.P., "Auditing ISO 9001:2000," Quality Progress, July 2001, p. 147, at http://www.asq.org.

^[10]The shoulds of the QS-9000 quality system requirements are to be treated the same as the SHALLS of ISO 9001. Should, in this case, indicates a preferred approach. It is not to be confused with the notes of ISO 9001 that are not mandatory, but are used as an interpretive aid.

^[11]Schlickman, Jay J., ISO 9000 Quality Management System Design: Optimal Design Rules for Documentation, Implementation, and System Effectiveness, Milwaukee, WI: ASQ Quality Press, 1998.

^[12]For an extremely lucid discussion of the ISO 9001:2000 and ISO 13485 issue, see Kimmelman, Edward R., "Is ISO Obsolete?" Medical Device and Diagnostic Industry, October 2001, p. 76. Mr. Kimmelman is currently the convener of the ISO/TC210, Working Group 1, on quality systems for the medical device industry.

^[13]Software development standards include the Carnegie Mellon University Software Engineering Institute capability maturity model for software, which has become a de facto standard for bids to the Department of Defense and NASA, as well as the IEEE/EIA 12207:Software Life Cycle Processes. All standards of this type can be analyzed and integrated into a QMS using this book's design techniques. See also, Rakitin, Steven R., Software Verification and Validation: A Practitioner's Guide, Norwood, MA: Artech House, 1997, p. 7.

^[14]Although it is possible to have the registrar declare the organization noncertifiable, I know of no such case in the hundreds of certifications with which I am familiar. The only situation under which this might occur, to my knowledge, is if the facility has obvious safety and/or hazardous waste nonconformances so that the assessors cannot perform their audit in a safe manner.