What is a major non-conformity?

In simple terms, a non-conformity is when a applicant or certificate holder (the client) fails to conform to a requirement for certification. All of the requirements that apply to the client should be written in the standard.

Grading non-conformities

One of the responsibilities of the auditor is to grade a non-conformity; that is to determine whether or not it is really serious or a small matter. Most schemes use the terms major and minor to grade non-conformities. Other schemes use terms such as critical to describe non-conformities that are so serious that it means the client automatically fails the audit.

There are two approaches that a scheme may uses to determine how to grade a non-conformity. And just to complicate things there are hybrid approaches that some schemes use which combine elements of these two approaches.

The first approach, and generally the most common, is for the auditor to take a decision about how serious the non-conformity is based on a set of criteria. An example of the criteria used to grade a non-conformity are as follows:

Major Non-conformity    

A Major non-conformity is normally raised when one or more of the following are found:

• The absence or total failure to meet a requirement and the failure has or is likely to result in the client failing to achieve the objective of a requirement.
• The non-conformity is highly likely to result in a breakdown of an requirement or materially reduce the ability to achieve the objective of a requirement.
• A minor non-conformity that is shown to continue over a specified period of time or occurs repeatedly.

Minor Non-conformity    

A minor non-conformity is usually raised when the client does not meet the requirement and the non-conformity does not jeopardize the integrity of the scheme. This includes one or more of the following:

• Where there is a failure to comply with a requirement is not likely to result in the breakdown of a system to meet a requirement and will not risk the integrity of the products or services that are being certified by the scheme. 
• Where the failure  does not meet the definition of a major non-conformity.

In brief an auditor must determine if the products or services that are being certified are likely or not meet the objective of the scheme. An example is for a scheme that requires that daily records be kept. If the auditor finds only a few records on file the non-conformity is likely to be graded as a major. If only a few days of records are missing, the non-conformity is likely to be graded as a minor.

In the second approach, the scheme determines in the standard how a non-conformity for each requirement is to be graded. 

In these schemes the standard is written so that each requirement is graded and for any non-conformity, no matter how serious is graded as specified in the standard. Schemes that use this approach sometimes use the category of 'critical' for some requirements.  A failure of a critical requirement means an automatic failure of the audit. For example for a non-conformity raised against a requirement that daily records be kept which is graded in the standard as a major in the standard would be graded as a major non-conformity even if only one day's records are missing.

For both of these approaches, an auditor may raise an observation for cases when the auditor finds cases where a non-conformity is likely to occur. Observations are intended to be helpful to the client so that they can address problems before a non-conformity occurs.

A Hybrid approach

Some schemes may wish to grade some requirements in their standard as critical or major, and leave the remaining requirements un-graded. The result is a scheme in which the auditor is required to grade certain non-conformities a certain way while giving the decision about grading for the rest to the auditor based on the severity of the impact.

Hybrid approaches are used most commonly for schemes that wish to specify which non-conformities will result in an automatic failure of the audit, that is to grade just the critical requirements and leave the rest to the auditor to grade according to the severity of the non-conformity.

Choosing an approach to grading non-conformities

Most schemes will decide which approach (or hybrid version) they will used based on their own needs. Often this is driven by the question of how the scheme owner chooses to approach an audit.  In cases where the scheme wishes to ensure that all non-conformities are graded identically across all clients, the second approach may be preferred. In other cases, the scheme owner may prefer to focus less on designing the standard and wishes to have a professional auditor take a decision about grading. 

Neither is better, neither is worse, it is a matter of preference.

Have you ever heard of "Root Cause Analysis"?

How do you make sure that a problem is fixed?

Throughout your certification system there are many points where non-conformities may be found these include:

• Certificate holders or applicants can have a non-conformity raised by an auditor or they could find an error through their internal procedures.
• Certification bodies can have a non-conformity raised by their accreditation body, or can find a error through their own internal audit process or as a result of a complaint.

• Logo license holders may be found to be out of conformity with the terms of licensing rules.

• And yes, even the scheme owner may find through an internal audit or complaint process that it has made an error.

Finding an error or having a non-conformity raised is not the end of the world. The most important factor is what is done about it.

Correction and Prevention

In the world of standards we talk about 'corrective' and 'preventive' actions. These are the things that are done by the organization that has been found to have a non-conformity or has identified an error that it has made.

'Corrective' actions are the things that are done to correct the error. For example if a certificate holder did not track the sales of certified product then it may 'correct' the non-conformity by going back into its records and making sure that sales that have been made are properly recorded.

'Preventive' actions are the things that are done to make sure that the non-conformity does not reoccur. This is where a challenge presents itself - How do you know what caused the problem in the first place? In the case of the sales records you could say the cause was the clerk that entered the data, the software used to record the information, the sales staff not specifying all the required the sale information, the office procedures or a whole bunch of other possibilities. If the action chosen does not fix the root cause then the problem may reoccur. Retraining the sales staff, for example will not fix a software problem.

Root Cause Analysis or Things may not be what they seem

Root Cause Analysis is a way to identify the real cause so that the fix can be directed to the source and ensure that the non-conformiety does not reoccur or so that a new problem does not occur from the same root cause.

There are number of methods available for identifying root causes, these are called root cause analysis methods. The one that fits best with you and your organization can be a subjective choice so I will not recommend that you choose a particular method. Some basic information can be found on Wikipedia or through a web search. There are also load workshops on root cause analysis on offer that your web search will pull up.

The most generic method that I have found is the '5 Whys'. In this method the question 'why' is asked first of the identified error, then the question is asked again of each answer at least 5 times.

On the sales records example:

Question 1: Why were the sales records wrong?

Answer 1: The clerk did not enter all the required information.

Question 2: Why did the clerk not enter all the required information?

Answer 2: The clerk was never trained on how to record sales information.

Question 3: Why was the clerk never trained?

Answer 3: The clerk has been on the job for 6 months and no training has been scheduled.

Question 4: Why has no training been scheduled?

Answer 4: Training is only offered once each year.

Question 5: Why is training only offered once each year?

Answer 5: The training budget has been cut each year for the last 5 years.

As a result of this Root Cause Analysis the proper preventive action would be to fix the training deficit one way would be to make sure that the budget allows for timely training of staff.

If your first thought was to fire the clerk then this action would not solve the problem and it is likely that it would would occur with the next clerk. It is also likely that other seemingly unrelated problems would occur in other parts of the organization because of lack of proper and timely training.

The root cause can sometimes be simple, but it is never a good idea to simply jump on the first idea you have and assume that the problem will be fixed. Often our first idea is more a reflection of our own bias (clerks are lazy) and not a considered analysis of the real cause (management is cheap).  

For-Profit or Not-For-Profit?

The design of governance structures is one of the key challenges that a new scheme will confront. This includes what new corporate bodies, if any, should be created, how they are to be structured (for-profit, not-for-profit, charity, co-op, benefit corporation, etc.), and how and by whom are they are to be governed.

Creating governance structures can be a challenge for a number of reasons, first it includes balancing a number of issues such as how to avoiding conflicts of interest, delivering quality services at a fair cost and maintaining stakeholder interest and support. Also, in most cases the folks that have been working to create a new scheme are experts in the topic addressed (i.e. agriculture, human rights, etc.) and are not usually well versed in how to set up of an international business operation. To make things worse, the learning curve can be steep and the pressure can be intense to make a decision quickly.

The first and over-riding concern is to make sure that the new operation is not hampered by built in conflicts of interests. This means that the part of the operation that is required to take independent decisions based on objective information is not the same part that is responsible for making the money. Whatever decisions are taken the priority should be on making sure that the organization is not seen to be taking decisions just to make more money or to gain market share. For this reason, often the body that owns the scheme is separate from the body that manages the money making side of the business, even if the money is made through grants from governments or foundations.

A second question is whether the new bodies should be for-profit or not-for-profit corporations. Some folks feel that the only choice is that all bodies should be not-for-profits, but that is not always the case. (As a side note we should be clear that not all not-for-profits are charities but all charities must be not-for-profit corporations.) This can become more complicated if more than one new body is created. Some schemes create a not-for-profit that is the scheme owner which itself owns one or more subsidiaries that are for-profits, especially when they are responsible for making the money. Another option is to contract these roles out to existing organizations that provide these services professionally.

Finally, there is the question of who makes the decisions. On this point I will be blunt. I am not in favour of managing an international business with a stakeholder board. Governance is a job that requires experience and normally stakeholders are selected based on their role in a network or group. A professional board for each organization is crucial. To ensure that the mission of the organization remains central for the new organization a small professional board can be supported by stakeholder or technical advisory boards. Some organizations create membership bodies (in the case of not-for-profits) or shareholders (in the case of for-profits) whose responsibility is to ensure that the organization remains focused on its mission.

The process of designing corporate structures for a new scheme can be confusing and involve lawyers, accountants and management consultants. It is best to start thinking early about how to structure the operations, how to govern any corporate bodies, who owns what and how to include stakeholders. A well thought out governance structure can make a launch easier and avoid problems that can be easily avoided with time, information and good advice.

Why don't auditors give advice?

I am hearing from some scheme owners and advocates for certification that they want auditors to help out the folks they are auditing by providing them with advice on how to fix the problems that the certification client is facing.

Under ISO, ISEAL and IAF approaches to auditing, impartiality is one of the highest values. The audit should be not benefit the auditor, the certification body or anyone that either the auditor or the certification body have an interest in gaining a benefit for. The goal should be to provide high quality, professional and impartial audit services. A certification decision should be based on the information gathered in the audit and nothing else.

Impartiality is maintained when there is no real or perceived conflict of interest that would lead a reasonable person to believe that an audit decision may be taken for any reason other than the audit evidence as observed by the auditor and a decision based on that evidence by the certification body. On the most obvious level it means that the auditor should not have other business with the certification client or have provided any products or services to the client within  a set amount of time before the audit (IAF sets a minimum of 2 years, other schemes have differing rules).

A key factor to consider in understanding impartiality is that the auditor should not under any circumstance be evaluating a product or service that he provided. First, she may be tempted to approve her own work because it looks better on her. Secondly, he may feel that since his work or product is in use that all is well and not properly evaluate how it is working.

Certification bodies are required to have a decision making entity that is independent from the part of the certification body that is responsible for or directly benefits from revenue generation. So for example a CEO of a certification body would normally not be involved with a certification decision since her job is to increase the revenue to the company. The decision makers must be individuals that neither benefit from or be penalized for a particular decision.

Providing advice to a client on how to best meet the requirements of a standard can lead to a number of conflicts of interest. For example, an auditor may try to sell her consulting services to the client and a client may feel pressured to hire the auditor out of fear that the client may fail the audit if they do not.

Also, a client may feel pressured to do what the auditor suggests even if it does not solve the problem or does not otherwise fit the needs of the client. This could mean installing equipment that is not fully compatible or spending more money than the client can afford.

If the client does what the auditor suggests and it does not work, how will the auditor provide an impartial audit when he returns? Will she feel the need to minimize the problems? Will he feel responsible and want to avoid costing the client more money by giving a negative recommendation? 

Certification bodies that provide services for schemes that require full impartiality may not want to provide services for schemes that want auditors to provide advice since it may complicate the certification body's own efforts to maintain clear and consistent rules for impartiality.

Any scheme that wants to design a program that includes auditors providing advice to clients will need to think through the implications for impartiality. How would you give advice and still conduct an impartial audit?

Another challenge is that some of the best auditors may be in fact really lousy at giving advice. They are trained as auditors to evaluate whether or not the requirements are being met, they may not be very good at recommending how to solve a problem and produce the result that the scheme requires. The same may be the case for the reverse, a really good problem solver may be a really lousy auditor.

One possible implication is that a scheme may become more expensive to clients since the auditors not only have to be top notch auditors but they also have to be great at coming up with solutions.

Finally, how would you design your program to make sure that the clients feel no pressure to implement the solution proposed by the auditor? Maybe there is a better, cheaper, and more effective solution available that may not be used.

The rules against auditors providing advice are in place for good reasons, because they solve real world problems. Any scheme that wants to change these rules will need to find another way to solve these real problems that does not just create new ones.

So now you're a Scheme Owner....

WHAT is a "Scheme Owner"?

Simply put, a scheme owner is the organization (individual, for-profit corporation, not-for-profit corporation, certification body, government department, agency or other body, trade association, group of certification bodies or other just about any other body or group of bodies) that is responsible for the development and maintenance of the scheme and owns the intellectual property, copyright, trademarks and other rights to a certification scheme. 

Ownership includes the copyright for the standard, the certification system, the name, trademarks, graphics and other identifying texts. Normally the name, trademarks, graphics, domains and other elements are registered in the jurisdictions where they are used.

Every certification scheme must have a scheme owner.  In short someone must be responsible for the development and maintenance of the scheme. Simply writing a standard and letting it loose is not sufficient to create a certification scheme. Standards must be interpreted, reviewed and revised; and certification systems must be maintained so that each use of the scheme is consistent and all audits and certifications are conducted to the same benchmarks.

WHAT does it mean to be a "Scheme Owner"?

Guess what?, there are standards and guidelines that apply to scheme owners (I bet you were not expecting that...).  ISO has the document 17067 titled "Conformity assessment — Fundamentals of product certification and guidelines for product certification schemes". There are a number of other ISO guidelines that are relevant to scheme owners but let's not list them here.... Also, ISEAL has a number of codes (standards writing, assurance, credibility, etc.) that are designed for scheme owners.  

Scheme owners are responsible for the development and maintenance of the scheme. That means that they are responsible for making sure that scheme is up-to-date, that questions about how to interpret the standard are answered and that all parties are equally informed about changes and interpretations of the standard and the scheme requirements.

The biggest worry for a scheme owner is avoiding conflicts of interest and when avoidance is not possible, managing the conflicts of interest that do occur.

WHAT are the major conflicts of interest and HOW they be managed?

First of all, the major conflicts of interest usually involve money but they are not limited to money. If a scheme owner is making money from the scheme then it is crucial that the making money part be as separate as possible from the scheme management part. It is crucial that there be no link between decisions about the scheme and making money, the risk that a scheme owner can be accused of changing requirements to favour one party over another or lowering requirements to get more folks certified just to make more money can dramatically undermine a scheme's credibility.

These risks are as present for for-profit corporations as they are for not-for-profits or other types of scheme owners. As a result the most preferred way to manage the risk is to separate the roles of the application and management of the scheme from marketing, promotion, advocacy or other efforts to sell or grow the use of the scheme. This means that the job of managing the scheme is best left to a 'utility-like' organization whose job it is to provide a professional and neutral service. The job of promoting and marketing the scheme can then be undertaken by others who do not have direct control over the maintenance of the scheme. 

A scheme owner should be impartial in all cases and so the management of real and perceived conflicts of interest should be its primary concern.

Whose afraid of the WTO?

The WTO (World Trade Organisation www.wto.org) is an intergovernmental organization that started in 1995 and replaced the badly named GATT (General Agreement on Tariffs and Trade that itself started in 1948). So much for history.

It's the job of the WTO that should be of interest to standards organizations. Its role is to be the guardian of the global agreements that all member countries have reached which sets the rules for what the countries themselves can do to regulate international trade.

For standards organizations, there is one major agreement that relates to your work.  That is Annex 3 to the WTO "Technical Barriers to Trade Agreement" (TBT Agreement).  Annex 3 is the "Code of Good Practice for the Preparation, Adoption and Application of Standards".

All well and good you say, but why should an obscure annex to an incomprehensible trade agreement be of concern to my little sustainability standard?

The first reason is the biggest.  Trade agreements are between governments and when a trade dispute arises anyone caught in the middle can get crushed - and since your standard may get caught you want to do everything to avoid it.  Even though it is rare that independent standards get caught up in WTO disputes the consequences can be devastating for your standard. Just have a look at the case of Dolphin Friendly Tuna (http://www.wto.org/english/tratop_e/envir_e/edis04_e.htm).  In this case the US government linked imports of tuna to Dolphin Friendly Tuna certification and the Mexican government objected.  In the end the US lost and the end result was pretty hard on the Dolphin Friendly Tuna scheme.

As a result of this case it became pretty clear that any standard should be developed in compliance with Annex 3 of the TBT Agreement. 

It is only a few pages long, so download it, read it and make sure that everyone in your organization that has a decision making role related to your standard is trained in its requirements.

It is fairly easy to conform to the Code of Good Practice and every standard should do so.  The bottom line is that it is in your best interest to know about this Code and make sure that everything you do conforms to it.

Of all the international benchmarks that you want to consider as important to your standard, this is the most important.

If I tell you, then we will both know...

A long time ago a colleague told me about a reporter going through a customs inspection at an airport in a developing country. As the customs officer emptied his suitcase and all other bags on the table, checking every pocket and container he could find the reporter asked: "If you tell me what you are looking for I'll let you know if I have it in my bags and show you where it is." The customs officer stared directly into his eyes and with a serious expression said: "If I tell you then we will both know" and then returned to tearing apart the reporter's bags.

Sometimes standards appear to be just like the customs officer, they appear to be reasonable but no body really knows what is required until the auditor finds what she is looking for. 

This can drive clients crazy because they want to make sure they know what they have to do and they want it all done before the auditor gets on site.

A key component of knowing what you have to do is knowing what the terminology in the requirements really means.  

I know it sounds boring but a good glossary of terms with clear definitions can turn your requirements from vague to precise.  Every scheme should have its own glossary of terms and acronyms to help the user understand what you really mean, not just what they think you mean.

Here are my high point recommendations on how to write good definitions for your scheme:

• There should be only one authoritative list of definitions; this avoids the eventuality of having several definitions for the same term.
• A term should be defined when it has more than one common definition, one or more of which can cause an inaccurate reading of the requirements.
• A term should be defined when it is unique to your scheme.
• There should only be one definition for each term.
• When more than one term is used for the same definition (e.g. ‘producer country’ and ‘country of origin’) there should be only one definition for the main term and the second term should be included in the glossary with a reference to the main definition (e.g. “country of origin: see producer country”).
• Terms should be defined in clear simple language, using short sentences.
• Definitions should not include examples, illustrations, similes or metaphors.

Clear definitions can make audits go more smoothly, encourage your scheme's users to be in compliance sooner and can help make sure that everyone knows what the requirement really means.

Now we all know.

Building a new scheme is like a puzzle.

I have had the chance to work with a number of new schemes; these are efforts to develop a certification or labeling scheme from scratch. What normally happens is a group of people get together, usually they are issue experts in the area. They start the process of writing a standard.  

Sometimes this process can go on for years. The number of experts as well as interested and affected parties begins to grow and the standard begins to take shape. They put a huge amount of energy into writing a standard that contains everything.  Then they run a field trial and test the standard out and once that has been done they start to put it into operation.

That is when things can go 'pear shaped'. They have built the perfect standard that issue experts understand but once they step out of that world the rest of the industry sector, processors, customers, retailers and other users cannot figure out how to use the standard.

Eventually they call someone like me and they are told that the standard has too much in it and that it does not play well with the other certification, business processes and the management systems used by most companies. This is hard news to hear and it can sound like they just wasted a huge amount of energy, good will and lost an opportunity. It can be fixed; it is just a lot of work.

The way to avoid this situation is to see the process of developing a new scheme as a puzzle. Puzzles are built one piece at a time and until all the pieces fit together it is not finished.

The standard, as important as it is, is not the only piece.  The key elements include:

• The standard,
• The rules that accreditation bodies must follow,
• The rules that certification bodies must follow,
• The rules that govern who can be an auditor,
• The rules that govern claims, trademarks and logos,
• The procedures that the scheme owner will follow, and
• Sometimes, even more sets of rules.

But, it can still get more complex than this. Each of these elements can be made up of a number of other pieces - it's becoming a rather large puzzle.

Let's look back at the standard. The best way to begin is to understand a bit about how the industry in which you work functions and the way the auditing industry functions.  

Many of the most successful schemes are built on the foundation of other standards. If, for example, your scheme is designed to produce a certified product then you may find that building your scheme on ISO 17065 can be useful. It is understood well by the auditing industry and it includes many of the process and administrative requirements already assembled and your scheme can add the specifics of your scheme right on top of it. Also, it is not necessary for your standard to describe every step that an auditor must take, much of that is included in ISO 19011 and most auditors are trained in its application. There are a number of other building blocks you can use to make your standard work.

In addition to using these building blocks you can modify them. If you feel that there are parts you do not like or that are not exactly what you want then your system can modify them. Your scheme requirements can specify that everything in a particular document applies to your scheme unless it is specifically changed in your requirements. If you think that 'periodic' internal audits are not enough you can require 'annual' internal audits.

The main advantage of this building block approach is that the industries, processors, traders, retailers and others are already used to many of these elements. They can readily understand them and they know the kinds of things that an auditor needs and what to expect in the process.

Finally, I always recommend that once you have your brilliant idea for a new scheme and you dive into the standard to make sure that you also start early to develop the rest of the puzzle pieces. Find the people on your team to lead these efforts and when you don't understand a new topic get some professional advice.

Yes, you are the issue expert that is working on the soul of the new scheme, the standard. To build the whole thing you will need all the puzzle pieces and they all must fit seamlessly together.

It is when you have a full and complete picture assembled that others, even those who are not expert in your field, will be able to see, understand and want to use your scheme.