What happens to your standard when the auitor goes to work?

So, you are now finished with your standard and all the rules that an audit firm (let's call it a CAB - a Conformity Assessment Body) has to follow.

Before you press send and go live with your new system, maybe it is time to sit back and think through how all of your work will be used in the field. Your system will work best if it fits the needs of the users, the client, CAB, Accreditation Body (AB), stakeholders and anyone else that needs to use your system.

Today, let's consider what happens to your system of requirements when they are used by a CAB. In brief, your system is dis-assembled and re-assembled into a new set of procedures, forms, checklists and requirements that fit the way the CAB is organized... 

  • All the requirements that apply to the CAB, including process and timelines are separated from those that apply to the client. The CAB builds the requirements that it must follow in its own internal systems and procedures.

  • All the requirements that apply to the client are put into checklists and other audit tools that are used to audit clients.

  • The CAB's auditors are then trained to use these checklists and audit tools when auditing a client.

It is important to realize that auditors are not walking into an audit carrying copies of your standard, audit rules or other scheme documents, they are carrying materials that the CAB has developed based on your system requirements that are merged with the CAB’s own procedures. Their goal is to ensure that the audit can be carried out in the most efficient and effective manner possible, at least from their perspective.

CABs conducts audits following a standardized template. Most CABs will use ISO 19011 as the basic template and will add a few tweaks to make it their own. ISO 19011 lays out how an auditor conducts an audit. It includes such wonders as the outline for an opening and a closing meeting and other such innovations. The steps in ISO 19011 mean that the auditor is not just using your standard to evaluate the client, they have a number of steps that they should follow. These steps are necessary to ensure that the auditor does a thorough job and that client knows what is going on and what they have to do.

Before a CAB is allowed to conduct audits, its system must be fully reviewed to ensure that all of your requirements made it into the CAB’s re-assembled system. This is done by the AB and it is a key role that the AB assumes. Unless you have built your own expertise to do this job, you will need a competent AB to fully review the CAB’s system.

If this sounds complicated and a little confusing, it is because it is.

Your standard, scheme requirements, rules for auditors and such are most effective when your system is designed to be taken apart and put back together. It needs to be designed so that an AB can fully check the CABs incorporation of your system into theirs. It needs to take into account how auditors work in the field.

It also needs to take into account time…

The time that an auditor has on site with the client - so as you review your system keep in mind some key questions:

  • How many hours does an auditor have per day to ask questions?

  • How many questions can an auditor ask (and get answered) in an hour?

  • Does the auditor have to travel from one site to another and how much time does that take?

  • Auditors need to eat lunch, so allow time for that.

  • Remember that the auditor has to include an opening and closing meeting with the client to explain what will happen in the audit and what the audit has found.

The time that it takes for each step in the process:

  • For your system to be fully developed;

  • For the CAB to incorporate your system into its own;

  • For the AB to assess the CAB and make sure that your system is fully incorporated; and

  • For the CAB to plan, execute and take a decision on any single client.

What happens during an audit is often as important (if not more important) than the specifics of what is being audited. I do not mean to suggest that what is in the standard is not important, but how the audit is conducted should be considered when your standard, requirements and other rules are developed.

Saving the World, one Checklist at a Time

One day a few years back I heard about some producer who wanted to do a reality TV show about environmentalists. They wanted real life environmenatlists; you know, the people who spend all their time chained to bulldozers to resist pipelines, swinging from construction cranes to hang banners or rescuing whales that are caught in fishing gear. This 'reality' TV idea came up while I was visiting a local environmental organization. Once the concept was explained the room was silent, then someone in the back of the room said: 'They want to film us sitting through endless meetings and while we read multi-volume reports?' Everyone started to laugh.

Needless to say, the show was never produced. 

Just like many other jobs, working in the world of standards, accreditation and certification can be dull. We create standards, guidelines and principles; then we turn them into checklists. With a blank checklist in hand we go through stacks of policies, procedures, practices and all kinds of other files; then we interview people and ask them all the same questions just to fill in the checklist and see if everything has been done and done right.

The work in standard systems is not going to get you a starring role in the hottest new reality show, but it is work that keeps us moving forward. Not just so that things continue to work, but to make them work better. We know that each application of the standard can help save an ecosystem, keep water clean, make sure workers are well treated, keep children in school, strengthen a community and so make the world a better place. 

Transforming markets takes dogged perseverance. It takes work over years to create innovative standards, conduct high quality accreditation, certification and auditing. It takes a hell of a lot of checklists.

So every morning when I start my day, I tell myself that I am saving the world, one checklist at a time.

You are too.

 

Accrediation - What is it & Why bother?

 

accreditation: third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks (3.1, ISO 17011:2004)
 

Accreditation is sometime described as 'auditing the auditors'...while that is not really correct it does hint at what an accreditation body (AB) does.

Accreditation is better described as a 'competency assessment' rather than an audit. You may feel that I am splitting hairs, stay with me for a few more paragraphs and I'll lay out why that is important.

First of all, let's focus on what competency means. Some folks think that a qualification to do a job is based on your education; in my case, I spent 2 years earing a masters degree in regional planning and a few more years qualifying for membership in a professional planning organization. As a result, I can call myself a 'Registered Professional Planner'.  I am a qualified planner!  In the twenty-eight years since I completed my degree in planning I have never worked as a land use planner or as a regional planner.  As a result, while I am qualified as a professional planner, I submit that I am no longer competent. Yes, I have the letters after my name and I use many of the skills that I learned in planning, but the degree and the designation does not equate to competency.

Competency is determined by demonstrating that a task or set of tasks can be done to a specified level of performance. In essence, it means the ability to do a job.

To assess the competency of a conformity assessment body (CAB) the AB looks at the systems, process and procedures used as well as how they are executed. Accreditation is granted to a CAB that demonstrates its competency in conducting conformity assessment work. In this context, competency is not a low bar to meet, it is a high bar. 

The areas that an AB would examine include a wide range of elements such as:

  • maintaining professional relationships with clients;
  • ensuring that all the staff of a CAB are competent to do their jobs;
  • the frequency and quality of the training and evaluation of auditors; 
  • planning and executing audits;
  • ensuring that non-conformities are addressed;
  • taking certification decisions; 
  • conducting regular management reviews and internal audits of CAB operations;
  • and a host of other elements.

Conducting an accreditation assessment is a specialized business, it is not just determining if the results of audits against your standard are producing the results you want. 

 

Who does what? - An Assurance Ecosystem

This blog focuses on the role of the scheme owner. But, let’s be clear that while the scheme owner is a central player, there are others that have crucial roles. Previous posts have included some elements related to standards development with some references to auditing. In this post I wanted to look at the big picture of how assurance is managed, in what I think of as an ‘assurance ecosystem’.

Before going any further, I want to be clear that while all well run standard systems include all the elements discussed below, who does what and how oversight is conducted can vary greatly. This can be due to the objectives of the scheme owner, the nature of the industry or in response to the way an industry sector or key resource is managed internationally or in specific countries.

A fully functioning assurance ecosystem includes several players, each with their own responsibilities. The diagram below is a highly simplified map of the players and the primary relationships of each.

Assurance Ecosystem

Kitbag - An Assurance Ecosytem - Page 1.jpeg

 

 For this post, as with the all the posts on this blog, the central player in our assurance ecosystem is the scheme owner.

In most cases the scheme owner is responsible for:

  • the standard itself, including its development, maintenance and interpretation;
  • the rules for the assurance system including the documented policies, processes and procedures that govern its application; and,
  • the day to day management of scheme operations including oversight, monitoring and evaluation, risk management, and business operations such as marketing, finance, personnel, etc.

Some schemes delegate some of these responsibilities to other organizations. However, the scheme owner is always ultimately responsible for the scheme. For example: 

  • In the case of organizations such as MSC and RSPO, they act as scheme owner and they have assumed all these three core responsibilities.
  • In the case of FSC, it established a global set of ‘Principles and Criteria’ that standards must address and works with affiliated national and regional organizations who assume the role of a standards development organization (SDO). FSC, the scheme owner, endorses those standards that conform to its system requirements.
  • In the case of ASC, the standard was developed by a group of semi-autonomous bodies, the aquaculture dialogues that were set up by WWF, and the completed standards were given to ASC who now undertakes all the responsibilities of the scheme owner.

The other three players on the assurance ecosystem map (AB, CAB and client) have their own roles to play.

The accreditation body (AB) has the job of providing "third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks (ISO 17011:2004 3.1)". In plain English, its job is to make sure that the CAB has the competence and capacity to carry out audits for a scheme. For some schemes, the accreditation body works in a single country, such as DAkkX in Germany or ANSI in the US, or the accreditation body works internationally, such as IOAS, SAAS or ASI.

The conformity assessment body (CAB) - sometimes referred to as the 'certification body' or the auditor - has the job of carrying out audits against the scheme owner's standard following the policies, practices and procedure specified by the scheme owner. CABs can range from large multinational organisations such as Bureau Veritas, SGS or TÜV to small organisations with only a few staff working in single country or region.

The client, that is the applicant for certification or, when certified, the certificate holder, has the job of making sure that the product, process or service conforms to the standard.

The assurance ecosystem is almost never a single organization that does everything. As a result, the task of assuring that each certificate means that the same minimum performance level has been achieved for each certificate is a challenge that requires skills of facilitation, negotiation, management, collaboration and communication.

In brief, the challenge of being a scheme owner includes the skill and capacity to partner with specialist organization in a range of roles and responsibilities in a way that respects the the scheme owner and their specialist roles so that the whole ecosystem delivers consistently high quality.

 

Planning for the future

There is a popular quote from Wayne Gretzky. He is reported to have said something like: "I skate to where the puck is going, not where it has been.” 

As cheesy as the quote may be, it draws attention to a fundamental problem that decision makers face - how to position yourself to take advantage of something that is going to happen rather than try to go after the rewards of something that is happening now or has already happened.

My cynical mind sees populist leaders of our age expending tremendous amounts of energy to recapture the glory of a bygone era, whether that is Trump seeking to recreate the hegemony of the US in the 1950s, Putin seeking to recapture the glory of the Byzantine Empire, or the Brexiteers seeking to reclaim the primacy of the British Empire, they all seem to me to be ‘skating to where the puck was…’

Business leaders can get caught in the same trap, that is focussing their resources on a recent trend only to be faced with all their competition going after the same, now diminishing opportunity.

This may not seem like it has much to do with certification; but it does, stay with me.

Certification systems face the pressing demand to solve the problems that they face now. This pressure can be overwhelming and can mean that leaders can be caught in the trap of focussing on where the puck is now, without devoting time and energy to where it will be. Limited resources can also truncate the capacity of leaders to think about the future.

The visionaries that were behind the leading social and environmental standards were thinking about the future and how to bend it toward creating a more just and greener world. They looked at long-term trends and tried to figure out how to take advantage of them.

While leaders in certification systems have to devote large amounts of energy into solving today’s problems, they also need to spend time thinking about were the world is going and how they can get out in front of the changes that are now developing.

All this comes down to the fact that certification systems are constantly changing and taking advantage of the opportunity to change can give us the opportunity leverage to broad changes that are occurring and are likely to occur.

These can include:

  • Growth in markets and trade in developing countries
  • The ever-increasing rate of technological change
  • The globalization of markets, especially capital markets
  • Market pressure from activist consumers
  • The growth of the size and importance of global megacities
  • The growing inability of national governments to control markets

I would not counsel adopting my list, but encouage looking to the research being done on future trends. Study the reports, evaluate the trends and ask the question: How could this impact my standard system?

Even if we just consider the growth in global trade and especially the growth in south-south trade, certification systems will need to adapt to a new shape in global markets. Most certification systems are designed to manage north-north and south to north trade. As a result, their efforts to develop markets for certified products in Europe, North America and Japan will not encourage growth in the rapidly growing market share in the developing world. Will your system be positioned to grow in Indonesia, Kenya and Brazil? Will you be able to deliver markets to your certificate holders for those that produce products in Africa and sell into Southeast Asia?
 

How to Build an agreement

Core to any standard system is the need to build a consensus, that is to reach agreements that can be supported by all your key stakeholders.

This sentence is self-evident...and impossible at the same time.

It is self-evident because agreements that have the support of all your key groups of stakeholders are the most durable and form a solid foundation for your system. It is impossible because getting that agreement on everything from everyone is, well, impossible.

Multiparty negotiation is one of the most challenging elements of building a consensus standard. It is necessary to get agreement between a wide range of interests, many of which are contradictory. (For more on what a consensus means see my earlier post: Who Can Write a Standard?)

In my experience multiparty negotiations follow a consistent pattern:

  1. Very quickly, the group can reach an agreement on close to 90% of the issues. This is the case because, despite our differences we all agree on wide common base (I know this does not feel like the case but it is really true!).
  2. Getting agreement on the next 7% of the issues will take a lot of work, trust building and creative thinking but it can be done.
  3. The last 3% is the killer. These are the issues where the most fundamental disagreements reside.

As I state in the title, agreements are built. The popular myths are either that the agreement is a compromise, that is all sides must ‘lose’ something or one side just gets its way at the expense of all the others. While these happen sometimes, the best agreements are built from innovative approaches to old problems. This is important because agreements we can actively support are more durable than those we can just accept.

Back in the early days of FSC, the original Principles and Criteria had a text of principle 9 – it dealt with the issues of ‘old growth forest’ and the parties could not agree to a text. After many tries to get an agreement a proposal was made to create a committee of representatives from the three FSC chambers  who would be charged to develop an agreed text for principle 9 (these included environmental, social and economic representatives from both the developed and developing countries).

To create the group, each of the chambers elected their developed and developing world representatives. To incentivise the group to reach a decsion, the FSC board which approved the process was clear, if this group did not solve the problem, the board would. This last point helped because very few folks liked any proposal that the board had produced to date.

When convened, the group started off with the all the expected concerns raised, the foresters from the economic chamber talked about ‘over-mature trees’, the environmentalist talks about the value of ‘old growth forests’, and the social chamber representatives talked about the need for sustainable employment and preservation of cultural sites in forest landscapes.

About halfway through the time we had together, a new idea emerged. It was proposed that we talk about ‘high conservation values’. That is, the concerns of each group could be mixed up in different ways. The question ceased to be “Is this an old growth forest?” and became “What are the high conservation values for this forest?”

Because of this change in approach a text of principle 9 was proposed that required protection of high conservation values. The text was clear that high conservation values could be found in just about any forest, from plantations to virgin forests and everything in between.

For FSC, the issue of principle 9 was the killer 3%. It took years to build an agreement, but it happened because a new idea was brought into the room, it was explored, and an agreement was built. The solution worked better for everyone’s concerns than any other. When the meeting ended, no one left feeling like they had lost.

A creative solution that allows the parties to think about an issue in a different manner is often the route to building agreements.

Writing Code for Humans

My father used to say: “If all else fails, read the instructions!” I think that this is an accurate way to describe how people function. The first thing we do when we open a box is to toss the instructions aside and try to figure out how to use our new toy. We will puzzle through a problem, press buttons on the new camera, and assemble the IKEA cabinet our own way before we find out that that we wasted time, did it wrong, or worse, broke it.

This natural tendency is at odds with how standards, audits and such are designed and implemented. Tossing the certification instructions aside is not going to ensure that your implementation of a complex system is going to be successful.

Standards, audits, certification, labelling, and other bits that are part of a certification scheme are detail oriented and require clear, step-by-step instructions and guidance for users. 

It really helps if the stuff is clear, easy to read and works with the way humans work. But, let’s be honest, most scheme documentation is confusing, not well organized and hard to follow.

We often think of instructions as a computer program in which each step, no matter how trivial must be written and placed in its proper order in the sequence. Humans, thankfully, are not computers and we are a bit more flexible than computers. With that said, clear instructions designed for humans do help and when properly prepared it is more likely they will be followed.

So how do you write code for humans? Here are a few things to keep in mind:

1.       Understand what your user needs

Find out who is going to use your instructions and guidance; write for them. If it is likely to be an intern or junior staffer, keep that in mind. Your writing may be their first real introduction to standards and certification.

2.       Use words and graphics

Some people read the words and ignore the graphics, others read the graphics and just look at the words as if they are footnotes to the graphics. Whenever you can, give presentations that both can read.

3.       Write in the positive

It is very hard to follow instructions that tell you what not to do. Instructions are so much clearer when you keep this in mind.

4.       Tell them how their work will be evaluated or audited.

Give a clear description of how their work will be used. If it is to be audited, explain the steps in how that works.

5.       Don’t think they are stupid or that they can read your mind.

Write for readers that are smart and want to learn, that is why they are reading your work. At the same time, be aware that what is ‘obvious’ to you may not be to others, so include all the steps in your descriptions.

Are voluntary standards really a form of environmental governance?

I have been reading studies, articles and such for years that describe voluntary standard systems (VSS) as a form of environmental governance – and for just as long I have been wondering if that really is the case.

The term ‘governance’ is very broad can include governments (national, state, provincial, county, municipal, etc…) and civil society (businesses, not-for-profits, charities, unions, churches, universities, partnerships, informal collaborations, the girl scouts, etc…).

I am a bit biased, I guess... I think of 'governance' as a role that includes a measure of direct control. This is informed by my experience as a member of several boards of directors and having worked directly for boards. The governance role of a board is focused on setting a direction and making sure that it is the goal of the organization. (Boards also have a budget and legal compliance role but let's put that aside for now.)

Most of the time when we hear the term ‘governance’ we think of a decision-making body that can create and enforce a policy, rule or regulation. As a result, most people do not think of voluntary or optional systems a form of governance because it lacks the capacity to compel compliance.

Since I am not a lawyer, I will skip over any musings on how legislation and regulation come to be; I will just focus on the voluntary side.

Standards are not legislation or regulation (see my earlier post “Conformity vs. Compliance”) although they can be easily confused. 

Voluntary standards are normally developed to codify norms of practice, that is they are designed to create specific outcomes, whether it is a common screw thread for bolts and nuts or how to manage watersheds.

Once they are developed, voluntary standards are either become the common practice or they are ignored. Those that become common practice create a new norm and nuts of a certain size all fit bolts of the same size. (These are the ones that can 'feel' like they are mandatory.) Those that are not adopted, fall by the wayside or sit in limbo until they are needed, if ever. The ‘decision’ is made in the market, and competing standards are used by different players until one is the clear winner. As a result, VHS became the norm despite the obvious superiority of Betamax (I know, I am showing my age now….)

Voluntary standards establish the norm by competition in the market. Winners and losers are not often clear because the winner may be selected for reasons that are not clear at the start.

A new norm, widely accepted may have the effect of regulation because once it becomes the new norm it is hard to use some other option, and have it accepted. Sometimes these other options can arise and disrupt the accepted norm, and then things change.

Well, what does this mean for a new standard system? I would see this as an opportunity to look at competing standards in the past and really understand why some succeeded and some failed. Shelves are full of standards that were rarely used (I should know because I spent lots of time writing some of them). Also, for every successful standard there are many competing ideas that were discarded.

I have my own ideas about why some succeeded and others fail. But that may just have to wait for a future post.

 

A NOTE: Yes, I agree that some governments adopt standards as regulation, effectively making them law in some jurisdictions. I consider these standards to have become regulations since they then have the force of law – they are no longer voluntary, even thought they were originally developed outside of legislatures and government departments.