What does it mean to be 'benchmarked'?

For a number of years some folks have been complaining that there are too many different certification marks, they feel that this confuses the consumer. 

One response to this perception is to 'benchmark' the schemes.

The definition of a benchmark is something like: "a standard or point of reference against which things may be compared or assessed". In short, benchmarking of certification schemes is a comparison of a number of schemes against a single set of criteria.

Three approaches to benchmarking

  1. An evaluation that focuses on comparing how each scheme addresses (or does not address) each individual element of the benchmark criteria. Usually this is done in a format that encourages each user to make their own decision on which one they like the best.
  2. An evaluation of a number of schemes to determine which one meets a minimum level of performance across all the benchmark criteria. Usually this is done to determine who is 'the best'.
  3. The awarding of an approval or endorsement to a scheme (or sometimes schemes) that meet a set level of performance set by the benchmarking organization, usually to say that all those that are endorsed are somehow 'equivalent'.

A note about ALL types of benchmarking systems

Before I look at these three types in depth, it is important to note that a benchmark is a type of a standard. It is applied to a scheme rather than an individual client but it is as much a standard as any other standard - it just looks a little different. Organizations that create a benchmarking system for schemes will often tell you that their system is 'neutral' or somehow without any bias - this is never true. Any selection of criteria for benchmarking reflects the interests and values of the folks that put it together. All benchmarking systems have some bias, some more than others.

I should also note that the more generic a benchmarking system is, the less helpful it is in comparing schemes. If I use the same set of criteria to evaluate a scheme for the responsible manufacturing of toasters with a scheme for the sustainable production of frogs legs; I will not really be able to say much about which one is 'better'. If I do the same thing for two schemes that both certify sustainable turnip production I can learn quite a bit about how the two compare.

Now to some examples

The first type of benchmarking systems do not try to award a gold star to the best one. They break each scheme into many elements (hundreds some times) and allow the user to look at each component of each scheme. I can, for example, see how various elements addressing workers' rights are treated in a number of  agricultural schemes. 

These systems produce lots of detailed information and leave it up to the user to draw their own conclusions. As a result they are rich in data but can be really hard to use. The user may need to spend many hours or days studying how two or more schemes compare. This amount of time makes sense if you are purchasing tonnes of fish for processing but hard to justify if you are doing the family shopping.

The second type of benchmarking systems do award a gold star to the best. These are often used by organizations that advocate for the adoption of a certain set of process or production methodologies by an industry sector. They can be done well when all the schemes are responsibly scored against the benchmark criteria or they can be done poorly by picking the winner before the scoring is done. If I know and trust the Responsible Green Bean Council, and they produce a balanced, well researched benchmarking of all green bean certification schemes, I am likely to accept their recommendation of the best scheme even if their choice is based on their own internally defined criteria.

Finally, the third type of system is often used at a commercial scale. It can be used to determine which schemes comply with a set of benchmark criteria (say food safety in the case of GFSI - the Global Food Safety Initiative) and says nothing about any other elements in the schemes. This enables food producers to know if the companies they are purchasing from have a certification that meets or exceeds a set of minimum requirements for food safety. It is important because the procedures for assuring food safety for dairy are very different from the ones that are used for fresh green beans. As a result the producer of frozen green beans with cheese can be assured that both are safe when green beans and cheese are purchased from companies that are certified using GFSI approved schemes. The GFSI system says nothing about any provisions in the green bean scheme that cover workers' rights or animal welfare provisions in the cheese standard.

What is good about benchmarking...

Benchmarking can be a very useful tool. It can provide information that is valuable to producers, manufacturers, processors and consumers. Even if the information is biased, it can be useful if the benchmarkers are honest and up-front about what they are tying to do.

What can go wrong...

When misapplied, benchmarking can be used a way to create a new certification scheme by trying to force existing schemes to do what the benchmarking group wants. This can be successful if a large enough group of organizations are able to influence a key step in the certification process.