Thinking deeply about Data...

One of the biggest challenges for new and growing schemes is dealing with data.  Early on in your operations there is often strong pressure to save money and postpone putting resources into a robust data management system that can grow with the scheme and provide a rich source of data for use in managing the scheme and understanding the impact your scheme is having.

Before thinking about how to start, let's look at the uses for the data.

For certification schemes there are two clear, high priorities for collecting data:

  1. Gathering data to be used in the ongoing management of the scheme.  This includes the operation of the business and the performance of your organization (i.e. financial management, numbers of certificates, results of audits, etc...) and all other related organizations that deliver your service (i.e. certification bodies, accreditation bodies, registered consultants, registered auditors, etc...).
  2. Gathering data to be used to understand the impact your scheme is having.  That is, how much, if any, is your scheme producing the changes that your scheme is seeking to make. This is to answer the question: "Is the environment or society better off because of your scheme?"

To address the first point, data is crucial to managing your scheme. Imagine trying to manage your scheme with no financial information, you would have no way of knowing whether or not you can make payroll each month or pay your bills.  Business management requires data. Business managers often speak about KPIs (Key Performance Indicators). These are the key bits that a manager needs to know in order to determine if the organization is achieving its operational goals.  

For business management the sequence of what can be done with raw data looks something like this:

Data - Information - Knowledge - Insight

Data is the raw set of numbers or other points that are gathered.  As you process that data you can produce information.  Using the information you can begin to ask questions, explore what is happening and gain knowledge about your operations.  From that knowledge you can gain insight that you can then use in making operational, tactical and strategic decisions about how to be more successful. For more information on this take a look at ISO 14031, Environmental Performance Evaluation - Guidelines (there are also loads of business books on KPIs).  In this area it is important to never confuse raw data with knowledge or especially with insight.

For the process of understanding the impact of your scheme the process looks something like:

Outputs - Outcomes - Impacts

Outputs are the basic statistics about your program (i.e. how many audits were conducted, how many people attended training seminars, how many products are certified, etc...).  Outcomes are the second or third order changes that may result from the outputs (i.e. how many people who took your training shared that information with other groups, how many companies sought certification because someone else got certified, etc...).  Impacts are the changes that you want your scheme to achieve (i.e. the water is less polluted, are more girls are going to school, is farm income rising, etc...).  For more information on this look at ISEAL Alliance's Impacts Code. The full progression is important because while you may be achieving high levels of outputs these are not necessarily a guarantee that you are ultimately having your desired impacts.

In  brief, you can run your scheme with limited data only for a short while.  Eventually you are going to need robust and reliable data that you can use as a basis for making decisions. One day you will get to the point that you need the data, at this point it is too late to start collecting it. Data collection is crucial and must be started before the board, CEO or a new strategy consultant asks for it.

The big challenge in starting the collection of reliable data is knowing what to collect.  To design a good system for collecting data it is important that you think deeply about what you are trying to do and what you are trying to achieve.  Your data collection should be focused on what you need to know about, understand and evaluate.  

My last point is to encourage you to be open to the possibility that what you learn from the data you collect may challenge some (or even all) of the fundamentals of your scheme.  You may be extremely successful in getting companies or products certified, that does not mean however that you are automatically achieving your desired impacts.  Use the data to understand, and if necessary, change how you and your partner organizations do your work. 

No matter where you are in the process of managing your scheme, but especially when you are just starting out, it is important to start early and to think deeply about data.