For companies, one of the most difficult (and frustrating) aspects of contributing to positive social, environmental and economic impact is finding realistic ways of measuring the difference their commitment make. To make things more difficult for them, in the midst of so many reporting procedures, there is often confusion and misuse of terminology when talking about the specifics of measurement.

Thankfully there are some simple ways to prevent getting lost in the myriad of terms, and our experience has shown us that the one companies find easier is to approach everything in a chronological way.

A programme that a company runs as part of its CSR will begin with aims and objectives. The aims of the work should clearly outline a change that is sought. From a strategic perspective, this is the most crucial step often skipped by companies. That usually takes between 5% and 15% of the whole project time, but without doing it, the whole project is aimless.

All too often aims run something along the lines of ‘our aim is to work with the local community’ or ‘improve employee engagement’ or ‘become a sustainable organisation’. Whilst the intention is well meaning, it misses out on the key aspect of an aim, which is about why the company is doing it and what kind of change the project intends to have.

We will often have an overarching aim that is backed up by one or a series of goals to contribute to the bigger picture. To set the objectives to these specific goals, a company needs to make sure there is a clear set of detailed activities.

A tip is to start by paying close attention to the language being used and combining it with specific changes and activities that will be delivered will make it easy for both internal and external stakeholders to understand.

When it comes to the measurement process itself, having clearly defined the aims, goals and specific objectives means the activity of the project should be clear. For example:

• Aim: To reduce the spread of HIV in young people in Gauteng, South Africa so that we increase our employee engagement in the local factories by reducing turnover of staff as we increase their level of satisfaction for being associated with our brand and lower the infection rate in our future staff;
• A goal: To increase the acceptance of HIV as a disease by young people in Gauteng, educate them about methods of infection and the implications of being HIV positive;
• A specific objective: To perform plays in front of 5 thousand young people in Gauteng by December 2010 to educate them about the risks and consequences of infection.

The activity then quickly becomes about the delivery of enough plays to reach 5 thousand young people. The resource and action that needs to be taken becomes the ‘input’ of our measurement process: £20k to fund a co-ordinator, actors and materials to stage performances in 10 schools across the Gauteng district.

Once this work has been done, the output should, if the programme has gone according to plan, be the objective of the project expressed in the past tense. For example, ’Twenty plays performed by the students themselves were performed, educating the audience about the risks and consequences of HIV infection, were performed across 9 schools in the Gauteng district of South Africa reaching 5,500 students within three months at a cost of Rand 50k’.

At the output stage there is room for clarification around what was over or underachieved with the resources that were made available as the input directly affects the quality and level of output.

Where it begins to get interesting however, is beyond the stage of input and output.

Documenting these early stages amounts to reporting the activity that was planned and what actually happened, but the vast majority of CSR investments take place because companies are looking for a change. And this change is what is documented at the final two stages of outcome and impact.

Outcome and impact are often the most misused of all of the terms because they are difficult to get right.

The outcomes of our project are the changes or benefits that take place as a result of the project activities and relate to the goals we set. Our specific goal in our example was to increase the acceptance of HIV as a disease, improve knowledge about infection and living with HIV. This means our outcome indicators would demonstrate the change that has taken place in the number of young people as a result of this work. For example:

Before the performance of the play 4 in 10 young people did not believe HIV existed, 5 in 10 did not know how it was transmitted and 3 in 10 did not know what the implications were for living with HIV. The outcome of this project is that 9 in 10 young people now believe HIV to exist, 100% know how it can be transmitted and 100% understand the implications for living with HIV.

The importance of measuring the change in attitude is crucial to understanding the project’s outcomes.

Having said that, this change in attitude, whilst a goal of the project, was not the overall aim, which was to reduce the spread of HIV infection in young people, and it is when we measure against this overall aim that we find the impact of the project.

The measurement process for the project impact is entirely different, and generally takes place over a much longer timeframe and is much broader (in our example above, the analysis of outcomes can be done the same day as the play, or shortly afterwards).

Achieving an impact from this project would amount to being able to say that the number of young people in Gauteng who were infected with HIV fell from 30% in 2009 to 15% in 2010. A great impact assessment for this project would be to be able to say that it fell from 30% to 15%, out of which the statistical analysis (or any other sort of reliable evidence) showed that half of the fall was due to this specific project. Although daunting, this can be done through many different scientific methods (e.g., control groups, regression analysis etc), and normally is less difficult than most people think.

When you work backwards through the process, it becomes clear as to why it is so important to set clear, specific aims, goals and objectives right at the beginning of a programme and to make sure you have selected the best indicators (and only the best) for your project and have a system in place to monitor the input, output, outcomes and the impact. Unless you know the details of your projects you will never truly know the extent of your impact, let alone your outcomes or outputs.

- Kim

Corporate social responsibility is this season’s fashionable word, probably second only to sustainability. And as it happens to every buzzword, there is a substantial amount of noise mixed up with reality. When it comes to CSR investment, there are things we know will work and things we know won't work. But, the vast majority are still things we suspect work and things we wish worked because it makes us feel good. But, frankly, this is just wishful thinking and a sure fire recipe for corporate disappointment.
 
There is still very limited academic research around CSR that is really relevant for companies. There are a lot of reviews of reviews, a lot of good-looking academic frameworks, but when it comes to really hard evidence, very few pieces of work – such as the one by Ioannis Ioannou and George Serafeim, we’ve discussed here not long ago – pass the tests of reliability and relevance.
 
And then there is the dangerous territory: things we want to believe, but are not necessarily true, let alone proven. We’ve recently spoken in this same space about how a survey found out that, contrary to our long held belief, people who invest on green products do not act more socially responsibly. Actually, they tend to act less socially responsibly once they’ve made that green commitment. And the more we research this area, the more we will find evidences against our wishful thinking.
 
And, finally, there are the things that we suspect based on observations, but where there is still no academic research to prove it.

In this group of things we suspect but can’t prove, probably the most relevant is that great CSR does not create great culture. This is because the concept of great culture is a subjective one . Suffice  to see how the recent financial crisis left many company corpses along the road, many of whom used to be regarded as bastions of great cultures.
 
If anything, our anecdotal evidence points in the opposite direction: great cultures (or, to be specific, great leadership cultures) create great CSR. But, frankly, there is no scientific evidence to prove it. What we know is based on our experience, but that does not make it scientifically proven. 

And this is also connected to the biggest problem middle managers – especially CSR managers – have in advocating CSR commitments within their companies: there is no proof that those commitments will work, and those very limited commitments that are scientifically proven to work (or not) are things that everybody else is already doing (and often have already been forced on them in form of laws and regulations by national governments). So, in one hand they have line managers demanding innovative approaches and reassurance that their investments will generated the expected results, but they can’t give that reassurance without first committing to invest on them. On the other hand, they have a very limited portfolio of potential commitments that they know will work because academic research has proven that they work; but the only reason why academics have been able to prove they work is because there is a wide enough pool of evidence constituted by the investments done by thousands of companies, and, in that case, the innovation element is no longer there: the organisation, at best, will play a me-too strategy.
 
And that is why great leadership cultures generate great CSR investments: because they are open to the risk involved in trying to innovate, and they know the risks are worth taking because they understand their own strategic objectives, the potential rewards and the underlying challenges.

- Gus

While working in Myanmar a few years ago, I learned from a Buddhist friend that the essence of meditation is to try to steer away from getting lost in digression and at the same time avoid getting stuck into trying to answer all questions. According to her, meditation is trying to stay creatively aware of what is essential.

One of the most interesting aspects of our work is that we get to work with a lot of different organisations, different people, and see a lot of different trends.

One of those trends we have started noticing is the desire of companies to get more focused on producing and presenting results. Some companies do it because they want to produce reports and portray their work to an wider audience. Others, because they need to justify their investments to their shareholders. Many, because they need to produce reports to show compliance to governmental regulations. And some, because they truly want to know more about what they are doing and what they are doing right.

What few still do, though, is to understand what exactly needs to be measured.

In order to know what needs to be measured, a company needs to know why it is committing to that specific initiative. In other words, what the ultimate purpose of that investment is.

To answer this question, the first step is to clearly define the primary audience. And, despite what many companies believe, their primary audience is often not the project’s beneficiaries. The beneficiaries might be just the channel/means to achieve some other purpose.

Some companies investing on equal opportunity for their employees do so not because they are particularly concerned about this issue or with the initiative’s beneficiaries (the employees), but because they want to comply with the governmental regulations. Therefore, the government is the primary audience. The positive impact on employees (secondary audience) is a very welcome side effect. Some other companies will incur the same investments because their primary concern is to motivate and engage employees. Employees, in this case, are the primary audience. The ability to comply and produce reports showing the compliance to governmental regulations are a welcome side effect.

Although the type of project might look similar, they are fundamentally different. Their setups are different because their objectives are different. And because they have different objectives, they need to be measured in different ways. Trying to apply the same metrics to both is simply a waste of time and money. The company will either not be able to collect the data that is really relevant for it, or it will have to collect more data than is required, waste significant resources and, even worse, by overloading itself with unnecessary data, not know what data is really relevant.

Following the example above, the first type of company should be trying to measure the outcomes that are relevant to the government. Things such as the gender ratio, minority ratio, salary difference per gender within each grade level, percentage of minority groups in senior positions etc. The second type of company should be much more focused on measuring whether this is an issue that actually increases employee engagement, whether employees believe there is a de facto equal opportunity environment, whether employees feel more motivated by specific initiatives the organisation takes to improve gender balance, how the non-minority group(s) react to different attempts to create a more equal working environment.

We are all tired of so many requests to fill surveys, research etc. Or to use the expression coined by a client recently, we are all ‘questionnaired out’. And, at the same time, we all believe that without quality data, we increase the likelihood of taking bad decisions and sticking to poor choices. The equilibrium is somewhere in the middle, where organisations focus on collecting essential data, and only essential data. And the only way to know what essential data looks like is to know why it is relevant and what it will be used for.

- Gus

Two of the most common problems strategic consultants find in any organisation trying to achieve something is that they generally start doing things before first taking the time to consider why they want to do it, and assess where they are right now. The same two issues also surface when we discuss problems with corporate social responsibility initiatives undertaken by companies.

I’ve discussed the why question recently. It is probably now a good time to talk briefly about the 'where' question.

Even some great companies who are clear about why they are undertaking some specific CSR initiatives and know how they want to achieve it, sometimes forget to set a baseline and pre-define how they want to measure their outcomes.

Often establishing a simple baseline that will allow companies to assess how much progress they have made represents less than 5% of the total project, and only takes a similar amount of time. In order to save time and money, however, companies start investing in developing and implementing their CSR initiatives and when the time comes to measure the outcome of their investment, they can measure where they are at the specific moment in time, but because they didn’t take the time to know where they were before they started investing, they have no way to know the real amount of progress they made. In other words, whether it was worth the investment at all.

Furthermore, because performance is generally measured using more than one type of outcome, they don’t know where they might be over and under investing and, therefore, are unable to fine-tune their investment. For instance, a company implementing an employee engagement program might measure its performance using employee turnover, employee performance and employee satisfaction. If they do not have a baseline, after the first cycle they will know where they are on that specific moment in time, but will not know whether employee satisfaction is actually improving or not, and if so, by how much. Have they overshot their objective? Or are they under investing?

Also, without pre-defining where you are, how can you know when you've got wherever you want to get? You might even be there already, before starting the initiative. Or you might be setting an objective that is not realistic with the resources (time, money, knowledge etc) committed, and at some point people will disengage. No one enjoys running a race they can't win.

The general consequences of that basic gap is that companies will either have to spend much more to try to understand what the baseline was (and trying to measure something in the past is generally a large project and generally provides only crude estimates), and/or will be unable to inform its future decisions based on lessons learnt from past success and mistakes.

The second problem we generally see with metrics is that companies simply do not know which metrics to use to assess the success of their corporate social responsibility initiatives. The majority of companies will end up defaulting to anecdotal evidence. But, as a managerial tool, anecdotal evidence is only good at providing examples to illustrate specific points (case studies), instigate further analysis, or act as a PR mechanism (internally and externally). Taking decisions based on anecdotal evidences is as good as building airplanes with feathers because birds have feathers. This is what is called simplistic inductive reasoning. It might work, but it will generally be out of luck and not out of rational ground; and even past successful outcomes will generally fail, and normally their failure will occur when investors will have raised their stakes based on their ill-founded confidence. Inductive reasoning is behind the Black Monday in 1987 and the demise of LTCM.

Defining which metrics to use to measure the success of a CSR investment upfront is crucial. The metrics to be used are a direct consequence of the why question I mentioned earlier. They will be developed around why you are investing on this initiative in the first place. If your investment on supply chain management is to reduce carbon footprint, it will make more sense to measure the transport mode and distance travelled than to measure their labour relations. On the other hand, if your objective is to deal with a bad string of publicity due to your suppliers using sweatshops in Asia, than the contrary is likely to make more sense.

The common argument we hear is that it is just too difficult to measure the outcomes of ‘fluffy’ projects or it is not adding real value to the beneficiaries. It is true that it is sometimes difficult to measure their outcomes, but our experience is that (a) it is generally easier to measure the outcomes than people think, and (b) even if you cannot directly measure the outcome, you can generally find a few proxies that provide reasonable indication of the progress. 

And that only serves to reinforce the need to define the baseline earlier on. If you have a baseline, you can at least keep the measurement consistent. Even if you are only using a proxy one.

And if reducing the risk of continually misplacing good money and learning what generates impact and what doesn't is not in the best interest of the beneficiaries and investors then, frankly, nothing is.

- Gus