Quanta Corporate Citizenship Defining CSR Output, outcome and impact 11/05/2010
For companies, one of the most difficult (and frustrating) aspects of contributing to positive social, environmental and economic impact is finding realistic ways of measuring the difference their commitment make. To make things more difficult for them, in the midst of so many reporting procedures, there is often confusion and misuse of terminology when talking about the specifics of measurement.
Thankfully there are some simple ways to prevent getting lost in the myriad of terms, and our experience has shown us that the one companies find easier is to approach everything in a chronological way.
A programme that a company runs as part of its CSR will begin with aims and objectives. The aims of the work should clearly outline a change that is sought. From a strategic perspective, this is the most crucial step often skipped by companies. That usually takes between 5% and 15% of the whole project time, but without doing it, the whole project is aimless.
All too often aims run something along the lines of ‘our aim is to work with the local community’ or ‘improve employee engagement’ or ‘become a sustainable organisation’. Whilst the intention is well meaning, it misses out on the key aspect of an aim, which is about why the company is doing it and what kind of change the project intends to have.
We will often have an overarching aim that is backed up by one or a series of goals to contribute to the bigger picture. To set the objectives to these specific goals, a company needs to make sure there is a clear set of detailed activities.
A tip is to start by paying close attention to the language being used and combining it with specific changes and activities that will be delivered will make it easy for both internal and external stakeholders to understand.
When it comes to the measurement process itself, having clearly defined the aims, goals and specific objectives means the activity of the project should be clear. For example:
Thankfully there are some simple ways to prevent getting lost in the myriad of terms, and our experience has shown us that the one companies find easier is to approach everything in a chronological way.
A programme that a company runs as part of its CSR will begin with aims and objectives. The aims of the work should clearly outline a change that is sought. From a strategic perspective, this is the most crucial step often skipped by companies. That usually takes between 5% and 15% of the whole project time, but without doing it, the whole project is aimless.
All too often aims run something along the lines of ‘our aim is to work with the local community’ or ‘improve employee engagement’ or ‘become a sustainable organisation’. Whilst the intention is well meaning, it misses out on the key aspect of an aim, which is about why the company is doing it and what kind of change the project intends to have.
We will often have an overarching aim that is backed up by one or a series of goals to contribute to the bigger picture. To set the objectives to these specific goals, a company needs to make sure there is a clear set of detailed activities.
A tip is to start by paying close attention to the language being used and combining it with specific changes and activities that will be delivered will make it easy for both internal and external stakeholders to understand.
When it comes to the measurement process itself, having clearly defined the aims, goals and specific objectives means the activity of the project should be clear. For example:
- Aim: To reduce the spread of HIV in young people in Gauteng, South Africa so that we increase our employee engagement in the local factories by reducing turnover of staff as we increase their level of satisfaction for being associated with our brand and lower the infection rate in our future staff;
- A goal: To increase the acceptance of HIV as a disease by young people in Gauteng, educate them about methods of infection and the implications of being HIV positive;
- A specific objective: To perform plays in front of 5 thousand young people in Gauteng by December 2010 to educate them about the risks and consequences of infection.
The activity then quickly becomes about the delivery of enough plays to reach 5 thousand young people. The resource and action that needs to be taken becomes the ‘input’ of our measurement process: £20k to fund a co-ordinator, actors and materials to stage performances in 10 schools across the Gauteng district.
Once this work has been done, the output should, if the programme has gone according to plan, be the objective of the project expressed in the past tense. For example, ’Twenty plays performed by the students themselves were performed, educating the audience about the risks and consequences of HIV infection, were performed across 9 schools in the Gauteng district of South Africa reaching 5,500 students within three months at a cost of Rand 50k’.
At the output stage there is room for clarification around what was over or underachieved with the resources that were made available as the input directly affects the quality and level of output.
Where it begins to get interesting however, is beyond the stage of input and output.
Documenting these early stages amounts to reporting the activity that was planned and what actually happened, but the vast majority of CSR investments take place because companies are looking for a change. And this change is what is documented at the final two stages of outcome and impact.
Outcome and impact are often the most misused of all of the terms because they are difficult to get right.
The outcomes of our project are the changes or benefits that take place as a result of the project activities and relate to the goals we set. Our specific goal in our example was to increase the acceptance of HIV as a disease, improve knowledge about infection and living with HIV. This means our outcome indicators would demonstrate the change that has taken place in the number of young people as a result of this work. For example:
Before the performance of the play 4 in 10 young people did not believe HIV existed, 5 in 10 did not know how it was transmitted and 3 in 10 did not know what the implications were for living with HIV. The outcome of this project is that 9 in 10 young people now believe HIV to exist, 100% know how it can be transmitted and 100% understand the implications for living with HIV.
The importance of measuring the change in attitude is crucial to understanding the project’s outcomes.
Having said that, this change in attitude, whilst a goal of the project, was not the overall aim, which was to reduce the spread of HIV infection in young people, and it is when we measure against this overall aim that we find the impact of the project.
The measurement process for the project impact is entirely different, and generally takes place over a much longer timeframe and is much broader (in our example above, the analysis of outcomes can be done the same day as the play, or shortly afterwards).
Achieving an impact from this project would amount to being able to say that the number of young people in Gauteng who were infected with HIV fell from 30% in 2009 to 15% in 2010. A great impact assessment for this project would be to be able to say that it fell from 30% to 15%, out of which the statistical analysis (or any other sort of reliable evidence) showed that half of the fall was due to this specific project. Although daunting, this can be done through many different scientific methods (e.g., control groups, regression analysis etc), and normally is less difficult than most people think.
When you work backwards through the process, it becomes clear as to why it is so important to set clear, specific aims, goals and objectives right at the beginning of a programme and to make sure you have selected the best indicators (and only the best) for your project and have a system in place to monitor the input, output, outcomes and the impact. Unless you know the details of your projects you will never truly know the extent of your impact, let alone your outcomes or outputs.
- Kim
Once this work has been done, the output should, if the programme has gone according to plan, be the objective of the project expressed in the past tense. For example, ’Twenty plays performed by the students themselves were performed, educating the audience about the risks and consequences of HIV infection, were performed across 9 schools in the Gauteng district of South Africa reaching 5,500 students within three months at a cost of Rand 50k’.
At the output stage there is room for clarification around what was over or underachieved with the resources that were made available as the input directly affects the quality and level of output.
Where it begins to get interesting however, is beyond the stage of input and output.
Documenting these early stages amounts to reporting the activity that was planned and what actually happened, but the vast majority of CSR investments take place because companies are looking for a change. And this change is what is documented at the final two stages of outcome and impact.
Outcome and impact are often the most misused of all of the terms because they are difficult to get right.
The outcomes of our project are the changes or benefits that take place as a result of the project activities and relate to the goals we set. Our specific goal in our example was to increase the acceptance of HIV as a disease, improve knowledge about infection and living with HIV. This means our outcome indicators would demonstrate the change that has taken place in the number of young people as a result of this work. For example:
Before the performance of the play 4 in 10 young people did not believe HIV existed, 5 in 10 did not know how it was transmitted and 3 in 10 did not know what the implications were for living with HIV. The outcome of this project is that 9 in 10 young people now believe HIV to exist, 100% know how it can be transmitted and 100% understand the implications for living with HIV.
The importance of measuring the change in attitude is crucial to understanding the project’s outcomes.
Having said that, this change in attitude, whilst a goal of the project, was not the overall aim, which was to reduce the spread of HIV infection in young people, and it is when we measure against this overall aim that we find the impact of the project.
The measurement process for the project impact is entirely different, and generally takes place over a much longer timeframe and is much broader (in our example above, the analysis of outcomes can be done the same day as the play, or shortly afterwards).
Achieving an impact from this project would amount to being able to say that the number of young people in Gauteng who were infected with HIV fell from 30% in 2009 to 15% in 2010. A great impact assessment for this project would be to be able to say that it fell from 30% to 15%, out of which the statistical analysis (or any other sort of reliable evidence) showed that half of the fall was due to this specific project. Although daunting, this can be done through many different scientific methods (e.g., control groups, regression analysis etc), and normally is less difficult than most people think.
When you work backwards through the process, it becomes clear as to why it is so important to set clear, specific aims, goals and objectives right at the beginning of a programme and to make sure you have selected the best indicators (and only the best) for your project and have a system in place to monitor the input, output, outcomes and the impact. Unless you know the details of your projects you will never truly know the extent of your impact, let alone your outcomes or outputs.
- Kim
1 Comment
Creative awareness 19/03/2010
While working in Myanmar a few years ago, I learned from a Buddhist friend that the essence of meditation is to try to steer away from getting lost in digression and at the same time avoid getting stuck into trying to answer all questions. According to her, meditation is trying to stay creatively aware of what is essential.
One of the most interesting aspects of our work is that we get to work with a lot of different organisations, different people, and see a lot of different trends.
One of those trends we have started noticing is the desire of companies to get more focused on producing and presenting results. Some companies do it because they want to produce reports and portray their work to an wider audience. Others, because they need to justify their investments to their shareholders. Many, because they need to produce reports to show compliance to governmental regulations. And some, because they truly want to know more about what they are doing and what they are doing right.
What few still do, though, is to understand what exactly needs to be measured.
In order to know what needs to be measured, a company needs to know why it is committing to that specific initiative. In other words, what the ultimate purpose of that investment is.
To answer this question, the first step is to clearly define the primary audience. And, despite what many companies believe, their primary audience is often not the project’s beneficiaries. The beneficiaries might be just the channel/means to achieve some other purpose.
Some companies investing on equal opportunity for their employees do so not because they are particularly concerned about this issue or with the initiative’s beneficiaries (the employees), but because they want to comply with the governmental regulations. Therefore, the government is the primary audience. The positive impact on employees (secondary audience) is a very welcome side effect. Some other companies will incur the same investments because their primary concern is to motivate and engage employees. Employees, in this case, are the primary audience. The ability to comply and produce reports showing the compliance to governmental regulations are a welcome side effect.
Although the type of project might look similar, they are fundamentally different. Their setups are different because their objectives are different. And because they have different objectives, they need to be measured in different ways. Trying to apply the same metrics to both is simply a waste of time and money. The company will either not be able to collect the data that is really relevant for it, or it will have to collect more data than is required, waste significant resources and, even worse, by overloading itself with unnecessary data, not know what data is really relevant.
Following the example above, the first type of company should be trying to measure the outcomes that are relevant to the government. Things such as the gender ratio, minority ratio, salary difference per gender within each grade level, percentage of minority groups in senior positions etc. The second type of company should be much more focused on measuring whether this is an issue that actually increases employee engagement, whether employees believe there is a de facto equal opportunity environment, whether employees feel more motivated by specific initiatives the organisation takes to improve gender balance, how the non-minority group(s) react to different attempts to create a more equal working environment.
We are all tired of so many requests to fill surveys, research etc. Or to use the expression coined by a client recently, we are all ‘questionnaired out’. And, at the same time, we all believe that without quality data, we increase the likelihood of taking bad decisions and sticking to poor choices. The equilibrium is somewhere in the middle, where organisations focus on collecting essential data, and only essential data. And the only way to know what essential data looks like is to know why it is relevant and what it will be used for.
- Gus
One of the most interesting aspects of our work is that we get to work with a lot of different organisations, different people, and see a lot of different trends.
One of those trends we have started noticing is the desire of companies to get more focused on producing and presenting results. Some companies do it because they want to produce reports and portray their work to an wider audience. Others, because they need to justify their investments to their shareholders. Many, because they need to produce reports to show compliance to governmental regulations. And some, because they truly want to know more about what they are doing and what they are doing right.
What few still do, though, is to understand what exactly needs to be measured.
In order to know what needs to be measured, a company needs to know why it is committing to that specific initiative. In other words, what the ultimate purpose of that investment is.
To answer this question, the first step is to clearly define the primary audience. And, despite what many companies believe, their primary audience is often not the project’s beneficiaries. The beneficiaries might be just the channel/means to achieve some other purpose.
Some companies investing on equal opportunity for their employees do so not because they are particularly concerned about this issue or with the initiative’s beneficiaries (the employees), but because they want to comply with the governmental regulations. Therefore, the government is the primary audience. The positive impact on employees (secondary audience) is a very welcome side effect. Some other companies will incur the same investments because their primary concern is to motivate and engage employees. Employees, in this case, are the primary audience. The ability to comply and produce reports showing the compliance to governmental regulations are a welcome side effect.
Although the type of project might look similar, they are fundamentally different. Their setups are different because their objectives are different. And because they have different objectives, they need to be measured in different ways. Trying to apply the same metrics to both is simply a waste of time and money. The company will either not be able to collect the data that is really relevant for it, or it will have to collect more data than is required, waste significant resources and, even worse, by overloading itself with unnecessary data, not know what data is really relevant.
Following the example above, the first type of company should be trying to measure the outcomes that are relevant to the government. Things such as the gender ratio, minority ratio, salary difference per gender within each grade level, percentage of minority groups in senior positions etc. The second type of company should be much more focused on measuring whether this is an issue that actually increases employee engagement, whether employees believe there is a de facto equal opportunity environment, whether employees feel more motivated by specific initiatives the organisation takes to improve gender balance, how the non-minority group(s) react to different attempts to create a more equal working environment.
We are all tired of so many requests to fill surveys, research etc. Or to use the expression coined by a client recently, we are all ‘questionnaired out’. And, at the same time, we all believe that without quality data, we increase the likelihood of taking bad decisions and sticking to poor choices. The equilibrium is somewhere in the middle, where organisations focus on collecting essential data, and only essential data. And the only way to know what essential data looks like is to know why it is relevant and what it will be used for.
- Gus
The good, the bad and the ugly. 02/03/2010
The press over the weekend has been filled with dialogue surrounding Apple’s disclosure of child labour, environmental abuses, excessive working hours and poor working conditions in supplier factories. Aside from the general public criticism we would expect of such activities, there has also been a wealth of commentary around whether Apple’s disclosure should instead be commended. More positive comments have included defence of their transparent reporting procedure and to the lead they are taking in their sector, since their competitors do not report at the same level.
The support for transparency is certainly an indicator of the growing appetite of consumers and media to really know the brands and to be able to make informed decisions about their purchases. Whilst this trend has been developing for some time and is being escalated by recent corporate behaviour, it seems that we are reaching a tipping point from which we will start seeing a far deeper level of public scrutiny of the reporting processes. Our work is showing us a substantial shift towards public scrutiny not only of the outcomes, but also of the reporting process itself.
Malcolm Moore, a journalist reporting on the Apple story was criticised so heavily for his article over the weekend that he felt it necessary to post another article defending his critical stance of Apple’s supply chain ethics. I mention this because in defending his reporting, Moore was forced to go straight to the detail. With Apple supplier reports being available from past years, he made cross comparisons and has detailed falling standards from Apple and inconsistency in their reporting processes. The debate focused not on the outcomes produced by Apple, but on the way the company portrayed those outcomes. The reporting hiccup served only to undermine the otherwise good practices of the company.
The logic applied by consumers and media is, ultimately, a very simple one: ‘If I can’t trust your process, I can’t trust your outcome’. Regardless of how good the outcome actually is. The consequence of this is that, from a pure PR perspective, especially for companies operating on B2C markets or B2B with high public exposure, investing on the outcomes but overlooking the reporting process, is at least as dangerous as not investing on the outcomes at all. In an age of public cynicism, publishing reports based on anecdotal, sporadical, superficial, confusing or irrelevant information is a dangerous game to play.
Transparency and the commitment to doing the ‘right thing’ will find few rewards if done halfheartedly. We live in the age of instant communication where Twitter is reporting the headlines before the news agencies and where companies are going to find a growing public and eagle-eyed analysis of their behaviour. Companies who really want to build strong brands and avoid the criticism that will abound from poorly carried out responsibility initiatives, are going to have to look more seriously at their responsibility strategies and reporting mechanisms. Without these, professional and amateur commentators alike will find more than enough ammunition to criticise 'good' company intentions and leave them looking ugly.
- Kim
The support for transparency is certainly an indicator of the growing appetite of consumers and media to really know the brands and to be able to make informed decisions about their purchases. Whilst this trend has been developing for some time and is being escalated by recent corporate behaviour, it seems that we are reaching a tipping point from which we will start seeing a far deeper level of public scrutiny of the reporting processes. Our work is showing us a substantial shift towards public scrutiny not only of the outcomes, but also of the reporting process itself.
Malcolm Moore, a journalist reporting on the Apple story was criticised so heavily for his article over the weekend that he felt it necessary to post another article defending his critical stance of Apple’s supply chain ethics. I mention this because in defending his reporting, Moore was forced to go straight to the detail. With Apple supplier reports being available from past years, he made cross comparisons and has detailed falling standards from Apple and inconsistency in their reporting processes. The debate focused not on the outcomes produced by Apple, but on the way the company portrayed those outcomes. The reporting hiccup served only to undermine the otherwise good practices of the company.
The logic applied by consumers and media is, ultimately, a very simple one: ‘If I can’t trust your process, I can’t trust your outcome’. Regardless of how good the outcome actually is. The consequence of this is that, from a pure PR perspective, especially for companies operating on B2C markets or B2B with high public exposure, investing on the outcomes but overlooking the reporting process, is at least as dangerous as not investing on the outcomes at all. In an age of public cynicism, publishing reports based on anecdotal, sporadical, superficial, confusing or irrelevant information is a dangerous game to play.
Transparency and the commitment to doing the ‘right thing’ will find few rewards if done halfheartedly. We live in the age of instant communication where Twitter is reporting the headlines before the news agencies and where companies are going to find a growing public and eagle-eyed analysis of their behaviour. Companies who really want to build strong brands and avoid the criticism that will abound from poorly carried out responsibility initiatives, are going to have to look more seriously at their responsibility strategies and reporting mechanisms. Without these, professional and amateur commentators alike will find more than enough ammunition to criticise 'good' company intentions and leave them looking ugly.
- Kim
