Getting started with impact measurement | Episode 4
PUBLISHED ON January 16, 2023
Purpose Champions is an official Goodsted podcast. Each month, I host guests who follow their purpose and strive to create social and environmental impact for their stakeholders and communities 🏆
These stories are full of inspiration and insights for our community to learn from and take action to progress on their own purpose journey. You also hear from leaders of impactful initiatives and have a chance to get involved in their mission by sharing your skills and time.
In this episode Oliver Kempton, Co-founder & Partner of Envoy Partnerships joined us to share more about what impact measurement is, how to get started, what to focus on when you’re early on this journey and many more.
Listen and watch the full episode here:
Selin: Hi, everyone. Welcome to the Purpose Champions Podcast, where we shine a spotlight on those championing social, environmental impact for their stakeholders and community. In this episode, we will talk about impact measurement and hopefully it will be an insightful episode for anyone who is interested in learning about this field and implementing it in their own work. Our guest for today is Oliver Kempton, who is a partner and co-founder of Envoy Partnership.
He leads social value, impact evaluation and training areas of envoy's work. He has trained over 1000 people and he is a visiting tutor at the Institute for Creative and Cultural Entrepreneurship at Goldsmiths, University of London. Hi, Oliver. It's great to have you here. I'm super excited to hear more about your story and also all about impact measurement.
Oliver: Hi. Thanks. Great to be here and looking forward to talking with you today.
Selin: You've been working on impact measurement for almost 12 years.
What inspired you to become a consultant in this field?
Oliver: So it's an interesting one. I never planned to be a social impact consultant. I guess it wasn't a dream I had when I was 10 years old or anything like that, but I always had an interest in using evidence to help make decisions, and that was in a number of different areas. I actually did some polling. political work quite some time ago now, and that carried on that approach to say, look, can we use particular evidence about what people think and how they feel and use that to inform decision making. And so I've got a social research background from that area, and I ended up working at the New Economics Foundation, at their consultancy arm, looking at exactly how you apply those kinds of disciplines to decision making.
I find that really interesting and I think if you look at the way that evaluation and decision making, the world has moved on over the past 25-30 years, however long it is. There's much more understanding now that we can use, actual kind of high quality evidence about how programs, initiatives, how they impact people's lives, how they impact that wellbeing, and particularly how you could use subjective measures to understand that. We no longer measure success entirely on money and how many people are employed and what the record of crime rate is, for example. It is a much broader set of indicators that we can use where we ask people questions and involve them and get that feedback. And the use of that kind of evidence I think is really interesting and it's a different kind of discipline and the way that we can use that to help us make better decisions I think is very important. So that's what I found exciting. And yeah, I've been doing it for quite a while now, and over that time the sector has really grown. So evaluation has grown. The whole social impact, social value that I know we’re going to talk about, that's grown over that time as well.
Selin: Amazing. That definitely sounds super interesting. And for those who are listening and who are new to impact measurement, how would you explain that in the simplest way?
Oliver: I think really it's trying to understand how successful we are and usually we're doing in the context of a program or a project or something. And what difference has that made? We'll talk about things like impact and outcomes, and there's lots of terms that we'll throw around there. But essentially it's not just saying how much activity we have done, it's not just saying, for example, how many volunteer hours we printed or how many donations we've made or how many food parcels we've given. It's actually moving beyond that and trying to say what difference has it made to people's lives? What are the outcomes that've been created? Or for organisations' work, or the difference for them, or environmental impact. But often it's the impact directly on people's lives. And that's why we get into the kind of world of social impact. A lot of it is related to this desire to think about a broader measure of success, define success more broadly. So it's not just about economic growth, for example, and it's not just about money, but it's also about those well-being, health aspects as well, and things we all know are important, but actually often don't get captured in the metrics that we use to judge success. That's what impact measurement is about.
I think many people listening today probably have heard terms like social value, social return investments, things with social in the front of it as well, very much related. What's interesting about the whole social value movement is that it's also trying to understand the value of those impacts and that value is often expressed in a monetary term. So we'll talk about creating a certain amount of social value and express that in pounds. Social Return On Investment is where we compare that with the investment that we've got in. And that's really interesting because we then get to the question of was it worth it? So if you think about people who are putting money into a project, it's a project that won't create a financial return, but it's helping people, it's having an impact on people, but is it worth the money that you put in or not? And how do we understand that better? How much social value do we create for every pound that we're putting in? Or if people are volunteering, maybe they're using the Goodsted platform and they are volunteering for organisations. What difference is it really making? Was it worth the time they put in there? Or I think, perhaps more usefully for decision making, which volunteering opportunities are creating the biggest impact for people? That kind of question is really interesting and it helps us understand, helps us decide where to allocate resources, and how to make decisions. So that's something I think we've seen really grow over the past 10 years, with things like the Social Value Act and so on. What is impact measurement? It's about understanding the impact that we have on people's lives, doing it in kind of a broader set of measures and trying to make decisions on the back of that.
Selin: I think that's a perfect summary and hopefully those who are listening also found it very useful. I definitely did. And so now, moving on to some examples.
What was the favourite project you worked on?
Oliver: I'm going to bring a couple of projects up, if I may. One of them is kind of a little bit different. We worked with Social Value International and put together a group for Social Value in Development and Humanitarian Assistance, an international development. So that's SVDHA, SVDHA.org if anyone's interested. What we tried to do there is say, well, look, we're seeing the social value principles being applied in these sectors in the development world and humanitarian assistance world. And we had done projects in that area, others were as well, but there's much more potential for it to be used there as well.
People are wanting to use it, but there wasn't really any kind of guidance in that area in particular. So the idea with SVDHA is it's a network, if you like, of practitioners across the world who are interested in social value. And we've done some work putting together some guidelines for applying social value in those sectors, and also some case studies. There's some great case studies on there, some of the best work that we've done, but work from other people as well.
So I think probably by the time people are listening to this, the most recent case study will be one around child labour in tobacco and helping to remediate and prevent child labour in the tobacco growing communities, essentially. I believe it's a great example of where thinking has moved on from just saying, let's quantify how many people at child labour, to actually saying, well, what difference are these community based initiatives making? Are they preventing it? But also, what's the impact of that prevention work? Yes, in terms of actual quantity of child labour, but actually then access to schooling and life chances of children who otherwise would have been involved in child labour. And also starting to get to some of the trade offs that are there as well, and the financial implications of the communities when these kinds of projects are run. So there's some really interesting stuff there. It's been great to be part of a network that's been looking at trying to spread that best practise, if you like.
A little bit closer to home, I think for ourselves, we did a piece of work with Lendlease, looking at Artworks Elephants, so Elephant and Castle part of London, it was a big redevelopment. There's a space there that was being used as part of the construction site, but was going to be empty for a while, so it was turned into a ‘meanwhile use space’ for 3 years, where a number of organisations were doing projects. So the council came together and developed a kind of hub really, a kind of community space, where there were shops with food, with drinks, with some offices as well. It really kind of contributed to that local community and supported local businesses to run their organisations there and so on. And so trying to understand the impact of that, again, not just looking at the kind of economic figures, but looking at that broader impact on the community I think is really interesting, really exciting and for us, what's really important about that kind of work is that we go to speak to people as well.
I think it's possible to do social value. Social Impact is a desk based exercise where you gather lots of data, put it into spreadsheets and so on and that's part of it, but actually going out and talking to people about their experiences, what they think is a really important part of it. We go out there and speak to people, get feedback, good and bad, what have you expected and plenty of unexpected feedback as well. So that's kind of a really fun project and there's a write up on that on the UK Green Building Council website. They did some work around Social Value and gathered some case studies, so that's up with them.
Selin: Yeah, it sounds super fun actually, just to go and speak to those people instead of only sitting behind the laptop all day. I know you are now working on some new
software tools to help organisations with their social value calculations.
Can you tell us a bit more about these new initiatives?
Oliver:Yeah, of course. So there's 3 things to talk about there really. The first one is a piece that we released last summer, summer of 2021, and that was a Social Value Return On Investment calculator. It's free to download from our website and we launched it at the Social Value UK Members Exchange. The idea with that particular piece of software is that a lot of people are approaching social impact, social value, SROI and so on, and they're approaching it and wanting to build an analysis framework basically spreadsheets, a model, and what have you. They're looking at the options that are out there and lots of online solutions that you can pay money for. There's solutions online or on Excel, sometimes free to use that lock you into a certain set of metrics and values and everything.
There's other ones like the Social Value UK Value Map that are free to use and don't lock you into a set of particular metrics or values. But we're seeing a lot of mistakes as well when they're being used and perhaps they don't always allow the sophistication analysis that people kind of realise they want, different ways of cutting your data, or representing changes. So we wanted to put something together that helped with all of that was free to use, that allowed you to use existing outcome frameworks that are out there, your listeners probably know some of your own outcomes or mix and match, and what have you. So we put together that Excel based calculator, basically. The idea there was it's transparent, it doesn't lock you into a certain set of approaches if you like, aligned with all of the social value best practise.So that was one thing and that was kind of really exciting.
What we've done more recently is that we started a partnership with Social Value UK to support with relaunching, something called the Global Value Exchange. Now the Global Value Exchange is something that has been around for a while, it's an online database of indicators, stakeholders, financial proxies of a like. Really useful resource for practitioners to find data that they might use. That however, went offline for a while and Social Value UK had difficulty getting it back online, it needed extra work and so on. So we've now partnered with them to bring that back online. That's really exciting. It's another free to use tool as well. You can register and do various things with it, but it's also without doing that, it's just a free to use searchable database. So I think that's, we found that useful in the past, I think as a practitioner as well as well. And I guess for us the next step is looking at how we can apply some of those principles that I've just described on an online portal version so we can have an online social value calculator. All the principles I just described about the Excel one, about being able to use your own outcomes or other outcomes, frameworks or mix and match, being able to use lots of different ways to measure your outcomes and to consider different things like impact.
I don't want to get too technical here, but we'll talk about things like how long your outcomes last benefit period, how long does that benefit last for your stakeholders, how much does it decrease over time and things like that. Your drop off, how much it changes over time. Sometimes it's a bit messy, the real world is a bit messy. You measure things and actually, things don't go down in a nice straight line or a nice curve. They change in slightly more unusual ways. And you want to be able to represent and take account of that when you're putting your work together, allowing you to do that kind of thing. So what we're doing with that is putting together an online calculator that will be available soon, so watch your space and I hope that will sticking to those principles. It's not going to be quite as free to use like the Excel one is because it costs a bit of money, but we're hopeful that it will make a contribution to the kind of practitioner work as well.
Selin: To be honest, that all sounds very useful, especially the Excel tool seems very flexible, how you can use it both with your own outcomes or with any existing outcome frameworks, I will definitely check that calculator out.
What would you advise organisations who are early in their journey with impact measurement?
Yeah, it's a good question and I think that it's interesting for organisations starting on this journey now because it's a very different world I think. Just 10 years ago, in terms of the amount of resources out there, the amount of different things that you can read, some of which people agree with often or they don't necessarily agree with each other on that. There are a number of consultants who are out there and happy to take your money to support you on this. And I should confess that I'm one of those as well, so they're not all bad, but I wouldn't start there.
I would start with trying to be really clear on what it is you're trying to achieve from doing social impact evaluation or social value or whatever you're calling it. How much is it a case that you're needing to demonstrate your impact to your funders? Maybe they have a very specific thing they want you to do. How much are you looking to try and be accountable for your impact to your stakeholders? And because they're important, and you want them to understand the impact for you to hold yourself to account for it. How much you're trying to improve what you do and actually find out where it's working, where it's not working and how you improve it? How much are you trying to make a decision about where you focus your time, your money, your resources? Which programmes are treating the most bang for their buck, as it were? So what are you trying to achieve from it?
That's I think the first thing and that can then guide our approach to it. So I would start there first. It's tempting to say, which framework should we go with? Look, here's 5 frameworks, here's 6 tools that we could use — let's pick that one and that one and off we go. I wouldn't start there.
I'd say, what are we trying to achieve, and then after that, I'd consider what are the principles we're trying to follow here? So rather than saying, what 10 things should we try to measure? I'd say, what are the principles that we're trying to underpin it? So that might be about what you're trying to understand. Are you trying to understand whether you met your objectives or not, or you're trying to think a bit broader than that and say, let's be accountable for all about impact, intended and unintended, positive and negative.
Lots of projects have unintended consequences as well. Some of your work I know with Goodsted is in housing, you're building new housing, there's going to be disruption impact in the local community. It's not intended, but it's going to happen though. Understanding that and being accountable for that, as well as the good stuff, I think is really important. I think it's a shame when social value just becomes, as it does sometimes — let quantify lots of good things that we've done on the side, rather than getting into the real core of an organization's business and positive, negative outcomes there. Anyway, I'll start with those principles. Social Value UK has a good set of principles, that's probably a good place to start. And then when it does come to measurement, I would say a couple of things. Firstly, focus on outcomes if you can, so it's much easier usually to measure your outputs, how much stuff that you've done. So how many apprentices did you work with? How many CV workshops have you supported? Important information, but really focus on the outcomes. What impact did you have on that apprentices' career progression, impact on their well being? When you do a CV workshop, does it actually make a difference? So focusing on those outcomes, but also when you're understanding what those outcomes are, then do that on the basis of fully engaging with the project as well. We spoke earlier about talking to people.
Talking to people first, before you ask them kind of these fixed quantitative questions, your metrics, if you like. Talking to them first is really important. So really get scripts of what you're trying to achieve. Focus on your principles first off, rather than grabbing metrics or frameworks or tools and so on. Then when it does come to the measurement, focus on those outcomes, but also be proportionate, I think, with this as well. So how much resource, money or volunteering time is going into the project you're trying to evaluate? And to be crude, the more that's going in, the more you want to invest in the evaluation as well. And I think two things can happen here, two extremes.
One, we don't want to invest much in the evaluation, so we get a bit of data that's easy to get, but really doesn't tell us enough about the project, given the scale that we're talking about, it's not really very useful. On the other hand, I sometimes see people do evaluations of really quite small projects where a lot of time and work and money has gone into evaluation and really quite a small project. Does that feel like it's proportionate or not? So it's got to be proportionate as well. This is trying to help us make decisions about how we, amongst other things, use our resources, but if all of our resources go to evaluation, it's not very helpful. There's a balance there, so be proportionate with all of that as well.
Selin: Amazing. Thank you so much for your time Oliver. All of this has been super insightful, and I'm sure everyone who's listening has been taking notes to start on their impact journey, or perhaps they've already started. And I'm sure they learned a lot too from this episode. So thank you so much, and I look forward to keeping in touch and continuing our talks about Goodsted's impact measurement.
Oliver: Indeed, thank you very much. It's been a pleasure to be with you today.