No, we should not.
Let me tell you why, with a story about how we keep track of our health.
Last week I attended the Financial Times Investing for Good USA Conference in New York. If you haven’t noticed, the FT is taking seriously the shift towards using money as a force for good. Their Moral Money reporting is getting a lot of column space and resources—they even have a newsletter (part of a paid FT subscription) with lots of great content.
During Q&A at the social impact measurement panel at this conference, a woman in the audience, sounding exasperated, asked whether social impact is like health: is it something so nuanced and complex that we can never fully understand it in a simple, clear and comparable fashion?
The implication seemed to be that until we can boil social impact down to a single number, like IRR, we can never really understand it.
I love the health analogy, but I disagree wholeheartedly with the conclusion.
How Do We Measure Health?
Let’s think for a moment about how we measure health. While a person’s health is complex, there are some basic, universal measures that indicate well-being: blood pressure, resting heart rate, BMI, cholesterol levels, respiratory function, and so on.
Think about the characteristics of these measures: they are easy to gather, we collect them directly from patients, and they can easily be compared.
We gather this data annually in a physical (and gather a subset of them every time we visit the doctor), and doctors and nurses use these data to get an overall sense patient well-being. If these measures are way off, a patient might be unwell and in need of further testing.
Some Core Principles of an Effective Measurement System
Let’s think about the core principles that are in evidence here, because they give us good guidelines for how to think about social impact measurement:
- Find measures that apply broadly
- Determine what good and bad ranges look like
- Regularly gather primary data to understand how individual patients are faring
- When those indicators are off target, go deeper with specialized measures
It feels obvious that doctors have a core set of things they can measure to understand well-being. At the same time, we are not scared off by the complexity underneath. Indeed, we recognize that we must master that complexity to truly help patients: the human body is complex, so we must be comfortable with complexity to understand it fully.
And so, in patient care, simplicity and complexity happily coexist.
The Core, Comparable Metrics of Social Impact
Similarly, for social impact measurement, there are broad indicators that can be easily compared (many of which align with the Impact Management Project and for which we’ve developed questions and benchmarks at 60 Decibels):
- WHO is being served: income levels, access levels, gender, members of excluded groups, etc.
- WHAT is their experience of the product: Net Promoter Score, customer effort score, etc.
- HOW MUCH does the product or service improve their lives: meaningfulness of impact, other indicators of changes in well-being (income, confidence, safety, empowerment, etc.)
All of these data can be easily gathered directly from the people experiencing (or not experiencing) social impact. And, just like blood pressure, gathering this data from the actual people being served is a prerequisite to understanding whether a specific product or service is making a difference.
From the People Being Served
“From the actual people being served” bears underlining: if my doctor wants to understand my health, she wouldn’t be satisfied knowing the BMI or blood pressure data of people like me. Instead, she would use population data to understand what the healthy range was and compare that range to what she reads on her dial when I’m standing right in front of her.
That might seem obvious, but in social impact measurement we seem too easily convinced that studying similar interventions is good enough—that we can simply extrapolate that data to our investment and be done. The fact is, as in medicine, studying other, similar interventions is the starting line, not the finish line. When I arm myself with that desk research, and then couple it with what I learn about the lived experience of the people my impact investment is serving, then (and only then) am I in a position to understand the impact performance of my investment.
Conversely, if we never listen to the customers being served by our investment, we’re saying the equivalent of (at my hypothetical doctor’s appointment), “typically, 46 year old white men have a blood pressure of 125 / 85.” That’s good to know, but it tells me nothing about whether I’m eating too much salt or at risk of heart disease.
Even more obvious, we would never expect that a blood pressure reading or BMI, alone, would tell us everything we need to know a person’s health. So why are we so obsessed with finding a single, one-number measure of social impact? These simplifying measures are, at best, directional indicators of a deeper reality that lies beneath. Importantly, that is not the same thing as saying that all that complexity must can and should boil up to that single number.
Finally, let’s not be frustrated that we can’t compare everything to everything. After all, we can only compare lungs to other lungs, not to livers or kidneys. Despite this limitation, we are not powerless to deduce whether one person is healthier than another.
Embracing Simplicity and Complexity
That’s good news.
It tells us that, in human health, we are comfortable with embracing both simplicity and complexity. We understand that the human body is itself a system with countless underlying organs and sub-systems. We recognize the need to understand these systems at a micro and a macro level. When we do so, we are in a position to successfully manage human health.
We can and should be just as comfortable with the notion that social impact happens as part of complex systems; and we should be optimistic that a core set of simple, easy-to-measure, comparable indicators can give us an enormous amount of insight about actual, on-ground social impact. Like in human health, we should also embrace the need to understand this deeper complexity if we are serious about managing social impact performance. That means deep, specific data about my specific intervention, coupled with cross-cutting, universal measures that apply to all interventions.
A Glimpse of the Future
This is all well within our grasp.
The first step is to stop telling ourselves that there’s some magical shortcut between here and there. Our work, and the people we aim to serve, are too important, and the amount of capital coming towards social impact is too big, for us to aim to skip steps.
Most important, let us never forget is that this work is about real, actual, living people. These people are the locus of change. It is a core part of our job to listen to them so that we can truly understand their perspective and their lived experience.
This is the only way we can manage social impact performance to achieve meaningful better outcomes.
And someday soon, doing all of this will be as normal and as natural as taking someone’s pulse or their blood pressure.
Thank you for always keeping people at the focus of what you do, Sasha. It is truly what sets you apart and allows your gifts to have a much broader impact than just throwing some money at problems and patting yourself on the back!
Hi Sacha,
Thanks for the great analogy. The part about gathering information directly from the people being served particularly resonates with me.
And yes, I think everyone agrees that social impact, like health, can’t be simply summed up by one particular metric. It would be a gross simplification of the impact of a company not to base impact assessment on a selection of relevant indicators.
I’m not sure I agree however with your saying that you can boil impact down to a few common metrics. I feel like impact is so complex that almost every case is very different. And then there’s the question of environmental impact which as we know has huge consequences on the social side, and shouldn’t be boiled down to one number either like it is too often the case with GHG emissions. That’s why we at impak Finance developed a rating because we feel that that’s a great tool to embrace all this complexity.
To come back to your health analogy: what if all these indicators you mention could be accurately translated into one number? This number being too low for instance would be a sign of the underlying indicators being too low as well. Doctors would then only need to look at one number to know whether or not they need to dig deeper.
Coming back to social impact, a low score would indicate poor performance in several sub-categories, and digging into the underlying data would give a better sense of the situation. That’s why we complement our scores with a full analysis of the IMP criteria, which contains all the complexity of impact analysis. You would then get all the advantages that my colleague Velina mentioned. And with enough data you could correlate impact with financial performance, think about what an efficient impact frontier would look like, etc.