How Much Impact Measurement is Enough?

Recently I was speaking with a sophisticated, experienced impact investor. She’s been investing impact capital for more than a decade. Her fund has a well-developed investment thesis and a clear impact measurement system.

This system, as I understood it, thoughtfully looks at preexisting research on social impact: things like whether a particular type of software improves learning outcomes for kids; or whether a given healthcare app results in better patient outcomes. In addition, post-investment, the companies she invests in study the efficacy of their offerings—it’s part of what is required by their (mostly) public sector customers. These studies test whether each company is replicating the results expected from the research.

However, in a few sectors, she’s not easily able to get this deeper data.  In that context, she asked me, “What’s your view? Do we really need more impact data?”

Meaning: in these sectors, they have good background research along with a general indication that the products being sold have a net positive impact on their customers. They just don’t know how those positive impacts translate into changes in customers’ lives. Isn’t that enough?

Why Are We Gathering Impact Data?

It depends, I replied, on why we’re gathering impact data.

If we’re doing it to gather evidence—to prove something—then by all means let’s gather only the data needed to cross the threshold of proof. After that, we should stop.

But what if that is not the right objective for impact measurement? The phrasing, “Do we really need more impact data?” assumes that gathering this data is at best a neutral activity for the business, and at worst it’s burdensome, a diversion of resources, and a distraction.

If this is the case, then managing it down to a minimum is the right thing to do.

Flipping the Question

My view, however, is that it’s high time we flipped her question from, “Do we really need more impact data?” to “Do I know all I need to know?”

“Do I know all I need to know?” about my customers, my beneficiaries, and how they experience my service?

“Do I know all I need to know?” to serve them better?

“Do I know all I need to know?” to create a deeper change in their lives, one that will both improve their well being and make them more likely to be loyal and to recommend my service to others?

We perpetually ask the wrong question because we’ve been trained to assume that social impact measurement will forever be a ponderous beast: large-scale, expensive studies that take years to deliver results. That heavyweight approach, the standard in our sector, is extremely useful in a very narrow set of cases. It’s also almost never the answer for growing, dynamic organizations that are still evolving how their solution can best serve customers.

Stop Taxing Social Businesses

For these sorts of social businesses, social impact measurement must be optimized for learning and improvement cycles. It must move as fast as these nimble, dynamic social businesses. It must feel like customer insights, and not like academic research.

If we fail to make this shift, impact measurement will forever be what it feels like today:  a compliance exercise that is a tax on social business, rather than a way to increase knowledge and insight. (More on this risk from my recent panel at the SOCAP conference)

Because, let’s be real: it’s hard enough to build a business that solves a social problem AND is financially viable. Adding a measurement tax onto that business makes no sense.

Conversely, if social impact measurement can help that business grow, improve, and better serve its customers (and yes, at any point feel free to substitute “nonprofit” or “community organization” for “social business”)…well then we’re really on to something.

A Real Example from Nigeria

Imagine, for example, that your social impact report helped you do real things, immediately. For Psaltry, a Nigerian company that helps smallholder cassava farmers, impact data gathered by our team at 60 Decibels helped Psaltry decide to open three new processing plants closer to customers (they discovered that customers’ earnings were taking a hit due to high transportation costs).  This same impact report uncovered that farmers had cashflow issues. Psaltry is using this data to help them get a loan from a local bank: the data helped them convince the bank of the need for this loan and how it would be used to help farmers. You can read their whole story here.

It’s Time to Stop Minimizing a Core Activity

The point, though, is not about this particular impact study—though the results, and the company’s responsiveness to them, are all outstanding.

The point is to ask ourselves: how have we allowed the data we gather about mission achievement to become peripheral to how we run mission-focused organizations? How have we created an approach to understanding our impact that should be minimized so we can “get on with our actual work?”

To the question, “How much social impact measurement is enough?” I’d give two answers:

One: if you’re doing it to prove something to someone, then gather the least data you can to demonstrate that proof, and then stop.

Two: if you’re doing it to learn, if you’re doing it to serve better, if you’re doing it to listen better, so that you can accelerate achievement of your mission, then I’d be really careful about marginalizing or minimizing it.

Instead, I’d bake it in right at the core.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.