This post originally appeared on Acumen Ideas, our new channel on Medium.com. If you’re interested in the nitty-gritty of impact measurement, be one of the first to sign up for Acumen’s new Impact Matters e-newsletter that will come out monthly. You’ll get great content at the cutting edge of impact measurement, and we’ll also make sure let you know when our full piece on Lean Data comes out in Stanford Social Innovation Review this winter.
In 2006, Sam Goldman and Ned Tozun set out to eradicate kerosene as a source of light in the developing world. As a Peace Corps volunteer in Benin, Sam witnessed the damage kerosene could do when an overturned lantern created a fire that nearly killed his neighbor’s son. They also saw what a scourge the dirty fuel was for poor, rural families without access to energy, eating up 15 percent of their spending.
Sam and Ned decided to start d.light design, a social enterprise that would solve this problem once and for all. With funding from Acumen and others, d.light set out to create a business providing low-cost solar lanterns to poor customers. Since then, the company has sold tens of millions of solar-powered lights across more than 40 countries.
So is d.light a success? By one measure, absolutely. They are seeing demand for their product and on track to reach 100 million customers by 2020. That’s nearly 10 percent of the more than 1.3 billion people globally without access to electricity. But for entrepreneurs like Sam and Ned — and all of us at Acumen with a mission to make a real dent in poverty — just reaching a large number of people isn’t good enough.
At Acumen, we’ve spent the last 15 years investing in social enterprises that provide critical goods and services to the poor. We invest in these businesses because they are hard-wired to reach large numbers of people: when a social enterprise gets its model right, it will reach more people per dollar funded than traditional aid or philanthropy.
But while it makes us proud to say we’ve helped a million people acquire a reliable solar light or 10,000 women give birth in a high-quality, low-cost hospital, we need more than just big numbers to tell us if we are actually changing people’s lives.
How can we know if we are making a real difference?
Over the last 10 years, impact investing has attracted lots of attention and dollars. Thanks to the success of d.light and other ventures like it, today there are hundreds of impact investors putting their money behind companies that aim to deliver a social and financial return.
Despite this growth, impact investors have done a terrible job of analyzing whether or not these enterprises are creating meaningful social impact.
For example, in June, the Global Impact Investing Network and Cambridge Associates published the Impact Investing Benchmark, the first comprehensive analysis of the performance of impact investors. The report does an outstanding job of analyzing the financial results of impact investing funds, but it says virtually nothing about social performance. That’s a problem.
You’d assume impact investors must be good at measuring social impact. How else could we call ourselves “impact” investors? Not surprisingly, 95 percent of impact investors say they measure impact. But, if you scratch the surface, you’ll discover their definition of impact is mostly limited to big, flashy numbers: number of farmers using an improved kind of seed, number of kids attending school or, as in the case of d.light, number of lights sold.
This is a start, but it’s not good enough. Typical impact investors may know how many farmers a company has reached, but they don’t have a clue if these farmers are better off. They may know how many kids attend schools, but they can’t tell you if the students are from low-income communities or just transplants from the private school down the street. They may know how many households bought a new solar lantern, but they don’t understand if the children in these homes are still dying from kerosene fires.
There’s a good reason impact investors have been falling short : the existing tools for measuring social impact are nearly useless to a social entrepreneur.
These tools, mostly inherited from large-scale, international development organizations, are cumbersome, expensive and typically take a matter of months or even years to produce any real data. For a cash-strapped, resource-constrained social entrepreneur trying to build a fledgling business in tough, emerging markets, these tools don’t make sense.
The good news is, we have an opportunity to change this. Unlike five or 10 years ago, the majority of the 2.5 billion people living in poverty now have access to a cellphone and, in another five years, virtually everyone will be reachable by phone or SMS. At Acumen, we’ve developed a new approach to impact measurement that takes advantage of this shift. Our approach is optimized for entrepreneurs building social enterprises in the developing world, and it capitalizes on today’s information revolution to gather data directly from low-income customers. Our goal is to use this infrastructure to understand our social impact and better serve the poor. We call this approach Lean Data.
Unlike traditional impact measurement, Lean Data is designed to quickly and affordably generate quality customer insights that can immediately drive entrepreneurs’ decisions.
It reframes impact measurement as customer feedback by applying Lean Startup experimentation principles to the collection and use of social impact data. While Lean Startup aims to understand product-market fit with questions like “Do you like this product?” and “Will you buy this product?,” Lean Data goes a step further by working to understand how a purchased product is — or is not — changing a customer’s life.
By asking questions via mobile phones and other existing customer touchpoints (such as a salesperson’s visit to a customer’s home or a company’s call center), Lean Data allows enterprises to get social performance data in a matter of weeks and at a fraction of the cost of traditional measurement approaches.
In the last year, Acumen has helped 12 of our companies measure their social performance by surveying more than 5,000 customers across seven countries. Each of these projects took weeks, not months, and cost thousands, not hundreds of thousands, of dollars.
Here’s how.
Lean Data leverages technology, so enterprises can communicate directly with their customers. It is now possible to get reliable, meaningful data directly from low-income customers either through calls or SMS messages. For example, we worked with Ziqitza, a healthcare company that provides low-cost emergency services in India to understand what percentage of its customers in Orissa and Punjab live below the local poverty line. Our results showed that 75 percent of customers live on less than $2.50 a day. In another case, we worked with Juhudi Kilimo, a microfinance enterprise servicing smallholder farmers in Kenya, to measure its social performance using a 10-question SMS survey. The survey showed that the loans Juhudi Kilimo provided to purchase dairy cows are helping farmers see an increase in milk yields of 60 percent.
Lean Data puts the customer first, not the investor. As an investor in social enterprises, Acumen needs impact data to manage its own performance. But we believe social enterprises should first and foremost be accountable “downward” to their customers before worrying about “upward” accountability to their funders. Social enterprises set out to solve meaningful problems for their customers, and they should only systematically collect impact data if that information helps them understand how their products or services are making a difference in their customers’ lives. The information should also be shared “upward” with funders, but that cannot be the primary reason for collecting data.
Lean Data gets underneath not just the “what” but also the “why” of product-market fit. Lean Startup principles focus on product-market fit: is there a demand for a new product in a given customer set? How satisfied are customers with the new product? Social enterprises can take this a step further, asking not just whether there is product-market fit, but why that fit exists. This is the first step towards understanding impact. When we discover why products are purchased, how well or often they are being used, and which problems they solve or fail to solve — like improved productivity, increases in household savings or fewer sick days — we empower customers to articulate what impact means to them. This kind of insight is invaluable to entrepreneurs looking to drive lifetime value, customer loyalty and social impact.
We’ve been developing Lean Data for a little more than a year and, while it is still in its early days, we see huge promise.
If we can give more entrepreneurs like Sam and Ned the right tools to understand their social impact and hear from their customers, they will, for the first time, have actionable data that can tell them, in real time, how to improve their products and create meaningful change.
The truth of this work is that the big, glossy numbers allow us to sing our own praises and raise more money, but they do little to help us improve the lives of the people we aim to serve. It’s time to dig deeper, to use technology to talk directly to our customers, so that our work can realize its full potential.