Building our impact intuition

I was speaking to an impact investor recently, and he was saying that investment decisions are ultimately based on an intuitive sense the investor has about the company: the deal, the team, the market opportunity. And shouldn’t we just use our intuition to assess impact?

This is the most common unspoken premise used by impact investors to justify not collecting impact data.

So, where does this intuition come from? And is there such a thing as good and bad impact intuition?

Intuition is subconscious pattern recognition. And patterns are the sum total of the information we’ve taken in. If that information, and our ability to understand and process it, is of high quality, then we develop good intuition. If not, not.

A good investor is awash in quantitative and qualitative data that inform her investment intuition. For example, on the quantitative side, she’ll know what she expected gross margins to be, the predicted length of a company’s working capital cycle, and how many years she forecasted it would take for the company to get to profitability.

But that original financial model will have a very short shelf life: after the investment, she’ll get reams of data to show whether her predictions were right or wrong.

But in the world of impact, she’ll handle things differently.

She’ll look at research and benchmarks to develop a thesis. And she’ll stop there and multiply products sold by those benchmarks [e.g. 10 lights sold * X predicted impact/light = 10x impact].

This is like creating a pre-investment financial model of a company and then, two years later, when asked how the company is performing, using the model’s original variables to answer that question.

Not only would this answer not be any good, but her impact intuition would never improve.

Why do we accept the idea that we can understand the impact we are creating in people’s lives by looking at comparables? Why do we nod when told that it’s hard to get better data (it’s not)? How can we say that “we know impact when we see it” if we don’t gather data to understand actual performance?

The only explanation is this: we are not the people whose lives are, or are not, improved by a given intervention; we are not personally affected by a positive or negative ROI on a “better” solution; and the difference between potential and actual impact doesn’t land on our doorstep, or in our pocketbooks, or in our child’s cough or the quality of the education he receives.

The only way we’ll create better impact intuition is if we apply ourselves seriously to the question of learning what does and doesn’t improve people’s lives.

We don’t settle for “it’s too hard” anywhere else but here.

Impact measurement over the next decade

I had the chance to participate on a panel at SOCAP about the future of impact measurement, and was surprised how challenging I found Karim Harji’s framing question:

Where is impact measurement headed over the next decade? What is it going to take to get there? 

After pondering on this question for a while, I ended up at the conclusion that the future is very bright at the level of company-customer interaction.

I say this because in the coming decade, social enterprises, like all companies, will necessarily begin accessing and managing much more customer data gathered remotely through devices. It’s a bit easier to see this future when one is in San Francisco staying at an Air BnB and taking Lyfts everywhere: as mobile phones become both the communications and transaction platforms for nearly everything in everyone’s lives, companies, no matter what their specific business or the customers they serve, will have more data about us. While it’s true that poorer, more remote customers will lag millennials in San Francisco in terms of how soon they get on this conveyor belt, the direction this is heading, for everyone, is clear.

With that in mind, the only question at the company-customer intersection is whether and to what extent companies will incorporate data about social impact into their growing data flow. My thesis is that doing so will be a competitive advantage, allowing companies that move first to better understand how well their products and services are improving their customer’s lives, thereby driving greater loyalty, share of wallet, and share of mind and voice.

I believe this because, as our Acumen impact team has worked with companies on Lean Data projects, it’s become increasingly clear that value creation is impact when you’re dealing with critical goods and services like electricity or education or healthcare: if the customer who buys her first solar lamp stops using kerosene, uses the lamp to keep her business open later at night, and also uses a second lamp for her kids to study at night, then that lamp is creating deep and meaningful value (impact) for her. And all our data show that this same customer is nearly always a net promoter of the company, a source of positive word of mouth, and a high-value and loyal customer.

If this thesis plays out over time, then we’re about to be riding a huge, powerful wave that we’ll simply have to redirect slightly to incorporate thoughtful impact data capture and to drive towards impact management. Soon, even resource-strapped, impact-focused companies in the developing world will have no choice but to gather and utilize more data (including impact data) from end customers if they want to serve these customers effectively.

The question I find harder to answer, interestingly, is: What is going to happen to the capital market for impact? Here, things seem a bit muddier.

In order for capital to increasingly flow towards high-impact opportunities, there has to be some standardization in terms of how impact is measured and communicated, so that an investor looking to compare impact performance can compare opportunity A and B in the same way she compares financial performance for these same two opportunities.

I believe this evolution is a very important one, indeed it might be the most important development that needs to happen if the impact investing marketplace is to realize its full potential. However, unlike the evolution at the company-customer level, it’s less clear to me that there’s forward momentum pushing us in the right direction. It seems possible that we are due for a step-change in terms of how investors deploy capital for impact, and it seems just as possible that five or ten years from now things will be as bespoke and hard to decipher as they are today.

My best guess is that what’s needed to make a shift here is that a handful of highly influential and interconnected players – those holding large amounts of capital that they distribute through a large ecosystem of connected funders – need to establish their own higher, clearer impact measurement standards that they will use to deploy capital, such that their new standards flow all the way down the chain and slowly shift expectations for, and raise the bar for, everyone in the space. This was the role that the U.S. Government played with LEED certification through the GSA, which owns 9,600 buildings in 2,200 communities across the U.S., and I suspect it’s the pattern that needs to play out in impact investing too.

For more on this topic, here’s the link to the SOCAP16 plenary I got to do with Karim Harji, Jim Fruchterman, Kelly McCarthy, and Paul DiLeo.

socap16-impact-measurement

Lean Data Podcast

On Monday, Tony Loyd was nice enough to include me in his great series of Social Entrepreneur podcasts. We covered a lot of topics but dug in most deeply on Lean Data, particularly on how we are using it at Acumen to amplify the voice of low-income customers so our entrepreneurs can better serve them.  It was a fun conversation.

(if you’re not seeing the embedded link click here)

 

If this kind of thing is up your alley, you might want to sign up to receive the specialized newsletter we’ve created to share hot-off-the-press insights on what we’re learning through Lean Data. We send it out once every six weeks or so, so it won’t clog your inbox, and it’s full of great stuff.

It’s called Impact Matters and you can sign up here.

 

#ImpactMatters Twitter Chat

Tomorrow, Wednesday, February 17th at 12 noon Eastern, I’m helping run a Twitter chat that Acumen is hosting to talk about Lean Data and measuring social performance. It’s all about the finding the next frontier in impact measurement, in a discussion with Acumen, Omidyar, Stanford Social Innovation Review, the Aspen Network for Development Entrepreneurs and Root Capital.

Here’s how it works: (aside: Twitter chat 101)

  1. You can follow the chat with the hashtag #ImpactMatters.
  2. Please submit your questions before the chat so we have good stuff to talk about.
  3. You’ll also want to follow @Acumen on Twitter and join the chat on Wednesday at noon Eastern.

I’ll be joined by a great group that of partners who have helped us develop and spread Lean Data, including:

Hope to see you there!

The Power of Lean Data

In the last few months, I’ve been writing more about the evolution in how we’re thinking about impact measurement at Acumen. We call in Lean Data.

Until now, there’s really not been a good way for social enterprises to measure their impact in a way that makes sense for them and adds values for their companies and for their customers.

I think we can change that.

For the full soup-to-nuts story of Lean Data, check out the article that we published yesterday in Stanford Social Innovation Review: The Power of Lean Data. I had the great pleasure of writing this piece with Tom Adams of Acumen and Alnoor Ebrahim of Harvard Business School.

SSIR_Lean Data

If you want to go out and use Lean Data, you still have time to sign up for our +Acumen Lean Data course, which starts on Monday. And don’t forget to print out and laminate your own version of our handy-dandy Lean Data Field Guide.

Geeking out Next Thursday

I’m looking forward to speaking at the Catalyst for Social Change event this coming Thursday, November 12. I’ll be speaking together with Jake Porway, the founder of DataKind and Samuel Sia, one of MIT’s Innovators under 35.

The event is at Fordham Law School at 7pm, and there are still a few seats left – you can get tickets here.

We’ll be talking about innovative approaches to data and measurement, and using them to make the world a better place. It should be a lot of fun.

While I don’t know exactly where the conversation will go, I suspect that if you’re the kind of person who finds this image funny then you’ll have a blast. Hope to see you there.

Numerator_denominator

Lean Data: Closing the Gap Between Entrepreneurs and their Customers

This post originally appeared on Acumen Ideas, our new channel on Medium.com.  If you’re interested in the nitty-gritty of impact measurement, be one of the first to sign up for Acumen’s new Impact Matters e-newsletter that will come out monthly.  You’ll get great content at the cutting edge of impact measurement, and we’ll also make sure let you know when our full piece on Lean Data comes out in Stanford Social Innovation Review this winter.

In 2006, Sam Goldman and Ned Tozun set out to eradicate kerosene as a source of light in the developing world. As a Peace Corps volunteer in Benin, Sam witnessed the damage kerosene could do when an overturned lantern created a fire that nearly killed his neighbor’s son. They also saw what a scourge the dirty fuel was for poor, rural families without access to energy, eating up 15 percent of their spending.

Sam and Ned decided to start d.light design, a social enterprise that would solve this problem once and for all. With funding from Acumen and others, d.light set out to create a business providing low-cost solar lanterns to poor customers. Since then, the company has sold tens of millions of solar-powered lights across more than 40 countries.

So is d.light a success? By one measure, absolutely. They are seeing demand for their product and on track to reach 100 million customers by 2020. That’s nearly 10 percent of the more than 1.3 billion people globally without access to electricity. But for entrepreneurs like Sam and Ned — and all of us at Acumen with a mission to make a real dent in poverty — just reaching a large number of people isn’t good enough.

At Acumen, we’ve spent the last 15 years investing in social enterprises that provide critical goods and services to the poor. We invest in these businesses because they are hard-wired to reach large numbers of people: when a social enterprise gets its model right, it will reach more people per dollar funded than traditional aid or philanthropy.

But while it makes us proud to say we’ve helped a million people acquire a reliable solar light or 10,000 women give birth in a high-quality, low-cost hospital, we need more than just big numbers to tell us if we are actually changing people’s lives.

How can we know if we are making a real difference?

Over the last 10 years, impact investing has attracted lots of attention and dollars. Thanks to the success of d.light and other ventures like it, today there are hundreds of impact investors putting their money behind companies that aim to deliver a social and financial return.

Despite this growth, impact investors have done a terrible job of analyzing whether or not these enterprises are creating meaningful social impact.

For example, in June, the Global Impact Investing Network and Cambridge Associates published the Impact Investing Benchmark, the first comprehensive analysis of the performance of impact investors. The report does an outstanding job of analyzing the financial results of impact investing funds, but it says virtually nothing about social performance. That’s a problem.

You’d assume impact investors must be good at measuring social impact. How else could we call ourselves “impact” investors? Not surprisingly, 95 percent of impact investors say they measure impact.  But, if you scratch the surface, you’ll discover their definition of impact is mostly limited to big, flashy numbers: number of farmers using an improved kind of seed, number of kids attending school or, as in the case of d.light, number of lights sold.

This is a start, but it’s not good enough. Typical impact investors may know how many farmers a company has reached, but they don’t have a clue if these farmers are better off. They may know how many kids attend schools, but they can’t tell you if the students are from low-income communities or just transplants from the private school down the street. They may know how many households bought a new solar lantern, but they don’t understand if the children in these homes are still dying from kerosene fires.

There’s a good reason impact investors have been falling short :  the existing tools for measuring social impact are nearly useless to a social entrepreneur.

These tools, mostly inherited from large-scale, international development organizations, are cumbersome, expensive and typically take a matter of months or even years to produce any real data. For a cash-strapped, resource-constrained social entrepreneur trying to build a fledgling business in tough, emerging markets, these tools don’t make sense.

The good news is, we have an opportunity to change this. Unlike five or 10 years ago, the majority of the 2.5 billion people living in poverty now have access to a cellphone and, in another five years, virtually everyone will be reachable by phone or SMS. At Acumen, we’ve developed a new approach to impact measurement that takes advantage of this shift. Our approach is optimized for entrepreneurs building social enterprises in the developing world, and it capitalizes on today’s information revolution to gather data directly from low-income customers. Our goal is to use this infrastructure to understand our social impact and better serve the poor. We call this approach Lean Data.

Unlike traditional impact measurement, Lean Data is designed to quickly and affordably generate quality customer insights that can immediately drive entrepreneurs’ decisions.

It reframes impact measurement as customer feedback by applying Lean Startup experimentation principles to the collection and use of social impact data. While Lean Startup aims to understand product-market fit with questions like “Do you like this product?” and “Will you buy this product?,” Lean Data goes a step further by working to understand how a purchased product is — or is not — changing a customer’s life.

By asking questions via mobile phones and other existing customer touchpoints (such as a salesperson’s visit to a customer’s home or a company’s call center), Lean Data allows enterprises to get social performance data in a matter of weeks and at a fraction of the cost of traditional measurement approaches.

In the last year, Acumen has helped 12 of our companies measure their social performance by surveying more than 5,000 customers across seven countries. Each of these projects took weeks, not months, and cost thousands, not hundreds of thousands, of dollars.

Here’s how.

Lean Data leverages technology, so enterprises can communicate directly with their customers. It is now possible to get reliable, meaningful data directly from low-income customers either through calls or SMS messages. For example, we worked with Ziqitza, a healthcare company that provides low-cost emergency services in India to understand what percentage of its customers in Orissa and Punjab live below the local poverty line. Our results showed that 75 percent of customers live on less than $2.50 a day. In another case, we worked with Juhudi Kilimo, a microfinance enterprise servicing smallholder farmers in Kenya, to measure its social performance using a 10-question SMS survey. The survey showed that the loans Juhudi Kilimo provided to purchase dairy cows are helping farmers see an increase in milk yields of 60 percent.

Lean Data puts the customer first, not the investor. As an investor in social enterprises, Acumen needs impact data to manage its own performance. But we believe social enterprises should first and foremost be accountable “downward” to their customers before worrying about “upward” accountability to their funders. Social enterprises set out to solve meaningful problems for their customers, and they should only systematically collect impact data if that information helps them understand how their products or services are making a difference in their customers’ lives. The information should also be shared “upward” with funders, but that cannot be the primary reason for collecting data.

Lean Data gets underneath not just the “what” but also the “why” of product-market fit. Lean Startup principles focus on product-market fit: is there a demand for a new product in a given customer set? How satisfied are customers with the new product? Social enterprises can take this a step further, asking not just whether there is product-market fit, but why that fit exists. This is the first step towards understanding impact. When we discover why products are purchased, how well or often they are being used, and which problems they solve or fail to solve — like improved productivity, increases in household savings or fewer sick days — we empower customers to articulate what impact means to them. This kind of insight is invaluable to entrepreneurs looking to drive lifetime value, customer loyalty and social impact.

We’ve been developing Lean Data for a little more than a year and, while it is still in its early days, we see huge promise.

If we can give more entrepreneurs like Sam and Ned the right tools to understand their social impact and hear from their customers, they will, for the first time, have actionable data that can tell them, in real time, how to improve their products and create meaningful change.

The truth of this work is that the big, glossy numbers allow us to sing our own praises and raise more money, but they do little to help us improve the lives of the people we aim to serve. It’s time to dig deeper, to use technology to talk directly to our customers, so that our work can realize its full potential.

Lean Data Goes Deeper

One of the most interesting questions we’re grappling with right now on the Impact team at Acumen is how to develop a more robust, rigorous, and transparent form of quantifying the social value our companies create. While I don’t believe we will ever fully understand all the social value created – and while there will always be room for debate and interpretation – I do think today we have the tools to get a lot closer to customers and hear how they value products.

And if we can figure this out in a clear and compelling enough way, I believe that would open the door to creating a true marketplace for social impact.

What we’ve learned with our Lean Data Initiative is that we can, thanks to the prevalence of mobile phones and other enabling technologies, now quickly and easily gather data directly from our customers in ways that drive insights for us and our companies – everything from customer loyalty metrics to poverty levels of who is being reached to customer satisfaction.

What we’ve been kicking around – and where I’d love your help – is the best approach to quantifying self-reported value.

Meaning, after a customer has purchased a product (a solar light, safe drinking water, an improved seed) and experienced the benefit, what are the best, most reliable ways to ask her how much she values that product? Because she is the one who is living it, accruing benefit from it, she is best placed to explain what it’s worth to her.

We’ve been having fabulous conversations on our team, conversations that get back to the basics about things like consumer surplus and why demand curves slope down; conjoint/discreet choice analysis to get to revealed preferences; and things as simple as asking how much, having experienced a product, someone would have been willing to pay for it.

To clarify the kinds of things we’re thinking about and where I’d love your thoughts:

  • How to best phrase questions that help Acumen’s customers accurately articulate the value they get from a product or service
  • Whether there’s a best way to ask “how much would you have paid” after people have a product to understand how much they value it
  • Prize-based, conjoint approaches where we give a subset of folks $100 to spend on one of a few bundles of goods / services, to understand real rank-order preferences

I’d appreciate ideas for approaches that might help us get the answers we’re looking for. Suggestions welcome in the comments or please just email me directly.