Uncorrelated Impact Understanding

Not long ago, I was speaking to a group of sophisticated impact investors from across the spectrum: everything from fully liquid, market-beating financial return expectations to market builders focused on creating social impact who are open to a broader range of financial returns.

The focus of my talk was Acumen’s work on Lean Data, which is our industry-leading approach to gathering customer data at scale. We’re cracking the nut on using technology to give voice to tens of thousands of customers in ways that allow companies to serve them better. I believe that this will, over time, help the sector as a whole deploy more capital to more opportunities that have more social impact. It’s exciting.

But before digging in to the details of Lean Data, I started the talk with an assertion:

The seriousness with which you work to understand impact should be uncorrelated with your expectations around financial return.

I actually said this twice, because we’re so used to talking about correlations (positive or negative) between social impact and financial returns that I wanted to be very clear what I was, and was not, talking about.

My point is, if you say you are in the business of creating impact, then, irrespective of the instrument you use, the financial returns you expect, and the risk you’re willing to take, you’ve got to be serious about understanding impact.

Interestingly, I heard some resistance on this point. The resistance mostly took the form of “I know impact when I see it” or, “why would I waste time on this, it will just distract me from doing the real work?”

I believe there are some cases in which we really understand impact, but I believe those are the exception. Indeed we are so quick to say “we know enough” in a world in which we know shockingly little.

For example, take the $800 billion spent annually by the U.S. government. Peter Orszag, and Jim Nussle, who successively ran the U.S. Office of Management and Budget, write in Moneyball for Government that “Less than one dollar out of every $100 the federal government spends is backed by even the most basic evidence that money is being spent wisely.”

Less than $8 billion of the $800 billion spent annually by the U.S. government is backed “by even the most basic evidence?” Wow. Color me unpersuaded by the argument that we generally know enough.

I think what’s really going on is that we:

Overestimate how much we know

Overestimate the cost of getting great data – because approaches that came before Lean Data typically cost 100x as much

Create an artificial distinction between “creating customer value” and “creating social impact”

Assume that, no matter what anyone says, this is about marketing and dealing with funders, not about learning

Underestimate the value of what we can learn.

On top of this, I worry that we say too lightly that we’re in the business of creating social change, or we assume that this “caring about impact” stuff should be left to the folks who are on the frontiers of solving tough, challenging problems in innovative ways.

The truth is, we are quick to celebrate and advocate for more money walking through the “I (also) want to create social impact” door and then get awfully timid talking about whether that impact is getting created or, more broadly, how much we understand about the connection between the investment, the intervention and the impact it creates.

Caring about impact doesn’t mean you don’t understand how to make money. It doesn’t mean you’re not a serious investor. It doesn’t mean that you’re giving something up.

It’s simply saying: this is who I am, this is what I do. I’m in the business of creating massive positive change in the world. And I know how to do that better than anyone.

You can say all of those things and not blink for a second when someone asks you what your financial returns are going to be.

If we are in the business of change, then we have to be in the business of understanding how change happens.

More what?

Read or attend any report or gathering in impact investing and you’ll be told that the impact investing market is growing fast. At last week’s Global Steering Group on Impact Investing Summit, for example, we heard a lot about the market reaching a “tipping point.”

How do we know whether or not this is a good thing?

We cannot answer that question by counting dollars, or by tracking how many philanthropists, foundations, and banks talk about the impact investing funds they are deploying and the impact they intend to have.

All that tells us is about changes in language and norms for product packaging.

The only “mores” that matter are more capital going to more initiatives and companies that make more of a positive difference in more peoples’ lives.

Until we are tracking that accurately, we have no way to know that we are making progress, or even if we are headed in the right direction.

Cogito Ergo Sum Ego Creo Impulsum

Of all the charts in the GIIN’s 2017 Annual Impact Investor Survey report, the one that struck me the most was this one.

Of 200 impact investors surveyed, 98% say their impact is in line with or outperforming their expectations.

This in a sector in which almost no one actually measures impact, a sector that still debates whether having the intention to create impact is an important description of an “impact” investor.

It’s like we have our own version of cogito ergo sum: “I call myself an impact investor, therefore, ipso facto, I create impact.”

 

 

Tomorrow: Facebook Livestream of Acumen Debates

Tomorrow (Wednesday) evening, at 6:30pm Eastern Time, I’ll be participating in the third in a series of Acumen Debates hosted by EY. The Debates are a fun format that create a livelier conversation than your typical panel. And the good news is that we’re livestreaming the event, so even if you can make it you can tune in on Facebook.

The debate topic is: Do impact investors need to compromise between financial and social returns?

I’ll be arguing, along with Debra Schwartz from the MacArthur Foundation, that there are many cases where compromises can and should be made. We will be debating against Greg Shell from Bain Double Impact and Hilary Irby from Morgan Stanley’s Investing with Impact Initiative, in a conversation moderated by EY’s Jon Shepard.

Here’s the link to watch on Facebook Live.

While I won’t tip my hand and share the points I plan to make tomorrow night, I do have one hope for the conversation, and for our sector as a whole. I hope we can all agree that, irrespective of the financial returns they can generate, if someone is going to be a truly great impact investor, they have to be passionate about impact.

This may sound like a truism, so let me explain by way of example. I read Fred Wilson’s blog every day, and the posts that make me smile the most are the ones in which he geeks out about technology: when he switches from iOS back to Android, or talks about the various wireless speakers he’s fidgeting with at home, or wades into conversations about Bitcoin and the blockchain. It’s apparent to anyone reading that Fred loves technology. It fascinates him. It’s what he’s passionate about. And I’m sure he can’t help but learn about the latest gadgets, all the time.

In the same vein, to be an “impact” investor (and putting aside if anyone still likes that term, which no one seems to these days), I would hope we’d be passionate about impact. And by “impact” I mean actual, tangible changes in people’s lives or in the environment. This passion would manifest, like Fred’s, in a deep, insatiable curiosity about what makes people’s lives better, and what leverage points might exist to make large-scale, lasting change. It would come across as profound attention being paid to improvements that happen in the real world, with all other indicators – including financial returns, which very well could be high – simply seen as means to an end.

These days, when I take part in and listen to conversations in our sector, I hear the most passion about funds, about returns, about fund structures, and about capital flows. It’s striking how comfortable we are talking about money. I’m looking forward to more days in which the passion we express, the deep curiosity we manifest, the conversations we can’t stop having, start and end with people, with communities, with what constrains and enables their lives.

 

What impact investing can learn from Vanguard’s 7x Growth

The Ford Foundation’s recent announcement of its plan to invest up to $1 Billion of its endowment into impact investments is yet another chance to ask if, or when, impact investing will become mainstream.

And by “mainstream” I mean normal.

One idea that’s been on my mind: rather than search for answers deep in the heart and soul of impact investing, we should look at the history of index funds.

The foundational academic article about indexing as an investment strategy was written 50 years ago, in a piece penned in 1966 by William Sharpe in The Journal of Business. Sharpe’s conclusion, among others, was that “The results tend to support the cynics: good performance is associated with low expense ratio.” Not shocking, but this was the opposite of the core logic of the mutual fund industry, one which justified high expenses by supposedly even higher performance.

Nine years after Sharpe’s article, in 1975, John Bogle founded the Vanguard Group, and in 1976 he launched Vanguard 500 (VFINX).  VFINX was and is a low-cost fund that mirrors the S&P 500. Investors in the fund are“buying” the entire S&P for a very low cost. The theory was simple: you won’t beat the S&P in the long run, so the smart thing to do is to get the S&P’s returns with as low an expense ratio as possible.

This one fund was a bet that, over time, investors would come to understand that consistently beating the market with an active stock-picking strategy is hard, and that beating the market by enough to make up for the cost of active management is nearly impossible.

(For some simple math to support this conclusion, think of it this way: historic equity returns are about 7% a year, and most mutual funds charge around 1.5% in fees (if not more). This means you’re eating up more than 20% of your return every year with the fees paid to an active manager. The assets that manager invests in needs to beat the market by more than 20% every year, forever, just to match the performance of an indexed fund. It turns out that this is very hard to do. A 1991 article by Sharpe drilled the point home: “To repeat: Properly measured, the average actively managed dollar must underperform the average passively managed dollar, net of costs. Empirical analyses that appear to refute this principle are guilty of improper measurement.” Now, this would be obvious if fund fees were sold as “20% of total expected return” instead of “1.5% of invested capital.” But I digress.)

Despite the analytical support of academics like Sharpe and others, Bogle and the Vanguard Group were often mocked, and even called un-American, for their tortoise-like strategy of mirroring the broader market instead of trying to beat it, all while charging investors spectacularly low fees. This strategy cut deeply against the story, heavily marketed by the big banks, that mutual fund managers added value to their clients, value that was more than offset by the fees they charged. The mutual fund industry exists to do many things, and one of those things to preserve itself. There was a lot to be lost with the growth of indexing, not only huge profits but also the very American story that it’s better to bet on the chance of winning than it is to be certain not to lose. These two reasons – marketing power and unchecked investor naiveté – are why it took so long for the facts to prevail.

And yet, despite institutional and cultural resistance, time is a powerful tool. Indexing has grown, ever so slowly, as a proportion of the market over the last four decades, reaching 20% of U.S. equity mutual fund assets in 2014. More interesting still, in the last three years, Vanguard’s assets under management have grown faster than the rest of the mutual fund industry combined. That’s right, according to a recent New York Times article, in the last three years, the entire mutual fund industry, more than 4,000 firms, took in $97 billion, and Vanguard took in $823 billion. Vanguard now manages $4.2 trillion in mutual fund assets, having quadrupled its assets under management in the last 7 years.

If this is analogous to what it will take to “mainstream” impact investing, then we have a few lessons to take away.

First, this is a long road we are walking. So let’s be prepared for a marathon, not a sprint. This means it’s time to stop, as we enter the end of our first decade, pretending that the tipping point is just around the corner.

Second, better data and facts, even for something as easily analyzed as public mutual fund returns, are hard to come by, easy to dispute, and alone they are not enough to tip the scales.

Third, economic incentives are powerfully aligned for the gatekeepers to keep things the way they are. Despite this, things can shift massively if clients speak loudly and uniformly.

Fourth, even a robust story that shouted “this is a better way to make money!” took 50 years to penetrate the prevailing wisdom. We’re twisting ourselves into knots to publish a handful of reports saying there aren’t financial trade-offs between impact investing and other approaches. It’s great to know that is possible. But data alone doesn’t tip the scales, what’s needed is a shift in mindset and a better story. And “we can make as much money as you” strikes me, in the face of the history of indexing, as weak sauce.

Finally, a request. I wasn’t able to uncover a proper analysis of the key milestones in the growth of indexing, but I’d love to find one. For anyone interested in growing the share of investment capital that takes social returns, stakeholders, the long-term view, the environment…anything beyond traditional thinking into account, I suspect that the rocky history of indexing holds more than a few clues about the pitfalls we can avoid and, hopefully, provides a map for shortening our 50-year journey just a bit. The world needs us to move faster than that.

The Third Plate and the Future of Impact Investing

Our food system is broken. While we have solved the problem of how to produce lots of calories for a low direct cost, this same food system has resulted in an obesity epidemic; it is why nearly 10% of the U.S. population has Type 2 diabetes; and, most recently, it likely is playing a role in the huge spike in colon cancer for people in their 30s and 40s.

What do we do about it?

I recently read Dan Barber’s The Third Plate. Dan is a famous chef, the co-owner of the acclaimed restaurants Blue Hill and Blue Hill at Stone Barns. The book’s title is Dan’s answer to the question, “what will the typical American dinner plate look like in 35 years?”

In response, Dan sketched three plates to show the evolution he imagines: the first plate, from the 1960s, had a large, corn-fed steak with a small side of industrial farmed vegetables; the second, from today, had a farm-to-table organic grass-fed steak with a side of organic heirloom carrots; and the third, futuristic plate, had a “steak” made of carrots garnished with a sauce made from leftover beef trimmings.

Dan’s point, with this third plate, is that the current high-end, farm-to-table, farmers’ market approach to food is a luxurious niche that doesn’t address the core issues of the food system: while the foods themselves may be natural and healthy, they are, in Dan’s words, “often ecologically demanding and expensive to grow,” and, by definition, they work at the edges of the system as a whole.

(Cue: impact investing theme music)

The Third Plate is Dan’s exploration of a better solution, a deep dive into whether the carrot-made steak really was the future of food, and what it would take to get there. The book recounts his exploration of Soil, Land, Sea and Seed – the book’s four sections – and what the future of each of these food categories might be.

Like all good narratives, this one is told through people. Each of the book’s many protagonists – whether Klaas Martens, a farmer in upstate New York, Miguel Medialdea, a Spanish biologist, Steve Jones, a seed breeder at Washington State University, or many others – are all rebels of sorts who reach the unavoidable conclusion that whether you’re growing a stalk of wheat, raising an acorn-fed pig, or cultivating the world’s most delicious fish, the only way to produce truly outstanding food is to create food that is in harmony with a broader food system.

Take Miguel Medialdea, the Spanish biologist who raises a bass so delicious that the first one Dan Barber tastes, which, unfortunately, was overcooked, is described thus:

The fish was incredible. Even overcooked and tough – even D.O.A. (“dead on arrival”), as line cooks like to say when a fillet has seen too much heat – it made my mouth water. It was so richly flavored, you’d be forgiven for comparing it to a slowly cooked shoulder of lamb or a braised beef short rib. I’d never known bass could be so delicious.

How does Miguel Medialdea’s Venta de la Palma produce such a bass? It’s a complex system of interplay between salt and fresh water, an 80,000 acre fish farm which feels like a loosely managed system in which Miguel has set up the major pieces, nudges things here and there, and then lets the system do most of the work.

I won’t attempt to describe all of the inner workings of Venta de la Palma – Dan does it better. But I was struck by a moment in Dan’s conversation with Miguel at the end of another meal, in which Dan tries to uncover the secret of what could make a bass so delicious. Was it the scale of the property, which meant no overcrowding and, therefore, almost no disease or parasites? Was it the intricate canal system, which provides a natural filtration system against pollution?

To try to make sense of it all, Dan casually asks Miguel how long it takes for one of his bass to mature.

‘Thirty months,’ Miguel muttered, seemingly to no one in particular.

‘Thirty months!’ I said. ‘It takes two and a half years to raise…a bass?’

‘Yes, that’s the average, which is more than twice the aquaculture average.’

I asked how the company could make money.

‘So far there’s profit, enough to keep us working at an optimum, not a maximum.’

This was the kind of answer Miguel, and Klaas, and Steve Jones kept on giving: that one of the fundamental constraints that had to shift in order to operate a healthy food system is a move from maximum profit to optimum profit. They propose that the only way to create the world’s best food is by creating and maintain a system in balance, and each one of them concludes that such a system is not one that is optimized for extracting every last bit of value that they, personally, can squeeze out of it.

To illustrate the point, at another juncture in this conversation, Dan is shocked by the 30,000-strong flamingo population on the farm. Since these flamingos eat 20% of the farm’s fish and fish eggs, wasn’t their presence a bad thing?

Miguel shook his head slowly, with the same calm acceptance shown in the face of losing half of his goose eggs to hawks.

“‘We’re farming extensively, not intensively,’ he said. ‘This is the ecological network. The flamingos eat the shrimp, the shrimp eat the phytoplankton. So the pinker the bellies, the better the system.’ The quality of the relationships matters more than the quantity of the catch.”

If Miguel’s job is to optimize the overall health of the system, then key indicators of success are the data, like the pinkness of flamingos’ bellies, that tell you about systemic health. Profit may result from this system, but the system is not engineered primarily to create profit.

What a fantastical notion, that profit might be a result and not the goal.

The parallels to our economic system are, I hope, obvious. When I compare the dialogue within impact investing with the conversations happening in the food system, I’m struck by how much we, in impact investing, have so far failed to have a rich, nuanced conversation about where profits fit in the new system we say we aim to create. In my experience, all conversations about profits – or returns – in impact investing quickly devolve into discussion of the financial return a given investment or strategy produces, with both sides losing when they debate the “right” level of return without a broader conversation about whether this return is a result of or the ultimate purpose of the investment.

The much deeper conversation we need to have is around whether to be a successful impact investor, or to be a successful player in an ecosystem funded by impact investments, one needs to have the willingness and the capacity to optimize for the health of the system, and not just one of its outputs (profits, or returns). Meet any of the colorful characters in Dan’s book and you come across rebellious tinkerers who bristle at the status quo at every turn, because they’ve learned, through a life’s worth of experience, that the traditional food system is broken.

Do we have a similarly clear point of view about whether the mainstream capitalist system works or is broken? Do we believe, as we watch everyone from Bain Capital to TPG to the Ford Foundation commit billions of dollars to impact investing, that we can create the kind of deep change we know the world needs if we are unwilling to confront this question head on? Are social entrepreneurs and impact investors the equivalent of food revolutionaries who see that we have no choice but to upend the whole system, or are we hangers-on to the edges of mainstream capitalism, excited to build out our small terrariums without ever questioning the bigger ecosystem?

My belief is that our breakthroughs will only come once we start saying out loud that our ultimate goal is to build a global economic system that is extensive, not intensive. And then, once we recognize that such systems can be built, to ask ourselves what it would take to move that from niche to mainstream.

My belief is that to get from here to there, we need more folks who are willing to think like Miguel. These are people who can deconstruct and reconstruct a food system (or any other system) and, in so doing, can reprioritize the factors they’ve been told to optimize. These are people who are willing to walk the long, hard, stupid road from nowhere to somewhere. These are people who won’t stop tinkering and experimenting and learning and failing and doing it all over again…until, one day, they can consistently produce an output that is better than anything that’s come before it and that enriches the health of all the players in that system.

It’s OK for us to acknowledge that we don’t yet know the right indicators of systemic health, as long as we say that we’re willing to put ourselves on the line to create them.

We start by asking: what is our equivalent of the pinkness of flamingo bellies?

Matthew Weatherley-White on How to Turn the Capital Markets Upside Down

In further proof that most important innovation involves the reassembling of existing ideas in new, surprising and powerful ways, I’d encourage you to check out this talk by Matthew Weatherley-White at SOCAP 2016.

Matthew’s big idea goes something like this:

In Austria, organ donation rates top 90%, in Germany they’re below 14%. Why? Because in one you have to opt in to donating your organs, and in one you need to opt out. And what could be more important, or more personal, than organ donation?

So maybe the way to make investing with a purpose mainstream is to stop fighting to change people’s minds and instead make it a default strategy.

Discuss.

[If you’re not seeing the embedded video below, you can click here to see it.]

If I were to describe the video in joke form, it would go something like…

Q: What do the Twilight Zone, organ donation and ESG investing have in common?

A: ???

(If you can make that into an actual joke, you win a prize.)

A confluence of impact and scale

I spent last week at the annual meeting of the Global Impact Investing Network (GIIN), and I was struck by three trends that could take our sector to the next level.

The first is around taking impact seriously. The second is how different the impact measurement challenge looks depending on where you sit. The third is the acceleration of the rate at which mainstream financial capital is entering our space.

Throughout the GIIN conference, impact — the role it plays in defining our work and how to improve the quality of our impact data — was front and center in a way that I’ve not felt before. For example, one of the first panels kicking off this year’s event was on market segmentation. While segmentation is not a new topic in impact investing, the panel was titled “Market Segmentation through an Impact Lens.” The panelists — from Skopos Impact Fund, Tideline, Athena Capital Management and Omidyar — discussed their research and client-facing efforts to make sense of impact investing from the perspective of impact objectives.

This shouldn’t be brand new, but it is. An orientation to start segmentation with an impact lens runs against the natural tendency to segment investors by asset class or sector strategy, and it’s certainly a far cry from accepting that “intentionality” (as in: my intention is to make such-and-such happen with limited accountability on the data to figure out whether or not real change is happening) is a high-enough bar to set for the sector in terms of impact.

If we could pull off organizing ourselves, as impact investors, by the change we’re trying to make in the world rather than by the investing strategies we’re using to make that happen, that would be a big step forward.

Second, we need much better impact data AND we need to help people who are drowning in too much indecipherable, low-quality data.

I had the chance to participate in two panels focused squarely on advances in impact measurement. What I learned from these panels is that better impact data isn’t enough — there’s a huge desire for simplification too.

At Acumen, our Lean Data work has focused relentlessly on going directly to the low-income customers we aim to serve so we can understand what they have to say. Our objective is to improve the quality of impact data we have by scaling up our capacity to listen to the voices these customers, so we and our investees can better serve them.

While I’m convinced that this kind of listening must to be the foundation of everything we do as a sector, it’s not enough. Listening to my fellow panelists — from Goldman Sachs, Zurich Re, Abraaj Capital and Leapfrog — I heard that big institutions with large, diverse portfolios of impact investments not only desire better impact data but they also need help simplifying and clarifying the reams of impact data they already feel they receive.

Ironically, these large institutions have too much data coming in and most of it’s not very good. Our job is both to improve the strength of the signal and also lessen the noise.

Lastly, it was impossible not to notice that more and more big-name financial players are coming to the table.

The simple fact of having an impact measurement conversation between Acumen and Leapfrog on the one hand (two organizations that are essentially growing startups, with between $100M and $1B in capital under management), and Goldman Sachs, Zurich Re and Abraaj Capital on the other means that there are innovations in impact management happening across the spectrum of impact capital. That’s hugely positive.

Then, at the end of the day, we got to hear Former Governor Deval Patrick and Deborah Winshel discuss the impact investing strategies they began implementing in the last year at Bain Capital and Blackrock. Both articulated their goals to fully integrate impact into the global practices of these uber-blue chip firms, firms that collectively represent more than $4.5 trillion in assets. While it’s early in the journey for both Bain and Blackrock, it’s clear that their actions could have a huge influence with other mainstream financial players and beyond.

As I left the conference and made my way back to New York, I was struck with the feeling that we are entering a new phase in our sector. Having passed through the teething pains of our early days and our loud, sometimes impulsive childhood, we’re ready to start growing up a bit. This means harnessing — rather than just shouting about — the increased momentum building in our space, thanks to the entrance of major new players, while also taking a much more sober and serious look at the ultimate goal of this work, which is to make a real, large-scale and lasting difference in the well-being of people and the planet.

If, in this next chapter, we can find a way to have impact investing go deeper on impact and bigger in terms of scale and reach, we will truly be in a position to take this work to the next level.

[Note: you can also follow the conversation about this post on Medium]

Raising the bar

10 years ago, if you wanted to get into the social impact/social enterprise sector, it was enough to say “I think I want to find ways to take a business approach to solving social problems. That makes so much sense!”

5 years ago, if you wanted to do this work you needed to show that you had some direct, relevant experience, a spike of some sort that allowed folks to connect the dots between things that you’ve done and the work you’re proposing to do now.

Today, the expectation is significant direct experience that matters.

If you want to work with social enterprises in the developing world, the expectation is that you’ve spent real time in the developing world doing related work – a couple of years, not a couple of months.

If you want to be a marketer for a great cause, the pool of applicants shooting for that job have been in the great cause marketing business for a while already.

If you want to invest overseas, the expectation is that you have both investing chops and a direct understanding of the markets and businesses you’d like to invest in.

The great news is, unlike 10 years ago, when you had to a make a giant leap, there are countless opportunities for smoother, more gradual transitions.

To start, it’s never been easier to form a group and take free online courses for social changemakers. Our +Acumen courses are designed for just this, and in the next month you can learn about Lean Data Approaches to Measure Social Impact, Storytelling for Change (available in English or Spanish), and Social Entrepreneurship 101.

Or maybe your path will take you to a mainstream firm that offers a rich set of pro-bono opportunities—like those offered by Bain, Ernst and Young, and PWC—or you’ll go to one of many progressive nonprofits that work with big companies—including Taproot Foundation, TechnoServe, Bankers without Borders and MovingWorlds.

Or you could work directly for a social enterprise: right now Burn, Esoko, and Seed Schools are all hiring.

And of course nearly all the top MBA programs now have social enterprise offerings, including Kellogg School of Management, Stanford Graduate School of Business, The Wharton School, Duke Fuqua School of Business, Yale School of Management and Harvard Business School.

This is what happens when a sector goes from “brand new” to “adolescent.”

Today, the bar is higher, but so are the opportunities to help you get over it.

 

(Big thanks to Duda and Ashley for helping me get this post over the line)

Lean Data: Closing the Gap Between Entrepreneurs and their Customers

This post originally appeared on Acumen Ideas, our new channel on Medium.com.  If you’re interested in the nitty-gritty of impact measurement, be one of the first to sign up for Acumen’s new Impact Matters e-newsletter that will come out monthly.  You’ll get great content at the cutting edge of impact measurement, and we’ll also make sure let you know when our full piece on Lean Data comes out in Stanford Social Innovation Review this winter.

In 2006, Sam Goldman and Ned Tozun set out to eradicate kerosene as a source of light in the developing world. As a Peace Corps volunteer in Benin, Sam witnessed the damage kerosene could do when an overturned lantern created a fire that nearly killed his neighbor’s son. They also saw what a scourge the dirty fuel was for poor, rural families without access to energy, eating up 15 percent of their spending.

Sam and Ned decided to start d.light design, a social enterprise that would solve this problem once and for all. With funding from Acumen and others, d.light set out to create a business providing low-cost solar lanterns to poor customers. Since then, the company has sold tens of millions of solar-powered lights across more than 40 countries.

So is d.light a success? By one measure, absolutely. They are seeing demand for their product and on track to reach 100 million customers by 2020. That’s nearly 10 percent of the more than 1.3 billion people globally without access to electricity. But for entrepreneurs like Sam and Ned — and all of us at Acumen with a mission to make a real dent in poverty — just reaching a large number of people isn’t good enough.

At Acumen, we’ve spent the last 15 years investing in social enterprises that provide critical goods and services to the poor. We invest in these businesses because they are hard-wired to reach large numbers of people: when a social enterprise gets its model right, it will reach more people per dollar funded than traditional aid or philanthropy.

But while it makes us proud to say we’ve helped a million people acquire a reliable solar light or 10,000 women give birth in a high-quality, low-cost hospital, we need more than just big numbers to tell us if we are actually changing people’s lives.

How can we know if we are making a real difference?

Over the last 10 years, impact investing has attracted lots of attention and dollars. Thanks to the success of d.light and other ventures like it, today there are hundreds of impact investors putting their money behind companies that aim to deliver a social and financial return.

Despite this growth, impact investors have done a terrible job of analyzing whether or not these enterprises are creating meaningful social impact.

For example, in June, the Global Impact Investing Network and Cambridge Associates published the Impact Investing Benchmark, the first comprehensive analysis of the performance of impact investors. The report does an outstanding job of analyzing the financial results of impact investing funds, but it says virtually nothing about social performance. That’s a problem.

You’d assume impact investors must be good at measuring social impact. How else could we call ourselves “impact” investors? Not surprisingly, 95 percent of impact investors say they measure impact.  But, if you scratch the surface, you’ll discover their definition of impact is mostly limited to big, flashy numbers: number of farmers using an improved kind of seed, number of kids attending school or, as in the case of d.light, number of lights sold.

This is a start, but it’s not good enough. Typical impact investors may know how many farmers a company has reached, but they don’t have a clue if these farmers are better off. They may know how many kids attend schools, but they can’t tell you if the students are from low-income communities or just transplants from the private school down the street. They may know how many households bought a new solar lantern, but they don’t understand if the children in these homes are still dying from kerosene fires.

There’s a good reason impact investors have been falling short :  the existing tools for measuring social impact are nearly useless to a social entrepreneur.

These tools, mostly inherited from large-scale, international development organizations, are cumbersome, expensive and typically take a matter of months or even years to produce any real data. For a cash-strapped, resource-constrained social entrepreneur trying to build a fledgling business in tough, emerging markets, these tools don’t make sense.

The good news is, we have an opportunity to change this. Unlike five or 10 years ago, the majority of the 2.5 billion people living in poverty now have access to a cellphone and, in another five years, virtually everyone will be reachable by phone or SMS. At Acumen, we’ve developed a new approach to impact measurement that takes advantage of this shift. Our approach is optimized for entrepreneurs building social enterprises in the developing world, and it capitalizes on today’s information revolution to gather data directly from low-income customers. Our goal is to use this infrastructure to understand our social impact and better serve the poor. We call this approach Lean Data.

Unlike traditional impact measurement, Lean Data is designed to quickly and affordably generate quality customer insights that can immediately drive entrepreneurs’ decisions.

It reframes impact measurement as customer feedback by applying Lean Startup experimentation principles to the collection and use of social impact data. While Lean Startup aims to understand product-market fit with questions like “Do you like this product?” and “Will you buy this product?,” Lean Data goes a step further by working to understand how a purchased product is — or is not — changing a customer’s life.

By asking questions via mobile phones and other existing customer touchpoints (such as a salesperson’s visit to a customer’s home or a company’s call center), Lean Data allows enterprises to get social performance data in a matter of weeks and at a fraction of the cost of traditional measurement approaches.

In the last year, Acumen has helped 12 of our companies measure their social performance by surveying more than 5,000 customers across seven countries. Each of these projects took weeks, not months, and cost thousands, not hundreds of thousands, of dollars.

Here’s how.

Lean Data leverages technology, so enterprises can communicate directly with their customers. It is now possible to get reliable, meaningful data directly from low-income customers either through calls or SMS messages. For example, we worked with Ziqitza, a healthcare company that provides low-cost emergency services in India to understand what percentage of its customers in Orissa and Punjab live below the local poverty line. Our results showed that 75 percent of customers live on less than $2.50 a day. In another case, we worked with Juhudi Kilimo, a microfinance enterprise servicing smallholder farmers in Kenya, to measure its social performance using a 10-question SMS survey. The survey showed that the loans Juhudi Kilimo provided to purchase dairy cows are helping farmers see an increase in milk yields of 60 percent.

Lean Data puts the customer first, not the investor. As an investor in social enterprises, Acumen needs impact data to manage its own performance. But we believe social enterprises should first and foremost be accountable “downward” to their customers before worrying about “upward” accountability to their funders. Social enterprises set out to solve meaningful problems for their customers, and they should only systematically collect impact data if that information helps them understand how their products or services are making a difference in their customers’ lives. The information should also be shared “upward” with funders, but that cannot be the primary reason for collecting data.

Lean Data gets underneath not just the “what” but also the “why” of product-market fit. Lean Startup principles focus on product-market fit: is there a demand for a new product in a given customer set? How satisfied are customers with the new product? Social enterprises can take this a step further, asking not just whether there is product-market fit, but why that fit exists. This is the first step towards understanding impact. When we discover why products are purchased, how well or often they are being used, and which problems they solve or fail to solve — like improved productivity, increases in household savings or fewer sick days — we empower customers to articulate what impact means to them. This kind of insight is invaluable to entrepreneurs looking to drive lifetime value, customer loyalty and social impact.

We’ve been developing Lean Data for a little more than a year and, while it is still in its early days, we see huge promise.

If we can give more entrepreneurs like Sam and Ned the right tools to understand their social impact and hear from their customers, they will, for the first time, have actionable data that can tell them, in real time, how to improve their products and create meaningful change.

The truth of this work is that the big, glossy numbers allow us to sing our own praises and raise more money, but they do little to help us improve the lives of the people we aim to serve. It’s time to dig deeper, to use technology to talk directly to our customers, so that our work can realize its full potential.