In case you missed it (thanks @beckystraw for sharing it), Anya Kamenetz at FastCompany recently wrote a piece titled “Why Social Investing is a Crock.”
It’s a pity that in the effort to grab attention, Ms. Kamenetz crossed the line from an attention-grabbing headline to misrepresenting the position of her story’s protagonist, Dean Karlan, author of More than Good Intentions. Dean’s not saying it’s a crock, he’s saying we haven’t done rigorous analysis yet. For example, Dean says in his book:
The social entrepreneurship world is in a weird spot, to be honest with you. It’s a world full of rhetoric about impact investing, yet I have very rarely seen an investor actually take that seriously. When you look at the actual analysis it lacks rigor.
Let’s be very clear: saying the analysis lacks rigor and saying the sector is “a crock” are wildly different things. Indeed the analysis mostly does lack the kind of rigor Dean is looking for, specifically because, as Dean rightly points out, “If you want to put a million dollars into a business, are you going to put 33% of that into an impact study?” The answer is often “no,” except when a funder wants to put up grant money to fund that study (which some are, and more should.)
More interesting, to me, than laying in to the lame title of this article is to think a bit about randomized control trials (RCTs) – the gold standard in the development space – and their application in the social investment space. I don’t know the answer here, but I’d like to throw out a question to kick off a conversation:
What would Dean Karlan say about the “evidence” around the cellphone?
Meaning, cellphones are an “intervention” (a product, really) that clearly provide benefit to customers. How much benefit is open to debate, of course, but the question is interesting because the cellphone is the first really new product to achieve mass penetration for low-income consumers (there are now 4.5 billion cellphones globally).
It feels to me that no one would bother asking in a serious way to do an RCT on cellphones, because it’s no longer anyone’s business to ask that question – because the market has taken over. The product is appealing enough, consumers have spoken, we can all agree that the ability to communicate via a cellphone creates positive externalities – some economic, some social – and since we’re not using up grant money to pay for it, it stops being the purview of development economists to fret about this.
I bring up this example because, put in the simplest terms, what social impact investors (at least the ones who really care about social impact) are trying to do is:
- Identify products and services that make a material positive impact on the lives of poor people
- Create business models that make the cost of providing each incremental product go down to zero…and beyond (meaning, you make a profit).
Without getting into all of the real complexities of profit maximization versus a “sustainable” rate of return at the firm level; of what to do about all the people who, no matter how cheap the product, won’t be able to pay the full cost; about the role of subsidy; etc; I’d like to posit that there is something fundamentally different about this construct and the construct of analyzing a purely philanthropic intervention.
No matter what scale a pure philanthropic intervention reaches, the total marginal cost of delivering the nth “thing” (any intervention) is always positive, so you’re in the business of figuring out how the impact relates to that cost and how the impact relates to other similar interventions. Not so if you find the “next cellphone” – except it’s not a cellphone, it’s safe drinking water or a bio-mass powered light on a mini-grid or a safe and affordable place for a mother to give birth.
What if we get to the point when it becomes pointless to ask if an intervention “works” because, like the cellphone, it will be ubiquitous, so the question will feel purely [sic] academic?
What do y’all think? Am I off my rocker here, or is it possible that creating a scalable businesses that serves the poor is fundamentally different than traditional charity?
For any given RCT, there is always the possibility that the evaluator/researcher asked the wrong questions or simply incompetently conducted the field research. I raise the possibility because it makes me terribly nervous when funders outsource their judgment and accumulated program knowledge. Even the best scientists make mistakes, test and retest experiments and research, etc.—all expensive, time-consuming processes. I note that on the basis of damn few RCTs, some academic centers have been quick to draw broad generalizations about microfinance. Will we end up creating an FDA for economic development? There are just no easy answers, as we all know.
Another analogy that may be useful is experimental new drugs. It is very difficult to find randomized studies in late stage terminal diseases. Who would want to be the parent whose child was randomly selected to the placebo group. Social businesses are addressing very serious issues and as the people motivated to start and fund them are trying to reduce suffering, objective detachment can be tough.
To me the challenge comes from the expectations. I don’t know of many people who have thought that microfinance is a magic bullet, or for that matter than anything is a magic bullet. When you drop your enthusiam into the binary pachinko machine that is the current media, you seem to get perfect or disaster as the two message outcomes. Hence “needs rigor” becomes crock, and “beats the placebo by 15%” becomes “miracle drug”. Actually there is a third outcome, a total lack of coverage.
Keep filling the gap Sascha!
This is a fantastic article and no, you are not off your rocker.
The good or at least appeal of a for-profit “intervention” is easy to measure because it’s measured in profits, which are quantifiable in a simple way.
Social good is extremely hard to measure, because it deals with changes in human lives, so we have to do RCTs to judge it. We have to triangulate to reduce bias. We have to deal with complex, qualitative changes over very long periods of time.
You might think that uptake of an intervention would serve as an indicator of social good done, but because supply and demand are in such disequilibrium in the developing world, it doesn’t. Increasing supply increases demand by increasing population through migration or organic population growth. Plus, the needy will take anything for free, in the hope of receiving more and even if it is not useful to them, of turning it into something useful. They cannot usually take the risk of declining the goods in the hope that the intervening agency or company will take their denial seriously and go rework the product.
So no, I do not think we will ever get to the point where the social good of an intervention is immediately obvious like it is in the for-profit sector. But that is why Harvard has a whole school dedicated to training leaders of the non-profit and public sectors.
Economics is like sociology, but you only deal with one teeny, tiny part of the human experience. In the private, for-profit sector, you use economics because you deal with one teeny, tiny part of the human experience (money).
Us? We deal with everything else.
Sasha- you may be off your rocker, but not on this.
Not sure if you have read Omidyar’s in new HBR, but I think he puts it well on the need for charity, philanthropy and investment: http://hbr.org/2011/09/ebays-founder-on-innovating-the-business-model-of-social-change/ar/1
There are a lot of challenges in the world and a growing tool chest of approaches to tackle them. RCT’s have a place, but as I have written* (another time Fast Company wrote about RCTs and JPAL), they are one tool (and I hope not a weapon) in figuring out how to effectively work on these challenges.
Fast Company has been great in the past at highlighting social entrepreneurs (and social capitalists) to a broader audience. So I think the original headline is probably not the view of Fast Company, either (or are you fighting provocation with provocation?). They are RCT fans, though, no doubt. Wonder when they will start using RCTs to measure Dropbox, facebook or some of their other fave companies.
Keep up the passionate posts.
Paul
*http://bopreneur.blogspot.com/2009/11/is-it-right-to-have-poor-pay.html
Paul, thanks for this. I haven’t read Pierre’s HBR piece yet but it’s sitting in my Inbox with a “must read” subject line!
My only small point is that I’m not sure I’m comfortable with “I think the original headline is probably not the view of Fast Company” – that is, in the end their editorial approves the headlines, right? (though of course I agree as a whole they’ve been highly supportive of our sector).
Taking your comment a step further (re: dropbox; facebook), I’d love our sector to develop a vocabulary around how RCT’s do/do not apply to our (Acumen and our ilk) work. Absent that, RCT = “gold standard” in common parlance, and any claims to the contrary look like the desperate pleas of someone afraid to subject their work to “rigorous” analysis. Thoughts?
Thanks for the link to your article. It’s great!
Where there are tools out there to measure to social return on investment (SROI) as ways of measuring impact ?