When we’re stuck with a problem we simply cannot solve, what we need is a great question.
The funny thing about a great question is that it is often hard to recognize.
It can feel slightly off topic.
Like a critique.
Or even downright irrelevant.
Like the person wasn’t fully hearing us and the core assumptions that we know to be true.
But sometimes, if we’re patient enough or stuck enough, we find ourselves sitting with a great question. It germinates in the back of our minds. We process it in the midst of doggedly re-treading the beaten path to our wrong answers.
And then, if we’re lucky, a eureka moment happens. We see something new. A door opens.
What that perfect question did was poke at the heart of a truth that wasn’t true, a strongly held assumption that was just plain wrong.
Suddenly, the impossible becomes possible.
We can’t predict when these moments will happen, but we can pay a bit more attention to off-beat questions before dismissing them.
Every day, more than 5 million new cellphones are sold. That’s more than 10 times the number of babies born each day. We are barreling towards a world where a cellphone will be in every pocket by 2020, and a smartphone in every pocket soon after that.
This revolution is making the unimaginable real— in the near future, we will have the opportunity to start a dialogue with literally every person on the planet. This new two-way conversation, where everyone participates, will pull billions of people into the mainstream by connecting them with one another.
Since starting this work in 2014, one of the most important lessons we’ve learned is that a cellphone in every pocket is just a starting point. The art of every Lean Data project is in the questions we ask. Ask the wrong questions, and you get back little of value. Ask the right ones, and you can move from data to information to actionable insights.
Great questions connect with customers and give them an opportunity to share their voice. But crafting a great question is no easy task. The slightest shifts in word choice can affect understanding; the smallest differences in intonation alter perceptions of sincerity. All of these nuances can bias the data and diminish its value.
For example, in trying to understand the usage of solar home systems in Kenya, we started with the question, “How often are you currently using (product/service)?” After testing this question over SMS, we received feedback suggesting we omit the word “often” and make the question more simple and direct. We quickly amended the question to “When do you use (product/service)?,” provided sample multiple choice replies, and received a higher level of understanding.
Getting questions right is not a new idea. Indeed, Angus Deaton’s recent Nobel Prize was largely the result of his foundational work on designing household surveys. What’s new is trying to gather rich data over a cellphone. While you can run an effective focus group with a loose guide of topics and you can cover a lot of ground in a 90-minute one-on-one interview, a typical SMS survey is limited to 10 questions and 150 characters per question. These constraints are a powerful pressure-cooker for the questions we ask. We’ve got to make every word and every question count.
So what makes a great question?
For us, a great question is one that is easily and consistently understood by customers. It’s one that makes the complex simple. And it’s one that yields insight around what matters to the customer and the social enterprise trying to serve them.
One of the biggest challenges in impact measurement and international development is understanding not just the breadth but the depth of impact. In Acumen’s case, depth is defined by the degree of change in their well-being a customer experiences from one of our investments’ products or services. For example, we know that a solar light is a better solution than a kerosene lamp, but exactly how much better and why is tricky to figure out. This isn’t an academic exercise for Acumen or our companies. Ultimately, we need to understand our customers’ needs to know where to direct our capital to drive the greatest impact, and without impact data we are simply flying blind.
Because we work across multiple sectors addressing a number of the problems of poverty, our challenge extends beyond just figuring out the quantitative impact of owning a solar light or sending a child to a low-cost private school. Our goal is to go one step further and understand the qualitative difference in value that our customers experience when comparing the various products and services available to them.
Photo by Joanne Schneider
Can we really compare the impact of a year of schooling to owning a solar home system? We’re not sure, but we think it’s worth a shot. We believe that trying to understand these comparisons from a customer’s perspective will push us to listen harder and deeper, and it will test the limits of our ability to get rich data through mobile phones.
We asked ourselves if we could create a question or a set of questions that get at this topic directly, helping our customers share what they value most and why.
While a single question to cut through the complexity of our work seemed far-fetched, we knew that similar attempts have been made before. Twelve years ago, Frederick F. Reichheld, Rob Markey and Bain & Company developed the Net Promoter Score® (NPS). According to the Harvard Business Review, the NPS “substitut[ed] a single question for the complex black box of the typical customer satisfaction survey.” Today, it’s become widely adopted by the Fortune 500 as one of the most effective ways to measure customer loyalty. Just as NPS provides companies with a method to effectively judge performance and generate qualitative customer feedback, we wanted to create a single, unifying question to compare social impact.
Photo by Joanne Schneider
We started by asking ourselves whether the NPS question — “How likely is it that you would recommend [product/service] to a friend or colleague?” [1–10 scale]” — could serve as a good proxy for how much impact a product had for our customers. We wanted to test this by asking NPS questions together with our depth of impact questions to see if products with a higher NPS also had a higher depth of impact.
We piloted this approach in Kenya and India in two surveys, and the initial results were not as promising as we had hoped.
Despite the proven success of NPS with more affluent, educated customers, the question didn’t seem to perform well with our customers who are typically poor, have limited formal education and little experience with surveys. In follow-up conversations, we heard that the 0–10 scale was hard for them to understand and the hypothetical “would recommend” language didn’t translate well.
Lean Data surveys are short and inexpensive to conduct, so it’s easy to test and refine questions. We experimented with four different versions of the question before landing on a question, inspired by NPS, that seems to perform well: “Have you ever recommended product/service to a friend?” We also played with three different answer scales and arrived at a workable solution. Instead of a 0–10 scale, customers choose between three responses: “Yes, I’ve told many friends;” “Yes I’ve told some friends;” or “No, I have not.”
Once we saw the effectiveness of this question, we wanted to go further, to learn not only whether or not customers recommended a product but also the drivers of meaningfulness of that impact. Drawing on the concept of Constituent Voice developed by Keystone Accountability, we developed a second question, asking customers to respond from “strongly agree” to “strongly disagree” to the statement: “There have been changes in my home because of (product/service).”
In the early tests we’ve run, we’ve seen correlation between reported depth of impact and the strength of agreement to this “meaningfulness” question. For example, owners of solar lights who “strongly agree” with the statement reported an 83 percent reduction in expenditure kerosene, while the customers who said “agree” only reported a 69 percent savings on kerosene. These are just preliminary results, but we’re starting to see that this question might allow us to compare across different interventions, so that customers can tell us what they value the most and why.
Photo courtesy of Joanne Schneider
While we’re still fine-tuning both of these questions, the progress we’ve made is exciting. Low-income customers are enthusiastic to engage in dialogue, and we are seeing that it’s possible — if you work at it — to develop new questions that capture rich, meaningful data about the wants and preferences of this emerging set of customers. At the end of one of our surveys, one happy customer expressed her satisfaction with the service she received at a health clinic and then added, “I really enjoyed being interviewed.” Clearly, we’re on to something.
While Lean Data is, today, being used mostly by startup social enterprises, our work in learning to ask the right questions over mobile phones is universal. The low-income customer of today is the low middle-income customer of tomorrow. Hundreds of millions of people in the developing world are poised to improve their well-being, but this depends on how well we, as a society, listen to them and adjust our efforts to meet their needs.
So much of this rests on the simple act of caring enough to ask the right questions.
One way to end a sales meeting is with the big push. You’ve done the work, you’ve made the pitch, you go to close the sale.
Before that moment, and in the meetings preceding that meeting, you’re having a different conversation.
And it IS supposed to be a conversation. That means questions are very often the answer. One of the biggest mistake people make in trying to make a sale is the rush to get out your “whole story:” your job is to make a pitch, and you’d better say everything you need to say the clock runs out on your meeting.
Of course the problem with that is that you can’t solve someone’s problem if you never bother to find out in the first place what their problem is.
The other day I was stopped cold by a great question I’ve never asked so directly (but wished I had):
“What factors are most important for you in making this decision?”
So simple, but I’d never actually paused to ask that clear, direct, transparent, non-threatening, and quite objective question.
I wouldn’t do it in every situation – this question can, if not asked in the right way, put your prospect in a “head” rather than “gut” or “heart” space in terms of her decision-making, which might not always be the right thing to do.
But if you’re in a complex, relationship-based, multi-faceted decision-making situation, asking directly how the decision is going to be made is probably going to help most of the time.
A close friend and loyal reader of this blog has asked me a question enough times that I thought I’d share it and take a stab at an answer:
“A number of your posts have questions without any answers, and you sometimes pose these to your readers as if THEY should have the answers.”
Implicit in the (paraphrased) question:
Your job, Sasha, is to give or find answers for your readers
If you don’t know the answers, how do you expect that your readers will?
Fair enough.
So here’s how I think about it. When I started blogging a year and a half ago, I had no idea what I was getting in to or how big a step I was taking. What does it take consistently to develop observations/thoughts/insights/questions and making them sharp enough that they’re worth sharing? This is the gift from readers to the blogs they follow, because without readers what you’d have is a journal (something I’m sure I would have abandoned long ago). Without the audience, the thoughts won’t get completed; the ideas would lie there, undeveloped and fallow. This is one of the many reasons I’m thankful to all of you for reading and for spreading the word.
Thank you.
The big unanswered question before you start blogging is: what’s “good enough” for a blog post? At the outset, you can share all the totally-unique-this-one’s-really-important thoughts you’ve been storing up for a while. But then you run out of those. And, for me and for this blog (which isn’t a “I read this blog/article, and here’s my take on it” kind of blog) I know that if I thought each post had to contain a world-changing insight, I’d never post anything.
For me, blogging is the discipline of continuing the conversation I’ve begun with my readers and fellow bloggers, constrained by the time I’ve allotted to blogging given my already too-full plate. So when it’s time to post I am where I am: sometimes I have an insight, sometimes I have an observation, sometimes I have a reaction, and sometimes what I have is a question I think is worth asking.
And here’s the secret: when I ask the question and I don’t share the answer, it’s because I don’t KNOW the answer. But forming the question and honing it into something worth posting requires refinement, it requires getting to the heart of something that’s I think is worth exploring.
And my hope is that you agree that figuring out which questions are worth asking is as important – maybe more important – as my take on answers. If I can help you discover a question worth asking, you’re confronted (and I’m confronted) with thorny issues you (and I) would rather run away from. By sifting through and distilling a good question, I hope to offer up something worth answering, a direction for focusing your (and my) energies.
So on the days I don’t have answers, success is a tough question that readers decide is worth sitting with, worth grappling with, worth sharing with a friend, and worth resolving.
(And when you come up with a great answer, by all means, let us know.)
