Lesson 25: Measure Customer Impact With Two Direct Conversations
Metrics are honest about direction and dishonest about cause.
A number moves and you construct a story for why. Usually the story flatters whatever you shipped last. I’ve done this more times than I’d like to write down – shipped a change, watched a number improve, told everyone the change caused it, and only found out months later that something completely unrelated was responsible.
The cure is embarrassingly cheap. Two phone calls a week.
The Rule: Aggregates Hide the Thing You Need to Know
Averages are built to hide outliers, and the outliers are where your problem lives. A dashboard showing neutral satisfaction can be one group of delighted customers and one group quietly deciding to leave, averaging into a flat line that tells you nothing.
Two conversations won’t give you statistical significance. That’s not what they’re for. They’re diagnostic, not representative – the same way a doctor listens to your chest instead of only reading your chart.
A Real Example: The Emails Nobody Received
A team I worked with had done everything right on paper. They’d mapped the onboarding flow, cut three unnecessary steps, and the completion metrics ticked up. Churn didn’t move.
Every meeting produced a new theory. Pricing. Competitors. Market conditions. Somebody suggested the product simply wasn’t sticky enough, which is the kind of sentence that ends useful conversation.
The flow owner started making two calls a week. One customer who was clearly happy, one who looked like they were drifting.
It took three weeks. A drifting customer mentioned, almost in passing, that she’d never received the welcome sequence. She’d assumed that was normal and worked it out herself. The team checked: for certain corporate email domains, the welcome emails were failing authentication and landing in spam. Silently. The send metrics showed them as delivered.
The fix was a DNS record and an afternoon. Churn started moving the following month, and the qualitative comments changed in a way the team could feel before the dashboard confirmed it.
No dashboard would ever have surfaced that. The system genuinely believed it had sent those emails.
How To Do This Yourself
The flow owner – not a researcher, not a delegate – makes two calls a week:
- One happy customer. Ask what specifically got better and when they noticed. Not “are you satisfied.” Ask what changed.
- One at-risk customer. Ask what broke, and then ask what they expected instead. The second question is where the real answer usually is.
- Keep them to ten or fifteen minutes. This is a habit, not a project. If it takes an hour nobody will do it in week four.
- Log verbatim quotes, not summaries. Your paraphrase will smuggle in your assumptions. Their words won’t.
- Two follow-ups every time: a thank-you note, and one action item somebody on the team owns by name.
Read the quotes out at your weekly review. Nothing changes a room’s opinion faster than a customer’s actual words.
The Honest Lesson
The reason this doesn’t happen isn’t time. Two fifteen-minute calls is nothing. It’s that talking to an unhappy customer is unpleasant, and we’re all very good at finding urgent work when something unpleasant is scheduled.
I put these calls off for years by telling myself I’d get better data from a survey. I wouldn’t have. Surveys tell you what people will admit in writing to a company. Conversations tell you what they actually experienced.
One warning. Two conversations are a compass, not a map. I’ve seen a leader take one vivid customer complaint and redirect an entire quarter around it. The discipline is to let conversations generate hypotheses and let the signal confirm them. Conversation tells you where to look. The number tells you whether you were right.
Book two calls this week. Put them in the calendar now, before something urgent arrives to save you from having to make them.
Next time, I’ll dig into “Share a One-Page ‘What Changed / Why It Matters’ With Execs and Frontline” – see you there.