I Ignored Our Dashboard and Bet on a Hunch. Then I Did the Opposite. Here's When Each Worked.

At Sonic Linker, our analytics said users loved our AI summarization feature. Usage was through the roof. 80% of active users hit it at least once a day. Classic engagement metric, right?

But retention was tanking. Week 2 drop-off was brutal. Week 4 was worse.

I spent two weeks staring at funnels, cohort charts, session recordings. Nothing made sense. High engagement, terrible retention. Then I just called 10 users and watched them work.

Turns out, people were using the summarization feature because our core linking workflow was confusing as hell. They'd summarize documents to figure out what to link them to. The feature wasn't beloved. It was a band-aid for bad UX.

The data told me one story. My gut, after talking to users, told me another. I trusted the gut. We rewrote the linking flow. Engagement on summarization dropped 40%. Retention jumped 28%.

When data is smarter than you

At Finvestfx, I had the opposite problem. We were building a treasury management dashboard for forex dealers. I thought the most important thing was real-time P&L tracking. Made sense, right? Traders love live numbers.

I pitched it to our enterprise clients. Got polite nods. One CFO even said it was "interesting." That's VC-speak for "no."

But our usage data told a different story. The feature dealers actually opened every single day? The compliance audit trail. Boring, unglamorous, zero dopamine. But it saved them 3 hours a week and kept regulators happy.

I wanted to build the flashy thing. The data said build the boring thing. We built the boring thing. Renewal rate on those accounts hit 95%. My gut would have chased the wrong problem.

Here's the pattern I've noticed: data is great at telling you what's happening. Your gut is better at telling you why.

If you see a metric move and you don't know why, the data's not lying. You just don't understand the user yet. Go talk to them. If you think you know what users need but the data says they're doing something else, you're probably projecting your own assumptions onto them.

The real question isn't which to trust. It's which question you're answering.

When I'm trying to understand what users are doing, I trust data. Full stop. How many people clicked this? What's the conversion rate? Which cohort retained better? Data doesn't have an agenda.

When I'm trying to understand why they're doing it or what to build next, I trust my gut, but only after I've talked to enough users that my gut is actually informed.

At Sonic Linker, we had to decide whether to build integrations with Notion and Google Drive or double down on our core AI linking engine. The data said integrations would spike signups. My gut, after 15 user interviews, said the AI engine was still too unreliable and people would churn anyway.

We ignored the growth data and fixed the engine. Took us 6 weeks. Signups stayed flat. But Week 4 retention went from 22% to 41%. That's the bet data alone wouldn't have told me to make.

The worst thing you can do is pretend you're being data-driven when you're not

I've seen PMs cherry-pick metrics to justify what they already wanted to build. I've done it myself. You tell yourself you're following the data, but really you're just finding the one chart that agrees with your opinion.

That's worse than just admitting you're going with your gut. At least then you're honest about the risk.

At NJ Group, I was coaching insurance advisors on a new digital onboarding tool. Adoption was low. The data said they weren't using the mobile app. My first instinct was to improve the app.

But when I sat with advisors in the field, I realized most of them were meeting clients at home, on sofas, where pulling out a phone felt weirdly impersonal. They wanted a tablet version they could hand over. The data said "mobile problem." Reality said "form factor problem."

I pushed for a tablet build. Took 3 weeks. Adoption doubled. The data would've had me optimizing the wrong surface.

Here's what I actually do now

I look at the data first. Always. If something's broken or working, the metrics will show it before users tell you.

But if I don't understand why the metric moved, I don't make a decision yet. I talk to users until the why becomes obvious. Then I decide.

And if I have a strong gut feeling that contradicts the data, I don't ignore it. I treat it as a hypothesis. I dig deeper. Maybe I'm wrong. Maybe the data's incomplete. Either way, the tension is useful.

The worst product decisions I've made came from trusting data I didn't understand or ignoring data because I liked my idea better. The best ones came from letting data show me the problem and my gut (backed by user conversations) show me the solution.