AI Let Me Interview 200 Users in a Week. Then I Realized I Was Still Asking the Wrong Questions.

I remember the exact moment I realized AI had changed product discovery for me. It was 2 AM, and I was reading through synthesized interview notes from 40 customer conversations that happened over three days. Normally, scheduling those calls alone would have taken two weeks.

But here's the thing: I was reading them, nodding along, and still had no idea what to build next.

That's when it hit me. AI didn't solve my discovery problem. It just exposed how shallow my questions were in the first place.

The Speed Trap (And Why I Fell Into It)

When we were building Sonic Linker, I got access to tools that could transcribe calls, pull out themes, even draft follow-up questions based on what users said. I thought I'd struck gold. Instead of talking to 5 users a week, I could talk to 20. Instead of spending hours in Notion tagging quotes, I could get a summary in 10 minutes.

So I did exactly that. I scheduled more calls. I ran more surveys. I had AI analyze support tickets and pull out feature requests. Within a month, I had this massive corpus of "user insights."

And our product still wasn't connecting.

The issue wasn't volume. It was that I was asking surface-level questions at scale. "What features do you want?" and "How do you currently solve this problem?" are fine questions, but they don't get you to the why behind the behavior. AI made me really good at collecting answers. It didn't make me better at asking the questions that mattered.

At Finvestfx, I'd spent months talking to treasury teams about their workflows. Those conversations were slow, manual, painful to synthesize. But because I couldn't do 50 of them, I had to go deep on each one. I'd ask follow-ups. I'd sit in on their actual processes. I'd watch them get frustrated with their tools and ask what was happening in that exact moment.

With AI speeding things up, I lost that. I got breadth, but I traded away depth without realizing it.

What Actually Changed (Once I Stopped Chasing Volume)

I started using AI differently. Instead of summarizing 40 calls into themes, I'd feed it one conversation and ask it to find the moments where the user hesitated, contradicted themselves, or got emotional. Those were the moments that mattered.

For example, we were building a feature for automated link generation at Sonic Linker. Users kept saying they wanted "more customization options." AI dutifully tagged that as a top request across 30 interviews. But when I went back and listened to the actual moments people said it, half of them were talking about branding and half were talking about URL parameters. Totally different problems, same words.

AI helped me catch that, but only because I asked it to look for inconsistencies, not consensus.

I also started using it to prep better questions. I'd upload past research, competitive teardowns, and usage data, then ask it to generate hypotheses about why users were churning or not adopting a feature. Those hypotheses became my interview guide. Instead of "Do you like this feature?", I'd ask "You used this twice in the first week, then stopped. What happened?"

Way more useful. Way harder to get at scale without AI helping me connect the dots across data sources.

The Real Shift Isn't Speed, It's Connecting the Dots Faster

Here's what I think is actually changing. Discovery used to be this linear thing: talk to users, synthesize notes, form a hypothesis, validate it, build. That cycle took weeks. AI compresses that cycle, but only if you use it to iterate on your understanding, not just collect more data.

At Sonic Linker, we'd ship a feature, watch the usage data, have AI pull out the users who dropped off, and within 24 hours, I'd have interview invites going out to those specific people with questions about what broke in their workflow. That feedback loop used to take a month. Now it's a week.

But the questions still had to be good. AI didn't write those for me. It just made it faster to test if I was asking the right ones.

I also found AI useful for sanity-checking my own biases. I'd draft a PRD, feed it my research, and ask it to tell me what I was missing or what assumptions I was making. Half the time, it'd point out that I was overindexing on power users or ignoring a segment that showed up in the data but not in my mental model. That's valuable.

What I'd Tell a PM Starting Discovery Today

Don't use AI to talk to more users. Use it to ask better questions of the users you already have.

Don't let it summarize everything into themes. Make it show you where users contradicted themselves, where their words didn't match their behavior, where they got stuck.

And for the love of god, don't trust a feature request just because AI tagged it 50 times across interviews. Go back and listen to why people said it. Half the time, they're describing the same symptom but want totally different cures.

AI changed discovery for me, but not because it made me faster. It made me realize that speed without depth just means I build the wrong thing in half the time. Once I figured that out, it became the best research partner I've ever had.