Every PM I know says they want to be more customer-centric. Most of them are also sitting on thousands of unread support tickets.
This is the gap I decided to close.
At Novanta, I was trying to build a defensible roadmap for a new sensor platform — medical and industrial customers, long sales cycles, very different needs. We had NPS surveys (response rate: 11%). We had sales call notes (inconsistently logged). We had quarterly business reviews with our top five accounts. And we had 4,200 support tickets from the previous 18 months, sitting in Zendesk, categorized only by "open" or "closed."
I processed all 4,200 in a weekend. Here's how.
Why support tickets are the best data source you're ignoring
NPS gives you a number. User interviews give you stories from the people willing to talk to you (selection bias: they're usually your fans or your angriest detractors). Sales calls give you what customers say when they're trying to buy something.
Support tickets give you what customers do when the product fails them. That's a different thing entirely. It's unfiltered, unsolicited, and — because someone was frustrated enough to open a ticket — it's emotionally honest.
The problem is volume. Nobody can read 4,200 tickets. Or 400. Or even 40 and extract reliable patterns. Humans are bad at synthesis at scale. We remember the last thing we read. We overweight the dramatic complaint. We pattern-match to what we already believe.
AI doesn't have those problems.
The pipeline I built
I used Claude (Anthropic's API) to process tickets in batches of 50. Here's the exact prompt structure I used:
Pass 1 — Categorization:
Each ticket got tagged with: primary pain area (from a predefined taxonomy I built based on our product areas), user type (inferred from language and context), severity (blocking / degraded / minor), and whether it mentioned a competitor.
Pass 2 — Theme extraction:
After categorization, I ran a second pass on all tickets in each category, asking the model to identify recurring themes, exact language patterns customers used, and any feature requests embedded in complaints.
Pass 3 — Opportunity ranking:
I gave the model the aggregated themes and asked it to rank them by two dimensions: frequency (how often does this come up) and severity (when it comes up, how bad is it for the user). The output was a 2x2 that our team could immediately act on.
The whole pipeline cost under $40 in API credits.
What it found that NPS completely missed
Our NPS surveys consistently flagged "product reliability" as the top concern. That's what customers said when asked directly.
The ticket analysis told a different story. The highest-frequency, highest-severity theme wasn't reliability — it was documentation. Specifically: customers couldn't find the integration documentation for a specific hardware interface, and when they couldn't find it, they tried workarounds that caused the reliability issues they then complained about in NPS surveys.
The root cause was a content problem, not an engineering problem. We were one pull request away from fixing the thing that was dragging our NPS down.
That's the kind of insight that changes where your team spends the next sprint.
The AI also surfaced:
- →Competitor mentions we'd never tracked systematically (two competitors came up in 340 tickets — our sales team knew about one of them)
- →Feature requests buried inside complaint language that our ticketing system had never captured as requests (tickets categorized as "bugs" that were actually users asking the product to work differently)
- →User segment patterns — enterprise users and SMB users were hitting completely different problems, but our support queue had been treating them as one group
Why sentiment analysis alone is useless
A lot of teams stop at sentiment. "Our ticket sentiment is 62% negative this month." Okay — and? Sentiment is a metric without a diagnosis. It tells you the temperature of the patient. It doesn't tell you what's wrong.
The move is to go from sentiment to *theme*, and from theme to *action*. Sentiment gets you a dashboard. Theme extraction gets you a roadmap input.
The other mistake is running this once and treating it as a research project. I set this up to run on a weekly cadence now. New tickets from the previous week get processed every Monday. By Tuesday morning, our product and CS teams have a digest: what broke, who it affected, what language customers used, and whether anything new showed up.
It's not replacing discovery. It's making discovery continuous.
What I'd do differently
The taxonomy matters enormously. My first pass used categories that were too broad ("performance," "UI," "integrations") and the themes that came back were too vague to act on. When I rebuilt the taxonomy around specific product areas and user jobs-to-be-done, the output became immediately actionable.
Also: don't skip the human review pass. AI is excellent at pattern recognition at scale, but it will confidently misclassify edge cases. I review the top 20 tickets in each category every week to make sure the model's categorization is still accurate. So far it's caught two systematic miscategorizations that would have skewed our priorities.
The goal isn't to remove humans from discovery. It's to make sure humans are spending their judgment on synthesis and decision-making — not on reading ticket number 3,847.