NPS Open-Text Analysis With AI: 6 Themes to Extract Weekly

After watching 30+ SMB founders run NPS surveys, my conclusion is brutal: most teams celebrate the score, paste 4 nice quotes in a board deck, and ignore the other 200 comments where the actual customer truth lives. The score is the receipt. The open text is the meeting.

Why does the NPS score lie?

Because it moves in response to sample composition, timing, and recency bias more than to product reality. A team can ship nothing for a quarter and watch the score climb because the unhappy cohort churned. A team can ship the best release of the year and watch the score drop because a billing bug went out the same week.

Definition: NPS open text — the free-form comment field attached to the "why did you give this score?" follow-up; typically ignored except for cherry-picked quotes.

The score answers "how do they feel today?" The open text answers "what would they tell a friend about us?" — and that's the question that actually predicts the next 6 months.

What's the cost of ignoring the comments?

In a typical 30-500-employee SMB, an NPS round produces 100-400 open-text responses. A human reading them properly takes 6-10 hours and is usually a contractor who summarizes them once a quarter. By the time the report lands, the customer who wrote the most useful comment has either churned or expanded — and the team missed the window to act.

This is exactly the gap AI closes. Not by writing pretty summaries, but by tagging every comment to a structured taxonomy weekly, so the right team sees the right signal within days.

The 6-theme taxonomy

Six buckets. Every comment gets 0, 1, or 2 tags. Sticking to 6 is the discipline — once you have 14 themes, nobody owns any of them.

1. Product feedback

Anything about features, UX, performance, reliability. Sub-tags: missing feature, bug, slow, confusing. Owner: Head of Product. Action cadence: feed into roadmap review every 2 weeks.

2. Pricing & packaging

Anything about cost, value-for-money, tier confusion, hidden fees, "got expensive." Owner: Founder or Commercial lead. Action cadence: monthly pricing review.

3. Support experience

Anything about response time, agent quality, self-service docs, escalation friction. Owner: Head of CS or Head of Support. Action cadence: weekly support standup.

4. Expansion signal

Anything that hints at "we'd buy more if…" — adjacent use cases, new team interested, integration request that unlocks a department. Owner: CSM assigned to the account. Action cadence: within-week outreach.

5. Churn risk signal

Anything that mentions "considering alternatives," "not sure we're renewing," "had to escalate to leadership." Owner: Head of CS. Action cadence: same-day flag.

6. Competitor mention

Any named competitor or category comparison ("vs Intercom," "switching from Zendesk"). Owner: Founder or marketing lead. Action cadence: monthly competitive review.

Definition: Taxonomy discipline — the rule that themes can be added only by removing one first; expanding the list past 6-8 turns weekly review into a sorting exercise nobody completes.

That's the whole taxonomy. Three years of B2B SMB NPS comments and you'll find ~95% fit cleanly. The 5% that don't are usually one-off complaints or compliments that belong in a "noise" bucket.

The weekly AI workflow

Sunday night or Monday morning, depending on when your survey ships. Four steps:

1. Pull the week's open-text responses from the survey tool into a CSV. Columns: respondent_id, account_id, score, comment_text, segment, ARR.

2. Run the tagging prompt — an LLM reads each comment, returns the 0-2 most fitting tags from the 6 themes plus a 1-line summary of the actionable point. For SMB volumes this finishes in minutes.

3. Aggregate into a one-page digest — count by theme, surface the top 3 verbatims per theme, flag any churn-risk or expansion signal as separate sections at the top.

4. Route to owners — CS gets the churn-risk and expansion sections, Product gets the feature/bug rollup, Founder gets the pricing and competitor view. One Slack thread per owner with the relevant slice; no group dump.

The whole thing is under 30 minutes of human time once the prompts are written. The first version costs you a Saturday to set up; after that it runs itself.

Copy/paste tagging prompt

This is the prompt that does step 2. Paste it into your LLM of choice; feed it 20-50 comments at a time.

You are tagging customer NPS comments using a fixed 6-theme taxonomy.

For each comment, return:
- 0-2 tags from: PRODUCT, PRICING, SUPPORT, EXPANSION, CHURN_RISK, COMPETITOR
- 1-sentence summary of the actionable point (max 20 words)
- Severity: HIGH if comment explicitly mentions cancellation, escalation,
  or "switching"; MEDIUM if dissatisfied but engaged; LOW otherwise.

Rules:
- Use NO tag if the comment is generic ("great product") or off-topic.
- CHURN_RISK and COMPETITOR are not mutually exclusive — many overlap.
- EXPANSION requires a concrete signal (new team, new use case, integration
  request) — not just "we love it."

Output JSON, one row per input:
{ "id": "...", "tags": ["..."], "summary": "...", "severity": "..." }

Comments:
[paste 20-50 numbered comments here]

The prompt is deliberately conservative on tagging. False positives create noise; missed comments are recoverable next week.

Tool tip (Course for Business): The reason most SMBs don't run this workflow is not the prompt — it's that nobody on the team has written one before, and the gap between "I should learn this" and "I shipped it" is months. Our 6-week program closes that gap in five days using the Augment, don't replace principle: every employee, including CS leads, builds their own version of this weekly digest as their week-1 automation. We've seen Head-of-CS roles ship a working NPS tagger by day three. Walk through the program at https://course.aiadvisoryboard.me/business.

Team scan (what AI champions report after week 1)

• One AI champion per ~17 staff covers a CS+Support+Product cluster; first NPS workflow lands in week 1
• Adoption: CS-lead, support-lead, and product analyst all using the same tagged dataset by day 5
• Use case: weekly NPS digest replaces quarterly consultant report; cost falls from €4-8k/year to near-zero
• Saved time: CS lead reclaims 4-6 hours/month previously spent skimming comments manually
• Quality lift: churn-risk flags surfaced within days rather than at quarterly review
• New behavior: Product team now reads CS-tagged comments before sprint planning
• Common stumble: teams over-tag in week 1 (4+ tags per comment) — week-2 calibration fixes it
• Champion role: weekly 15-min sync to refine the prompt as edge cases appear
• Failure mode caught: support team initially tagged every "slow response" as PRODUCT, not SUPPORT
• Next week unlock: same prompt structure adapts to G2/Trustpilot review streams

Micro-case (what changes after 7-14 days)

A 200-person B2B SaaS ran NPS quarterly with a contractor synthesizing 250 comments into a 30-slide deck two months after the survey closed. Net useful actions per cycle: about three. They moved to a weekly AI tagging workflow with the 6-theme taxonomy. Week 1: prompt set up, first digest shipped Monday morning, 4 churn-risk flags surfaced — 2 of which the CS team had not tracked. Week 3: CSM reached out to one of the silent flags and uncovered a pricing objection that turned into a renegotiated 18-month renewal instead of a churn. By week 8: the product team had reordered the sprint based on a clustering of "confusing onboarding" comments, support had revised three macros, and the founder had two competitor signals he hadn't seen before. Annual cost of the workflow: roughly €600 in LLM calls. Cost of the previous quarterly report: about €6k plus the opportunity cost of acting two months late.

Note on this case: This example is illustrative — based on typical patterns we observe with companies of 30-500 employees, not a single named client. Specific numbers are rounded approximations of common ranges, not guarantees.

Tool tip (Course for Business): The shift from quarterly to weekly NPS analysis is a behavior change, not a tool change — and behavior changes need the Shoulder-to-Shoulder approach. Week 4 of our 6-week program is the workflow-installation session: the CS lead and a champion sit together, run the tagging prompt on real survey data, route the digest to the actual owners, and watch the first action happen by Friday. Most teams that try to install this alone stall at "the prompt works but nobody reads the digest." Book a 30-min mapping call at https://course.aiadvisoryboard.me/business.

FAQ

Won't the AI miss nuance a human reader would catch? For tagging, no — LLMs are very good at this category of work at 2026 quality levels. For the 3-5 highest-severity comments per week, a human still reads the full text. The AI handles volume; the human handles judgment on the few that matter.

Should we tag positive comments too? Yes — positive comments are where the EXPANSION signal lives, and they're also useful sales testimonials if the customer agrees. Don't treat the open text as a complaint box.

What if our NPS volume is too low — under 30 comments per round? Then run it monthly instead of weekly, and rely on the human reader for now. The 6-theme taxonomy still works; you just don't need the AI tagging layer yet.

How is this different from sentiment analysis? Sentiment gives you positive/negative — that's already in the score. What you need is the what about — the structured topic. Sentiment without topic tagging is the same problem as the NPS score itself: directional but unactionable.

Should the same prompt run on support tickets and app-store reviews? The taxonomy carries over with one substitution: replace EXPANSION with INTENT-TO-CANCEL for support tickets, and add an ONBOARDING bucket for early-tenure tickets. Same structure, slightly different ownership routing.

Conclusion

The NPS score is the wrapper; the open-text answers are the substance. A 6-theme taxonomy and a weekly AI tagging workflow turn surveys from a quarterly ritual into a CS+Product+Founder feedback loop that runs every Monday.

Pick the 6 themes. Write the tagging prompt this week. Ship your first weekly digest next Monday.

If you want every employee to ship their first AI automation in five days — including a working NPS tagger by day three — book a 30-min call and we'll map your team's first week at https://course.aiadvisoryboard.me/business.