Chatbot Survey Template
What you'll get: a 2-minute post-chat survey that tells you whether the bot resolved the issue and why it did or did not (usability, usefulness, trust, and handoff quality). Customize it for your bot by swapping in the support, sales, or internal IT/HR module without changing your core scoring. Set it up now: trigger at end-of-chat, show within 0-15 minutes, and sample 10-30% of eligible sessions with a 7-day per-user cooldown.
Trusted by 5000+ Brands
The 8 core chatbot survey questions (plus add-on modules)
"Did you accomplish what you came here to do today?"
Why it matters: This is your Resolution score. You can fix tone later, but you cannot ignore sessions where the job-to-be-done failed.
When to use: Include in every run. Trend it before vs. after prompt/model/workflow changes.
"How satisfied are you with this chatbot session?"
Why it matters: Use this as your top-line KPI for dashboards. It is easy to trend and easy to explain.
When to use: Include in every run. Use an internal starter investigation threshold based on your baseline variability (for many teams, a sustained shift of roughly 0.3-0.5 on a 1-5 scale is a practical place to start), then tune by volume, seasonality, and channel mix.
"The chatbot was easy to use."
Why it matters: This is your first usability anchor (ease). It catches confusing UI, unclear system messages, and awkward turn-taking.
When to use: Keep wording stable so your trends are comparable. Use a consistent Likert scale question design (1-5 or 1-7) across releases.
"It took too much effort to get what I needed."
Why it matters: Effort is where users feel loops, repeated questions, and long back-and-forth. This item usually moves first when fallback handling improves.
When to use: Use as your "friction" measure. Flip-code it so higher is better if you roll it into a Usability score.
"I felt confident I was using the chatbot correctly."
Why it matters: Confidence flags unclear instructions, unclear boundaries, and inconsistent behavior. It is also a leading indicator for repeat use.
When to use: Pair with the effort item to diagnose whether people struggle because of bot limitations or because the flow is confusing.
"The chatbot's answer was helpful."
Why it matters: Helpfulness is your Usefulness score anchor. Useful experiences tend to drive satisfaction in chatbot use, so a drop here usually explains a CSAT drop faster than UI tweaks do.
When to use: Include in every run, even for internal bots, so you can compare usefulness across intents and channels.
"Using this chatbot saved me time."
Why it matters: Time saved is a plain-language productivity item. It is especially important for internal IT/HR bots and workflow assistants.
When to use: Keep it as-is when you want a stable trend line. If this drops after a release, check latency, refusals, and missing next steps.
"I trust the chatbot's answers."
Why it matters: Trust is your release-safety check. Users will stop using a bot that feels unreliable, even if the UI is smooth.
When to use: Include in every run. If you add citations or "why" explanations in the bot, watch whether this item improves.
Add-on modules (swap wording, keep scoring): Keep your 8-item core stable, then add 2-3 questions only when you need extra diagnostics. If you want inspiration for chatbot-specific usability wording (without rebuilding your entire survey), review the item styles in the Chatbot Usability Scale paper.
- Support bot (Resolution + handoff): Add "Did you need a human agent to resolve this?" and (if escalated) "The agent had the context from the chatbot."
- Sales bot (Helpfulness + next step): Swap Q1 to "Did you get the next step you needed (quote, demo, product match)?" Add "I know what to do next after this chat."
- Internal IT/HR bot (time saved + policy clarity): Add "The policy/process was clear." and "I would use this chatbot again for a similar request."
Keep your mapping consistent: Use 2-3 "ease/effort/confidence" items for Usability and 2-3 "helpfulness/productivity/reuse" items for Usefulness. Useful chatbot experiences are strongly tied to satisfaction, so treat usefulness drops as primary fix candidates, not cosmetic issues (usefulness and satisfaction in chatbot experiences).
When to trigger the survey (3 proven moments in the chat flow)
Trigger 1: End-of-conversation (default)
If the user taps "Done" or the session hits inactivity timeout, then show the survey within 0-15 minutes. This captures fresh recall without interrupting task completion.
Trigger 2: After escalation or handoff request
If the user requests an agent (or the bot routes to one), then ask handoff questions right after the human resolution step. Log whether context carried over and time-to-resolution.
Trigger 3: After an unresolved-intent pattern (avoid "happy path" only)
If the session hits N turns without progress (start with N=6-10), repeated fallbacks, or a refusal, then show a short version of the survey. This prevents survivorship bias from surveying only successful chats.
Sampling rules you can turn on today
- Default: Randomly sample 10-30% of eligible sessions.
- Cooldown window: 7 days per user (or 30 days for internal employee bots) to control fatigue.
- Release focus: Oversample new/changed intents for 7-14 days after a release, then revert to baseline sampling.
- Bias control: Make all sessions eligible (including drop-offs, fallbacks, refusals, and escalations), then apply your sample and cooldown. Use these rules as your practical way to apply how to reduce response bias.
Checklist: Pick your default trigger, add the escalation trigger, add an unresolved-intent trigger, set 10-30% sampling, set a 7-day cooldown.
In-chat vs post-chat: which deployment fits your bot?
| Decision point | In-chat micro-survey (1-2 taps) | Post-chat survey (5-10 items) |
|---|---|---|
| Typical length | 1-2 questions (thumbs up/down, 1 item CSAT) | 8-question core + optional 2-3 question modules |
| Response rate | Higher (low effort, immediate) | Lower, but you get richer diagnostics |
| Recall accuracy | Best for immediate reaction to the last answer | Best for end-to-end outcomes (resolution, handoff, time saved) |
| Interruption risk | High (can break task flow if asked too early) | Low (shown after completion or timeout) |
| Bias risk (who sees it) | Often only users who reach a visible "end" state see it | Easier to include drop-offs, fallbacks, refusals, and escalations in eligibility |
| Channel fit | Best: web widget, in-app chat where a quick tap is natural | Best: web/in-app follow-up modal, email; use caution in SMS where 8 items feels long |
| Data you can safely ask | Outcome + one diagnostic (e.g., "Why?" with preset options) | Outcome + usability/usefulness/trust + optional open-text (with strict guardrails) |
| Identity handling | Often anonymous by default; hard to follow up | Anonymous, confidential, or identified. For internal employee bots, default to confidential unless you have a clear follow-up workflow and you disclose it upfront. |
Rule of thumb: If you need quick regression monitoring, start with an in-chat pulse. If you need to diagnose why resolution dropped (fallbacks, handoff, trust), use the post-chat core.
How to score and act on chatbot survey results (release-ready playbook)
- Define your scores (one KPI + four sub-scores)
Set a top-line KPI (pick one): "Overall satisfaction" or "Worked as expected". Then compute four sub-scores as simple averages: Resolution, Usability (ease/effort/confidence), Usefulness (helpful/time saved/reuse), and Trust.
Start with 1-5 scoring for most items. If you use a reverse-coded effort item, flip it before you average.
- Trend weekly and set an investigation threshold
Trend scores week-over-week and annotate releases (prompt changes, model swaps, new intents, routing changes). Use a configurable starter threshold based on your baseline variance and sample size. As a starting point, many teams flag a sustained shift of roughly 0.3-0.5 on a 1-5 scale (or about 5-10 points on a 0-100 scale), then tune once you learn what "normal" looks like for your bot.
Notes for internal and regulated contexts: Document your chosen threshold and who can approve exceptions so an alert is interpreted as an ops signal, not a universal standard.
- Segment the way your bot actually runs
Break down results using operational segments so you can take action fast:
- Intent category: billing, returns, password reset, benefits, etc.
- Escalation: contained vs escalated vs user-requested agent
- Channel: web widget vs in-app vs SMS vs Slack/Teams
- Language/region: especially if you localize prompts and policies
- User type: new vs returning, employee vs customer
- Prioritize fixes with Impact x Frequency
Rank problems using two numbers you already have. Define Impact as the score drop or high dissatisfaction rate. Define Frequency as how often that intent/path occurs.
Fix high-impact, high-frequency items first (example: a common intent with low Resolution and low Trust). Treat Trust drops as release blockers because trust and perceived reliability strongly shape satisfaction and continued use (research on trust in customer service chatbots; evidence linking satisfaction and loyalty for service chatbots).
- Tag open-text and connect it to intents (without creating a mess)
Use one open-text prompt at the end (optional): "What went wrong or what should we improve?" Then apply a small tag set so your team can sort quickly. Follow open-ended question best practices to keep comments usable and reduce sensitive-data risk.
- Default tags (start with 8): wrong answer, didnt understand, missing info, too slow, handoff failed, refusal, confusing flow, privacy concern
- Weekly routine: review top 20 negative sessions by Impact, then scan tags by Frequency
- Repro workflow: pair the tag with intent + a transcript snippet only if your privacy rules allow it
Checklist: Pick 1 KPI, compute 4 sub-scores, trend weekly, segment by intent/escalation/channel/language, rank fixes by Impact x Frequency, tag comments with 5-10 labels.
Privacy, consent, and data handling checklist for chatbot feedback
- Use a plain consent line in the invite: Copy/paste: "Help us improve this chatbot. This survey takes about 1-2 minutes. Please do not enter passwords, payment details, health information, or other sensitive information."
- Decide identity mode up front: Default to anonymous or confidential. For internal employee bots, keep it confidential-by-default unless you will follow up 1:1 and you disclose that clearly.
- Put guardrails on open-text: Show a reminder above the comment box ("No secrets or account numbers"). Follow open-ended question best practices and consider turning off open-text in high-risk flows (billing, medical, HR cases).
- Separate survey data from transcripts unless you disclose linkage: If you plan to join responses to chat transcripts, say so in the invite. In regulated contexts, keep survey storage separate from chat logs unless you have a documented reason and user notice.
- Set retention and access controls: Pick a retention default that fits your policies (internal starter default: 90-180 days, then adjust for legal/regulatory requirements and your analysis cadence), restrict access by role, and log exports. Use your existing privacy and data handling guidance to align survey access with your bot ops workflow.
- Make redaction and deletion actionable: Create a queue for "please remove my data" requests and document who handles it and how fast (internal starter target: 7-14 days, then adjust based on risk and staffing). Keep your process consistent with recognized survey best practices like AAPOR best practices for survey research.
Frequently Asked Questions
How long should a chatbot survey be?
Keep your default survey to 5-10 items so most users finish in under 2 minutes. Add modules (handoff, trust/safety, multilingual) only when you need that extra diagnostic detail. If interruption risk is high, run a 1-2 tap in-chat pulse and save the full survey for post-chat.
Should I trigger the survey inside the chat or after the chat ends?
Use an in-chat micro-survey when you want the highest completion and a quick signal on the last answer. Use a post-chat survey when you need to diagnose resolution, effort, and trust across the full session. As a rule of thumb: web/in-app can handle either, SMS should stay short, and internal Slack/Teams bots usually work best with a post-chat follow-up.
How do I avoid bias if I only survey successful conversations?
Do not make a "success state" a requirement for eligibility. Include drop-offs, fallbacks, refusals, and escalations in your eligible set, then randomly sample 10-30% with a per-user cooldown. Add a separate trigger for unresolved-intent sessions so failure modes show up every week.
How do I measure resolution for a chatbot?
Ask one direct outcome item like "Did you accomplish what you came here to do?" and keep it in every run. Then add an escalation/handoff question so you can separate "contained and solved" from "escalated and solved". Segment resolution by intent category to see where your bot fails most often.
Can I use SUS or TAM concepts without running a formal research study?
Yes. Borrow plain-language items that map to usability (ease, effort, confidence) and usefulness (helpfulness or time saved, intention to reuse) and keep them consistent across releases. Focus on actionability and trend tracking, not perfect wording tweaks every week.
What should I do with open-text feedback from chatbot surveys?
Use a small tag set (5-10 labels) and review it on a weekly cadence. Pair tags with the intent and, if your rules allow it, a short transcript snippet so your team can reproduce the issue. Keep the comment prompt last and remind users not to enter sensitive information.