Chatbot survey questions: 14 items for the chat that just ended
A chatbot survey built around one question the usual chatbot questionnaire skips: did the bot finish the job, or did it hand you to a person, and how did that go. Answer it below to see it working, or open it in the editor and put your own bot in.
- 14questions
- about 3 minto answer
- 5-pointagree scale
- No nameor account asked
What a chatbot survey should measure
Fourteen questions about one conversation. The form is organised around containment: whether the bot finished the job, handed it to a person, or left somebody to work it out alone.
The outcome comes before the rating. Most chatbot questionnaires open with satisfaction and never establish what happened, which leaves you with a number that moved and no idea what to change. This one asks what the person came to do, how the conversation ended and whether the thing is now sorted, and only then asks how it felt.
Containment is the question the metric cannot answer. A chat nobody escalated looks the same in a dashboard whether the bot solved the problem, left the person to finish it themselves, or wore them out until they closed the window. Thematic put it plainly in July 2026: containment "rewards not-escalating rather than resolving".[3] Question 2 splits those five endings apart, and it is the most useful item on the form.
The handover is the other half of the outcome. Two items ask what happened when somebody wanted a person: how hard it was to reach one, and how much they had to repeat once they got there. Both start with the option for somebody the situation never applied to, so nobody rates an event that did not happen.
What this form is not. It is not a usability score for the chat interface, which is what the usability survey template is for. It is not a ticket-level survey either: when the queue is staffed by people, the IT help desk questionnaire asks better questions.
The 14 chatbot survey questions
Every question, the answer it takes, and why it is worded the way it is. Copy one group or the whole form.
What happened
Three questions before any rating, because a score with no outcome attached cannot be acted on. The first sets the intent, the second is the containment question, and the third is the one a support team can check against its own records.
- What did you come to the chat to do?Swap these five for your own top intents. They exist so a low score can be read per job rather than per bot.
- How did the conversation end?The item this whole form is built around. Note that two of the five options are failures that a containment metric counts as successes: a person who was left to finish the job themselves, and a person who gave up. Thematic makes the same point about the metric, which "rewards not-escalating rather than resolving".[3]
- Has the thing you came to do been sorted out?Ended and finished are different things. The fourth option separates a person who is waiting from a person who is stuck, and only one of those needs a call.
The chatbot itself
Five agreement items, all running the same direction, so nothing has to be reverse-scored before you average it. Each one is a separate thing a bot can fail at, kept apart because an item asking whether the bot "worked well" cannot tell you which.
- The chatbot understood what I was asking.Understanding, not answering. A bot can grasp the question perfectly and still have nothing useful to say, and the fix for each is in a different team.
- The answers the chatbot gave were correct as far as I can tell."As far as I can tell" is deliberate. The person answering often cannot verify the answer, and asking them to pretend otherwise gets you a guess dressed as a fact.
- It was clear from the start that I was talking to a chatbot.Disclosure, asked plainly. Read a low score here next to the free text: people who thought they were talking to a person describe the same chat very differently.
- The chatbot was honest about what it could not help with.The alternative to a confident wrong answer is a bot that says it cannot help and moves you on. This item is how you find out which one yours does.
- It was easy to get this handled.Effort, written positively so it runs the same way as the other four. The case for measuring effort separately from satisfaction was set out in 2010.[6]
Getting to a person
Two questions about the moment the bot stops being the answer. Both are single-choice rather than rating scales, and both open with the answer for somebody the situation never applied to, so nobody is asked to rate something that did not happen.
- If you wanted a person at any point, how easy was it to reach one?The last two options are the ones worth watching. A person who asked and never reached anyone has had a worse experience than a person the bot simply could not help.
- If you were passed to a person, how much did you have to repeat?Repeating yourself is the part of a handover people remember. Answers here point at the transcript passing between systems rather than at the bot.
Next time, and in their words
A behavioural question instead of a recommendation score, two open answers before the end, and one overall rating to close on. The opens sit second-to-last because a person who has just described a specific problem writes a better sentence than one asked to comment cold.
- If you had the same thing to sort out tomorrow, what would you do?What somebody would do next time is easier to answer honestly than how likely they are to recommend a support channel. Two of the options are quiet failures: going straight to a person, and not getting in touch at all.
- What did the chatbot get wrong, or what did you have to repeat?Optional. This is the box that names the intent your bot handles badly, which no rating scale can do.
- What would have made this quicker for you?Optional, and phrased as a request for a fix rather than an invitation to complain.
- Overall, how satisfied are you with this chat?The satisfaction item goes last so it is answered after the person has thought about what actually happened, not before.
Cutting it down. If you are sending this inside the chat window rather than after it, keep questions 2, 3, 9, 10 and 14 and move the rest into a follow-up. Those five carry the outcome, the handover and one overall rating, which is the smallest set that still says something.
Two swap-ins. A sales or booking bot wants question 1 replaced with your own intent list and one item on whether the person got a price or a slot. An internal bot answering staff questions wants question 11 rewritten around what people do when the bot fails, since going to a person means a colleague rather than a queue.
All fourteen are already loaded and working in the preview above, so if you would rather start from the form than from this list, the free online survey editor opens with every question in place and the bracketed names left blank for you.
Reading the answers: the containment split
Question 2 has five endings. Split the responses by that item first and read every other answer inside the split, because the same satisfaction score means different things in different rows.
| How the conversation ended | What to read next | What it points at |
|---|---|---|
| The chatbot finished it on its own | Question 3, then question 5. | Nothing, unless question 3 disagrees with question 2. |
| The chatbot did part of it and left the rest | Question 8, and both written answers. | A content or workflow gap: the bot knows the topic and cannot complete the action. |
| I was passed to a person | Questions 9 and 10 together. | A handover problem if people are repeating themselves, not a bot problem. |
| I asked for a person and did not get one | Question 9 on its own. | A routing or staffing gap, and the fastest thing here to fix. |
| I gave up and left | The first written answer, read one by one. | The rows a containment dashboard has been counting as wins. |
Read the five agreement items as five separate things. Understanding, accuracy, disclosure, honesty about limits and effort fail independently and are fixed by different people, so averaging them into one bot score hides the only useful information in them.
Question 11 is the one to trend. What somebody says they would do next time is a behaviour rather than an opinion, and two of its options are quiet failures: people who would go straight to a person, and people who would not get in touch at all.
Compare the same bot over time, not two bots against each other. Intent mix, page placement and how visibly the escape hatch is offered all move these numbers. If you are scoring a mobile product rather than the chat inside it, the app feedback form is the better instrument.
Sending a chatbot survey
Four decisions before the first send, then the note that goes above question 1.
Decide which chats are eligible before you look at any results. The temptation is to survey the conversations that reached a tidy ending, and that is the one choice that guarantees a flattering answer. Fallbacks, refusals, escalations and abandoned windows all belong in the eligible set. If they are not in it, question 2 cannot do its job.
Sample rather than send to everyone, and set a cooldown. Both numbers are yours to choose, and they depend on your chat volume and how often the same people come back. Pick them deliberately and write them down, so a change in the results later can be told apart from a change in who was asked.
Send it after the chat, not during it. Question 3 asks whether the thing is sorted, and inside the chat window nobody knows yet. A short delay costs you responses and buys you the only answer on the form a support system cannot already tell you.
Say who reads it, once. One line above question 1 is the ceiling. Name the team, say the answers are read together rather than one at a time, and give people somewhere else to go if they still need the original problem fixed.
Two neighbours are worth checking first. If a person finished the job rather than the bot, that interaction has its own form in the after-service survey. If people reach the chat because they could not find the answer anywhere else, the website feedback form measures the cause rather than the symptom.
Do not use this form as a support channel. Some people will answer question 12 with the problem they still have. Decide in advance who reads those and how quickly, and put the real route to help in the note above question 1. Question wording throughout follows published guidance on asking one thing at a time and on how answer options shape a response.[1]
The note that goes above question 1
Before you send it This survey is about the chat that just ended, not about [COMPANY]. It takes about three minutes. There are 14 questions and you can skip the two written answers. We do not ask for your name or your account number here. Answers go to the team that looks after this chat, and they are read alongside everyone else's rather than one at a time. If you still need this sorted out, do not use this form to tell us. [WHAT TO DO INSTEAD] will get you to somebody who can act on it.
Chatbot survey questions FAQ
What questions should I ask in a chatbot survey?
Start with what happened, not with how it felt. Ask what the person came to do, how the conversation ended, and whether the thing is now sorted. Then ask about understanding, accuracy, honesty about limits and effort, one item each. Then ask what happened when they wanted a person. A rating with no outcome attached tells you a number moved without telling you what to change.
How many questions should a chatbot survey have?
Fewer than this if you send it inside the chat window, and about this many if you send it after. Fourteen items at roughly three minutes is at the long end, so the two written answers are optional and the form is built to be cut. For a short version, keep questions 2, 3, 9, 10 and 14.
Should the survey appear inside the chat or after it?
Inside the chat gets you more answers and shallower ones, because the person is still in the middle of something. After the chat gets you fewer answers from people who have had time to find out whether the thing was actually resolved. This template is written for after.
Should I ask people to rate the chatbot, or rate the outcome?
Both, in that order, and keep them apart. Questions 4 to 8 rate the chatbot; questions 2 and 3 record the outcome. Keeping them separate is what lets you find the uncomfortable case where people are content with a polite bot that did not get anything done.
Is there a validated chatbot usability scale I could use instead?
Yes, and it is worth knowing about. The Chatbot Usability Scale, published open access under a CC BY licence by Borsci and colleagues, covers accessibility, interaction quality, information quality, privacy and response time.[4] Use it when you want a comparable usability score across bots or releases, and use this form when you want to know whether a job got finished. They answer different questions.
Does a chat count as contained if the person just gave up?
By most containment metrics, yes, and that is the problem with reading containment on its own. A conversation that nobody escalated looks identical to a conversation that worked. Question 2 exists to separate them, and Thematic makes the same argument about the metric: containment "rewards not-escalating rather than resolving".[3]
Should the survey ask for a name or an account number?
Not in this form. If you already know who the chat belonged to, you have the identifier without asking, and asking again suggests the answers will be read next to the person. If you do not know, adding the field will cost you responses and will not reliably get you the truth. Say plainly who reads the answers instead, in the note above question 1.
How often are people using chatbots at all?
Often enough that the channel is no longer optional. Pew Research Center found that about half of US adults now use AI chatbots, up from a third in 2024, and about a quarter use them daily, in a survey of 5,119 US adults fielded in February 2026.[5] That is general chatbot use rather than customer service specifically, so treat it as context for why the channel matters, not as a benchmark for your own bot.
Methods and sources
What this template is based on
Fourteen items ordered as the reader's own account of one conversation: what they came for, how it ended, whether it is done, then the bot, then the person, then what they would do next time. Every agreement item runs in one direction so nothing needs reverse-scoring, and the three situational items are single-choice with an explicit "this did not happen to me" option so that no question depends on an answer somebody skipped. Wording follows published guidance on asking one thing at a time, on answer-option effects and on question order.[1][2]
Question licensing
Nothing here is reproduced from a licensed instrument. The Chatbot Usability Scale[4] is published open access under a CC BY licence and is recommended as the alternative for readers who want a comparable usability score, but none of its items appear on this page. The idea of measuring effort separately from satisfaction is credited to its 2010 publication[6] and the item here is written from scratch.
Limits and disclosure
This measures one conversation as one person remembers it, so it is a read on the experience rather than on what the transcript contains. It cannot verify whether an answer the bot gave was actually correct. It supports comparison of the same bot over time far better than comparison between bots, because intent mix and placement move the numbers as much as quality does. The one population figure quoted is Pew Research Center's and describes general chatbot use among US adults, not customer service[5]; it is context, not a benchmark.
SuperSurvey builds the editor this template opens in. Nothing on this page depends on using it.
References
- Pew Research Center. Writing Survey Questions. Methods. pewresearch.org
- American Association for Public Opinion Research. "Best Practices for Survey Research." aapor.org
- Thematic. "How Do You Analyze What Customers Are Saying About Your AI Chatbot?" 15 July 2026. getthematic.com
- Borsci, S., Malizia, A., Schmettow, M., van der Velde, F., Tariverdiyeva, G., Balaji, D. and Chamberlain, A. "The Chatbot Usability Scale: the Design and Pilot of a Usability Scale for Interaction with AI-Based Conversational Agents." Personal and Ubiquitous Computing 26(1), 95-119 (2022). Open access, CC BY. doi.org
- Pew Research Center. Americans and AI 2026: Chatbots, Smart Devices and Views on Impact, 17 June 2026. Survey of 5,119 US adults, American Trends Panel, fielded 17-23 February 2026. pewresearch.org
- Dixon, M., Freeman, K. and Toman, N. "Stop Trying to Delight Your Customers." Harvard Business Review, July-August 2010. hbr.org
Put your own chatbot in it
Fourteen questions, about three minutes, and an answer at the end to the question a containment dashboard cannot reach. Open it, change the intent list, send it.
Use this template