Concept Testing Questions, One Concept Per Person
Fourteen concept testing questions for an idea that does not exist yet. Take the survey here, copy the questions, or open it in the editor and paste your own concept in. Monadic by default, because that is how somebody will meet your idea.
- 14questions
- About 4 minto complete
- 1 conceptper person
- 2 questionsasked before the reveal
Monadic testing or a side by side comparison?
The same two ideas can hand you two different winners, depending on whether people meet them together or one at a time. That is a published result, not a matter of taste. So the design is the first decision, before a single question gets written.
Three designs, and the names carry weight. Monadic means one person sees one concept and nothing else. Sequential monadic means one person sees several, one after another. Comparative means every concept is in front of the person at once, rated against the others.[3]
What changes between them is what a person can judge. Shown one idea on its own, somebody weighs it against whatever they already do. Shown two together, they can also weigh the things that only mean anything in comparison. A review of joint and separate evaluation found that options are evaluated differently in the two modes.[1] It also found that preferences between the same two options can reverse.[1]
The worked example in that review is a second hand music dictionary. Asked about one at a time, people paid more for the undamaged copy with 10,000 entries. Put the two side by side and they paid more for the torn one with 20,000 entries.[1] An entry count means nothing until there is another entry count beside it.
| Design | What one person sees | What you can say afterwards | What it costs you |
|---|---|---|---|
| Monadic | One concept, with nothing beside it | How the idea lands against whatever people already do | A separate group of people for every concept |
| Sequential monadic | Two or more concepts, one after another | The same, plus a ranking, if you correct for the order | Every concept after the first is judged next to the one before it[2] |
| Comparative, side by side | All of the concepts at once | Which one people pick when the choice is put in front of them | Attributes nobody could judge alone start deciding the answer[1] |
| A single preference pick | A list of names | Which one won | No reason attached to the win |
For product concept testing, monadic is the default here. Nobody will meet your idea in a line-up beside its two rivals. They will meet it on its own, next to what they already do, and the monadic design puts them in that position. A side by side test answers a different question, which is which of your candidates wins a contest you arranged.
This page is about the idea itself: a description, a positioning, a pack design, anything judged before it exists. Which name to launch it under is a separate test with its own scorecard, in the naming survey template. Where the product already exists and people have used it, start from the product research survey and let it route you. To put two brands that both already exist on the same attributes, that is the brand perception survey template.
The 14 concept testing questions
Every question, its scale, and what a poor answer is telling you. Copy one block, copy all 14, or open the survey in the editor and paste your idea into it.
Before the idea appears
Two questions asked before anybody sees the concept, so they can still screen. A category question placed after the reveal cannot screen anyone out.
- How often do you buy or use [CATEGORY]?People who never buy the category can still like your idea. Keep them in a separate column or you are counting applause from a different audience.
- The last time you needed something in this area, what did you do?This is what the idea has to beat. Doing nothing is on the list on purpose, because leaving it off would hide the alternative that wins by default.
First reaction, before any rating
One open answer and one rating, both about whether the idea was understood. Both are asked before any liking question, so a high score cannot be a guess wearing a smile.
- In your own words, what is [CONCEPT]?Read these before anything else. An answer describing something you did not build means the description failed, and every rating after it is about a different product.
- The description made it clear what this is.A low score here invalidates the rest of that one response rather than telling you the idea is weak.
Does it fit their life
Five statements on one agree scale, each naming a separate reason an idea gets taken up or ignored. All five are asked before the decision question, so the decision has something to explain it.
- This is meant for someone like me.Low here with a high appeal score means you have built something people admire and will not buy.
- This would deal with a problem I have.The item that separates an interesting idea from a needed one, and the hardest of the five to fix later.
- I believe it would work the way it is described.Belief, not desire. Where people want it and do not believe it, the work in front of you is proof rather than persuasion.
- There is nothing else available to me that does this.Answered from what they know exists, which is the only comparison a concept test can honestly ask for.
- This is worth going out of my way for.A stand-in for effort without naming a price, so the answer is not anchored on a number you have not decided yet.
The decision
Two questions that turn an opinion into something countable. One intent item, with no argument here about which intent scale is best, and one about how often the thing would be used.
- If [CONCEPT] were available today, would you get it?Count the top box on its own and the top two together, then report both and name the one you are quoting.
- How often do you think you would use it?An idea people would get once and never open again is a different business from one they would use every week.
In their words, and the closing rating
Two open answers set before the end, while somebody still has a reason to type. One rating last, because a rating is a single tap.
- What would have to be true before you would get it?The most useful box on the page. It names the missing piece that no rating can point at.
- How would you describe this to a friend in one sentence?The sentence written here is the one your launch copy has to survive standing next to.
- Overall, this idea appeals to me.The number everyone will ask you for. Put it under the five fit items, never instead of them.
Three placeholders to replace before you send. [CATEGORY] in the first question, [CONCEPT] in questions 3 and 10, and [YOUR ORGANISATION] in the first-screen text further down. The wording is ours, so copy it, edit it and translate it, then leave it alone: a changed word ends the comparison with your next wave.
One question this survey refuses to ask. Several concept testing templates ask how the product compares with similar products the person has used. Nobody has used a concept, so that item has no honest answer and every response it collects was invented on the spot. Question 8 asks instead what somebody knows already exists, which is a question they can answer.
What the running order does to your scores
If everybody has to see more than one concept, the order they see them in stops being a detail and becomes part of the result.
Position moves the score. A study of real sequential monadic concept tests asked each person to evaluate three concepts. It found clear order effects: the same concept was evaluated more highly in first position than in later ones.[2]
The concept before yours moves it as well. The same work found that the quality of a concept influenced the evaluation of the one after it.[2] Concepts following a good one were evaluated less positively, and concepts following a weak one more positively.[2] So a score in position two is not a property of that concept. It is a property of that concept, in that slot, after that neighbour.
What to do about it. Split the list into as many groups as you have concepts. Give each group a different running order, so every concept sits first for somebody. The editor will not rotate them for you: its randomise setting shuffles the questions inside one survey, which is a different thing. Rotating means separate copies of the survey with the blocks swapped over, one group sent to each.
Or run monadic, and the problem does not arise. One concept per person removes an order effect rather than correcting for it. That is the argument for the design in a single line.
How to send it so each person sees one idea
The instrument is the easy half. Monadic is a sending decision, and there are two ways to make it.
Write the idea down in five lines or fewer
One paragraph covering what it is, who it is for and what it replaces, plus one picture where a picture is part of the idea. Give every concept in the test the same length, the same tone and the same amount of detail, or what you are measuring is the writing.
Make one copy of the survey per idea
Duplicate the survey, paste idea A into one copy and idea B into the other, then send half your list each link. Nobody meets two ideas, and the size of each half is yours to decide rather than a coin flip.
Send both halves inside the same week
Two ideas tested a month apart are also two different weeks. Whatever happened in between arrives in your data dressed as a difference between concepts.
Stop at the number you planned, not the number you like
Write down how many answers each idea gets before you open the first one. Then close both links together. Watching them arrive and stopping when one pulls ahead is how a coin flip turns itself into a finding.
Or do the whole thing in one link. Put a question everybody answers at the top. Give it two logic rules, both with the condition Randomly and the action Skip to question. Point one at the first question of block A and the other at block B. Then add a rule on the last question of block A, condition Always, action End quiz, or block A runs straight on into block B. Each person is sent down one branch and stays on it. What this will not do is even the two groups up. The split is a coin flip per person with no quota behind it, so two links and a list you divide yourself is still the version you control.
How many people per concept. That is a power calculation, not a number anybody can hand you. The answer moves with how small a difference you need to see. The arithmetic lives on the market research survey template. What this survey asks of you is the same number on both sides, decided in advance.
What the idea will cost belongs in a later study, so this survey never names a price. When a concept survives, take it to the product pricing survey. To measure intent properly inside a real buying window, that is the buying intentions survey template. All fourteen questions are already loaded in the online form and survey builder, so paste your idea in and share one link.
First-screen text
About this survey We are testing an idea. It does not exist yet, so there is nothing here to buy and nothing to install. Two short questions come first, then one idea described in a few lines, then twelve questions about it. It takes about four minutes and there are no right answers. Answers go to [YOUR ORGANISATION], who are running this test. You are not asked for a name or an email address.
How to read what comes back
Fourteen answers per person and a decision at the end of them. Read them in this order.
Throw out the misreads first. Take question 3, the open one, and sort the answers into people who described what you built and people who described something else. Everyone in the second pile rated a different product, and their scores belong in a column of their own rather than in the average.
Then read the five fit items as five answers, not one. Averaging them into a single number is the quickest way to lose the finding. The five say different things. An idea can be understood, believed and admired, and still be built for somebody else. Report all five, with the count of people behind each.
Then the decision questions. Show the top box and the top two boxes from question 10 beside each other, and say which one you are quoting. Two concepts run monadically means two independent groups with nobody in common. A gap between them carries the noise of both, so a narrow gap is a tie.
Read the open answers before you open a chart. Question 12 names the thing that stops people, and it surfaces in no rating on this page. Question 13 hands you the sentence somebody uses when nobody is watching. That is worth more than the appeal score sitting beside it.
| The claim you want to make | What this test supports | Write this instead |
|---|---|---|
| Concept A beat concept B | Two separate groups answered, so the gap between them carries the noise of both | Report both sets of numbers, with the size of each group beside them |
| Sixty percent would buy it | An intent answer from people who have not been shown a price | Say it is intent, name the box you counted, and stop there |
| People love it | An average taken across five items that measure different things | Give the five items separately |
| The market wants this | Answers from a list that already knows you | Say who answered before you say what they said |
Concept testing survey FAQ
What is monadic testing?
One person sees one concept and nothing else, so nothing they tell you is a comparison with your other candidates. It is the design this template is built for, because it matches the way somebody will actually meet the idea.
How many concepts can I test at once?
As many as you have groups of people to send to. Monadic costs one group per concept. If one person has to see more than one, the running order becomes yours to control. Every concept then needs a turn in first place.
How many people do I need per concept?
There is no honest single number, and the figures other pages print for this are power calculations presented as facts. It depends on how small a difference you need to be able to see. Fix the number before you send, and use the same one for every concept.
Can I run a concept test with no product and no prototype?
That is what a concept test is. The idea is a few lines of description and, where it helps, one picture. Every question in this template can be answered by somebody who has never seen, bought or used the thing.
What counts as a good score?
Nothing published applies, because the answer moves with your category, your description and who you asked. The first test you run becomes the benchmark for the second. Until then the only comparison you have is between your own concepts, measured the same way.
A concept is judged from a description on a screen, with nothing in the room. Once there is a physical sample to put in somebody's mouth, the taste test survey handles the sitting, the sample order and what a coded sample can honestly tell you.
A concept has no price yet. Once one is set and customers are living with it, the price evaluation survey asks whether it still reads as fair.
Methods and sources
What this template is based on
The choice between monadic and side by side follows the published review of joint and separate evaluation[1], and the caution about running order follows a study of real sequential monadic concept tests[2]. The three design names are used the way the field uses them[3].
Question licensing
Nothing licensed appears. There is no rights-held concept testing scale, so all 14 items are original and free to copy, edit, translate and redistribute.
Limits and disclosure
Neither cited study reports an effect size this page could quote, so no magnitude is attached to either finding. No sample size is recommended, because that is a power calculation whose answer depends on the difference you need to detect. Sent to your own list, the answers come from people who already know you, and nothing here extends to the category.
SuperSurvey makes the editor this template opens in and sells no respondent panel, so the advice about splitting your own list costs us nothing to give. The template is free either way.
References
- Hsee, C.K., Loewenstein, G.F., Blount, S. and Bazerman, M.H. "Preference Reversals Between Joint and Separate Evaluations of Options: A Review and Theoretical Analysis." Psychological Bulletin 125, no. 5 (1999): 576-590. doi.org
- Friedman, M. and Schillewaert, N. "Order and Quality Effects in Sequential Monadic Concept Testing: Methodological Details Matter in Concept-Testing Practice." Journal of Marketing Theory and Practice 20, no. 4 (2012): 377-390. doi.org
- Qualtrics. Concept Testing Questions To Ask Survey Respondents. Qualtrics Strategy and Research. qualtrics.com
Put the idea in front of people this week
Fourteen questions, about four minutes, one concept per person. Paste your idea in, divide your list, and send one link to each half.
Use this template