How to Ask Questions During Playtesting Without Hinting at the Answer: Turn "You Get It, Right?" into Neutral Questions

The goal of playtesting questions is not to get the player to say the answer you expect, but to have them reconstruct what just happened in their own words. The approach can be compressed into three steps: first have the player retell their actions, then ask which clue they based it on, and finally ask why they chose this one between two options. Put questions after the action, do not interject while they are hesitating, and do not evaluate any route as better. As long as a question contains "is it because," "you get it, right," or "this is obvious," what you get is mostly agreement, not understanding.

The following uses a fictional teaching example throughout: in the interactive novel Letters from Fog Harbor, the player plays a postman who must deliver an unsigned letter to one of three addresses. The clues include tide stains on the envelope, traces of the recipient's surname having been crossed out, and the dock bell ringing twice before the hour. The following dialogue, records, and numbers are made up to illustrate the method; they are not actual player interviews and do not represent any sample effect.

Right after the player finishes delivering the letter, do not first ask "why did you deliver it correctly." Have them describe the order of operations, and you only record verbs and objects:

• "I first looked at the envelope and found tide stains in the lower left corner."
• "I opened the address book and flipped through all three addresses."
• "I heard the bell, and then I chose the old lighthouse."

This retelling itself is evidence. It tells you that the player noticed the tide stains and the bell, and it also tells you that they flipped through the address book. At this point, do not follow with "right, the tide stains are the key." Once you confirm, later answers will drift toward your tone.

The left column of the table below is common phrasing, and the right column is the rewrite. The rewritten questions are all placed after the action and do not praise any route.

What the rewrites have in common is: do not presuppose an answer, do not evaluate a route, and do not summarize for the player. Keep the words the player themselves used in the question; for example, if they say "tide stains," you ask about "tide stains," and do not switch to "watermark clue."

Feeling-type questions easily get "it's okay" or "pretty interesting," with low information value. You can change them into evidence recall: have the player point out the specific text, image, or sound they relied on. Still using Letters from Fog Harbor as an example:

• "You said the bell rang twice; in which scene did you hear it?"
• "In the address book, which entry made you look at it one more time?"
• "Did you rule out the dock because you saw something, or because you did not see something?"

Note that "did not see something" is also valid evidence. The player may say "I did not find any words related to the dock," which is closer to their judgment process than "I felt the dock was wrong."

When the player is hesitating between two options, any question becomes a hint. You can wait until they click, then ask:

• "You finally chose the old lighthouse; which other one were you almost choosing?"
• "If the old lighthouse option did not exist, what would you choose?"
• "When you chose, did you more want to verify a guess, or more want to avoid a certain outcome?"

The third question splits "reason" into two common motives: verification and avoidance. It does not hint at which route is correct, and it can also show whether the player is reasoning or avoiding risk.

During playtesting, it is recommended to use a fixed table and fill it in while listening, rather than filling it in afterward from impression. The template is as follows:

The last column is self-check. If "Whether I evaluated the route" is filled in as "yes," the conclusions about understanding in this record should be discounted.

The player finishes delivering the letter, and the system displays "The letter has been delivered to the old lighthouse." The following is fictional dialogue:

You: What did you just do? Player: I first looked at the envelope, there were tide stains, then I flipped through the address book, heard the bell, and chose the old lighthouse. You: What did the tide stains make you think of? Player: A place near the sea. Two of the three addresses are near the sea, so I looked at the surname again. You: What did you see in the surname? Player: It was crossed out, but the upper half of the character "Lin" was exposed. You: Between the dock and the old lighthouse, which did you rule out? Player: The dock. The dock entry in the address book said "discontinued," and I did not want to deliver to a discontinued place. You: If the old lighthouse did not exist, which would you choose? Player: Maybe the bell tower, because the bell rang twice.

In this segment, the player themselves stated four pieces of evidence: tide stains, crossing out, discontinued, and the bell. The questioner did not say "right" or "wrong," and did not interrupt while the player was flipping through the address book.

After one playtesting session ends, check with the following five items:

1. Every question appeared after the action, and there was no interjection while hesitating.
2. The questions did not contain "is it because," "obvious," or "you get it, right."
3. At least two clues pointed out by the player themselves were recorded, rather than summarized by you for them.
4. The reason for the choice asked about "based on what," not "did it feel good."
5. In the record table, "Whether I evaluated the route" is "no."

If item 3 cannot be met, first look back at whether the question was too broad, such as "what do you think." Change it to "where did you look first just now," and you can usually get a more specific action.

When AI-generated text is involved, it may add backstory that was not in your setting. Playtesting questions should only target what the player actually saw; do not use details added by the AI to ask the player back, otherwise you will treat information that does not exist in the current version as a clue. When recording, take the player's exact words as the standard, and do not treat AI output as player understanding.

Next playtesting session, first change only one question: change "you get it, right" to "where did you look first just now."