Synthetic users are a rehearsal, not a result
Simulated customers love your prototype. Of course they do. Use them to practise, then write down what only a real person can tell you.
What changed
You can now ask a model to be your customer. Give it a persona, show it the prototype, ask what it thinks. It will answer in character, at length, instantly. Teams do this and call it validation.
It isn’t. The model has never been confused by a bill. It has never ignored a letter because it arrived on a bad day. It gives you a plausible customer, and plausible is exactly the thing a test is supposed to get you past.
But it’s not useless either, and the teams who dismiss it entirely waste a decent rehearsal tool. The skill is knowing where the line is.
The skill
Use simulated responses to rehearse: try your questions, catch the obvious breakages, explore what a sceptic or a hurried person might say. Then write down what you still don’t know that only a real customer can tell you. Then design a realistic task for five of them that gets at exactly that.
Fig. 8 — Rehearse, find out, decide. The model lives in step one.
The “what only a real person can tell you” list is the heart of it. In the high-bill sprint it came out as:
- Do they open a letter from us at all, in the week after a bill?
- When they see “+£61 vs last year”, do they read on or do they reach for the phone?
- Does a call from an unknown number get answered, and does “we’re ringing before your bill” land as helpful or alarming?
None of those can be answered by a model playing a pensioner. All three were answered on Thursday by five actual customers, with a realistic task: here’s a bill, here’s a letter, here’s a phone, show us what you’d do. Two of the five rang the number before reading the letter. That one observation changed the order of the whole thing.
What goes wrong
Opinion tests. “What do you think of this?” gets you politeness from real people and enthusiasm from synthetic ones. Neither is evidence. Give them a task and watch what they do. The gap between what people say and what they do is the entire reason testing exists.
And the quiet one: a team that’s rehearsed with the model for an hour starts to feel like it’s already tested. The synthetic session was so fluent, so detailed, that real testing feels like a formality. Name this. Say out loud at the end of the rehearsal: “We now know nothing we didn’t know this morning. We know what to ask.”
Run it Monday
- Rehearse with two or three synthetic personas. Twenty minutes, no more. Note every question that felt weak.
- Write the “only a real person can tell us” list. Three to five items.
- Turn each into a task, not a question. “Here’s X. What do you do?”
- Five real customers, recorded, the team watching. Thursday if the sprint started Monday.
- Say the sentence: “The rehearsal taught us what to ask. It didn’t answer anything.”
Humans decide. AI accelerates. Customers settle it.
The test-script prompt and the task template, how we go from rehearsal to a five-person task in an afternoon, are in the facilitation deck.
