Synthetic Users and Why We Talk With Real People When Doing User Research

4 Feb

There are a lot of forces, some recent and some historical, that work against talking with real people during user research. The rise of synthetic users is the most recent and perhaps the most powerful. But even when considering the apparent efficiency and cost-effectiveness of synthetic users, we still advocate talking with real people.

Why do we take this position? It all comes down to the ethics and value of working with real people. But to explain, let’s start with the basics, ‘What are synthetic users?’

How Generative AI and Large Language Models build synthetic users and how are they used?

Generative AI is the dramatic new movement in the field of AI we’re sure you’ve heard about. The technology news covers little else lately. You may even have used one of AI’s many recent tools: DallE, Deepseek, OpenAI, ChatGPT, Gemini, CoPilot, Claude and more.

The core technology at the heart of Generative AI are the Large Language Models that make up the AI’s interior architecture. Think of these Generative AI as wrappers, including interfaces and some linking components around the core of a large model.

Large Language Models (LLMs) are pretty much what they sound like; large and interconnected models of language. A bit like a tremendous mind-map, but all built in the math of statistical relationships between words and word-fragments.

They are built by training the emerging AI on billions of sources of text. A good chunk of the internet. It’s been proven that many copyrighted works have been caught up in the training set. Something we’ll come back to later.

In the training, the software learns about what words (or more accurately) what word-fragments relate together. Then, when asked a question, it works… frag… ment… by… frag…ment… to… dis… play… the most likely next few characters. Eventually building what appears to be a comprehensive answer.

This is really key. These tools build their comprehensive answer ‘bottom-up’, with no knowledge of the world or any real ‘idea’ or ‘concept’ of what they’re talking about. That’s why they’re often called ‘Statistical Engines’ or ‘Stochastic Parrots’ (stochastic is just a fancy way of saying probability - match). They parrot out the most likely match to input strings, based on their massive internal database of statistical matches. Nothing more, nothing less. Any internal structure of seeming hint of genius comes from the size of the training set and the number of matches made.

In turn, synthetic users are custom tools build on-top of these Large Language Models (LLMs). When interrogated, they produce profiles of synthetic ‘people’. These synthetic people serve as research proxies, able to prompt design teams about apparent user needs, answer questions about feature fit and explain their demographics.

Again, all of these are based on an incredibly large and averaged data set of patterns, derived from our world of words.

A key argument for synthetic users centres on their representativeness of an average human population… and therefore, they are suitable replacements for product, service or experience research with the users, customers or stakeholders.

On the surface, this new shiny user and design research toy is an attractive proposition. Synthetic users are not real people so they can eliminate recruitment times, recruitment costs, booking arrangements, suitable interview spaces, travel time and cost for researchers, all reduced to a software-as-a-service fee, all produced at the click of a button.

What’s wrong with synthetic users?

There are four key problems with this new approach, which we’ve summarised as Demographic Illusions, Idiosyncrasy, Context and Ethics.

Challenge #1 Demographic Illusions

Firstly, you’d expect generative artificial intelligence, based on its large language models, to demonstrate some form of correspondence between a human population, as represented in a massive volume of training data and a synthetic population.

There is some early research that shows these synthetic users can answer questions in a way that matches demographic averages of real populations of people. It’s the sort of correspondence you’d expect, given the models are trained on lots of human words. But remember the phrase, ‘the map isn’t the territory’? Well, the model isn’t the territory either.

And it can’t be taken as a sign that the synthetic users are actually representative of real people. The models being used to produce these artefacts are full of hallucinations, errors, bias et al. They are not the biases and errors of people, but of training data sets. Even subtle distortions could lead to details that are utterly wrong.

Challenge #2 Idiosyncrasy

Contrary to what synthetic users seem to be promoting, good user research often lookS for idiosyncrasies, not the answers from a demographically normal population.

That’s how User Research differs from its origin point in the Social Sciences (especially psychology). Psychology does often care about the averages of a population, because it’s looking for patterns around which to create theories of behaviour.

But in design, it’s rarely a good idea to design for the average. Considering how these large models are trained, that’s exactly what sort of synthetic user opinion you’re getting. You end up designing for no-one because in real-world use, an average doesn’t really exist. Everyone is different in some subtle way and specific user populations might be very different.

Synthetic users are just not idiosyncratic, they are normative averages generated by a ‘Stochastic (Probability) Parrot’ that squawks out the answers based on the most likely frequency of the next word, sentence or paragraph.

It might be cheap and quick, but it won’t be the rich insight you want to base expensive and important product, service or experience design designs on.

Challenge #3 Context

Good user research is often based on a real living or working contexts. The places where the people participating in the experience, actually do the job, task, workflow or personal activity that is in focus.

Put in other words, we’re looking for the contextual, real-world juice. We’re looking for the little things, often physical hints or clues. Like sticky notes around a computer monitor at work, with reminders how to use a complicated piece of software. Like little guides bolted onto a piece of equipment to ensure it doesn’t move past some safe position of operation. Like the idiosyncratic quirks of how a service is delivered at once specific branch.

These can’t be found in a statistical pool of averages. Remember, the models don’t really know anything about the real world. They’re not samples of real world behaviour. They’re statistically trained models that make up their own ‘mirror-world’ to our own world.

Challenge #4 Ethics

Finally, considering how these models are trained, there are some deep ethics issues with using synthetic users in their current form. Ethics issues (corporate or reputation risk) in using models based on copyright data. Risk of having copyright data show up in outputs, which are then unknowingly used.

In the other direction, there are ethical risks of having private information leak back into the models on use.

And, as already noted, these models can be full of biases, errors, hallucinations and other cruft. There is an ethical risk of embedding these errors into new products, services and experiences.

It’s not a great bet.

We’re sticking with real people

So, for now, we continue to research with real people in real settings and do real work with real things. It might cost a bit more and take a bit more time, but the results will be higher quality, safer and more valuable to the longer term development of products, services and experiences.

As I noted in my much deeper review of this topic on my seperate think-a-zine, “Adventures in a Designed World,”

“Using synthetic humans for direct research is like navel gazing, but not at our own navels, but into the great messy, cluttered and often disgusting navel of the Internet-with-a-capital-I. The problem isn't to make up fake people. It’s getting better access to real people. ” (Christopher Roosen, Using Generative AI To Make Synthetic Users is not Design Research, it’s a Misleading Form of Navel Gazing)