Evidence in the Face of the Extraordinary Claims We Make About Products, Services or Experiences

It sometimes feels like the product and service-making community over-complicates our defense of research in service of design. The questions and observations about research come fast and thick…

“Why conduct any research at all? We already know what our customers want.”

“We’re agile. We don’t have time for research.”

“Innovation is about creativity and creativity is just about your gut.”

“We prefer to throw up ideas, like spaghetti on a wall, and see what sticks.”

I especially hate the last example, especially because well-cooked spaghetti is much more likely to fall to the floor in a wet clump. These ideas miss the key point—because carrying out research in service of, or to inform design, is about gathering evidence.

The emphasis is intentional.

It’s not just about finding any evidence, but evidence that supports, or cannot support, our hypothesis. These can build, brick-by-brick, our theories of behaviour.

‘Hold it,’ you might cry out. ‘What is the purpose of talking about hypotheses and theories? I’m just concerned with developing an amazing product, service or experience to support my organisation. I’m here to solve problems, not mess about with evidence.’

Give me a minute to explain. Imagine that a product, service or experience is not a thing, but a theory we have; a theory that something we’ve built will solve a problem for someone.

But how can we know?

Well, if we follow the spaghetti theorists, we hope that something we build is ‘sticky’ enough to stand out from all the ideas we consider. That seems weak.

We can just as easily dismiss, in no specific order: not doing any research, relying on our gut (which is an organ for digestion, not hypothesis testing) or winging it. None of these approaches appeal to me. They’re risky, both for the people we serve and the organisations investing time, effort and money to build things.

Advocating for research-free design is like guessing, and often the result is ending up with products, services and experiences that are full of biases and assumptions. You know what they say about assumptions.

Instead, let’s take a hint from the late Carl Sagan, astronomer, science communicator and tireless advocate for critical thinking, who noted:

“What counts is not what sounds plausible, not what we would like to believe, not what one or two witnesses claim, but only what we support by hard evidence rigorously and skeptically examined. Extraordinary claims require extraordinary evidence.” ("Cosmos/ Encyclopaedia Galactica". Documentary, www.imdb.com. 1980.)

Design-oriented research gathers evidence on behavior, context, technologies, problems, and promising ideas. Research is the spotlight that illuminates and cuts through the darkness of ignorance.

Crucially, as Sagan noted, the more extraordinary the claim we’re making, the more extraordinary the evidence required. So with products, services and experiences, the more we’re expecting our systems to influence behaviour, the more evidence we need about how things will work; both ahead of time and as we’re building out our sequence of prototypes. Both to understand what influence our systems will exert (or cannot exert) and whether there will be any dangerous consequences.

Evidence underpins Sagan’s famous ‘Bologna Detection Kit,’ a set of tools to help enhance critical thinking, which includes references to independent confirmation, debate avoiding arguments from authority and invalidating hypothesis.

In the world of science, we gather evidence to prove or disprove our hypotheses, which gives us ever increases confidence that our grand theory is sound. In the world of product, service and experience design, we conduct research and run experiments to solve the right problem, for the right people and for the right reasons.

Sometimes it feels like we’re all so busy rushing around trying to defend the need for research that we forget the basic premise…

…Evidence exists to help us understand how and why something works, and therefore to make better and clearer decisions.

Anything else is just sloppy thinking.