Wizard of Oz

The product looks finished and automated to the person using it. Behind the curtain, you're doing the work by hand. They type a request to your "AI assistant," and the assistant is you, typing back fast from behind the screen. They never know. In the movie, the great and powerful Oz is a man pulling levers behind a curtain. Here, the levers are you.

The question it answers

Wizard of Oz tests desirability (do people actually want this feature) and usability (once it's in front of them, can they work it).

It tells you both because it puts a real-looking thing in front of real users inside your real product. They're not telling you what they'd do with a feature you described. They're doing it. You watch behavior, not opinion, and behavior is the only signal worth trusting.

What it does not test is feasibility, the can-we-actually-build-it question. The full warning lives below. The short version is that a glowing result here says people want the magic button, not that the magic is buildable at a sane cost.

How to run it cheaply

The minimum version is one button and a human.

1. Pick a feature that would be expensive to build, sitting inside a product you already have. The cost is the whole reason to fake it first.
2. Put a real-looking entry point into the live product. A button, a link, a chat box. It should look as finished as the real feature would.
3. Wire the back of it to a person, not a system. When a user triggers it, someone on your team does the work by hand and sends the result back fast, while the user waits on a plain "working on it" state.
4. Write the success number and the end date down before you turn it on. Usage, not money, is the currency. Do real users use the faked feature, and do they come back to it?
5. Run it on a small slice of real traffic for a fixed window, then read the number.

If people keep reaching for the magic button, you've earned the hard build. If they try it once and never return, you just saved yourself a project for the price of some hand-done work.

A worked example

The way I teach Wizard of Oz is plain: create a fake front of the product and operate it by hand, without the user knowing, so you test whether the thing works before you build it. The company and numbers below are invented to make the mechanics concrete. The method is real.

Picture a project-management app that gets constant requests for an AI that writes status updates from a project's activity. Building that is a real, hard project. So before any of it, you add the button.

A user clicks "Draft my status update," sees a "generating..." spinner, and ten seconds later a clean draft appears. There's no model. A person on your team read the project's recent activity and wrote that draft by hand, fast, while the spinner ran.

You set the bar before you start. 40% of users who try it once use it again within a week. (Illustrative.) If they keep coming back, the demand is real and the hard build just earned its place on the roadmap. If they tried it once and bounced, you spent a few weeks of hand-written drafts instead of a quarter of engineering on a feature nobody wanted. Either way, you learn whether they want a hard feature, and whether they can use it, for the cost of some typing.

When to reach for it, and when not

Reach for Wizard of Oz when you already have an app or site with live users, and you want to test a feature that would be expensive to build before you build it. The fake feature has to live inside something real, or the behavior you watch won't be honest.

Don't confuse it with Concierge, a test where the user knows a human is doing the work by hand. That's the whole difference between the two: who knows. In a Concierge test, the work is white-glove and in the open. In Wizard of Oz, they think it's the software. Concierge learns whether a mostly-human service is worth doing at all. Wizard of Oz learns whether an automated-feeling feature pulls real, repeated usage, and whether people can actually work it, before you automate it. Same hand-done work behind both. The curtain is the only difference.

Don't reach for it to answer "can we build this." It can't. You faked the build, so a glowing result still leaves feasibility untouched. Pair it with a feasibility check, or you'll greenlight a hard project on a signal that never tested the hard part.

And don't climb to it before the cheaper rungs have spoken. If a fake-door test, a button that just records the click, can tell you whether anyone reaches for the feature at all, run that first. Climb to Wizard of Oz once you've cleared "do they want it" and need the sharper answer. Will they use the thing, again and again, when it's finished in front of them?

The taxonomy underneath this catalog comes from Testing Business Ideas by David Bland and Alex Osterwalder. Wizard of Oz is one of the 44 they map. One rule ties the full catalog together. Find the riskiest assumption, name the question under it, and run the cheapest experiment that answers it. For a hard-to-build feature people might want, that's often this one.