Step 5 — Test & Validate — The Product Playbook

You have a problem worth solving, a hypothesis, and a few concepts you could test. Step 5 is where you find out if you're right. Not by building the real thing. By running cheap experiments in front of real customers, and watching what they actually do.

We all think we have great ideas until we get them in front of our customers. The idea felt obvious in the room. The deck was clean. Everyone nodded. Then a real person sees it and shrugs, and you learn the idea was only ever obvious to you.

That's not failure. That's the whole point of this step. You're not here to be right. You're here to find out which of your ideas a customer will actually pay for.

You're buying confidence, not proof

Get this straight before you run a single experiment: you are never going to prove your solution is the correct one. That's not what an experiment is for. There are no right answers, just confidence levels.

The goal is to raise your confidence that the solution you have will actually solve a need for the customer, enough that you'd bet engineering time on it. That's it. You're not searching for a yes. You're moving a dial. A good experiment nudges the dial up or down, and either direction is a win, because either one tells you what to do next.

So stop trying to win the experiment. Design it to tell you the truth, and let the truth land wherever it lands.

Currency is the signal

How do you know if someone actually wants the thing? You watch what they're willing to give up for it. In every experiment, you're looking for one signal: a customer handing over something that costs them. Call it currency. It takes a few forms here:

Money
Time
Email or personal information

Each one carries different weight, and the order matters. An email is cheap. People hand over an email to make you go away. Time is more. Someone sitting through a fifteen-minute call, filling out a real survey, showing up to a session, that costs them something, so it tells you more.

But money is top. The largest boost to confidence is money changing hands. If you can get someone to actually pay you during an experiment, before the thing even exists, that outweighs every email and every survey you could ever collect. People say a lot of things. People sign up for a lot of things. Paying is the one that's hard to fake.

So ladder your experiments up that hierarchy. Start cheap, an email capture, a fake-door click (a fake door is a real-looking button or page that leads nowhere, so you can count who taps it). Then climb. The closer you can get to real money, the more your confidence is worth.

The test they never got to run

Let me show you what climbing that ladder actually looks like, including what happens when you stop one rung short.

In my Intro to Design Thinking course at UIUC, students spend the semester working through the DT process on a challenge of their choosing. One semester, a team wanted to tackle hunger on campus, specifically the food deserts. For example: you have a class on one side of campus, the cafeteria is out of the way, and you can't get there and back before your next class. So you either go hungry or grab a candy bar.

The team had an idea to solve it: vending machines in those deserts that served hot, nutritious meals. Not snacks. Real food. They started testing it the cheap way, with sign-up pages. People signed up. They iterated, ran another page, and the sign-ups kept climbing. Confidence kept climbing with them. The last page they ran said "we're launching next month, sign up to be notified," and people signed up and filled out a survey.

Good signal. Real interest. But all of it was free. Email and time, no money. So the next move I was steering them toward was the climb: relaunch the page as a pre-sell, which means selling "vending machine credits" and taking real money. Tell people the machine is coming in a month. Let them lock in discounted meals by buying credit early. Start a Stripe account, the payment service that lets you take a credit card without building your own checkout. Take the payments for real, then refund everyone at the end of the test. The refund doesn't weaken the test. The moment someone pays, you have your answer. The money goes back because the machine isn't real yet.

They never got to run it. The semester ended first. But that was the test that mattered. A vending-machine company carries real overhead: machines, food, restocking, all of it paid up front. If you don't fully understand the real problem students have getting a meal, that overhead gets expensive fast, and the company loses money. Sign-ups can't tell you whether the money side works. Money can.

A survey tells you people like the idea. A credit card tells you they'll buy it. Those are not the same thing, and you only find out which side of that gap your idea is on when you ask for the money.

I ran this on my own book

I'll give you one more, because I put my own money and ego on the line with it.

When I had the idea for what became 5 Weeks to Hired, I didn't write a book. I started from a problem I knew was real, framed the same way you've been framing problems since step 1:

When applying for jobs, I want my effort to get me in front of real people, So I can land a role instead of disappearing into an applicant pile, But the process works differently now and the old playbook doesn't.

What I didn't know was the right thing to build for it. So I ran a scaled test instead, putting several product ideas in front of real traffic at once.

I built sales pages for a few different products, a course, a book, a newsletter, paid coaching, and ran ads to all of them. None of the underlying things existed. Just pages. The ads told me what people actually wanted, and out of everything, the book won.

So I climbed the ladder on the book. I outlined it, designed a cover, and launched a page where people could buy it. I hadn't written a single page. I told people it was launching in the next 30 days, and that buying early got them some free bonuses. Then I tested. Different price points. Different marketing angles. Different bonuses.

I was looking for one thing: could I get roughly 1% of the people who landed on the page to actually buy, before a word of the book existed. That's the confidence threshold I'd set.

One percent sounds small. For this kind of test, it isn't. The conversion benchmarks you see thrown around (6, 7, 8 percent) are mostly counting form fills and email signups, not purchases. Littledata analyzed about 2,800 Shopify stores and found the average store turns 1.4% of all its traffic into actual purchases, and that's a blended number propped up by warm traffic: returning customers, people searching the brand by name.

A cold ad sends a total stranger to a checkout page for something they've never heard of. If 1% of those strangers pull out a credit card for a book, real demand is there.

I even ran lean experiments on the name. I put ads behind over 50 different program and book names to see which one pulled. 5 Weeks to Hired won.

The famous one: Dropbox

You don't have to invent this approach. The textbook example is Dropbox, and it's a clean smoke test: a fake-door-style test of whether people want the thing before it exists. Before they built the product, they made a short video showing how it would work, put it on the internet, and waited to see if anyone cared. People did. They signed up to be beta users off a thing that didn't exist yet. It was smoke and mirrors, except nobody knew it, and the beta waiting list jumped from 5,000 people to 75,000 overnight. If you want to learn more, check out this article Eric Ries wrote about it for TechCrunch.

Same logic as the vending machine, same logic as my book. Get the exchange of currency, money, time, email, before you build. The video was the experiment. The signups were the signal.

The menu of experiments

There's a whole toolkit here, and the names matter less than the mechanics underneath. The standard ones are catalogued well in Testing Business Ideas by David Bland and Alex Osterwalder, all 44 of them, if you want the full menu.

One you'll hear named a lot is the Wizard of Oz test: the customer thinks they're using a finished, automated product, but behind the curtain you're doing the work by hand. It's a strong move when you already have an app or a site to test inside of, a real surface where customers expect the product to just work while you fake the engine behind it. Its close cousin is the concierge test, where you don't hide the manual part at all: you deliver the service by hand, in the open, instead of building software for it, and learn whether it's even worth automating. You'll get the full toolbox, Wizard of Oz, concierge, fake door, the prototype types, in Part 06, where you can pull whichever one fits your test. Here, stay on the principle.

The principle is the same across all of them. You're not building a small version of the product to sell. You're building the smallest thing that gets you a real answer, and most of the time it isn't software at all. It's a page, a video, a manual workaround, a Stripe link.

When the answer is no

Sometimes you climb the ladder and the dial drops. People won't pay. The survey loved you and the credit card didn't show.

When that happens, you may have to rewrite something: the hypothesis, the problem statement itself, any part of the work you did to get here. Do it. And keep it as the same card on the opportunity backlog (each opportunity lives on one card there), as long as you're still trying to solve the same problem for the customer. A pivot isn't a new card. It's the same problem, a sharper guess.

That's the move that protects you from the build trap, pouring months of engineering into something nobody validated. You learned the idea was wrong for the price of a landing page and a Stripe account, not the price of six months of engineering.

It's supposed to be hard

I won't pretend this part is fun. This process takes work, and it's punishing. The feedback can be demoralizing. You'll watch ideas you loved get rejected by the very people you built them for.

But without that feedback, you can't grow. Getting proper confidence in a solution is arduous, and the effort is worth it every time, because the alternative is shipping on a hunch and only finding out you were wrong once a customer was supposed to show up and didn't.

Run the experiment. Read the dial. Next you decide what the result actually means: build it, rework it, or let it go.