How to Validate a Feature by Shipping a Small Version (2026)

A customer has asked for a specific feature, and your team is about to spend weeks on it. One option is interviews, mockups and click-through tests. The other is building the full feature on faith. This guide describes a third option, the Ship First loop, developed by Spencer Shulem, cofounder and CEO of BuildBetter. You build the smallest working version, give it to the person who asked, and decide from what they actually do. Below are the eight steps, a cost comparison, a worked example, and the cases where you should not ship first.

What it means to validate a feature by shipping a small working version

Validating a feature by shipping means building the smallest version that actually performs the requested job, giving it to the person who asked, and watching them use it on their own information at the next real occasion. The team then decides, by a date agreed in advance, whether to expand, revise or retire it.

This differs from opinion-based validation in what it measures:

• A mockup, survey or interview tests whether someone likes an idea. That is stated preference.
• A working version tests whether the solution does the job better than the workaround they already have. That is revealed behavior.

The method works because a customer can sincerely request a feature and still leave it unused. They aren't lying. Requests describe a frustration, and the spreadsheet, the copy-paste routine or the colleague who handles it is often good enough to survive a new feature. Real use shows you this while the investment is still small.

Expert teams are often wrong about what users will value. In Trustworthy Online Controlled Experiments, Ron Kohavi, Diane Tang and Ya Xu report that at Microsoft roughly a third of tested ideas improved the target metric, a third had no significant effect and a third made it worse. Those were large-scale A/B tests, not single-requester B2B experiments, but the lesson carries over: predictions are weak evidence.

In 2026 this approach is practical for more teams. AI-assisted development has made small working releases feasible for more of the work teams encounter. Product managers, researchers, designers and support leads can now contribute directly to software instead of handing everything to engineering.

Why mockups, surveys and big builds fail as validation

Most feature validation fails in predictable ways, and each failure mode has a specific cause.

1. Stated preference is not use. People approve a concept in an interview and then keep their spreadsheet workaround. Hypothetical questions like "Would you use this?" produce false positives. Both Rob Fitzpatrick's The Mom Test and Teresa Torres's discovery work warn about this.
2. The buyer is not the user. The ops lead who asked for an export may never run it. What a purchaser prefers and how the actual user works can be different things.
3. Misread problem. The team interpreted the request poorly, so even a well-built feature misses.
4. Delay kills memory. By the time a researched, designed and built feature arrives weeks later, the customer no longer remembers the exact occasion that triggered the request.
5. Building on faith. If you ship the full feature without a written assumption, disappointment quietly turns into another month of work. Nobody agreed on what failure would look like, so nobody calls it.
6. The temporary feature that never leaves. An experiment with no decision date becomes a permanent feature with a temporary label, and you keep paying to maintain it.

None of this means research is waste. Research has value because decisions have consequences. The real question is how to get useful evidence at a reasonable price, and sometimes a working release costs less than a study.

Compare the price of an answer before you pick a test

The release route and the research route both have full costs, and you should compare them before choosing a test. Teams often count the engineering cost of shipping and ignore the calendar cost of research, or the reverse.

Consider an export request. The research route means recruiting participants, running sessions, building a simulation and revising it. The release route means building a working export and watching the requester use it on the report they actually need to send. If the export reuses existing data and permissions, the second route is often cheaper and gives a stronger answer.

Your existing system sets the price. Permissions, document fidelity and downstream commitments can make a small-looking feature expensive. An "export" that touches a new permission model is not a two-day job.

Be skeptical of general claims about AI speed. In METR's early 2025 randomized trial, experienced open-source developers took 19% longer on real tasks when using AI tools, even though they expected a 24% speedup and afterward believed they had been about 20% faster. METR's February 2026 update reported results suggesting speedup, but METR cautioned that selection effects limit conclusions about its size. Measure your own costs:

• Request-to-release time for small changes
• Review burden per change
• Defects introduced by small releases
• Support cost of helping requesters adopt them

If a feature creates an enduring obligation, such as a contract term, a public API or a migration, include that obligation in the price.

The Ship First loop: 8 steps to validate a feature with real use

The Ship First loop is an eight-step method for validating one customer request through real use, with a dated decision to expand, revise or retire. It comes from Ship First: How to Build Self-Improving Products by Spencer Shulem, cofounder and CEO of BuildBetter, a short companion to Customer-Led Development. Shulem writes that at BuildBetter the team creates over 100 pull requests per engineer per week and deploys to production three to six times a day. Shulem presents this as the reason the loop matters, not as a benchmark you should match.

The method needs no special tool. It runs on a document, a feature flag or private preview, and a calendar reminder.

Step 1: Keep the request and the person attached

Capture the request in the customer's own words. Record the most recent occasion it came up, their current workaround and the outcome they wanted. If a buyer or admin asked, find the person who will actually use the feature.

Step 2: Make a product decision

Ask three questions. Does this job belong in the product? Is it useful to the customers you intend to serve? Can it fit the existing experience without turning a shared product into a set of one-off exceptions? A clear no is often the fastest useful response.

Step 3: Choose the test by the uncertainty

Use grounded personas, a targeted survey or a prototype when one of those resolves the question within budget. Ship when real use is the most useful next test. Not every idea needs the same sequence of steps.

Step 4: Write an observable assumption

Too broad: "Customers want reporting." Observable: "The requesting team will use this report for its next weekly review instead of compiling it manually." A good assumption names a person, an upcoming occasion and a behavior that replaces the workaround.

Step 5: Set the experiment's terms

Before release, write down:

• Who gets access
• The behavior that would support the assumption
• The next realistic occasion
• The decision date
• What triggers expansion, revision or retirement
• What you will not build
• The rollback route
• The owner

The criteria don't have to be numbers, but they must be unambiguous.

Step 6: Build the smallest version that performs the job

The version needs correct permissions, understandable output and acceptable reliability, and it should pass normal code review. Instrument the meaningful action before release. If the version fails at basic execution, nonuse tells you nothing.

Step 7: Release, tell and help

Limit exposure with a feature flag or private preview. Tell users what works, what is limited, and that this is an experiment. Never quietly put an irreversible change into someone's workflow. Prepare documentation, a short video or an onboarding call. Tell the requester personally, connect the release to their original problem, and help them reach the first meaningful result. For how to send that message, see how to close the loop with customers when you ship a feature. Record any help you gave, so you can tell guided use from independent use later.

Step 8: Observe, then decide on the agreed date

Watch the next occasion. Did they start the job, finish it and use the result? At the occasion after that, check whether they came back, and ask what work they still do elsewhere. Then make one of three decisions: expand, fix an obstacle, or stop investing. Extend the observation only if the customer hasn't had a fair chance to use the feature. Whatever you decide, record what you learned and what it changes.

Ship First loop checklist

1. Keep the request and the requester attached.
2. Decide whether the job belongs in the product.
3. Choose the test that resolves the current uncertainty.
4. Write an observable assumption: person, occasion, behavior.
5. Set terms: access, evidence, occasion, decision date, triggers, exclusions, rollback, owner.
6. Build the smallest version that completes the job, with the key action instrumented.
7. Release behind a flag, tell the requester, help them, and log the help.
8. Observe one or two occasions, then expand, revise or retire on the agreed date.

When a mockup or a conversation is still the right test

Pick the activity that resolves the uncertainty in front of you. Shipping is one test among several. It is the best one only when the open question is about actual use. This matches Teresa Torres's guidance on assumption testing: test the riskiest assumption, not the whole idea.

Worked example: one request, two cycles, one keep-or-remove decision

This example is illustrative. The company, account and numbers are invented to show the mechanics. They are not real customer data.

Setup. A B2B operations app. The ops lead at a 40-seat account asks for a "weekly report export." The team interviews them about the last time they needed it, and the real job comes out. Every Monday, a team coordinator spends about 90 minutes copying task status into a spreadsheet for a 10am review. The coordinator is the real user, not the ops lead who asked.

Price comparison. The research route: recruit six customers, run sessions and test a clickable mockup. That takes roughly three weeks before any help arrives. The release route: the export reuses existing report permissions and needs about two days of build, half a day of review and a 30-minute onboarding call. The team ships.

Cycle 1 terms.

• Assumption: "The coordinator will use the export for next Monday's review instead of compiling manually."
• Access: flag on for one account, three users.
• Out of scope: scheduling, PDF formatting.
• Decision date: the Tuesday after the second Monday.

Cycle 1 observation. With guidance on the onboarding call, the coordinator starts and completes the export on Monday. Then they reopen their old spreadsheet to add "owner" and "due date" columns, which still takes about 25 minutes of manual work. Decision: revise. The team adds the two columns. There is no reason to expand yet.

Cycle 2 observation. The next Monday, the coordinator exports on their own with no help. Manual work drops to about 10 minutes, and they come back the following week without prompting. Asked what they still do elsewhere, they say: "Just a quick note at the top."

Decision: keep. The team expands to five accounts with the same Monday-review workflow, adds a short help article and sets a new decision date. The removal criterion is written down in advance: if fewer than three of the five accounts use the export for a real review within two weeks of onboarding, it gets retired to the existing report screen.

The counterfactual remove path. Suppose the coordinator had ignored the working export in cycle 2. The team would have retired it and recorded that the real need was something else, such as a shared live dashboard. That lesson would have cost three days instead of a quarter.

Is shipping first the same as an MVP?

No. A minimum viable product usually tests whether a product or market exists for a broad audience. The Ship First loop tests one specific job for one real requester, inside an existing product, at their next real occasion, with a dated decision.

The main differences:

• Unit: a single request, not a whole product.
• Requester: the person who asked stays attached throughout.
• Context: the version ships into a live workflow and runs on the customer's own data.
• Decision: expand, revise or retire is agreed before release.

They share two things. Both value learning from real behavior over opinion, and both require the version to actually work. For the broader concept, see what an MVP is.

How small is small enough?

Small enough to ship within the experiment's appetite, but complete enough that the job can be done end to end. In practice that means correct permissions, understandable output and acceptable reliability. Ryan Singer's idea of appetite in Shape Up helps here. Decide what the answer is worth first, then shape a version that fits that budget.

Use the interpretability test. If the version failed, would nonuse tell you something about the solution, or only about the bugs? If it would only tell you about the bugs, the version is too small.

Cut scope by listing what you won't build, such as scheduling, formatting variants or support for other segments. Never cut the core path.

Watch for one warning sign. If the "small" change turns out to need a major migration or a new security model, reconsider the test. Don't pretend the budget still holds.

When not to ship first

Ship first only when a working release is the cheapest reversible way to answer whether people will use the feature. Skip it when:

• The job doesn't belong in your product. Say no.
• The change is irreversible or creates an obligation you can't retire, such as a data migration or a contractual commitment.
• Permissions, compliance or document fidelity make the small version expensive or risky.
• You can't reach the requester, or there is no realistic upcoming occasion to observe.
• The open question concerns scope, interaction or problem understanding rather than use. A sketch or conversation answers those more cheaply.
• You can't roll back or limit exposure with a flag or private preview.

Common mistakes when validating features by shipping

Most failed validations trace back to a handful of avoidable mistakes.

• Detaching the request from the person. The ticket says "export," but nobody knows who asked or what triggered it, so nobody can observe the right occasion.
• Treating the buyer's request as the user's workflow. You build for the person who signs the contract and miss the person who does the work.
• Writing an unobservable assumption. "Users want X" can never be confirmed or refuted.
• Skipping the decision date or retirement criteria. The experiment turns into a permanent feature by default.
• Shipping something too broken to interpret. Nonuse then reflects bugs, not the value of the solution.
• Shipping silently. If nobody tells the requester, nonuse only means they didn't know the feature existed.
• Counting guided use as independent use. Use during an onboarding call shows the feature can work. It doesn't show the customer will use it alone.
• Measuring clicks instead of the job. The real questions are whether they completed the job, used the result, and what they still do elsewhere.
• Extending observation forever. Delay protects feelings and wastes the cycle.
• Skipping the cost review. After several cycles, check where the time went. Customer access, specs, release safety, analytics or work in progress may be the bottleneck before coding capacity is.

When you need tooling (and which tools help)

The loop runs manually with a doc, a flag and a calendar, and that's enough for most teams to start. Manual capture breaks down past a handful of concurrent experiments. It also breaks down once requests arrive through calls, Slack and tickets faster than one person can keep each one attached to its requester, which usually happens when you're handling hundreds of pieces of feedback a month. Observation breaks down when you can't see whether the job was completed without asking every time.

1. BuildBetter: keep requests attached to people and conversations

BuildBetter keeps each request linked to the person and conversation it came from, across call recordings, Slack, support tickets and surveys. It also drafts the specs, Linear or Jira tickets and loop-closure messages for the release. One caveat: it is not a session-replay or feature-flag tool, so pair it with one.

2. Unleash or Flagsmith (open-source feature flags)

These limit exposure to the agreed group and make rollback a single switch.

3. GrowthBook (open-source flags and experimentation)

Useful once you move from a single requester to a defined cohort.

4. OpenReplay (open-source session replay)

Lets you see whether the requester actually completed the job, without asking them.

Tools reduce bookkeeping. They don't make the product decision or write the assumption for you.

FAQ: validating a feature with a small working version

What is the Ship First loop?

It is a validation method from Spencer Shulem's book Ship First: How to Build Self-Improving Products. You take one real request and keep the requester attached. You build the smallest working version and put it in their workflow with their own information. Then you observe the next real occasion and decide by an agreed date whether to expand, revise or retire the feature.

Is shipping a feature better than a mockup for validation?

Only when the remaining uncertainty is about actual use: will they use it instead of their workaround? Mockups and conversations are cheaper for other questions, such as whether the job belongs in your product, how the workflow works, or what the scope and interaction should be.

Is shipping first the same as an MVP?

No. An MVP usually tests whether a product or market exists for a broad audience. The Ship First loop tests one job for one real requester, inside an existing product, using their own data at their next real occasion, with a pre-agreed decision. Both value real behavior over opinion, and both require the version to work.

How long should a feature validation experiment run?

Until the requester's next one or two real occasions for the job, for example the next two weekly reviews. Set the decision date before release. Extend only if they haven't had a fair chance to use it, and never just to avoid a disappointing decision.

What counts as evidence that a feature is validated?

The requester completes the job independently with the feature, uses the result, and returns to it at the next occasion without prompting. Asking what work they still do elsewhere shows how complete the solution is. Clicks or page views alone are not evidence.

Do I need AI coding tools to ship first?

No. AI makes small working releases practical more often, but the loop depends on measuring your own request-to-release time, review burden and support cost. Ship first wherever that cost is lower than the research route.

What should happen to a feature that fails validation?

Retire it, record what you learned and what it changes, and tell the requester. An experiment left in place becomes a permanent feature with a temporary label.

Sources and further reading

• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
• METR, We Are Changing Our Developer Productivity Experiment Design (February 2026)
• Ryan Singer, Shape Up, Chapter 3: Set Boundaries
• Teresa Torres, Product Talk, Assumption Testing
• Wikipedia, Feature toggle
• Wikipedia, Minimum viable product
• Ron Kohavi, Diane Tang, Ya Xu, Trustworthy Online Controlled Experiments (Cambridge University Press, 2020)

Make churn optional.

Every Ship First cycle starts with a request you can trace back to a real person and a real occasion. BuildBetter captures those requests from calls, Slack, tickets and surveys, keeps them tied to their source, and turns them into specs, tickets and follow-ups when you ship. Make churn optional. Book a demo.

Get Ship First

This method comes from Ship First: How to Build Self-Improving Products by Spencer Shulem: 24 short chapters on going from a customer request to a working release. Get the free PDF or EPUB, or read it on Kindle.