No pop-up format wins everywhere. Across the brands we work with, a quiz beat a standard pop-up at one brand. A scratch-off beat a quiz at another. Gamification hurt signups at a third. Three things mattered more than format: who signed up, what the offer trained them to expect, and whether the pop-up got in the way on a phone.
So the scoreboard matters more than the format. Signup rate is the number every pop-up tool shows you first, and it’s the easiest one to inflate. The tests that told us something were judged on revenue per visitor and blended CAC. Below is what those tests showed, where mobile friction quietly cost more than any offer, and how to run a pop-up test you can trust.
Key Takeaways
- In a test with about 77,000 views per arm, a scratch-off pop-up beat a zero-party-data quiz by 11% on signups and 19% on revenue.
- A book-and-coffee box swapped its email platform’s default pop-up for a quiz. It got 90 welcome conversions in under a week, against 220 in the previous year.
- Gamification isn’t a sure thing. A grooming brand saw signups fall with it, and its best result so far is a plain 10% offer on first visit.
- Giveaway and sweepstakes signups grow the list, not revenue. A 3,000-profile test send to one brand’s giveaway list got clicks and zero purchases.
- Fix mobile timing first. Pop-ups that fired after add-to-cart, or 15 seconds into a product page, caused rage clicks and blocked carts.
What did pop-up tests across brands actually show?
Every format won somewhere and lost somewhere. Here’s what we saw from our client and discovery calls between mid-2025 and early September 2026. Client results are specific outcomes, not typical or guaranteed.
| Test | Brand type | Result | Caveat |
|---|---|---|---|
| Default ESP pop-up → quiz pop-up (product match, 10% off) | Book-and-coffee subscription box | 90 welcome conversions in under a week vs 220 in the prior 12 months; email capture about doubled; blended CAC down about 10% | Not a split test; before-and-after |
| Quiz pop-up vs scratch-off pop-up | Edibles brand, heavy paid traffic | Scratch-off: +11% signups, +19% revenue | Split test, about 77,000 views per arm |
| Spin-to-win wheel | Dog subscription box | ”Those emails are terrible,” per the team | Qualitative; low-intent signups |
| Gamified pop-up vs plain offer | Men’s grooming brand | Gamification hurt signups; 10% off on first visit best so far | Signup rate only |
| ”Who are you shopping for?” profile builder | Children’s products brand | Very low response, even with an incentive | Response rate only |
| Newsletter-style content vs promo-specific signup offer | Hardgoods brand | Newsletter-style content converted much better | Directional |
Three patterns hold across the table.
The default pop-up is usually the weakest option. The book-and-coffee box had used its email platform’s built-in pop-up for a year. A quiz that matched visitors to a product, with 10% off, outperformed it immediately. Its box sold out with about 30 left, where the team had expected a couple of hundred unsold.
Questions cost signups. Every extra step is a place to drop off. The children’s brand barely got answers to its profile builder. At the edibles brand, the quiz lost to a one-tap scratch-off. The team also noticed a gap between what people said in the quiz and what they bought.
Games attract players. The dog box’s verdict on spin-to-win signups was blunt. Those visitors came for the wheel, not the product. The scratch-off win shows a game mechanic can work. But it worked as a fast way to reveal an offer, not as entertainment.
Why signup rate is the wrong scoreboard
A pop-up’s job is to lower what a customer costs you, not to grow a list. Signup rate is a means. The end is first orders from signups, at a contribution margin you can live with.
Benchmarks for signup rate do exist. Klaviyo puts its general benchmark submit rate at 3%. A Klaviyo strategist describes 3–8% as typical for standard pop-ups and 10–13% for exit-intent forms (Klaviyo, August 2026). Use those to spot a broken form, not to declare a winner.
Here’s why a higher signup rate can still lose:
- It can cost site conversion. Your funnel conversion rate is the number a pop-up most often hurts. One brand saw its list grow after installing a new quiz pop-up while site conversion “tanked.” It moved the pop-up to exit intent.
- It can buy the wrong people. One brand sits on about 150,000 emails from giveaways. A 3,000-profile test send got clicks and no purchases. A book-and-coffee box saw sweepstakes and bot signups spike its bounces. Those lists also cost money. Klaviyo bills on active profiles, so importing the 150,000-profile list as active produced a $2,500 bill.
- It can teach a discount habit. Every pop-up offer is your new customer’s first price. We showed how offer depth shapes cohorts in discount depth and cohort quality. Our working guidance is to keep a first-order offer under about 25%. Beyond that, we see more buyers who don’t return and more refunds and chargebacks.
There’s a selection trap in the other direction too. A skincare brand found quiz completers spend about 2.5× more over 12 months. That doesn’t mean the quiz created the value. People willing to finish a quiz were already more committed. Don’t credit the format for the visitor.
Raffles fit this logic. A book-and-coffee box refunds every 10th order during a promo window. Each run brings in 20–30 new subscribers, and its September run was its best yet. That’s a raffle aimed at people already at checkout, which is why it works. A giveaway aimed at strangers fills a list with people who want free things.
What to measure instead
Judge every pop-up variant on these, in this order:
| Metric | How to calculate | Why it matters |
|---|---|---|
| Revenue per visitor exposed | Revenue from everyone shown the variant ÷ visitors shown it | Catches pop-ups that win signups but cost orders |
| Signup-to-first-order rate | Signups who order within your window ÷ signups | Separates buyers from collectors |
| Discount cost per first order | Average discount × first orders from signups | Shows what the offer really costs |
| Blended CAC | Total marketing spend ÷ new customers | The number that has to move |
| Signup rate | Submits ÷ form views | Diagnostic only; spots a broken form |
Revenue per visitor is the one most teams skip, and it’s the one the edibles brand used. The scratch-off won on signups and on revenue. If it had won only on signups, it wouldn’t have been a win. For the break-even math behind the CAC line, see break-even CAC for small DTC brands.
Mobile friction costs more than any offer test
Most pop-up damage we’ve seen came from timing and placement on phones, not from the offer. It’s also the cheapest problem to fix. Here’s what showed up on our calls:
- A full-screen mobile pop-up at the book-and-coffee box produced rage clicks.
- At a vinyl subscription club, the close button on mobile confused visitors.
- At another brand, a pop-up appeared after add-to-cart on mobile. Session replays showed rage clicks and rage quits. It moved to exit intent only.
- A pop-up that fired 15 seconds into a product page blocked carts at a subscription health brand.
- At a beverage brand, the pop-up fired before the visitor had scrolled at all. Another brand settled on a 30-second delay plus exit intent.
- The vinyl club’s prepaid-upsell pop-up broke and was switched off. New-member conversion “shot through the roof.” Its “$170 in free gifts” message had read as a scam.
There’s a search cost too. Google said in August 2016 that pages showing “a popup that covers the main content” to mobile searchers may rank lower, from January 2017 (Google Search Central). Legally required banners and reasonably sized, easily dismissed banners were exempt.
The current practitioner default is similar. The Klaviyo strategist recommends against showing forms on page load, suggesting a 5-second delay combined with a 20% scroll. She also suggests excluding pages like checkout (Klaviyo, August 2026). Our own rule is simpler: never interrupt someone who has just added to cart.
How to run a pop-up test you can trust
Change one thing, split by views, and judge on revenue. Most pop-up “tests” we review fail one of those three.
- Baseline first. A dog subscription box with no site capture is starting with its email platform’s native form and a simple “mystery offer” test. Our advice was to run a few months of baseline before paying for a pop-up tool. You can’t measure a lift without a stable before.
- Test one variable. Format or offer or timing, never all three. The book-and-coffee box changed format and offer together, so it can’t say which one did the work.
- Split by views, and wait for volume. The edibles brand’s test had about 77,000 views per arm. A small store won’t reach that quickly. Run long enough to see first orders, not just signups.
- Segment by traffic source. Visitors from Google searches usually already want the product. They often need a softer incentive than cold traffic from Meta. Use UTMs to show different offers.
- Ask one question, at most. If you want zero-party data, ask one question in the pop-up and the rest after purchase. For the children’s brand we suggested store credit instead of a discount, since buyers tend to perceive it slightly better.
- Check the plumbing. At one brand, 1,115 pop-up signups landed in Shopify as “not subscribed,” so they couldn’t be emailed. Submit your own form before you read any results. Our silent flow failures audit covers the full check.
One more measurement note: judge welcome-flow results on clicks and orders rather than opens. Email platform attribution can credit email for orders it didn’t drive. We covered that in Klaviyo click-only attribution.
What to do this week
- Open your store on your phone in a private window. Note when the pop-up fires, what it covers and how easy it is to close. If it appears after add-to-cart or before a scroll, fix that first.
- Pull last month’s signups by form. For each one, count how many placed a first order within 30 days. If you only know submit rates, you don’t yet know which form works.
- Calculate revenue per visitor exposed. Do it for your current pop-up, so your next test has a baseline that isn’t signup rate.
- Audit your offer. If your pop-up discount is over about 25%, or stacks with another code, write down what it costs per first order.
- Pick one test. Format, offer or timing. Write down the metric that decides it before you launch.
The pop-up is the first price you show
A pop-up is the first conversation you have with most future customers. It sets their first price and the kind of buyer you attract. It also decides whether the next email ever reaches them. Test it like a pricing decision, not a design tweak. Then make sure the second and third orders are ready for the people it brings in.