Guide · Subscription apps
Paywall tests, in the order worth running them.
The paywall is where acquisition spend turns into revenue or does not. Most teams test the cosmetics first because those are easiest to change, and never reach the two decisions that actually move the number.
Test in this order
Roughly by how much each variable moves the outcome, largest first. If you only ever run two tests, run the first two.
| Rank | What to test | Why it comes here | Typical effort |
|---|---|---|---|
| 1 | Placement – when the paywall appears | Decides whether the user has understood any value yet. Biggest single lever. | Medium |
| 2 | Offer – trial length, payment gating | Trades trial volume against conversion rate in opposite directions. | Low |
| 3 | Plan mix and pre-selection | Moves revenue per trial rather than trial count. Durable. | Low |
| 4 | Price framing | Per-day framing, annual saving, decoy tier. | Low |
| 5 | Copy – the value proposition | Matters most when the ad promised something specific. | Low |
| 6 | Visual design | Real but smaller. Clarity beats polish. | High |
1. Placement is the decision, not a setting
Three viable positions. Immediately on first open – highest trial volume from people who already knew what they wanted, lowest from everyone else. After the first moment of value – usually the best trade, and the hardest to build, because someone has to define what that moment is. After N sessions – builds a large retargetable base and delays revenue, which suits products with slow-emerging value.
This choice usually moves trial start rate more than every subsequent test on this list combined. It is also the one most often inherited from whatever the first version of the app did.
2. The offer, meaning trial length and card gating
Requiring payment details up front dramatically raises trial-to-paid and lowers trial starts. Not requiring them does the reverse. Neither is correct in the abstract – what matters is which side of the trade produces more paying subscribers per thousand installs, and that depends on how quickly your product proves itself.
Test it as a pair with trial length, because the two interact. A seven-day trial without card details and a three-day trial with them are two coherent strategies; mixing the properties usually produces the worst of both.
3. Plan mix and pre-selection
Which plan is highlighted when the screen loads changes what most people pick, and most people do not deliberate. Moving the default from monthly to annual changes cash timing and lifetime value without changing trial count at all, which makes it one of the cheapest meaningful tests available.
4. Price framing, not price
The same $59.99 annual plan reads differently as "$59.99/year", "$4.99/month billed annually", or "less than a coffee a week". Framing tests are quick and often produce a few points of movement. Changing the actual price is a different exercise with revenue consequences well beyond the paywall.
How to know when a test has finished
Decide the sample size before you start, and write it down. Roughly 300–400 conversions per variant is a workable floor for the differences that matter here; below that you are reading noise with a nice interface.
Also decide the end date in advance. Almost every disputed test result in this field comes from someone stopping early when it looked good, or extending until it did.
Questions people actually ask
- How much traffic does a paywall test need?
- Enough conversions, not enough users. At a 20% trial start rate you need roughly five times the traffic to get the same number of events as at 100%. As a rough floor, aim for 300–400 conversions per variant before reading a result, and expect two to three weeks at typical volumes.
- Can I test more than one thing at once?
- Only if you are willing to accept "this whole variant won" as the answer, without knowing which change caused it. That is often fine early on when you are looking for a big move. It stops being fine when you want to build a model of what works.
- What if traffic is too low to test at all?
- Then do not test – decide. Pick the variant your judgement and category norms favour, ship it, and spend the effort on getting more traffic instead. A test you cannot power is worse than no test, because it produces a confident wrong answer.
- Does paywall design matter more than copy?
- Copy and offer structure usually move the number more. Visual design matters mostly through clarity – whether a person can tell in two seconds what they get and what they are charged. Beautiful paywalls that hide the price underperform plain ones that state it.
- Should the paywall differ by acquisition source?
- Often, yes, and few apps do it. Someone arriving from a specific search query has a specific problem; someone from a broad social campaign may not know what the app is yet. Remote paywall configuration makes this cheap to try.
What the paywall does to your acquisition ceiling.
Trial-to-paid is the input the whole model is most sensitive to. The CPI calculator shows how much a few points of paywall improvement changes what you can afford to bid.