Step three: wait longer than you want to
Every test looks decisive on day two. Small numbers swing wildly, one version leaps ahead, and the temptation to call it is enormous.
Two rules that will keep you honest. Run it for at least two full weeks, so you cover the weekly rhythm. People behave differently on a Sunday evening than a Tuesday morning. And wait until each version has had a few hundred visitors before you look at all.
If your renewal traffic is small, this takes a whole cycle. That's genuinely fine. A test that takes one renewal run and gives you a real answer beats four tests that each took a week and told you nothing.
Step four: read the result honestly, including the boring one
Three outcomes, and two of them are useful.
One version clearly wins. Make it the page. Write down what you changed and what happened, somewhere the next person will find it.
They're basically level. This is a real result and people treat it as a failure. It means the thing you tested doesn't matter, which is worth knowing. You can stop arguing about button wording and go and test something structural instead.
It's close but one is slightly ahead. The honest answer is that you don't know. Either run it longer or accept it's a coin flip. Do not report a two-point difference on four hundred visitors as a finding; somebody will build a strategy on it.
Check your dates against the renewal cycle before concluding anything. A version that ran mostly during the first week, when your keenest members renew, will look better than one that ran during the chasing phase. Same page, different population.
The three tests worth running first
In this order, because each one teaches you something you need for the next.
- Strip the page back. Remove navigation and sidebars from the variant so there's one thing to do. This tells you how much you're losing to distraction.
- Lead with the amount, or lead with the value. Does a member want to see what they owe immediately, or be reminded why first? Organisations are usually confident about this and usually split down the middle when tested.
- Button wording. Cheap, fast, and the last one worth doing because it only matters once the page around it is right.
What makes this compounding rather than a one-off: each result is a fact about your members that applies elsewhere. If stripping distractions wins on the renewal page, it will probably win on your join page too.
Do this yourself, or have it done
This is a backoffice job. Engage's test setup is built for marketers, the judgement about what's worth testing is yours, and the discipline of changing one thing at a time is a habit rather than a skill.
Where we help is what testing eventually asks for: showing a different page to members inside their renewal window than to everyone else, which needs your membership data joined to your website, and making sure the goal you're measuring is a genuine completed renewal in your CRM rather than a click that looked like one. That's Maintain and Evolve. And if you've been hand-swapping page versions and comparing numbers in a spreadsheet, keep the spreadsheet. It's a record of what you've already learned, and it's more useful than starting over.