What four years of A/B testing taught us about ecommerce conversion

Most ecommerce advice is somebody’s opinion. A/B testing is the part where you find out whether it is true for your customers.

EquipOutdoors has been one of New Zealand’s top destinations for outdoor gear for over 20 years. Geoff and the team stock the best Kiwi and Aussie brands for camping, tramping and everything in between, and while there is a physical store in Hamilton, the online store does a lot of the heavy lifting.

We have been running a conversion rate optimisation programme on that store for four years, across more than 50 split tests. Here is what actually moved the numbers, what quietly backfired, and what we would tell any ecommerce owner thinking about CRO.

CRO is not a redesign

The temptation with an underperforming store is to rebuild it. Change everything, hope the numbers go up, and never really know which part did the work.

CRO is the opposite. You change one thing, split your traffic between the old version and the new one, and let real customers decide. Every test starts with a hypothesis about why people are dropping out, runs long enough to be trustworthy, and ends with a decision: keep it, kill it, or test a better version of it.

EquipOutdoors’ site was already working when we started. The question was whether it could work harder, and the only honest way to answer that is to measure.

Seven changes that lifted transactions

These are all live on the site today, and each one came out of a test that beat the existing version.

None of those is a dramatic change. Stack a few percent on top of a few percent across a year of trading and it becomes a meaningful number.

Search was the single biggest lever

If you take one thing from this case study, take this one. Every improvement we made to site search won.

That makes sense when you think about who uses search. A shopper browsing a category is still deciding. A shopper typing “merino base layer” into your search box has already decided, and the only thing standing between them and a purchase is whether your site can find the product.

For EquipOutdoors that meant upgrading the search engine itself, adding predictive query suggestions as people type (the +7.87% above), putting category filters inside the search results so shoppers can narrow down without starting again, and more recently testing an AI assistant in search for people who describe what they need rather than naming it.

Most stores treat search as plumbing. On a catalogue of any size it is a sales channel.

Social proof does a lot of the selling

Outdoor gear buyers research. They want to know that someone else took this tent camping and it held up.

Putting review stars on the product cards in category listings was worth +3.55% on its own. Pulling a featured customer review up the product page on mobile, where there is no room for anything below the fold, won as well. So did adding the brand’s Instagram feed to the footer, which is not a sales element at all, but does tell a first-time visitor that real people buy here.

The reviews you already have are the cheapest conversion asset on your site. Most stores bury them.

Mobile and desktop are two different shops

This is the one that surprises people, and it is the reason we never report a single blended result.

Showing products two across on mobile category pages lifted transactions +4.44%, because it suits thumb scrolling. A sticky navigation bar was worth +5.39% overall and noticeably more on phones, where getting back to the menu otherwise means scrolling all the way up.

Go the other way and the same pattern holds in reverse. Hover-for-more-information tooltips won on one device and lost badly on the other. Predictive bundle suggestions on the product page won on desktop and lost on mobile, because mobile shoppers want to find bundles before they have committed to a product, not be upsold once they are there. Seasonal promotional overlays on category images won on desktop and hurt on mobile, where they simply added clutter.

Had we looked only at the combined numbers, we would have shipped several changes that made things worse for most of the traffic.

The tests that lost taught us the most

Roughly a third of the tests in this programme lost. That is normal, and it is not wasted money: a losing test stops you permanently shipping something that costs you sales.

The losers had a pattern. A product comparison tool, a quick view on category pages, a swipeable image gallery and an expanded mega menu all added a decision or a step where there had not been one. Moving the product description into a sidebar to sit closer to the add to cart button sounded sensible and performed badly, because it broke the layout shoppers expected. Even adding buy-now-pay-later logos to the cart page cost conversions, by introducing a new choice at the exact moment the shopper was ready to pay.

Almost everything that lost here did so for the same reason: it asked the customer to think harder.

What this means for your store

Want to know what your site is leaving on the table?

We run CRO programmes for ecommerce brands that are already trading well and want the same traffic to produce more revenue. No rebuild, no guesswork, just a prioritised list of things worth testing and the discipline to find out which ones work.