Section 1
Does your catalogue justify a model at all
The value of a recommender scales with the difficulty of finding things. A store with sixty products has no discovery problem. A well-organised category page and a few hand-picked pairings will outperform any model, and they never embarrass you. Models start earning their cost when the catalogue is large enough that no shopper can see all of it, when items vary in a way that makes the right match non-obvious, and when you have enough traffic that behaviour patterns are stable rather than anecdotal. Below that threshold, spend the effort on merchandising rules written by whoever knows the products best. Above it, a model finds combinations no merchandiser would have guessed. The market-sizing discipline behind this kind of judgement call is covered in [AI-Powered Market Research for Startups](/blog/ai-powered-market-research-for-startups).
Section 2
The three recommendation jobs are not interchangeable
Similar items, shown on a product page, help a shopper who has not decided. This is the easiest to build and the least valuable, since it often delays a purchase they were already making. Complements, shown at cart or after purchase, add to the basket. This is where most incremental revenue actually comes from, and it works with simple methods: what do people who bought this also buy, filtered by a human so the pairings make sense. Personal recommendations, on a home page or in email, are the hardest. They need enough history per shopper to say anything, which most stores do not have outside their repeat buyers. Build in that order, the sequencing argued for in [AI Automation in Product Development: From Idea to Launch](/blog/ai-automation-in-product-development-from-idea-to-launch). Teams that begin with the personalised home page build the hardest thing first and abandon it.
Section 3
Cold start and the popularity trap
Two structural problems shape every recommender. Cold start: new products have no interaction history, so the system cannot recommend them, so they get none. New launches quietly starve. The fix: reserve slots for new items, and use product attributes rather than behaviour until history accumulates. Popularity bias: the model recommends what sells, which makes it sell more, which reinforces the recommendation. Within months your recommendations are a bestseller list with extra steps, and the long tail of the catalogue is invisible. Both are correctable, but only if someone is watching. Track the share of your catalogue that ever appears in a recommendation. If that number is small and falling, the system is narrowing your store.
Section 4
The holdout is the only honest measurement
Attributed revenue from recommendations is the most inflated number in e-commerce reporting. It counts every purchase that followed a click, including the ones that were always going to happen. Run a holdout. Randomly select five or ten percent of shoppers who never see recommendations, and compare revenue per session, basket size, and repeat rate against everyone else. The difference is what the system is worth. It will be lower than the attributed figure, and it is the number you can defend in a budget conversation. Keep the holdout running permanently rather than for one experiment. Recommenders decay as catalogues and seasons change, and a permanent control group is the only thing that shows you the decay while it is happening.
Section 5
Where recommendations damage trust
The failure modes are specific and avoidable. Recommending the item just purchased, which tells the shopper the store is not paying attention. Recommending across sensitive categories, where an inference about health, finances, or family circumstances arrives in front of whoever is holding the phone. Recommending out-of-stock items. Recommending in a way that exposes what someone bought on a shared device or in an email preview. Hard rules solve most of it, written before the model launches: exclusion lists, category boundaries no recommendation may cross, a stock check at render time. Set them as constraints the system cannot override, since a model optimizing for clicks will find these edges itself.
Section 6
The measurements that matter
Click-through on a recommendation widget is a diagnostic, not a result. Judge the system on revenue per session, average items per order, catalogue coverage, and repeat purchase rate, each compared against the holdout. Add two guardrail metrics: return rate, since recommendations that push poorly matched items show up as returns weeks later, and unsubscribe rate on recommendation emails. You are ready for an engine if your catalogue is large, your traffic is steady, and you will hold out a control group. You are not ready if the catalogue is small, in which case editorial pairings will beat a model and cost nothing to run. Presenting either result convincingly is a communication problem, discussed in [Storytelling in the Age of AI and Automation](/blog/storytelling-in-the-age-of-ai-and-automation).