The E-Commerce ML Playbook — Algorithms That Actually Move Metrics

I work in e-commerce analytics, and here's what frustrates me about most "ML in e-commerce" content: it's either a vendor listicle that names zero algorithms, or an academic paper about one narrow problem. Neither answers the question a working professional actually has:

"Which ML systems actually move business metrics, which specific algorithms run them in production today, and what should I learn first?"

This post answers exactly that. Every section follows the same structure: the business metric at stake → the problem → what actually runs in production → why that algorithm wins → what to learn. No hype, and — importantly — honest notes on where deep learning is not worth it.

1. The Metric-to-Model Map

E-commerce runs on a small set of metrics. Every ML system exists to move one of them. Start with the metric, not the model — this table is the whole post in one view:

2. Search & Ranking — Where the Money Is Highest ROI

The business problem

30–50% of e-commerce revenue flows through the search box. A user types "running shoes" and you have 200ms to order 50,000 candidates so that the thing they'll buy sits in the top 4 positions — because position 1 gets ~10x the clicks of position 10, and most mobile users never scroll past the first screen.

What actually runs in production

Modern e-commerce search is a two-stage cascade, and understanding this architecture matters more than any single algorithm:

1. Retrieval (candidates): BM25 keyword matching (still! it's fast and hard to beat) plus two-tower embedding retrieval — one neural tower encodes the query, another encodes products, and approximate nearest neighbor search (HNSW / FAISS) finds semantic matches so "comfy sneakers for jogging" finds running shoes without sharing a single keyword.
2. Ranking (ordering): a learning-to-rank model re-orders the top ~1,000 candidates using hundreds of features — text relevance, price competitiveness, historical CTR, conversion rate, rating, delivery speed, margin.

And here's the part vendors won't tell you: the ranking stage in most major e-commerce companies is still LambdaMART — gradient-boosted trees optimizing NDCG directly — usually via LightGBM's lambdarank objective. A 2025 industry study from OTTO (major German e-tailer) found their production LightGBM ranker was only recently matched by a deep model, a decade after deep learning "won" everything else.

# The production workhorse of e-commerce search, in ~10 lines
import lightgbm as lgb

ranker = lgb.LGBMRanker(
    objective="lambdarank",      # optimizes NDCG directly
    metric="ndcg",
    ndcg_eval_at=[4, 10],          # positions that matter on mobile
    n_estimators=500,
    learning_rate=0.05,
)

# X: features per (query, product) pair — relevance scores, CTR,
# price rank, rating, delivery speed...
# y: graded relevance (0=ignored, 1=click, 2=add-to-cart, 3=purchase)
# group: number of products per query (ranking is per-query!)
ranker.fit(X_train, y_train, group=query_groups)python

What to learn

• Metrics: NDCG, MRR — and why NDCG@4 matters more than NDCG@100 on mobile
• Models: LightGBM lambdarank, then two-tower architecture conceptually
• Concepts: retrieval vs ranking split, position bias, query understanding

3. Recommendations — The Second Storefront GMV Driver

The business problem

"Customers also bought", "Similar items", homepage feeds. McKinsey's much-quoted estimate attributes ~35% of Amazon's revenue to recommendations. The economics: recs increase basket size (AOV) and surface the long tail that search never shows.

What actually runs in production

# A production-credible recommender baseline in one afternoon
import implicit
from scipy.sparse import csr_matrix

# rows = users, cols = items, values = interaction strength
# (view=1, cart=3, purchase=5 is a common weighting)
user_items = csr_matrix((weights, (user_ids, item_ids)))

model = implicit.als.AlternatingLeastSquares(factors=64, iterations=20)
model.fit(user_items)

# Top-10 personalized recs for a user, excluding already-bought
ids, scores = model.recommend(user_id, user_items[user_id], N=10)python

What to learn

• Concepts: implicit vs explicit feedback, cold start, exploration vs exploitation
• Models: ALS (build one!), item2vec, then read the two-tower paper (YouTube 2019)
• Metrics: recall@k for retrieval, NDCG for final ranking — and why offline metrics routinely lie (see section 8)

4. Demand Forecasting — The Unsexy Giant Working Capital

The business problem

Every unit you overstock is trapped cash plus storage cost plus eventual markdown. Every unit you understock is a lost sale plus a customer who tried a competitor. Forecasting drives purchasing, replenishment, warehouse allocation, and markdown timing. It is the least glamorous and most financially consequential ML in retail — this is where inventory provisions and NRV write-downs live or die.

What actually runs in production

Here's the honest answer the "AI trends" posts won't give you: the M5 forecasting competition (Walmart data, 30k+ series) was won by LightGBM with engineered features, and that's still what most retailers run. Not LSTMs, not transformers — gradient boosting over lag features:

• Lags: sales 7/14/28 days ago (same weekday matters)
• Rolling stats: 7/28-day mean, std, min, max
• Calendar: day-of-week, payday, Ramadan/Diwali/White Friday, school holidays
• Business state: price vs base price, promo flag, stockout flag (crucial — a zero-sales day during stockout is not zero demand!)

The single most important upgrade over "predict the average": quantile loss. Inventory decisions don't need the expected demand — they need "the level that covers demand 95% of the time," because the cost of understocking and overstocking is asymmetric:

# Forecast the 95th percentile of demand, not the mean —
# because a stockout costs more than excess stock
import lightgbm as lgb

p95_model = lgb.LGBMRegressor(
    objective="quantile",
    alpha=0.95,          # service-level target
    n_estimators=800,
)
p95_model.fit(X_train, y_train)

# safety_stock = p95_forecast - p50_forecastpython

What to learn

• Concepts: quantile/pinball loss, service levels, hierarchical reconciliation (SKU forecasts must sum to category forecasts), censored demand (stockouts hide true demand)
• Models: LightGBM with lag features; the M5 competition write-ups are the single best free education in applied forecasting
• Metrics: WAPE (not MAPE — MAPE explodes on low-volume SKUs), bias tracking

5. Pricing, Promotions & Uplift — The Margin Lever Most Underrated

The business problem

A 1% improvement in price realization is worth more to the P&L than a 1% traffic increase — with no extra marketing spend. But most companies burn promo budgets on customers who would have bought anyway. This section is the highest-leverage, least-understood ML in e-commerce.

What actually runs in production

Price elasticity

The foundation: how does demand respond to price? Log-log regression gives you elasticity directly (the price coefficient is the elasticity), with gradient boosting for the non-linear version. The hard part isn't the model — it's confounding: prices historically dropped during promos, which came with banners and emails, so the naive model attributes the banner's effect to price. This is why pricing teams increasingly borrow causal inference tools (double ML, instrumental variables).

Uplift modeling — the most underrated skill in e-commerce

A propensity model answers "will this customer buy?" An uplift model answers "will this discount change whether they buy?" — completely different question, and the one that actually decides where promo money goes. Segment your customers by treatment response:

The workhorse implementations are refreshingly simple: T-learner (two gradient boosting models — one on treated customers, one on control — uplift = difference in predictions) and X-learner for imbalanced treatment groups. Libraries: causalml (Uber) or EconML (Microsoft). The prerequisite is a randomized holdout in your past campaigns — if your company doesn't keep one, championing it might be the single most valuable thing you do this year.

Bandits for promo allocation

When you can't wait weeks for an A/B test (flash sales, White Friday), Thompson sampling shifts traffic toward winning variants as evidence accumulates. Vastly simpler than reinforcement learning and covers 95% of the "adaptive" use cases in practice.

What to learn

• Concepts: elasticity, confounding, incrementality, cannibalization (your discounted SKU steals from full-price neighbors)
• Models: T-learner uplift with LightGBM, Thompson sampling (implementable in 20 lines)
• Key mental shift: from "predicting outcomes" to "estimating treatment effects" — correlation-grade ML spends money badly here

6. Customer Models — LTV, Churn, Propensity CAC Efficiency

The business problem

You can't spend the same acquisition cost on a one-time buyer and a future top-1% customer. LTV models set CAC ceilings, churn models time win-back campaigns, propensity models pick the audience for every push notification you send.

What actually runs in production

• RFM (Recency, Frequency, Monetary) is still undefeated as a baseline. Three SQL-computable numbers segment your base surprisingly well. Any model that can't beat RFM should not ship — and plenty don't.
• BG/NBD + Gamma-Gamma (the lifetimes library): elegant probabilistic models of "is this customer still alive and what will they spend?" — great for non-contractual commerce where churn is never observed directly, just inferred from silence.
• Gradient boosting on behavioral features for churn/propensity: sessions, days since last order, category breadth, discount dependency, delivery incidents, support tickets. The last two are routinely the most predictive features nobody thinks to include.

What to learn

• Start: build RFM in SQL, then lifetimes BG/NBD, then a boosted churn model — in that order
• Concepts: censoring, calibration (a 0.7 score should mean 70% probability — critical when scores drive spend)
• Trap to avoid: optimizing AUC when the business needs precision@top-decile — you only ever act on the top of the list

7. Fraud & Abuse — The Silent P&L Drain Loss Prevention

The business problem

Payment fraud, promo abuse (100 accounts, one device, every welcome voucher), refund abuse, seller fraud on marketplaces. It's basis points of GMV — which at marketplace scale is millions — and it compounds: weak enforcement attracts more abuse.

What actually runs in production

• Rules + gradient boosting hybrid. Rules catch known patterns instantly and give compliance teams explainability; the model catches novel combinations. Every mature fraud stack is this hybrid — pure-ML fraud systems exist mainly in conference talks.
• Extreme class imbalance (1:1000+): use scale_pos_weight, evaluate with precision-recall curves (never accuracy, and even ROC-AUC flatters you here).
• Graph features are the modern edge: shared devices, addresses, payment instruments, wifi networks across accounts. Fraudsters fake individual features easily but connections are expensive to fake. Even simple "count of accounts sharing this device" features are powerful before you touch graph neural networks.
• Cost-sensitive thresholds: a false positive (blocking a real customer) costs the order plus lifetime trust; a false negative costs the item. The optimal threshold comes from this cost ratio, not from maximizing F1.

What to learn

• Concepts: precision/recall trade-offs as cost decisions, adversarial drift (fraud patterns mutate when you deploy — retraining cadence matters more than architecture)
• Practice: the IEEE-CIS fraud detection dataset on Kaggle is genuinely representative of the feature-engineering style used in industry

8. Experimentation — The Meta-Skill The Gatekeeper

Here's the uncomfortable truth that separates people who've shipped ML from people who've read about it: most offline model improvements die in the A/B test. A recommender with +10% offline NDCG regularly ships flat or negative revenue. Reasons: offline metrics reward predicting the popular items users would find anyway; position bias contaminates the labels; and the offline test can't measure how users react to a different experience.

What every e-commerce professional should know:

• A/B testing fundamentals: power analysis (most "failed" tests were just underpowered), one primary metric decided upfront, guardrail metrics (latency, cancellations, support contacts)
• CUPED — using pre-experiment data to cut variance, often 30–50% faster experiments for free; standard at every major tech company and shockingly little-known outside them
• Peeking is the cardinal sin: checking significance daily and stopping at p<0.05 inflates your false positive rate severalfold. Use sequential testing methods if you must monitor.
• Interleaving for ranking changes: mix results from ranker A and B in one list and see which gets clicked — orders of magnitude more sensitive than user-level splits

9. What Production Actually Looks Like

Patterns you only learn by being inside — the stuff that separates practitioners from paper-readers:

1. Baselines are brutally hard to beat. Popularity-ranked recs, last-4-weeks-average forecasts, RFM segments. Teams routinely spend quarters failing to beat them. Always build and report the dumb baseline first — it's also your fallback when the model service goes down.
2. The feature pipeline is 80% of the work. Training/serving skew — where the feature computed offline doesn't match what's computed at request time — is the most common silent killer of production models. This is the problem feature stores exist to solve.
3. Models rot. Assortment changes, seasons shift, competitors move, fraud adapts. Monitoring prediction distributions and retraining cadence matter more than architecture choice.
4. Latency is a feature. A brilliant ranker at 800ms loses to a mediocre one at 80ms. This is why two-stage retrieval-then-rank architectures exist at all.
5. Explainability buys deployment. A merchandising team will act on "demand up 30% because price cut + payday weekend" and will quietly ignore a black box. SHAP values on a LightGBM model are often the difference between a model that ships and one that doesn't.

10. Your 90-Day Roadmap

Goal-oriented, by month. Each project is chosen to be portfolio-credible and interview-defensible:

Month 1 — Foundations with real data

• Master LightGBM end-to-end: regression, classification, ranking objectives. It's the workhorse of literally every section above.
• Build RFM + a churn model on any transactions dataset (Olist's Brazilian e-commerce dataset on Kaggle is excellent and messy in realistic ways).
• Learn WAPE, NDCG, precision@k, calibration — the metrics vocabulary of the industry.

Month 2 — The two big systems

• Forecasting: work through M5 competition solutions; build a LightGBM forecaster with lag features and quantile loss.
• Recommendations: build ALS with the implicit library, evaluate recall@k, then read the YouTube two-tower paper to see where it scales.

Month 3 — The differentiators

• Uplift modeling: T-learner with causalml on a simulated campaign. Almost nobody has this on their CV; every promo-heavy business needs it.
• Experimentation: design a full A/B test — hypothesis, power analysis, guardrails, CUPED. Write it up.
• Learning to rank: LightGBM lambdarank on any click log dataset.

Do this and you won't just "know ML concepts" — you'll be able to walk into any e-commerce room and map its P&L to specific, current, production-grade techniques. That's the difference between following the industry and operating in it.

Sources & Further Reading:
OTTO: Deep Learning vs GBDT for E-Commerce Ranking (2025) • LambdaMART Explained • M5 Forecasting Competition • CausalML (Uber) • implicit (ALS library) • lifetimes (BG/NBD)