The Search Analytics Playbook — Running the Surface Where Half Your Revenue Lives

The ML playbook covered how search ranking works — retrieval, LambdaMART, two-tower models. This post covers the other half that determines whether any of that matters: the operating layer. Because the best ranker in the world can't fix a query that returns nothing, a catalog that calls a hoodie a "knit upper garment," or a merchandising rule from 2023 that still buries your best seller.

1. Search Is a P&L Surface, Not a Feature

The industry numbers that justify the attention:

• Site searchers are roughly 24% of visitors but ~44% of revenue, converting at 2–3x the rate of category browsers — they arrive with intent
• 10–20% of all searches return zero results at a typical retailer — every one a customer announcing money and being shown an empty page
• Search abandoners don't browse instead — a large share exit entirely; a bad search experience is a bounce, not a detour

The framing that changes the conversation: multiply your search sessions × the CVR gap × AOV, and price a one-point improvement in search conversion. At most mid-size retailers that number is large enough to fund a team — and unlike traffic acquisition, it's all margin-side improvement on demand you already paid for.

2. The KPI Stack

3. Zero Results — The Cheapest Money in E-Commerce

The weekly zero-results report is the highest ROI-per-hour artifact in search analytics: the top 50 zero-result queries by volume, classified into four buckets, each with a different owner:

Industry experience says 40–50% of zero-result queries are fixable through product data enrichment alone — no search-engine replacement required. And instrument the zero-results page itself: popular-category fallbacks and a "notify me" capture convert a dead end into a signal and occasionally a sale.

4. Query Mining — Head, Torso, Tail

Query distributions are brutally head-heavy, and each zone gets different treatment:

• Head (top ~100 queries, often 20–30% of volume): these deserve individual attention — a human looks at the results page for each, monthly. A relevance bug on one head query can outweigh a quarter of tail improvements. These are also your merchandising surfaces and, per the pricing playbook, a strong KVI signal.
• Torso (next few thousand): rule- and synonym-driven improvements, monitored via aggregate reformulation and exit rates by query cluster.
• Tail (everything else, half your queries): this is where semantic/embedding retrieval earns its keep (ML playbook, section 2) — no rule set scales to millions of unique long-tail queries.

One query pattern worth automating — the trending-query detector:

-- queries growing fast week-over-week: demand arriving before your buyers know it
SELECT
  query,
  this_week,
  last_week,
  SAFE_DIVIDE(this_week - last_week, last_week) AS growth,
  zero_result_share
FROM weekly_query_stats
WHERE this_week >= 100                    -- volume floor
  AND SAFE_DIVIDE(this_week - last_week, last_week) > 0.5
ORDER BY this_week * SAFE_DIVIDE(this_week - last_week, last_week) DESCsql

A trending query with a high zero-result share is the purest signal in retail: demand you're not serving, quantified, days before it shows up anywhere else.

5. Search Logs as Assortment Intelligence

This is the most under-used asset in e-commerce: search logs are a free, continuous market-research program. Customers type exactly what they want, in their own words, with volume attached. The quarterly assortment-gap report: aggregate zero-result and low-click queries into product concepts (an LLM clusters these well), size each by search volume × category AOV, and hand buying a ranked list of "demand we're bouncing." It converts search analytics from a UX function into a commercial one — and it's the report that gets search teams invited to assortment meetings.

6. Search Merchandising — Rules Without Chaos

Every search platform allows boosts, burials, and pins. Every retailer eventually drowns in them: hundreds of rules, authors long gone, quietly fighting the ranker. The discipline:

• Every rule gets an owner, a reason, and an expiry date. No permanent rules — a boost that's still correct in 90 days gets renewed in 30 seconds; one that isn't dies automatically. This single policy prevents the archaeology problem.
• Boost on business state, not vibes: availability (never boost low-stock items into position 1), margin tiers, deal participation, delivery speed. These translate strategy into ranking without hand-picking products.
• Don't hijack high-intent queries. A banner or sponsored placement on a precise query ("iphone 15 pro case magsafe") that displaces the exact match costs more conversion than the placement earns. Reserve editorial control for broad/inspirational queries.
• Audit rule-vs-ranker conflicts quarterly: list queries where manual rules override the top-5; for each, would the ranker alone do better now? Rules written for the 2023 search engine often handicap the 2026 one.

7. The Search Quality Process

How to know search is getting better — beyond anecdotes from the CEO's last bad query:

1. Build a golden query set: 200–500 queries stratified across head/torso/tail, navigational/category/attribute/problem queries, both languages if bilingual. Include the embarrassing ones from support tickets.
2. Judge results on a rubric (per query: are the top-4 relevant? is the exact item first for navigational queries?) — human judgment quarterly, LLM-as-judge weekly with human audit (evaluation discipline from the LLM playbook, section 8).
3. Regression-test before every ranker or rule change: run the golden set, diff the results. A change that helps aggregate metrics but breaks 10 head queries is a net loss you'd otherwise discover via Twitter.
4. Ship ranking changes through interleaving, not user-split A/B tests — mixing results from both rankers in one list needs orders of magnitude less traffic (experimentation playbook).

8. The Weekly Ritual

The whole playbook compresses into one recurring 45-minute meeting with search, catalog, and category:

• KPI stack vs last week — decomposed by query segment, not aggregates
• Top-50 zero-result queries → bucketed → actions assigned
• Trending queries review → assortment flags
• Head-query spot check (rotate through the top 100 monthly)
• Expiring merchandising rules → renew or release

Sources & Further Reading:
Algolia: 40+ Search KPI Stats • Algolia: Null Results Optimization • Bloomreach: Fixing Zero Results