Ranking NDCG offline +12%, online flat — где divergence

Ситуация: New ranking offline NDCG +12%. Online A/B: flat or slightly negative. Common ML pain point. Find root cause.

Search ranking. Offline eval: replay historical queries. Online: real user behavior.

Доступные данные

• offline_eval: query_id, model_version, ndcg
• online_log: query_id, user_id, ranking_version, clicks, deals
• historical_clicks: query_id, listing_id, position, clicked

Задачи

• Hypothesize divergence causes (5+).
• Diagnostic queries.
• Process improvements.

Все кейсы для подготовки → ← Все кейсы