Search ranking ML system design на собеседовании Data Scientist

Постановка задачи

Search engine. User query → relevant documents.

Constraints:

• Index 100M-1B docs.
• p99 latency < 300ms.
• Real-time indexing для new docs.

Retrieval

Goal. Из 1B → 1000 candidates.

BM25 / sparse. Term frequency-based. Industry standard (Elasticsearch).

Dense retrieval. Embedding-based via two-tower + ANN.

Pros / cons:

• BM25 — exact keyword match strong, semantic weak.
• Dense — semantic strong, exact match weak.

Hybrid retrieval

Combines both.

Score = α · BM25_score + (1-α) · dense_score

Or Reciprocal Rank Fusion (RRF).

score(d) = Σ 1 / (k + rank_method(d))

Often better recall чем either alone.

Reranking (cross-encoder)

Top 100 from retrieval → cross-encoder.

Cross-encoder: input (query + doc) → score

Heavier чем bi-encoder (no precompute), но much better accuracy на small set.

Modern: monoT5, ColBERT, BERT-based rerankers.

Learning-to-rank

Учим ranker на user feedback.

Pointwise. Predict relevance score per (query, doc).

Pairwise. Loss на pairs (relevant > not).

Listwise. Loss на full list (LambdaMART, ListNet).

Features:

• BM25 score.
• Embedding similarity.
• Document quality (PageRank, freshness, length).
• Click history.
• User-document features.

Метрики

Offline.

• NDCG@10.
• MAP.
• MRR.

Online (A/B).

• CTR top results.
• Time to click.
• Search abandonment rate.
• Reformulation rate.
• Long-clicks (читает > 30s).

Связанные темы

• Two-tower DSSM для DS
• Vector search optimization для DS
• Ranking metrics NDCG для DS
• BERT vs GPT для DS
• Подготовка к собесу Data Scientist

FAQ

Это официальная информация?

Нет. Статья основана на индустриальных search practices (Google, Yandex, Elastic).

Тренируйте Data Science — откройте тренажёр с 1500+ вопросами для собесов.