Vector quantization для recsys на собеседовании Data Scientist

Зачем VQ в recsys

Catalog 100M-1B items. Embedding каждого 256 floats × 4 bytes = 1 KB. Total: 100-1000 GB embeddings.

VQ — compress embeddings 32-128× с small accuracy loss.

Product Quantization

Split d-dim vector на M sub-vectors of d/M dims. K-means каждый отдельно с K codes.

128-dim vector → 8 sub-vectors of 16 dims.
For каждого — k-means K=256 codes.

Storage: 8 indices × 1 byte = 8 bytes per vector (vs 512 bytes original).

Distance approximated:

d(x, y) ≈ Σ d(quantize(x_i), quantize(y_i))

Pre-computed lookup table.

Residual VQ

Hierarchical PQ — quantize, then quantize residual, recurse.

v ≈ codebook_1[c_1] + codebook_2[c_2] + ... + codebook_K[c_K]

Better fidelity, comparable memory.

Multi-stage retrieval

Production recsys.

Stage 1: ANN with VQ — 1B → 1000 candidates (fast).
Stage 2: Reranking — 1000 → top-100 with full vectors / cross-encoder.
Stage 3: Final ranking — heavy model, business rules.

VQ для bulk pruning. Exact compute только для top-K.

TIGER (semantic IDs)

Recent (2023+). Encode item как sequence semantic IDs (codebook indices).

Item → 4 IDs ([42, 17, 99, 3]) — каждый level codebook hierarchical RVQ.

LLM-style decoder generates next item IDs sequentially. Replaces classical retrieve-rerank.

Used в Google YouTube experiments.

Связанные темы

• Vector databases на собесе DS
• Vector search optimization для DS
• Two-tower DSSM для DS
• Collaborative filtering для DS
• Подготовка к собесу Data Scientist

FAQ

Это официальная информация?

Нет. Статья основана на работах Jegou 2010 (PQ), Rajput 2023 (TIGER), документации FAISS.

Тренируйте Data Science — откройте тренажёр с 1500+ вопросами для собесов.