Bayesian neural networks на собеседовании Data Scientist

Зачем Bayesian NN

Standard NN — point estimate weights. Bayesian — distribution.

Get uncertainty. Не just prediction, но «насколько confident».

Critical для:

• Medical (uncertain prediction → defer to doctor).
• Self-driving (uncertain → slow down).
• Active learning (label uncertain examples).

Posterior weights

P(w | D) ∝ P(D | w) · P(w)

Posterior — distribution weights given data. Intractable для real NN — millions weights.

Approximations needed.

MC Dropout

Trick (Gal 2015). Train с dropout. Inference — keep dropout enabled, run много times.

model.train()  # dropout on, even at inference
predictions = [model(x) for _ in range(100)]
mean_pred = predictions.mean()
uncertainty = predictions.std()

Cheap approximation Bayesian inference. Practical.

Variational Inference

Replace true posterior с simpler family (e.g., Gaussian per weight). Optimize KL divergence.

Bayes by Backprop. Each weight — mean + variance. Train both.

Pros: principled. Cons: complex, costs.

Применения

Calibrated predictions. «70% confidence positive» actually means 70%.

OOD detection. High uncertainty → out-of-distribution.

Active learning. Label samples max uncertainty.

Anomaly detection. Anomaly → high uncertainty.

Reinforcement learning. Exploration via uncertainty.

В практике — MC dropout популярен (cheap). Full Bayesian — рарко в production.

Связанные темы

• Bayesian методы для DS
• Bayesian optimization для DS
• Active learning для DS
• Dropout и BatchNorm для DS
• Подготовка к собесу Data Scientist

FAQ

Это официальная информация?

Нет. Статья основана на работах Gal 2015, Blundell 2015 (Bayes by Backprop).

Тренируйте Data Science — откройте тренажёр с 1500+ вопросами для собесов.