Source schema drift: vendor added column, ETL silently broken

Ситуация: Source vendor add column. ETL пропустила column. Downstream broken silently 2 weeks. How prevent next time?

ETL: vendor API → raw → cleaned → analytics. ~50 sources, varied freedoms.

Доступные данные

• etl_runs: source, ts, rows_loaded
• schema_snapshots: source, ts, schema_json
• incidents: ts, source, type

Задачи

• Detection: schema diff vs prev.
• Alert: when meaningful change.
• Action playbook.
• Backfill on detected change.

Все кейсы для подготовки → ← Все кейсы