消息与事件
kafka-topic
更新于 2026-11-01
Kafka 工程规则
Kafka rules: idempotent producers, schema evolution and consumer lag budgets
- 技术栈
- Kafka
- 主题
- 工程基线
- 条目
- 9 条
- 版本
- v0.2.0
✓ 应当这么做
- ✓ Version datasets and pipelines together so a model and its inputs are traceable
- ✓ Validate schema and null-rate drift on every ingestion run
- ✓ Write idempotent transformations that can be re-run safely
✕ 常见反模式
- ✕ Silent schema drift between source and warehouse
- ✕ Pipelines that mutate the raw layer in place
- ✕ Dashboards built directly on non-validated raw tables
◎ 交付前验证
- ◎ Row counts and key uniqueness compared against the previous run
- ◎ Backfill path tested on a partition
- ◎ Lineage graph updated for the new source