Hover a node · click to focus · ←/→ steps
Promote only after eval. Canary a traffic slice on K8s; auto-rollback on SLO breach; GitOps is the audit trail.
When you need this
- Enterprise buyers ask which model version is live
- Data science ships models without platform review
- Rollback takes hours because nothing is versioned
- EU AI Act evidence needs version → eval → change ticket linkage
Prerequisites
- MLflow tracking server or managed Databricks
- CI runner with model eval script and holdout set
- Agreed Staging → Production stage names with owners
Tools
Start with registry + experiment tracking before full deployment automation.
Trial when teams outgrow notebook exports but are not ready for full KServe.
Steps
- 1
Register every production candidate
Log experiments with params, metrics, and artifacts. Only models with a registered name can reach staging — no ad-hoc pickle deploys from laptops.
- 2
Define promotion stages and required tags
Use Staging → Production with required tags: eval_passed, owner, change_ticket, data_snapshot_id. Block transition if offline AUC/F1 or business metric falls below threshold.
- 3
Wire eval gates in CI
On merge to main, run the eval suite against a frozen holdout. Fail the pipeline on regression; MLflow logs the run and links the gate result to the candidate version.
- 4
Tie serving image to registry version
Serving manifests (BentoML, KServe, or custom) must reference the registry version ID. Promotion updates the image tag via CI — never by hand in prod.
- 5
Document for EU AI Act evidence
Export registry history and eval reports quarterly. Map model versions to change records for high-risk use cases; store under the system’s evidence owner.
Adoption pitfalls
- Registering after deploy — gate becomes theater
- Eval set drifts from production traffic distribution
- Multiple ‘shadow’ registries (notebook, W&B, MLflow) with no source of truth
Adoption checklist
- All live models have a registry name and version
- Promotion requires passing eval gate
- Serving image tag matches registry version
- Rollback procedure documented and tested in last 90 days
- Owner tagged on each production version
SEER REAL assessment / sprint
Assessment maps who can promote and whether gates are enforced. A 2-week sprint typically wires CI → MLflow stages → rollback drill for one production model family.