Hover a node · click to focus · ←/→ steps
Promote only after eval. Canary a traffic slice on K8s; auto-rollback on SLO breach; GitOps is the audit trail.
When you need this
- Ad-hoc FastAPI wrappers per model
- No canary — big-bang deploys to 100%
- Need reproducible container builds from registry versions
- Want runners for both classic ML and LLM adapters
Prerequisites
- MLflow (or equivalent) registry version to serve
- Container registry + K8s or BentoCloud target
- Latency/error SLOs for canary promotion
Tools
Trial when teams outgrow notebook exports but are not ready for full KServe.
Start with registry + experiment tracking before full deployment automation.
Works best once model configs live in Git alongside app manifests.
Steps
- 1
Package the model as a Bento
Declare service API, resources, and dependencies. Build from the exact registry artifact — pin versions.
- 2
Push image and update GitOps manifest
CI builds on registry promotion, opens PR with new tag for Argo/Flux. Staging auto-deploys.
- 3
Canary to production
Route 5–10% traffic; watch error rate, latency, and business metric. Auto-promote or rollback on threshold.
- 4
Rollback drill
Revert manifest / previous Bento. Time the drill; document owner. Link rollback to MLflow previous Production version.
Adoption pitfalls
- Building images from local checkout instead of registry
- Canary without business metric — only HTTP health
- Manual traffic split with no timeout to promote/rollback
Adoption checklist
- Serving image built only from registry artifacts
- Canary thresholds defined and automated
- Rollback tested in last 90 days
- Prod traffic split visible on dashboard
SEER REAL assessment / sprint
Assessment maps serve path and rollback. Sprint packages one model with BentoML, GitOps deploy, and a canary with rollback drill.