Hover a node · click to focus · ←/→ steps
Versioned rails in Git. Input → dialog → output checks; high-severity detections fail closed before the provider.
When you need this
- Prompt injection incidents in support bots
- Need deny-list topics for brand/legal
- Security wants guardrails reviewed like policy-as-code
- Output schema must be validated before UI render
Prerequisites
- Gateway or app middleware hook
- Threat model for prompt injection and data exfil
- Git repo for rail configs
Tools
Trial on customer-facing bots; keep rails versioned in Git with eval coverage.
Use at response boundary; combine with Presidio for pre-prompt scrubbing.
Assess vs OSS rails when you need managed updates for attack patterns.
Deploy as single ingress before adding a second LLM vendor.
Steps
- 1
Threat-model the surface
List untrusted inputs (user chat, retrieved docs, tools). Classify risks: injection, PII leak, off-topic, schema break.
- 2
Express rails in versioned config
NeMo Colang / Guardrails AI specs in Git. Input rails (detect jailbreak), dialog rails (topic), output rails (PII, schema). PR review required.
- 3
Enforce at the gateway
Run rails before/after the model call (LiteLLM callback or sidecar). Fail closed on high-severity detections.
- 4
Add specialized injection defense
For high-risk apps, add Lakera (or equivalent) as a complementary scanner — do not rely on prompt-only defenses.
- 5
Test rails in CI
Red-team corpus: known jailbreaks, PII payloads, off-topic asks. Fail PR if rails regress. Log blocks for compliance.
Adoption pitfalls
- Rails only in the system prompt — easily overridden
- False positives with no tuning → teams bypass the gateway
- No logging of blocked attempts for forensics
Adoption checklist
- Rail configs in Git with owners
- CI red-team suite green on main
- High-severity detections fail closed
- Quarterly rail review with security
SEER REAL assessment / sprint
Assessment scores injection exposure and whether controls are code-reviewed. Sprint wires NeMo or Guardrails AI on one app with a CI red-team corpus.