In this article
Problem: LLMs receive malicious inputs (prompt injection) and produce harmful or malformed outputs. A single unguarded LLM call is a security and quality risk.
Solution: Apply a three-layer "sandwich": input guardrail → LLM call → output guardrail. Each layer is independent and specialised.
Implementation:
- Input guardrail: Check for prompt injection, PII, off-topic requests, and policy violations before calling the LLM
- LLM call: Execute with system prompt constraints and JSON mode if structured output is needed
- Output guardrail: Validate output schema (Zod/Pydantic), check for harmful content, verify factual grounding (citations present), and apply PII redaction
Layer options:
- Input: Rebuff (prompt injection), Azure Content Safety, custom classifiers
- Output: Guardrails AI, NeMo Guardrails, RAGAS faithfulness check, schema validation
Trade-Offs:
- ✔ Pro: Defence-in-depth — multiple independent layers catch different failure modes
- ✔ Pro: Each layer is independently testable and replaceable
- ✖ Con: Adds 100-500ms latency per layer
- ✖ Con: False positives in input guardrails block legitimate requests
When To Use: Any production, user-facing LLM application. Non-negotiable for regulated industries. When to avoid: Internal prototype tools with trusted users.
Related Articles
the ai gateway pattern