LLM Input Validation Strategies

Problem Context

Input validation for an LLM application is not the same problem as input validation for a REST API. The threats are different — prompt injection, jailbreaks, PII leakage upstream, off-topic abuse — and the defences must run before a single token reaches your model. Skipping input validation is the #1 reason "safe" LLM apps end up in screenshots on Twitter.

The Five Things to Check Before Calling the LLM

• Length / token cap — bound the input so a single user can't blow your context window or your bill.
• PII / secrets detection — refuse or redact credit cards, SSNs, API keys, JWTs before they reach a third-party model.
• Prompt-injection / jailbreak detection — a classifier that recognises the "ignore previous instructions" family.
• Topic / policy gate — is this request even something your product is supposed to answer?
• Rate / quota check — per-user and per-tenant. Cheap, catches abuse early.

flowchart LR
    R[Request] --> L[Length/token cap]
    L --> S[Secret/PII scan]
    S --> J[Injection classifier]
    J --> T[Topic gate]
    T --> Q[Rate limit / quota]
    Q --> LLM[LLM call]
    L -. fail .-> X[Typed 4xx + audit]
    S -. fail .-> X
    J -. fail .-> X
    T -. fail .-> X
    Q -. fail .-> X
      

Cheap Before Expensive

Order matters. Each step should be cheaper than the next:

1. Length check — bytes/regex (microseconds).
2. Secret detector — regex over high-signal patterns (Stripe key, AWS key, JWT, IBAN).
3. Lightweight injection classifier — local model or managed (Azure Content Safety, Lakera, Protect AI) — milliseconds.
4. Optional LLM-as-judge for ambiguous cases.

PII / Secret Detection

Use a layered detector: regex for high-confidence patterns (credit card with Luhn, AWS access key prefix, GitHub PAT) plus a Presidio/AWS Comprehend / Azure AI Language NER pass for names, emails, phone numbers, addresses. Two policies are useful:

• Redact-then-call — replace with placeholders ([EMAIL]) before sending to the LLM, optionally re-insert in the response.
• Block-and-warn — for secrets (API keys, credentials), refuse outright and surface a clear message.

Prompt Injection Detection

Modern injection classifiers operate on the request and on any retrieved context (RAG documents are a primary injection vector). Patterns they catch:

• Direct: "ignore previous instructions and …".
• Indirect: instructions hidden in retrieved markdown, PDFs, scraped web pages.
• Encoded: base64, leetspeak, foreign-language equivalents of override phrases.
• Multi-turn social engineering — needs conversation-aware classifiers.

A 90% detection rate with a 1% false-positive rate is realistic in 2026. Tune toward false-positives you can survive — over-blocking creates worse UX than under-blocking creates risk for most consumer apps; the calculus reverses for high-stakes domains.

Topic Gating

For domain-specific assistants, a topic classifier (or a small embedding-similarity check against your scope description) blocks off-topic traffic before the expensive model runs. This is also where you catch users trying to use your finance assistant for medical advice.

Quota / Rate Limiting

• Per-user rate (e.g. 30 requests/min) — blocks scrapers and runaway client loops.
• Per-user token budget (e.g. 200K tokens/day) — caps individual cost.
• Per-tenant burst + sustained — protects multi-tenant fairness.
• Always emit a typed 429 with Retry-After and a budget-remaining header — clients can adapt instead of retrying immediately.

Failure Modes

• Validation that runs after the model call — defeats the purpose; cost and risk already incurred.
• Static denylist of injection phrases — bypassed by paraphrase. Use semantic classifiers.
• Silent block — user has no idea why their request failed. Always return a clear, typed error.
• No audit trail — you can't tune false positives without a log of every block and why.

Production Checklist

• Five-layer pipeline: length → PII → injection → topic → quota.
• Cheap-first ordering with explicit fail-stop on each layer.
• PII detection on both inputs and retrieved RAG context.
• Typed errors, audit log, dashboard for block rate per layer.
• Periodic red-team set in CI to catch regressions in your detectors.

Key Takeaways

• Input validation for LLMs is not a webhook concern — it is a security boundary specific to generative AI.
• Layer cheap checks before expensive ones; classify, don't denylist.
• Treat retrieved context as user input — indirect injection is the most under-defended attack surface.