Constrained generation forces LLM outputs to conform to a specific format — JSON schema, regex pattern, BNF grammar, or enumeration. Without constraints, models may generate text anywhere in the output, include explanations, or produce invalid JSON. OpenAI Structured Outputs (json_schema), Anthropic tool use, llama.cpp grammar, and Outlines/LMQL/Guidance frameworks provide different levels of constraint enforcement. Constraints are essential for machine-parseable outputs in production pipelines.
Each stage in order — click any step to read what it does.
Where constraints can be enforced in the generation stack.
The trade-offs worth knowing before you build this.
Old approach: 'return JSON format' in prompt (fails ~5% of requests). Structured Outputs: enforced at generation time via constrained decoding. Zero parse failures. Use it by default for any machine-parsed output.
For classification tasks, constrained generation prevents the model from inventing new categories ('somewhat negative', 'ambiguous_positive'). Enum constraints guarantee output is one of your defined classes.
Can't use cloud APIs? Outlines + vLLM applies JSON schema, regex, and grammar constraints to any HuggingFace model. Get OpenAI Structured Outputs quality from self-hosted Llama/Mistral.
Sign in to share your feedback and join the discussion.
