Orchestration layers, structured output libraries, and developer SDKs for building LLM apps.
Composable LLM application framework — chains, agents, RAG
Battle-tested for RAG pipelines and agent orchestration. Large ecosystem but high abstraction cost. Use LCEL (LangChain Expression Language) — avoid legacy Chain classes.
Data framework for LLM applications — indexing and retrieval
Best-in-class for document ingestion, chunking strategies, and advanced retrieval (HyDE, hybrid search). Prefer it over LangChain when the core workflow is RAG-heavy.
Stanford framework for programming (not prompting) LMs
Game-changer for prompt optimisation. Define signatures and let DSPy optimise them with a dataset. Steep learning curve but worth it for production pipelines that need consistent quality.
Deepset open-source NLP framework for production LLM pipelines
Excellent for enterprise document QA. Clean pipeline abstraction that is testable and type-safe. Haystack 2.0 rewrite is well-designed with great component model.
Microsoft's enterprise SDK for AI apps (.NET, Python, Java)
Production-ready for .NET and Azure shops. Native Azure service connectors with managed identity. Python and Java SDKs maturing fast. Best choice for C#-first teams.
Structured outputs from LLMs using Pydantic schemas
Cleanest way to get typed, validated structured output from any LLM. Built on Pydantic. Automatic retries on validation failure. Should be in every production RAG stack.
TypeScript toolkit for building AI-powered web applications
Best framework for AI-powered web apps in the React/Next.js ecosystem. Streaming, tool calling, and generative UI built in. Provider-agnostic with OpenAI, Anthropic, Google support.
Agent framework from the Pydantic team — type-safe and Pythonic
Clean, type-safe agent builder from the team behind Pydantic. Result validators, dependency injection, and structured outputs feel natural. Best for Python teams wanting minimal magic.
Spring framework for building AI applications in Java/Kotlin
Brings the Spring ecosystem to AI. Familiar paradigms for Java teams — auto-configuration, dependency injection, and testability. Supports all major model providers. Growing fast.
Microsoft's constrained generation library for structured LLM output
Powerful constrained decoding — guarantees LLM output matches your grammar or regex. Works at the token level for maximum control. Best for complex structured extraction tasks.
TypeScript-first AI agent framework with built-in workflows
Clean TypeScript SDK for building agents with workflows, tools, and RAG. Good DX with VS Code extension. Newer framework with growing ecosystem. Worth watching for TS-first teams.
Query language for LLMs — SQL-like syntax for constrained generation
Interesting approach: write LLM interactions as queries with constraints, decoding, and control flow. Good for complex multi-step prompting patterns. Academic origin with practical applications.
