In this article
Context
We need to select a primary LLM for our platform's AI features (article assistance, search, code generation). The choice affects cost, quality, and vendor dependency.
Options Evaluated
GPT-4.1 (OpenAI)
Pros
- +Strong coding and structured output
- +Largest ecosystem (libraries, tools, community)
- +Azure OpenAI available for enterprise compliance
- +Function calling is mature
Cons
- −Higher cost than Claude for long-context tasks
- −Less nuanced in creative/explanatory writing
- −Rate limits on newer models
Claude 3.5 Sonnet (Anthropic)
Pros
- +Superior long-context handling (200K tokens)
- +Better at nuanced, educational content
- +Competitive pricing
- +Strong safety alignment
Cons
- −Smaller ecosystem
- −Function calling less mature than OpenAI
- −No equivalent of Azure OpenAI for compliance
Multi-Model with AI Gateway
Pros
- +No vendor lock-in
- +Route by task type (code → GPT, writing → Claude)
- +Fallback if one provider is down
Cons
- −More complex prompt engineering (per-model tuning)
- −Higher operational overhead
- −Inconsistent voice across features
Decision
We chose the Multi-Model approach via an AI Gateway pattern. GPT-4.1 via Azure OpenAI for code generation and structured output tasks. Claude 3.5 Sonnet for content assistance and long-context RAG. This gives us resilience and lets us play to each model's strengths.
Consequences
- •Must implement AI Gateway pattern (Azure APIM or LiteLLM)
- •System prompts must be tuned per model
- •Cost tracking per model and per feature required
- •Team needs familiarity with both APIs
Related Articles
the ai gateway pattern