On this page
The problem
We need to select a primary LLM for our platform's AI features (article assistance, search, code generation). The choice affects cost, quality, and vendor dependency.
What we weighed
GPT-4.1 (OpenAI)
Pros
- +Strong coding and structured output
- +Largest ecosystem (libraries, tools, community)
- +Azure OpenAI available for enterprise compliance
- +Function calling is mature
Cons
- −Less nuanced in creative/explanatory writing
- −Rate limits on newer models
Claude 3.5 Sonnet (Anthropic)
Pros
- +200K-token context window (GPT-4.1 offers about 1M)
- +Better at nuanced, educational content
- +Strong safety alignment
Cons
- −Smaller ecosystem
- −Function calling less mature than OpenAI
- −Higher per-token price than GPT-4.1 ($3/$15 vs $2/$8 per million input/output tokens)
- −Not on Azure OpenAI; enterprise cloud access is through Amazon Bedrock or Google Cloud Vertex AI
Multi-Model with AI Gateway
Pros
- +No vendor lock-in
- +Route by task type (code → GPT, writing → Claude)
- +Fallback if one provider is down
Cons
- −More complex prompt engineering (per-model tuning)
- −Higher operational overhead
- −Inconsistent voice across features
What we chose
We chose the Multi-Model approach via an AI Gateway pattern. GPT-4.1 via Azure OpenAI for code generation and structured output tasks. Claude 3.5 Sonnet for content assistance and long-context RAG. This gives us resilience and lets us play to each model's strengths.
What it committed us to
- •Must implement AI Gateway pattern (Azure APIM or LiteLLM)
- •System prompts must be tuned per model
- •Cost tracking per model and per feature required
- •Team needs familiarity with both APIs
- •Superseded: GPT-4.1 and Claude 3.5 Sonnet have both been replaced. The multi-model gateway approach still holds, but choose the models again from the current lineup.