API Gateways in 2026: YARP, Azure APIM, and the AI Gateway PatternOfficial sources11 of 11 sources come from the companies and standards bodies that own the technology.This rates where the sources come from. It doesn't prove every claim.
YARP for in-process east-west traffic, Azure APIM north-south with policies for auth + rate limit + transform, BFF per channel, defense in depth, and the 2026 AI Gateway: token caps + semantic cache + token metrics.
By Amit UpadhyayPublished Updated 15 minArticle 1 of 8 in API Design
AI assistant · answers from this article
1 of 6
A single service
Nothing to add yet
With one service, authentication, limits and logging live in that service. A gateway would be one more thing to run.
On this page
API gateway, in one sentence
An API gateway is one front door for many services, where the jobs every service would otherwise repeat, such as checking tokens and limiting traffic, are done once.
You reach for it when several services are exposed to external clients, each needing authentication, limits and logging. Not before: it costs another hop with its own latency and failure modes, and a component that must be highly available.
The cheat sheet
What to use when
You do not need all of this on day one. Each row is one signal and the piece that answers it. Stop at the row your system has reached, or pick one to see it in the picture above.
Where you areThe signalWhat to useWhat it costs
None. One service faces the outside world.Nothing. Handle it in the serviceNothing
Each team wrote its own token validation, rate limiting and CORS handling, and a review finds one of them wrong.A gateway for the cross-cutting jobs: TLS, token checks, limits, routing, loggingStep 2 ↓A shared dependency that every request passes through
A request that passed the gateway reads another tenant’s data, because no service checked.Keep resource-level authorisation in each serviceCommon mistakes ↓Checks at the edge and again in the service
A pricing rule lives in a gateway policy, and nobody can test it.Keep business logic out of the gatewayWhat a gateway does ↓Saying no to quick fixes at the edge
One slow service ties up every gateway worker.Timeouts and circuit breakers for each backend, and several gateway instancesStep 6 ↓Thresholds to tune from measured latency
A buggy retry loop spends thousands overnight against a model endpoint.Token-per-minute caps and a semantic cache at the gatewayStep 5 ↓Limits to size for each consumer
Start With One Service
With a single service behind your API, handle authentication, limits and logging in that service. A gateway in front of it would be one more thing to run, for no gain. It earns its place when several services face the outside and each one starts solving the same problems separately.
Each team behind your public API wrote its own token validation, rate limiting and CORS handling. They mostly agree. Then a security review finds one service that checks the token’s signature but not its expiry, and revoked tokens have been working there for weeks.
An API gateway is the single front door that handles these cross-cutting jobs once: TLS, authentication, rate limits, routing and edge logging. It should stay thin. Business rules and resource-level authorisation belong in the services behind it. Below is what to put in the gateway, what to keep out, and how to run it so it does not become the single point of failure.
Signs you need a gateway
Every microservice re-implements JWT validation slightly differently
You have CORS bugs in three services with three different fixes
Your "rate limit" lives in 12 controllers and one of them has a typo
An LLM call accidentally cost $2,000 last month with no way to cap it
Pro deep dive15 min full read
Keep reading with Pro
This is one of our advanced production deep dives. Pro unlocks the rest of it, every other deep dive, and the See, Animate, Test and Build modes on every topic.
What it covers
YARP for in-process east-west traffic, Azure APIM north-south with policies for auth + rate limit + transform, BFF per channel, defense in depth, and the 2026 AI Gateway: token caps + semantic cache + token metrics.