Rate Limiting: Token Bucket, Sliding Window, and Distributed Counters

Problem Context

Without a rate limiter, one bug, one runaway script, or one motivated bad actor saturates your API and breaks it for every honest user. With a poorly designed one, your honest users get 429s during normal traffic spikes while the actual abuser routes around it. Rate limiting is the difference between a service that degrades gracefully under hostile load and one that falls over.

The 2026 reality: ASP.NET Core ships a first-class rate limiting middleware, Azure Front Door / API Management / App Gateway WAF can rate-limit at the edge, and Redis-backed sliding-window algorithms are basically free to implement. The hard part isn't the algorithm — it's deciding the key, the limit, the window, and what to do when you exceed it.

Concept Explanation

Four algorithms cover almost every use case:

• Fixed window — N requests per minute, reset on the minute boundary. Cheapest, but allows a 2N burst at the boundary.
• Sliding window — N requests in the last 60s, computed precisely. Smooth, more state.
• Token bucket — bucket of N tokens, refills at R/sec. Allows controlled bursts up to bucket size, then steady rate. Default choice for APIs.
• Leaky bucket / concurrency — at most N in-flight requests. Best for protecting downstream finite resources (DB connections, GPU).

flowchart LR
    C["Client"] --> EDGE["Edge limiter<br/>Front Door / WAF<br/>(coarse: by IP, geo)"]
    EDGE --> APIM["API Mgmt<br/>per-subscription quota"]
    APIM --> APP["App rate limiter<br/>(fine: per user, per route)"]
    APP --> DB["Backend"]

    APP -.->|exceeded| R429["429 + Retry-After<br/>+ X-RateLimit-* headers"]

    style EDGE fill:#0078D4,color:#fff,stroke:#005a9e
    style APP fill:#dc2626,color:#fff,stroke:#b91c1c

Implementation

Step 1: ASP.NET Core built-in rate limiter (the .NET 9 default)

using System.Threading.RateLimiting;

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
    options.OnRejected = async (ctx, ct) =>
    {
        if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var ra))
            ctx.HttpContext.Response.Headers.RetryAfter = ((int)ra.TotalSeconds).ToString();
        await ctx.HttpContext.Response.WriteAsync("Too many requests.", ct);
    };

    // Token bucket per authenticated user (fall back to IP for anonymous)
    options.AddPolicy("per-user", httpContext =>
    {
        var key = httpContext.User.FindFirst("sub")?.Value
                  ?? httpContext.Connection.RemoteIpAddress?.ToString()
                  ?? "anon";
        return RateLimitPartition.GetTokenBucketLimiter(key, _ => new TokenBucketRateLimiterOptions
        {
            TokenLimit       = 100,                       // bucket size
            TokensPerPeriod  = 100,                       // refill amount
            ReplenishmentPeriod = TimeSpan.FromMinutes(1),// every minute
            QueueLimit       = 0,                         // reject immediately
            AutoReplenishment= true
        });
    });
});

var app = builder.Build();
app.UseRateLimiter();

app.MapGet("/api/search", () => "...").RequireRateLimiting("per-user");

Step 2: Distributed limiter (multi-instance) with Redis

// Built-in limiter is per-process. With N instances, the effective limit is N×N.
// Use a Redis-backed sliding-window for accurate global limits.
public class RedisSlidingWindow(IConnectionMultiplexer redis)
{
    private const string Lua = """
        local key = KEYS[1]
        local now = tonumber(ARGV[1])
        local window = tonumber(ARGV[2])
        local limit = tonumber(ARGV[3])
        redis.call('ZREMRANGEBYSCORE', key, 0, now - window)
        local count = redis.call('ZCARD', key)
        if count >= limit then return 0 end
        redis.call('ZADD', key, now, now)
        redis.call('PEXPIRE', key, window)
        return 1
    """;

    public async Task<bool> AllowAsync(string key, int limit, TimeSpan window)
    {
        var db = redis.GetDatabase();
        var result = (long)await db.ScriptEvaluateAsync(Lua,
            new RedisKey[] { $"rl:{key}" },
            new RedisValue[] { DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
                               (long)window.TotalMilliseconds, limit });
        return result == 1;
    }
}

Step 3: Choose the partition key carefully

Bad keys:
  ❌ Client IP only          → corporate NAT = thousands of users on one IP
  ❌ User-Agent              → trivially forged
  ❌ Auth token raw          → token rotation bypasses the limit

Good keys, in priority order:
  ✅ Authenticated user/sub  → first choice for logged-in APIs
  ✅ API key / subscription  → for B2B; tier the limits per plan
  ✅ Tenant id               → per-tenant fairness in multi-tenant apps
  ✅ IP + path bucket        → fallback for anonymous endpoints
  ✅ Composite (sub + route) → expensive endpoints get tighter limits

Step 4: Tier the limits by endpoint cost

// Cheap endpoints: generous limits
app.MapGet("/api/products/{id}", ...).RequireRateLimiting("standard");   // 600/min

// Expensive endpoints: tight limits
app.MapPost("/api/search/full-text", ...).RequireRateLimiting("expensive"); // 30/min

// Auth/login: very tight + per-IP fallback to defeat credential stuffing
app.MapPost("/api/auth/login", ...).RequireRateLimiting("auth-strict");  // 5/min/IP

Step 5: Return useful headers — clients should self-throttle

HTTP/1.1 200 OK
RateLimit-Limit: 100
RateLimit-Remaining: 23
RateLimit-Reset: 37            # seconds until the window resets

# When rejected:
HTTP/1.1 429 Too Many Requests
Retry-After: 12                # seconds; clients MUST honor this
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 12

Step 6: Defence in depth — limit at every layer

Layer 1  Azure Front Door / WAF
         • Rate-limit by client IP (coarse, anti-DDoS)
         • Block bad geos / known abusive ASNs

Layer 2  Azure API Management
         • Quota by subscription key (1M calls/month)
         • Spike-arrest at the gateway (50 req/sec per key)

Layer 3  App-level (ASP.NET RateLimiter)
         • Per-user, per-route, with cost weighting
         • Returns proper 429 + Retry-After

Layer 4  Backend (DB connection pool, semaphore)
         • Bulkhead: max in-flight ops per dependency
         • Last line of defence; protects the data tier

Pitfalls

1. Per-instance limits in a multi-instance app. The built-in RateLimitermiddleware is per-process. With 10 pods, your "100/min" is actually 1000/min. Use Redis-backed limits for global accuracy or accept the multiplier and tune accordingly.

2. IP-only keys behind a proxy. If you forget UseForwardedHeaders, every request looks like it came from the proxy IP and the entire fleet shares one bucket. Configure X-Forwarded-For properly.

3. Fixed-window burst attack.A client can fire 100 requests at 12:00:59.999 and another 100 at 12:01:00.001 — that's 200 in 2ms. Use sliding window or token bucket for limits that matter.

4. Returning 429 without Retry-After. Clients (and SDKs) without this header retry immediately, amplifying load. Always emit Retry-After and the RateLimit-* headers.

5. Counting all routes equally. A GET /healthzand a full-text search both count as "1 request" — but the search costs 100x. Either separate policies per route or assign request cost (token bucket lets you take N tokens per request).

6. No rate limit on auth/login. Credential stuffing attacks fire millions of login attempts. Tight per-IP and per-username limits + CAPTCHA on suspicious patterns are mandatory for any login endpoint.

Practical Takeaways

• Default algorithm: token bucket per user. It allows fair bursts and a steady rate.
• Key by authenticated identity first, API key second, tenant third, IP last.
• For multi-instance apps, use a Redis-backed sliding window or accept that built-in limits are per-process.
• Limit at multiple layers: edge (DDoS), gateway (subscriber quota), app (per-user), backend (bulkhead).
• Always return Retry-After + RateLimit-* headers. Document them.
• Tier limits by endpoint cost. A search shouldn't share a bucket with a healthcheck.
• Auth endpoints get the strictest limits + CAPTCHA fallback. Treat them as the prime target.