What this prompt does
This prompt directs the AI to design a distributed rate limiter tightly enough to build, with real tradeoffs rather than generic advice. You provide the [limit_dimensions], the [tiers], the [backend_store], and the [failure_policy]. It returns an algorithm comparison (fixed window, sliding window, token bucket), a store-backed token bucket that enforces limits across many app servers, per-dimension keys and quotas, client-facing headers with a Retry-After contract, the failure behaviour when the store is unreachable, and an atomicity strategy so concurrent requests don't over-admit.
The structure works because a sloppy limiter either lets abuse through or takes the whole API down when the store hiccups. Deciding fail-open versus fail-closed deliberately is the call most people skip, and making the bucket update atomic prevents two requests both winning the last token. [limit_dimensions] defines the keys, [tiers] map to refill rates, [backend_store] backs the shared counter, and [failure_policy] dictates behaviour during an outage.
When to use it
- You're building an API limiter and need it consistent across many servers.
- You're choosing between fixed window, sliding window, and token bucket.
- You need per-user and per-IP limits with tier-based quotas.
- You want correct rate-limit headers and a Retry-After contract for clients.
- You must decide fail-open versus fail-closed when the store is down.
- You need atomic bucket updates so concurrent requests can't over-admit.
Example output
You get the design plus sketch code for the core limiter check: an algorithm choice comparing fixed window, sliding window, and token bucket with tradeoffs for [tiers]; a [backend_store]-backed token bucket enforcing limits consistently across many app servers; per-dimension keys and quotas for [limit_dimensions] with tiers mapping to refill rates; client-facing headers (limit, remaining, reset) and a Retry-After contract; the [failure_policy] behaviour when the store is unreachable, fail-open versus fail-closed, and why; and an atomicity strategy such as a Lua script so concurrent requests don't over-admit.
Pro tips
- Decide
[failure_policy]deliberately — fail-open keeps the API up but lets abuse through during an outage, while fail-closed protects the system but can block legitimate traffic; pick the one your service can live with. - Make the bucket update atomic from day one (Lua or equivalent), or two concurrent requests will both win the last token and you'll over-admit under load.
- Set
[limit_dimensions]to exactly what you key on, since each dimension needs its own counter and quota. - Map
[tiers]to clear refill rates so free, pro, and enterprise behave predictably and quotas are easy to reason about. - Match
[backend_store]to what you actually run; the atomicity approach depends on the store's scripting or transaction support. - Return the rate-limit headers consistently and document the Retry-After contract, so well-behaved clients can back off instead of hammering you.
