Limits that shed the abuse, not the customer.
A rate limit is easy to add and easy to get wrong: too loose and one client takes everyone down, too blunt and your best customer gets a 429 mid-checkout. We build a per-key token bucket that lets normal traffic burst, throttles a genuine flood with a Retry-After a client can back off to, and suppresses the keys that keep hammering, all counted in one place so the limit is real across every server.
Drive it: burst the key past its budget and watch the 200s turn to 429s with a real Retry-After, then refill. Flip to the pro tier and the same burst sails through.
burst drains the bucket, then it's 429s until it refills · flip to pro for more headroom
Where every request gets weighed.
What you just operated is this. Each request is keyed, checked against its own refilling bucket, and either served or thrown back with a Retry-After. The keys that ignore the throttle and keep hammering get suppressed, so an abusive retry loop can't drown the clients behaving.
Every request is bucketed by a key, an IP, or a user, so one client can't spend another's budget.
The key's bucket refills at a fixed rate. A request costs a token; normal traffic bursts freely.
Tokens left, the request is served, with the remaining budget in the response headers.
Bucket empty, it's a 429 with a Retry-After, the exact time until a token frees up.
A key that keeps slamming an empty bucket earns a short, escalating ban instead of endless 429s.
A token bucket, not a blunt counter.
A fixed window punishes a client that happens to arrive at the boundary and lets a burst slip through at the edges. A token bucket refills continuously, so it allows a real burst and still holds the average. The whole decision is a few lines, run atomically so ten servers agree.
1// limiter.ts — token bucket, per key, atomic in Redis2export async function allow(key, cost = 1) {3 const { cap, refill } = tierFor(key); // per-key limits4 const now = Date.now() / 1000;56 const b = await bucket.get(key); // { tokens, ts }7 const filled = Math.min(cap, b.tokens + (now - b.ts) * refill);89 if (filled < cost) // not enough tokens10 return { ok: false, retryAfter: (cost - filled) / refill };1112 await bucket.set(key, { tokens: filled - cost, ts: now });13 return { ok: true, remaining: filled - cost };14}Tiers are yours to set. A key carries its limits with it, so upgrading a customer is a config change, not a deploy.
The parts that make the limit real.
A limit that only holds on one server, or drops your best customer to protect against a bot, is worse than none. These are the parts that make it fair and dependable.
One noisy client can't spend another's budget. Every key, IP, or user gets its own bucket.
The 429 carries the real seconds until a token frees up, so a good client backs off exactly right.
A key that ignores the limit and keeps slamming earns a short, escalating ban, not infinite 429s.
The count lives in one store, not per-process memory, so ten servers enforce one real limit.
If the limiter store blips, you decide up front: fail open to stay available, or closed to stay protected.
Allowed, throttled, suppressed, per key, so you can see who's hitting the wall and tune the tiers.
A rate limit is one if-statement. A rate limiter is a small system.
The if-statement works until you have more than one server, and then two machines each count to the limit and let through double. Or a client retries the 429 in a tight loop and turns your defense into the attack. Or the limiter store goes down and takes the whole API with it. The real work is the shared atomic counter, the honest backoff, the suppression, and deciding what happens when the limiter itself fails. We build that part, because it's the part that actually holds under load.
Rate limits that hold under a real flood.
A per-key token bucket with honest Retry-After backoff, suppression for repeat abuse, and one shared count across every instance, so the limit is fair and real.