Sizing API Rate Limits So They Actually Protect You

Say your typical customer makes 200 API calls a day, but your single heaviest customer, running a legitimate batch sync job overnight, makes 8,000 in a ten-minute window. A limit set from the average would throttle that customer constantly. Pull your actual top-percentile usage from logs, not from what you assume "normal" looks like, and set the limit above that, with headroom for growth.

This single step catches the most common rate-limiting mistake: building the limit around a mental model of typical usage instead of the real distribution, which is almost always far more skewed than people expect.

Pull at least 30 days of logs before setting anything, since a shorter window can hide a legitimate monthly or quarterly batch job that only runs once. A limit tuned on two weeks of data will throttle that job the first time it actually runs, and the resulting support ticket is a worse way to discover the gap than checking the data upfront.

An IP-based limit punishes every user behind a shared corporate NAT or VPN for one bad actor's behavior, and it does nothing to stop an attacker who simply rotates IPs. Tie limits to the authenticated caller's identity instead, whether that's an API key, an OAuth client, or a user session, since that's the identity zero trust already requires you to establish before the request is processed anyway.

Keep a lightweight IP-based limit as a backstop against unauthenticated abuse, but treat identity-based limiting as the primary mechanism, not the fallback.

A read of a cached, lightweight resource and a call that triggers an expensive downstream AI model or a heavy database query shouldn't share the same quota. Set limits per endpoint category based on actual cost to serve, not a single blanket number applied everywhere, or you'll end up either overprotecting cheap endpoints or underprotecting expensive ones.

This matters most for anything that calls out to a metered third-party service, where a burst of legitimate-looking traffic can generate a real, unexpected bill before anyone notices a spike.

A rate limit caps request volume; a spend cap caps dollars, and they're not the same protection. A caller making requests within their rate limit can still generate a large bill if each request is expensive enough, especially for endpoints backed by usage-priced infrastructure. Set an explicit dollar ceiling, checked independently of the request-count limit, so a cost spike gets caught even when the volume looks unremarkable.

Alert well before the cap is hit, not only when it's reached, so someone can investigate whether the spend is legitimate growth or something worth stopping. For example, if an endpoint proxies calls to a metered third-party model and a caller's usage jumps from $40 a day to $400 a day, that's worth a look before the monthly bill arrives, not after.

A caller that exceeds its limit should get a clear, documented response, a specific status code and a retry-after header, not a generic error that looks like an outage. Decide this behavior and test it deliberately, including for your own internal callers, so a burst of legitimate traffic degrades predictably instead of producing a confusing failure that looks like a bug in the API itself.

Review your limits again after any customer's usage roughly doubles, the same way you'd revisit any other capacity assumption, rather than waiting for a complaint to prompt the review.

Size and roll out a limit in this order:

1. Pull a long enough window of real usage logs to include monthly or quarterly batch jobs, and find your heaviest legitimate callers.
2. Set the limit above that top-percentile usage with headroom for growth, instead of basing it on an average customer.
3. Tie the limit to the authenticated identity, such as an API key, OAuth client or user session, rather than an IP address.
4. Give expensive endpoints their own lower quotas based on the cost to serve them.
5. Add a separate dollar spend cap, then decide and test what callers see at the limit, including a clear status code and retry-after header.