Setting Rate Limits That Protect Budget, Not Just Uptime

A request-per-minute limit protects your infrastructure from being overwhelmed. It does nothing to protect your bill if each request is cheap to serve but expensive to compute, like a call that triggers a large batch job or an external API charge per call. Set a cost-based cap alongside the request-based one, tracked in dollars or a proxy unit, not just request count.

This matters most for anything metered by a third party: usage-based infrastructure, external data providers, or any pay-per-call dependency. A request limit set correctly can still let a cost overrun through if nobody separately capped the spend.

A single global rate limit either throttles your biggest, most valuable customers or leaves the door open for abuse from a free-tier account, because one number cannot serve both cases well. Set limits per plan tier, with room to grant a documented exception for a specific customer who has a legitimate reason to exceed the default.

Make the exception process lightweight enough that a support engineer can grant a temporary bump without waiting on engineering, but log every exception so nobody forgets to revert a limit raised for a one-time event, and review that exception log on the same cadence as your other quarterly housekeeping tasks.

For example, imagine a free-tier account and an enterprise account sharing one global limit. Set it low enough to stop the free account from abusing the service and the enterprise customer gets throttled during a legitimate bulk import. Set it high enough for the enterprise customer and the free account can run up costs unchecked. Per-tier limits solve both problems, and a documented exception lets support raise the enterprise customer's ceiling for the length of the import. The common mistake is granting that bump and never reverting it, so record an expiration date at the moment you grant it.

A rate limit that fails silently, dropping requests without a clear error, turns a protective control into a confusing outage from the customer's side. Return a specific, well-documented status and a retry-after header so a well-behaved client can back off correctly instead of hammering the endpoint harder.

Test what your own client libraries and internal services do when they hit their own limits. It is common to build careful limits for external customers and forget that an internal service can trigger the exact same failure mode against another internal service.

A predictable limit response includes these pieces:

• Return a specific status code when a limit is hit, instead of silently dropping requests.
• Include a retry-after header so a well-behaved client can back off correctly rather than hammering the endpoint harder.
• Test how your own client libraries and internal services behave when they hit their own limits.
• Document the status and headers publicly, so customer engineers know how to handle the limit automatically.

By the time a hard limit is hit, the damage from a runaway process or a compromised key may already be done. Alert on unusual growth in usage or spend against a customer's normal baseline, not only on the moment a cap is reached, so a spike gets a human look before it maxes out and gets silently throttled or, worse, before the cap was set too high to catch it at all.

A daily or hourly spend anomaly check against each customer's typical usage pattern catches most of the expensive incidents days before a hard cap ever would have.

A rate limit set when your pricing plans were defined a year ago rarely still matches what those plans mean today. A tier that used to be your smallest offering can quietly become a mid-market plan as your product matures, and the original limit, sized for a much smaller workload, starts throttling exactly the customers you most want to keep happy.

Treat limit values as part of your pricing and packaging review, not a one-time engineering decision made in isolation. Whoever owns pricing changes should have a standing checklist item to confirm the technical limits still match what each plan is supposed to allow, and engineering should be in that conversation, not just informed of the outcome after the plan page already changed.