A Runbook for Surviving Upstream API Rate Limits in Production

Most APIs return their current quota state in response headers, commonly something like a remaining-requests count and a reset timestamp. Log these on every call and graph them, so your team can see a quota approaching zero hours before the first rejected request, not after. If the provider doesn't expose headers, track your own call volume against the documented limit and alert at a conservative threshold, since undocumented limits tend to be stricter in practice than in the docs. Attribute usage by caller, not just by endpoint, so when the limit does trip you can tell in seconds which service or job is actually driving the volume instead of guessing across every integration that touches that provider.

A naive retry on failure turns one rate-limited request into a thundering herd that keeps the limit tripped indefinitely. Use exponential backoff with jitter: wait a short interval, then double it on each subsequent failure, with some randomness added so multiple clients don't retry in lockstep. Respect a Retry-After header when the API sends one, since it's telling you exactly when the window resets rather than making you guess.

Put non-urgent calls behind a queue with a worker that respects the rate limit, so a burst of requests gets smoothed out over time instead of slamming the API all at once. For calls that are time-sensitive and can't wait in a queue, decide in advance what gets shed first: a background sync job should lose its turn before a customer-facing checkout flow does. Writing that priority order down before an incident means nobody has to invent it under pressure.

Settle these load-shedding decisions before an incident, not during one:

• Put non-urgent calls behind a queue with a worker that respects the provider's rate limit, so bursts are smoothed out over time.
• Decide in advance which calls get shed first, and write the priority order down where the on-call engineer can find it.
• Rank background sync work below customer-facing flows such as checkout, so the sync loses its turn first.
• Keep quota-sensitive background jobs separate from customer paths, since an account-wide limit affects both at once.

When a dependency is throttled, the rest of your product should degrade gracefully rather than hang or error opaquely. Cache the last known good response where staleness is acceptable, show a clear status to the user when it isn't, and make sure your own error responses distinguish 'we're rate limited upstream' from 'something is broken,' since the two need very different responses from your on-call engineer. A dashboard that lumps every upstream failure into one generic error metric hides exactly the pattern you need to see during an incident: is this one throttled dependency, or is something actually down.

If a single provider is a recurring bottleneck, the fix isn't always more engineering. Many providers offer a higher tier or a dedicated quota for paying customers with a documented use case, and a short email describing your traffic pattern often gets a limit raised faster than another week of backoff tuning. Keep a running note of which dependencies you've had to throttle around, since that list is exactly what to bring to a renewal or contract conversation.

Say a nightly reconciliation job calls a payments provider's API in a loop with no backoff, and one night it runs long and starts tripping the account-wide rate limit. Checkout calls to the same provider start failing too, because the limit isn't scoped to the job, it's scoped to your account. The fix isn't a faster batch job. It's separating quota-sensitive background work onto its own throttled queue, so a slow overnight job can never starve a customer-facing request for the same shared quota.