A Runbook for When an Upstream API Starts Throttling You

A generic error-rate alert usually won't distinguish a provider's rate limiting from any other failure, and the response is different for each. Alert on 429 responses from each upstream dependency as their own signal, and if the provider publishes quota headers (remaining calls, reset time), track those directly instead of waiting for the first rejection. Say a dashboard shows you're most of the way through your quota with an hour left in the window; that gives you time to react before the first request actually gets rejected.

Match your client's outbound rate to the provider's published limit with a token bucket or similar limiter, so you're shaping traffic before it hits the wall instead of reacting after. When a request does get throttled, retry with exponential backoff and jitter rather than a fixed delay; a fixed delay means every client that got throttled at the same moment also retries at the same moment, which just recreates the spike. If calls are bursty by nature (a batch job, a webhook fan-out), put a queue in front of the caller instead of letting it fire requests as fast as the work arrives.

Worker count matters here too. Adding more workers to drain a queue faster feels like the obvious fix for a backlog, but against a fixed upstream quota it just means more callers competing for the same limited pool of calls, so throughput doesn't actually improve; the limiter, not the worker pool, sets the ceiling. Size the limiter to the quota first, and only then decide how many workers you need to keep it saturated.

Wrap the dependency in a circuit breaker so a throttled provider doesn't back up worker threads or connection pools that other, unrelated requests also need. Where the feature can tolerate it, degrade gracefully instead of blocking: serve cached or slightly stale data, disable a non-critical feature temporarily, or queue the work for later instead of making the user's request wait on a dependency that's currently refusing calls. The goal is to contain the blast radius to the one feature that depends on the throttled API, not let it take down request handling for everything else.

Batch calls where the provider supports it instead of making one request per record; a single request that fetches or updates fifty records instead of fifty separate calls buys you the same headroom as a fifty-fold increase in your limit, without asking anyone for anything. Cache aggressively for data that doesn't need to be fetched fresh on every request, especially reference data that changes rarely but gets requested on every page load. If usage is genuinely growing, ask the provider for a higher limit or a different pricing tier rather than engineering around a ceiling that's meant to be negotiated. Spreading load across multiple API keys is sometimes technically possible, but check the provider's terms first; several treat that as a violation and will revoke access for all of them at once, which turns a throttling problem into an outage.

A common response to hitting a rate limit is to tune the retry count and backoff on the calling code and stop there. That reduces failed requests without reducing the number of calls being made, so the same quota gets consumed just as fast, and the throttling recurs at the same volume threshold. The retry logic controls how gracefully you fail; it doesn't touch how often you're asking. Fixing the volume, through batching, caching, or a higher limit, is the part that actually moves the ceiling.

When an upstream API starts throttling you, work through the runbook in order:

1. Alert on 429 responses from each upstream dependency as their own signal, and track quota headers such as remaining calls and reset time if the provider publishes them.
2. Match your outbound rate to the provider's limit with a token bucket, and retry throttled calls with exponential backoff and jitter.
3. Wrap the dependency in a circuit breaker, and degrade gracefully with cached data or a disabled non-critical feature where you can.
4. Cut the number of calls at the source by batching and caching, rather than only tuning retry counts.