Diagnosing Slow Requests Before You Blame the Database

Every request has three places to lose time: getting to your server, your application code doing its work, and any database or downstream call it makes along the way. Trace a slow request end to end and log a timestamp at each handoff, rather than guessing which layer is responsible.

Say a checkout request takes four seconds. If three of those seconds are spent waiting on a single database query, no amount of front-end optimization or CDN configuration will touch that. Find the layer first, then work on it.

An average response time hides the requests that actually matter. If your median response is fast but your 95th or 99th percentile is far slower, you have a specific subset of requests, often a particular query, a particular customer's data size, or a specific endpoint, causing real pain for a real slice of users while your dashboard looks fine.

Pull your percentile breakdown by endpoint before you decide anything is or isn't a problem. A slow p99 on your busiest endpoint usually matters more than a slow average across everything.

If you're on shared or burstable infrastructure, a slowdown can come from something else entirely sharing the same underlying hardware, not your own code at all. Check your provider's own resource metrics (CPU steal time, throttling, or credit balance, depending on the platform) before you spend a day rewriting a function that was never the problem.

This step gets skipped constantly because it feels like giving up on finding a real answer. In practice it's often the fastest one to check and the one most likely to save you from a wasted week.

Caching is the most common overcorrection. It makes a slow query feel fast without fixing why the query was slow, which is fine until the cache misses, the data changes, or you add a new endpoint that needs the same data in a slightly different shape and the whole problem reappears.

Use caching to buy time or handle genuinely expensive, rarely-changing lookups, not as a substitute for finding out why a query or a call is slow in the first place.

When it happens again, work through this in order:

• Trace one slow request end to end and time each handoff
• Pull the p95 and p99 for the affected endpoint, not just the average
• Check host-level resource metrics for contention outside your own code
• Only then decide whether the fix is a query, an architecture change, or a cache

Skipping straight to step four is how the same slowdown comes back in a different form a few weeks later.

Sometimes tracing points at something that isn't a quick fix at all: a service that has to call three other services sequentially to answer one request, a database that's being asked to do both fast lookups and heavy reporting queries at once, or a queue that backs up under normal load rather than only during spikes. Adding an index or a cache doesn't touch a bottleneck built into the shape of the system itself.

When you find one of these, resist patching around it with a bigger instance, since that buys time but not much of it. Put the redesign on the roadmap as a real project with an owner and a timeline, and be honest with the rest of the team about the fact that a quick fix isn't coming. A clearly scoped architectural project, even a slow one, beats a string of temporary patches that each buy a few weeks before the same complaint comes back.