Load Balancer or API Gateway? What Each One Does

A load balancer accepts connections and forwards them to a pool of backends. Its core jobs:

• Health checks: it stops sending traffic to instances that fail them.
• Distribution: round robin, least connections or similar strategies to spread work.
• TLS termination: decrypting traffic once at the edge so backends don't each manage certificates.
• Connection handling: keeping client connections stable while backends are replaced during deploys.

Some load balancers work at the connection level and are unaware of HTTP. Others read the request path and headers and can route `/api` to one service and `/app` to another. Path-based routing is often enough for a monolith or a few services.

A gateway understands APIs as products. It adds behavior that has nothing to do with spreading load:

• Authentication and authorization checks before requests reach your services.
• Rate limiting and quotas per key, plan or tenant.
• Request and response transformation, such as translating an old API version to a new one.
• Usage analytics per client, and sometimes a developer portal and key issuance.
• Central policy: one place to add logging, CORS rules or a new header.

Products like Kong and Apigee sit in this category. They earn their place when many services or many external clients need the same policies applied consistently.

Work through these questions in order:

1. Do you have more than one instance of a service? You need a load balancer.
2. Do outside developers or customers call your API with keys? You probably want a gateway, or at least gateway-style rate limiting and key management.
3. Do you have several services that each reimplement auth and throttling? A gateway removes that duplication.
4. Do you have one web app and one backend, and only your own front end calls it? A load balancer alone is usually enough.

Say you run a single Node service behind two instances. Adding a gateway means another hop, another thing to configure and another failure mode, with no problem solved. Say instead you expose a public API with paid tiers to a hundred integrators. Key management and per-plan quotas are gateway work.

The common layout is client, then a load balancer or CDN at the edge, then the gateway, then your services. The edge balancer terminates TLS and spreads traffic across gateway instances. The gateway applies policy and routes to the right service, and each service may sit behind its own internal balancer.

Notice that the gateway itself must be highly available, so it typically runs as several instances behind a load balancer. That's why the choice is rarely either-or once you outgrow a single service. Look at the gateway comparison guide when you're ready to compare products.

If you add a gateway to a live system, don't switch everything at once. Route one low-risk API through it first. Keep the existing authentication in the service until the gateway's checks have run in parallel for a while and you've compared their decisions in logs. Only then remove the duplicate check. This staged approach turns a risky cutover into a series of small, reversible steps.

A few patterns repeat:

• Adopting a gateway to solve a scaling problem. Scaling is a load balancing and capacity issue.
• Putting business logic in gateway configuration, where it is hard to test and review.
• Running a single gateway instance, which turns a convenience layer into your biggest outage risk.
• Forgetting the extra hop's latency and timeouts. Set timeouts at each layer, shortest at the inside.
• Duplicating auth in both the gateway and every service without deciding which is authoritative.