Load Balancing

Intermediate · 17 min read · High-Level Design, Scaling the Application Tier

L4 vs L7, balancing algorithms, health checks, and avoiding a single point of failure.

A load balancer sits between clients and a pool of servers and distributes incoming requests across them. It's the component that makes horizontal scaling work: clients talk to one address, while behind it any number of servers can be added, removed, or replaced.

A load balancer provides four things:

Where load balancers appear

Load balancers appear at several layers of a typical architecture:

L4 vs L7 load balancing

Content-based routing lets one entry point serve many services:

Load-balancing algorithms

These algorithms don't look at server state.

Health checks

A load balancer must know which backends can serve traffic.

Removing the load balancer as a single point of failure

A single load balancer is itself a single point of failure. Common solutions:

Other responsibilities of load balancers

Load balancer vs reverse proxy vs API gateway

These terms overlap heavily:

Client-side load balancing

Instead of a central balancer, each client gets the list of healthy instances from a service registry (Consul, Eureka, etcd, Kubernetes endpoints) and picks one itself.

Interview questions

  • When would you choose an L4 load balancer over L7?
  • All of your servers suddenly fail health checks at the same time. What could cause this and how would you prevent the outage?
  • How does a load balancer support zero-downtime deployments?
  • What's the problem with using hash(client_ip) mod N for session affinity?

Practice

Write your answer before opening the suggested one. Anything you mark "Partly" or "Missed it" comes back in your review deck.

Further reading

Key takeaways

  • Load balancers enable horizontal scaling, provide failover via health checks, and act as a control point.
  • L4 balances connections and is fast and protocol-agnostic; L7 balances requests and can route on content.
  • Choose the algorithm based on the workload: round robin for uniform requests, least connections for long-lived ones, hashing for affinity.
  • Health checks must not depend on shared dependencies, or a partial failure becomes a total outage.
  • The load balancer itself must be redundant: active-passive with a VIP, active-active, anycast, or a managed service.

Previous: Scalability: Vertical vs Horizontal · Next: Choosing a Database: SQL vs NoSQL