Load Balancing Interview Questions and Answers
Last updated 2026-11-19
Load balancers appear in nearly every system design answer, so interviewers check that you know more than "it spreads traffic". The follow-ups are about the OSI layer it works at, which algorithm fits which workload, how it detects dead servers, how sessions survive and how the balancer avoids being a single point of failure. This page covers each with a worked example.
Pair it with system design questions and system design for freshers. The caching layer is in caching questions, and the round plan is in how to crack a system design interview.
| Layers | L4 (TCP/UDP) and L7 (HTTP) |
|---|---|
| Algorithms | Round robin, least connections, hash, consistent hashing |
| Level | Easy to medium |
A short example configuration
In nginx a pool of servers is declared as an upstream block and requests are proxied to it. The weights and the backup server below show two ideas at once: capacity-aware distribution and a standby that receives traffic only when the others are down.
upstream app {
least_conn;
server 10.0.0.1:8080 weight=3;
server 10.0.0.2:8080;
server 10.0.0.3:8080 backup;
}
server {
listen 80;
location / { proxy_pass http://app; }
}Questions that follow naturally
- What happens when a server dies mid-request?
- How do you deploy a new version without dropping connections?
- How does the balancer itself stay available?
- Where do you terminate TLS?
Load Balancing Interview Questions and Answers: questions with worked answers
1 What is a load balancer and why do we need one?
easyA load balancer distributes incoming requests across several servers. It increases capacity by letting you scale horizontally, improves availability by routing around failed servers, and allows rolling deployments by draining servers one at a time. It also gives clients a single stable address while the pool behind it changes. It can additionally terminate TLS, compress responses and apply rate limits.
2 What is the difference between Layer 4 and Layer 7 load balancing?
mediumA Layer 4 balancer works on TCP or UDP connections, choosing a backend from IP addresses and ports without reading the payload. It is fast and protocol agnostic. A Layer 7 balancer understands HTTP, so it can route by URL path, host header, cookie or header, terminate TLS, rewrite requests and cache. It costs more CPU but enables content-based routing, such as /api to one pool and /static to another. Use L4 for raw throughput or non-HTTP protocols and L7 for smart routing.
3 Explain round robin and weighted round robin.
easyRound robin sends each new request to the next server in a fixed cycle: A, B, C, A, B, C. It assumes all servers are equal and requests cost about the same. Weighted round robin gives bigger servers more turns: with weights A 3 and B 1 the cycle serves A three times for each B. Nginx uses a smooth variant that spreads them out, in the order A, A, B, A, so the heavy server does not receive a long burst.
4 When is least connections better than round robin?
mediumWhen requests have very uneven durations, such as long uploads or streaming and short API calls mixed together. Round robin may pile several long requests on one server by chance, while least connections routes each new request to the server currently handling the fewest active connections, adapting to real load. Weighted least connections accounts for different server sizes. Least response time further considers measured latency.
5 What is IP hash, and how does consistent hashing improve on it?
mediumIP hash maps a client address to a server with hash(ip) mod N, so a client reaches the same server, which gives crude stickiness. When a server is added or removed N changes and most clients are remapped. Consistent hashing places servers and keys on a ring, so a change moves only a small share of clients, which matters when servers hold local state, such as caches. See the cache-cluster discussion in caching questions.
6 How do health checks work?
mediumThe balancer probes each backend periodically. An active check sends a request, such as GET /health every few seconds, and marks the server unhealthy after a number of consecutive failures or timeouts, restoring it after consecutive successes. A passive check watches real traffic and ejects servers that return errors. A good health endpoint verifies the dependencies the server really needs, such as database connectivity, without being so strict that one slow dependency ejects the whole fleet at once.
7 What are sticky sessions and what are the alternatives?
mediumSticky sessions (session affinity) route a client to the same server using a cookie or IP hash, so in-memory session data keeps working. The drawbacks are uneven load and lost sessions when that server dies. The better design is stateless servers with session data in a shared store such as Redis or in a signed token like a JWT, so any server can handle any request. Use stickiness only when state cannot be moved, such as WebSocket connections.
8 The load balancer is itself a single point of failure. How do you fix that?
hardRun at least two balancers. In an active-passive pair the standby takes over a shared virtual IP using a protocol such as VRRP (keepalived) when the active one stops sending heartbeats. In active-active, DNS returns several balancer addresses, or anycast routing advertises the same IP from several locations. Cloud balancers are already distributed across zones. Also consider global server load balancing, which uses DNS or anycast to send users to the nearest healthy region. DNS caching delays failover, so keep TTLs short for such records.
9 What is the difference between a load balancer, a reverse proxy and an API gateway?
mediumA reverse proxy sits in front of servers and forwards client requests to them, adding caching, TLS termination and compression. A load balancer is a reverse proxy that specialises in distributing traffic over several backends. An API gateway is a reverse proxy aimed at APIs, with authentication, rate limiting, request transformation and routing to microservices. In practice one product, such as nginx or Envoy, can play all three roles. See microservices questions.
10 Where should TLS terminate?
mediumTerminating at the load balancer takes the encryption work off the application servers and lets the L7 balancer inspect requests for routing, but traffic from the balancer to servers is plain unless you re-encrypt. TLS passthrough keeps encryption end to end but the balancer cannot read HTTP, so it works at L4 only. Compliance needs often require re-encryption to the backend inside the private network. Certificates are managed in one place when terminated at the edge.
11 How do you deploy a new version without dropping requests?
mediumUse connection draining: mark a server as draining so it receives no new requests, let in-flight requests finish within a timeout, then update and return it to the pool after it passes its health check. Repeat server by server (rolling deployment). Blue-green deployment keeps two full environments and switches the balancer from old to new at once, with instant rollback. Canary deployment sends a small percentage of traffic to the new version first and watches error rates.
12 How does a load balancer handle a backend that is slow rather than dead?
hardA slow server passes simple health checks but harms users. Remedies: set request timeouts and retry idempotent requests on another server (never blindly retry non-idempotent ones such as payments); use least-response-time balancing or passive outlier detection that ejects servers with high latency or error rates; limit concurrent requests per backend with a connection cap; and add circuit breakers in clients. Retries must be bounded with backoff, otherwise they multiply the load and make an overload worse.
13 How does autoscaling work with a load balancer?
mediumA scaling policy watches a metric, such as average CPU, requests per instance or queue depth, and adds instances when it exceeds a threshold and removes them when it falls. New instances register with the balancer once they pass the health check, and removed ones are drained first. Practical points: set a warm-up period so new servers are not judged before they are ready, keep a minimum capacity for failures, and scale on a metric that reflects user-visible load. Stateless servers make this far easier.
14 What is DNS load balancing and what are its limits?
easyDNS returns several IP addresses for a name, or rotates them, so clients spread across servers or regions. It is simple and works globally. Its limits are caching: resolvers and clients keep answers for the TTL, so failed servers keep receiving traffic until entries expire, and DNS has no real knowledge of server load or health unless a managed service adds it. It is usually combined with load balancers rather than replacing them.
Frequently asked questions
Is nginx a load balancer?
Yes, among other roles. It can act as a web server, reverse proxy and L7 load balancer, and with the stream module also at L4.
Which load balancing algorithm is best?
There is none. Round robin suits equal servers and short uniform requests, least connections suits uneven durations, and hashing suits cache locality or stickiness.
Does a load balancer improve security?
It can: it hides backend addresses, terminates TLS, filters traffic and absorbs some floods. It is not a replacement for a firewall or web application firewall.
What is the difference between hardware and software load balancers?
Hardware appliances offer dedicated throughput at high cost. Software (nginx, HAProxy, Envoy) and cloud services are flexible and cheaper and are far more common now.
Related guides
More in CS Fundamentals Interview Questions
- Low-Level Design Interview Questions with Code
- Memory Management Interview Questions with Numericals
- Message Queue and Kafka Interview Questions
- Microservices Interview Questions and Answers
- MongoDB Interview Questions and Answers
- Network Security Interview Questions: Cryptography, Firewalls and Attacks
- Normalization Interview Questions with Worked Examples
- NoSQL Interview Questions and Answers
- OOPs Interview Questions and Answers
- Operating Systems Interview Questions with Worked Answers
- OSI Model Interview Questions: Layers, PDUs, Devices and Encapsulation
- Paging and Virtual Memory Interview Questions with Numericals