Distributed Systems Interview Questions
Last updated 2026-11-19
Distributed systems questions probe whether you understand what goes wrong when work is spread across machines: messages are delayed or lost, clocks disagree, nodes fail halfway. This page works through the usual questions with small examples, including a Lamport clock trace, a vector clock comparison, quorum sizing and the blocking weakness of two-phase commit.
Context comes from system design questions and NoSQL questions. Single-node transactions are in transactions and ACID, and applied patterns in microservices questions. The plan for the round is in how to crack a system design interview.
| Core topics | CAP, consistency, consensus, replication, time |
|---|---|
| Worked examples | Lamport clocks, vector clocks, quorums, 2PC |
| Level | Hard |
The fallacies to quote
Peter Deutsch's fallacies of distributed computing are a good opening line: the network is reliable, latency is zero, bandwidth is infinite, the network is secure, topology does not change, there is one administrator, transport cost is zero and the network is homogeneous. Every design decision in this area is a response to one of them.
Failure model vocabulary
- Crash-stop: a node fails and never returns.
- Crash-recovery: a node fails and later restarts, with durable state intact.
- Omission: messages are lost.
- Byzantine: a node behaves arbitrarily or maliciously.
- Partition: groups of nodes cannot communicate with each other.
Distributed Systems Interview Questions: questions with worked answers
1 What is a distributed system and why is it hard?
easyA distributed system is a set of independent computers that cooperate over a network and appear to users as one system. It exists for scale, availability and locality. It is hard because of partial failure (some parts fail while others continue), unreliable and variable networks, no shared clock and no single global state. You cannot distinguish a slow node from a dead one, which underlies most of the difficulty.
2 Explain the CAP theorem and common misreadings.
mediumDuring a network partition, a system must choose between consistency (linearizable reads return the latest write or fail) and availability (every non-failed node answers). Partition tolerance is not optional because partitions happen. Misreadings: that you pick two of three at all times (without a partition you can have both), that CP means unavailable always (only for the minority side during a partition), and that CAP consistency equals ACID consistency. PACELC adds that even without partitions there is a latency versus consistency trade-off.
3 Compare strong, causal and eventual consistency.
mediumStrong (linearizable) consistency makes the system behave as if there is one copy and every operation takes effect at a single point between its start and end, so a read sees the latest completed write, at the cost of coordination latency. Causal consistency preserves cause-and-effect order: if you read a post and then reply, nobody sees the reply without the post, but unrelated updates may appear in different orders. Eventual consistency promises only that replicas converge if updates stop, with the lowest latency and highest availability.
4 How do Lamport clocks work? Trace an example.
hardEach process keeps a counter. On a local event or before sending, increment it; a message carries the sender's counter; on receive, set the counter to max(local, received) + 1. If P1 has an event (clock 1), then sends a message (clock 2), and P2 has clock 0, then P2 receives it and sets clock to max(0, 2) + 1 = 3. The rule guarantees that if event a happened before b, then clock(a) is less than clock(b). The reverse does not hold: a smaller clock does not prove causality, which is why Lamport clocks cannot detect concurrent events.
5 How do vector clocks detect concurrent updates?
hardEach node keeps a vector with one counter per node. A node increments its own entry on an event and merges vectors on receive by taking the element-wise maximum, then incrementing its own entry. Vector A happened before B if every entry of A is less than or equal to B and at least one is smaller. If neither dominates, the events are concurrent. For example [2,0] and [0,1] are concurrent, so a store like Dynamo keeps both versions and asks the application or a merge rule to reconcile, while [1,0] and [2,1] are ordered.
6 Explain how Raft elects a leader and replicates logs.
hardNodes are followers, candidates or leaders, and time is divided into terms. A follower that hears no heartbeat before a randomised election timeout becomes a candidate, increments its term and requests votes; a node grants one vote per term, and a candidate with a majority becomes leader. The leader appends client commands to its log, replicates them with AppendEntries, and commits an entry once a majority holds it, then applies it. Voters refuse candidates whose logs are less up to date, which guarantees a new leader has all committed entries. Randomised timeouts avoid repeated split votes.
7 How many failures can a cluster of N nodes tolerate with majority quorums?
mediumA majority is floor(N/2) + 1, so the cluster tolerates floor((N-1)/2) crash failures: 3 nodes tolerate 1, 5 tolerate 2, 7 tolerate 3. Even sizes add no tolerance: 4 nodes need a majority of 3 and still tolerate only 1. This is why consensus clusters (etcd, ZooKeeper) use odd sizes. Majorities always overlap, so two partitions can never both form a majority, which prevents split brain.
8 How does two-phase commit work, and why can it block?
hardA coordinator asks all participants to prepare; each writes its changes durably and replies yes or no. If all vote yes the coordinator logs the decision and tells everyone to commit, otherwise to abort. Blocking arises because a participant that voted yes must wait for the decision and cannot unilaterally commit or abort: if the coordinator crashes after collecting votes, participants hold locks until it recovers. Three-phase commit reduces blocking but assumes bounded delays. Microservices usually avoid 2PC and use sagas; see microservices questions.
9 Compare leader-follower, multi-leader and leaderless replication.
mediumLeader-follower sends all writes to one leader that replicates to followers: simple and consistent for writes, but the leader is a bottleneck and failover needs care; asynchronous replication can lose recent writes. Multi-leader accepts writes in several places, such as one per data centre, improving write latency and availability but requiring conflict resolution. Leaderless (Dynamo-style) lets any replica accept writes and uses quorums with read repair, and tolerates node failures well but gives weaker consistency guarantees.
10 What is split brain and how do you prevent it?
mediumSplit brain happens when a partition leaves two parts of a cluster each believing it is the leader, so both accept writes and diverge. Prevent it with majority quorums for leader election, so only the side with a majority can act; with fencing tokens, where each leader term has an increasing number and storage rejects writes with an older token; and with STONITH (shooting the other node in the head) in traditional clusters.
11 What does exactly-once delivery mean in practice?
mediumTrue exactly-once delivery over an unreliable network is impossible, because a sender cannot know whether a lost acknowledgement means the message was lost or just the reply. Systems offer at-most-once (may lose), at-least-once (may duplicate) and the practical goal of effectively-once processing, achieved with at-least-once delivery plus idempotent handlers or transactional writes that deduplicate. Kafka's "exactly-once" covers its own read-process-write transactions, as described in message queue questions.
12 How are data partitioning and rebalancing handled?
mediumPartition by hash for even distribution or by range for scans, and map partitions to nodes. A fixed number of partitions, far more than nodes, lets you rebalance by moving whole partitions, as Kafka, Cassandra vnodes and Redis Cluster slots do. Consistent hashing limits data movement when nodes change. Handle hot partitions by splitting them or salting keys. See NoSQL questions for shard-key choice.
13 What is a CRDT?
mediumA conflict-free replicated data type is a data structure whose replicas can be updated independently and merged automatically into the same state, because the merge is commutative, associative and idempotent. A grow-only counter keeps one count per node and merges by taking the maximum per node; its value is the sum. Sets and registers have CRDT forms too. They give strong eventual consistency without coordination, used in collaborative editors and some databases, at the cost of limited operations and metadata growth.
14 How do gossip protocols and failure detectors work?
mediumIn gossip, each node periodically picks a few random peers and exchanges state, so information spreads exponentially, reaching N nodes in about log N rounds, with no central coordinator and tolerance of failures. Failure detectors use heartbeats and timeouts; because slow and dead look the same, detectors are probabilistic. The phi accrual detector outputs a suspicion level that adapts to observed network delays. Cassandra and Consul use gossip for membership.
Frequently asked questions
Is Paxos or Raft asked more?
Raft, because it was designed to be understandable. Know leader election and log replication in Raft, and that Paxos solves the same consensus problem.
What is the FLP result?
In a fully asynchronous system, no deterministic consensus protocol can guarantee termination if even one node may crash. Practical protocols use timeouts and randomisation to work around it.
Do I need to implement consensus in an interview?
No. Be able to explain how it works and when to use a system such as etcd or ZooKeeper rather than building your own.
What is the difference between availability and reliability?
Availability is the fraction of time the system serves requests. Reliability is that it behaves correctly, including not losing data, over time.
Related guides
More in CS Fundamentals Interview Questions
- DNS Interview Questions: Resolution, Record Types, Caching and Troubleshooting
- ER Model Interview Questions
- File System Interview Questions: Inodes, Allocation and Disk Scheduling
- Git Interview Questions and Answers with Commands
- HTTP and HTTPS Interview Questions: Methods, Status Codes, TLS and Caching
- Inheritance and Polymorphism Interview Questions
- IP Addressing and Subnetting Interview Questions with Worked Solutions
- Linux Commands Interview Questions with Examples
- Linux Interview Questions and Answers: Concepts, Permissions and Processes
- Load Balancing Interview Questions and Answers
- Low-Level Design Interview Questions with Code
- Memory Management Interview Questions with Numericals