What you'll learn
Quick Answer
Leader election is how a distributed system automatically picks one node to be in charge of a task that only one node should do at a time, and re-picks a new one if that node dies. The Bully algorithm does it by always picking the highest surviving ID; Raft does it with randomized timeouts so one node usually becomes a candidate first; many production systems delegate the whole problem to ZooKeeper or etcd.
Why Bother With a Leader at All
Some jobs in a distributed system go wrong if more than one node does them at the same time. Two nodes both thinking they're allowed to accept writes to the same partition can produce conflicting data; two nodes both assigning work can double-assign it; two nodes both handing out “the next sequence number” can hand out the same number twice. The straightforward fix is to designate one node as the leader for that job and have every other node defer to it.
That creates an obvious new problem: the leader is now a single point of failure. If it crashes, hangs, or gets cut off by a network partition, the rest of the system needs to notice and agree on a replacement — automatically, without a human intervening at 3 a.m. That agreement process (detect the leader is gone, pick exactly one replacement, get everyone to accept the same replacement) is leader election, and it's a genuinely hard problem because “the leader is gone” and “the network is just slow right now” look identical from the outside.
The Bully Algorithm: Highest Surviving ID Wins
The Bully algorithm is one of the oldest and simplest election algorithms, named for the fact that the highest-numbered live process always wins and “bullies” everyone else into accepting it. Every process has a unique, fixed ID, and the rule is: the highest-ID process that's currently alive is the leader. It uses three message types — ELECTION (announcing an election is happening), ALIVE (a higher-ID process answering to say it's still around), and COORDINATOR (the winner announcing itself).
When a node notices the current leader isn't responding, it sends ELECTION to every node with a higher ID than itself. If nobody higher answers, it declares itself the winner and broadcasts COORDINATOR to everyone. If a higher-ID node does answer, that node takes over running the election upward, and the original node just waits for the eventual COORDINATOR message. Simulating a 5-node cluster confirms the cascade this produces when the leader dies:
=== Bully algorithm: initial cluster {1,2,3,4,5}, node 5 is leader ===
node 1 notices the leader is unreachable, starts an election
node 1 -> ELECTION -> nodes [2, 3, 4, 5]
node 1 receives ALIVE from [2, 3, 4, 5], stands down and waits
node 5 hears no ALIVE from anyone higher -> declares itself leader
node 5 -> COORDINATOR -> all live nodes
Leader IDs recorded by each live node: {1: 5, 2: 5, 3: 5, 4: 5, 5: 5}
=== Node 5 (the leader) crashes; node 2 is first to notice and starts an election ===
[event] node 5 crashes
node 2 notices the leader is unreachable, starts an election
node 2 -> ELECTION -> nodes [3, 4]
node 2 receives ALIVE from [3, 4], stands down and waits
node 4 hears no ALIVE from anyone higher -> declares itself leader
node 4 -> COORDINATOR -> all live nodes
Leader IDs recorded by each live node: {1: 4, 2: 4, 3: 4, 4: 4}Killing node 4 next and having node 1 notice produces exactly the same pattern one level down, with node 3 taking over — the algorithm always converges on whichever surviving node has the highest ID, no matter which node happens to notice the failure first.
Where Bully Falls Short
Bully is easy to reason about, but it has two real limits that keep it out of most modern systems. The first is message volume: in the worst case, where the lowest-ID node is the one that notices the failure, that node contacts every higher node, several of those may each in turn contact the nodes above them, and the eventual winner then broadcasts to the entire cluster — roughly quadratic in the number of nodes for a single election.
The second, more fundamental limit is what the algorithm depends on: a fixed, comparable ID per node, and a reliable way to tell “this node is dead” apart from “this node is just slow right now.” That second part is genuinely hard in an asynchronous network — a node that's merely overloaded looks exactly like a crashed one from the outside. Set the failure-detection timeout too aggressively and you get unnecessary re-elections triggered by ordinary network jitter; set it too conservatively and a real failure takes longer than it should to be noticed and fixed.
Raft: Let Randomness Break the Tie
Raft, the consensus protocol behind systems like etcd, takes a different approach entirely: instead of comparing static IDs, it relies on timing and majority votes. Every node starts as a follower. The current leader sends periodic heartbeats; if a follower doesn't hear one before its own election timeout expires, it assumes the leader is gone, increments a counter called the term, promotes itself to candidate, and requests votes from everyone else. A candidate that gets votes from a majority of the cluster becomes the new leader for that term.
The critical detail is that each node's election timeout isn't fixed — it's randomized, commonly in a 150–300ms range. That randomization is deliberate: if every node's timeout were identical, a leader's death would make every follower become a candidate in the same instant, splitting the vote across all of them with nobody reaching a majority. A short simulation of five servers' randomized timeouts shows why randomizing avoids that:
5 servers, election timeout randomized uniformly in [150ms, 300ms], 3 independent rounds:
Round 1: timeouts=[('server-2', 166), ('server-4', 180), ('server-0', 184), ('server-3', 215), ('server-1', 295)]
-> server-2 fires first, 14ms clear of the next server (enough time to request votes before a rival becomes a candidate)
Round 2: timeouts=[('server-0', 164), ('server-2', 171), ('server-1', 173), ('server-4', 193), ('server-3', 242)]
-> server-0 fires first, 7ms clear of the next server (enough time to request votes before a rival becomes a candidate)
Round 3: timeouts=[('server-2', 183), ('server-0', 210), ('server-3', 244), ('server-4', 271), ('server-1', 289)]
-> server-2 fires first, 27ms clear of the next server (enough time to request votes before a rival becomes a candidate)Compare that to what happens with a fixed timeout instead of a randomized one:
Now show what happens WITHOUT randomization (fixed 200ms timeout for everyone):
timeouts={'server-0': 200, 'server-1': 200, 'server-2': 200, 'server-3': 200, 'server-4': 200}
-> all 5 become candidates in the same instant -> vote splits 5 ways
-> nobody gets a majority -> election times out -> must retry, possibly foreverIn every randomized round, one server's timer fires meaningfully earlier than the rest — in these three runs, by 7 to 27 milliseconds — which is enough time for it to request votes and become leader before anyone else even becomes a candidate. If a split vote does still happen occasionally (it can, if two timeouts land close enough together), Raft doesn't need a special tie-break rule for it: the term simply times out, every node picks a fresh random timeout, and the process repeats until one candidate gets a clean head start.
Delegating It: ZooKeeper and etcd
Most engineering teams don't implement Bully or Raft from scratch — they delegate the whole problem to a coordination service built on top of one.
- ZooKeeper offers leader election via ephemeral sequential znodes: every candidate creates one, and whoever holds the lowest sequence number is the leader. If that node's session expires — it crashed, or simply got disconnected — ZooKeeper deletes its znode automatically, and the next-lowest candidate takes over without anyone having to run an explicit election protocol themselves.
- etcd, itself built on Raft, typically implements leader election through a lease: a candidate acquires a lease and renews it periodically, and whoever holds a currently-live lease is the leader. Kubernetes' own controller manager uses exactly this mechanism against etcd, renewing its lease roughly every 10 seconds against a 15-second TTL by default — so a crashed controller gets replaced within about 15 seconds without any custom election code in Kubernetes itself.
Kafka is a useful real-world illustration of this whole space evolving. For years, Kafka elected its cluster controller through a ZooKeeper ephemeral /controller znode — the first broker to create it won. As of Kafka 4.0 (released March 2025), ZooKeeper has been removed entirely in favor of KRaft, Kafka's own internal Raft-based metadata quorum, where the metadata log itself elects the leader with no separate coordination service required at all.
The Danger After Election: Split-Brain
Winning an election doesn't fully solve the problem, because the “old leader” a system elects around might not actually be dead — it might just be unreachable, stuck on the wrong side of a network partition, still very much alive and still convinced it's in charge. If it keeps accepting writes while a newly elected leader also accepts writes, the system now has two leaders disagreeing with each other: split-brain.
The standard defense is fencing: attach an increasing epoch or term number to every election (Raft already does this as part of its protocol), and have the actual resource being protected — a shared disk, a database, a downstream service — reject any request tagged with an older epoch than the highest one it's already seen, even if that request genuinely comes from what used to be the real leader. Lease-based systems add a second layer on top: a leader's authority automatically expires unless it's actively renewed, so a partitioned leader that can no longer reach the coordination service loses its authority within one lease TTL, whether or not it's still technically alive.