The recipe
ZooKeeper gives you a hierarchical store of small nodes called znodes. A lock is built from two flavors:
- Ephemeral nodes vanish when the creating session dies, so a crashed holder cannot deadlock the system.
- Sequential nodes get a monotonically increasing counter appended by the server.
To acquire a lock a client creates an ephemeral sequential child under a lock parent, say lock slash n. The client with the lowest sequence number owns the lock. Everyone else waits.
Avoiding the herd
A naive design watches the parent, so every release wakes every waiter at once. That is the herd effect. The fix is to watch only the node with the next lower sequence number. When that single predecessor disappears, exactly one waiter is notified and re acquires.
Why it is correct
- Ordering comes from the server assigned counter, giving FIFO fairness.
- Liveness comes from ephemeral cleanup on session expiry.
- Safety holds because only one node can be lowest at a time.
Key idea
Ephemeral sequential znodes plus a watch on your immediate predecessor give a fair, deadlock free lock with no thundering herd.