How does a Consul agent know it is the leader of a cluster?

Question

In Consul you can have many agents as servers or clients. Amongst all servers one is chosen as the leader. From the agent\'s point of view, how does it know it is the leader?

Answer 1

The Consul leader is elected via an implementation of the Raft Protocol from amongst the Quorum of Consul Servers. Only Consul instances that are configured as Servers participate in the Raft Protocol communication. The Consul Agent (the daemon) can be started as either a Client or a Server. Only a Server can be the leader of a Datacenter.

The Raft Protocol was created by Diego Ongaro and John Ousterhout from Stanford University in response to the complexity of existing consensus protocols such as Paxos. Raft elects a leader via the use of randomized timers. The algorithm is detailed in Ongaro and Ousterhout's paper.

Consul instances that are configured as Clients communicate with the cluster via the Gossip Protocol which is based on Serf. Serf communication is eventually consistent. The Serf cluster has no central server, each node is considered equal. All nodes (Clients and Servers) in participating the Gossip/Serf Protocol spread messages to their neighbors, which in turn spread messages to their neighbors until the message has propagated to the entire cluster. Sort of like the infection path of a zombie apocalypse. This is done to greatly reduce communication overhead in the cluster as it scales to potentially tens of thousands of nodes.

Consul Clients can forward messages to any Consul Server which will then forward the message to the Leader. Consul Clients do not need to care which Consul Server is the Leader. Only the Servers need to care.

The Consul HTTP API running on any Consul Server will tell you which Server is the leader at $ANY_CONSUL_SERVER/v1/status/leader. However, this has nothing to do with how Consul Agents do leader election.