About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.
Decide which side may accept new work before either side accepts it. A site that can still reach its users is not, by that fact, allowed to write. For Raft, a valid leader must obtain the required voting quorum to commit new entries. A written decision or stopping another site cannot replace that quorum. A non-consensus single-writer service instead needs its own supported promotion and fencing process. If a Raft partition lacks the required quorum, it cannot commit new writes. Silence is a result. It is the result the consensus rule is built to produce.
This is a decision article. It does not place a witness, it does not change cluster membership, and it does not repair a database. The sources are the Raft consensus paper, for election and commit quorum, and NIST SP 800-34 Rev. 1, for recovery validation and examples of alternate-site designs. Neither defines every product’s multi-site semantics.
Why it matters
A partition leaves both sides looking healthy from the inside. Each side can see its own disks, its own applications and a share of the users. A correctly operating consensus service prevents both partitions from committing conflicting histories. Uncoordinated promotion, independent clusters or applications outside that protection can instead create divergent writes. Merging them later is a data repair, which is a worse problem than the outage you were trying to shorten.
NIST SP 800-34 Rev. 1 describes concurrent processing as one possible recovery-validation arrangement. Its high-impact template explicitly says concurrent processing is not required. Where used, it means running a system at two separate locations until there is a level of assurance that the recovered system is operating correctly and securely. After validation testing, the plan's people declare that reconstitution is complete and the system is operating normally. The declaration can sit in a recovery log. It is an act with names on it, not a feeling that both sites seem up.
That does not classify every dual-site deployment as a temporary test. The same publication describes a different, designed pattern. A hot site can be two identical systems at alternate locations, both in production, serving different regions or sharing a load, each able to carry the full workload, with data synchronised continuously between them. That pattern has a condition: the synchronisation is happening. If a required synchronisation link fails, assess the design’s documented partition behaviour. Some designs stop a partition, some remain read-only, and others support explicitly conflict-resolving writes; NIST’s example alone does not choose the policy.
How it works
What "a majority" is a majority of
Raft is a consensus algorithm for a replicated log. The paper states the availability rule directly. The servers are fully functional as long as any majority of them are operational and can communicate with each other and with clients. A typical cluster of five can therefore tolerate the failure of any two. Servers are assumed to fail by stopping. They may later recover from state on stable storage and rejoin. The conditions the algorithm claims to handle include network delays, partitions, and lost, duplicated or reordered packets, short of Byzantine fault, where a server lies rather than stops.
Leadership requires a majority of the configured voting members in a stable configuration. Non-voting learners do not count. During the paper’s joint-consensus membership transition, both old and new configurations require their respective majorities. A candidate that receives votes from a majority of the full cluster becomes the leader. A candidate has to contact a majority of the cluster to be elected. Election safety says at most one leader can be elected in a given term. In some situations an election produces a split vote. That term then ends with no leader, and a new term begins with a new election.
Those sentences are the gate. "Majority" is not "most of the servers we can see". For a stable configuration, it is a majority of the full voting membership. A side that contains two servers of a five-server cluster cannot elect a leader, because two is not a majority of five, however healthy those two look. A cut that leaves three and two does have a majority on the three side, and the paper's rule makes that side the one that can elect. A cut that leaves neither side with a majority of the full membership leaves both sides unable to elect. The split vote, ending with no leader, is a defined outcome. It is not a fault to be patched by appointing a writer on each side so that users see something.
The paper covers non-Byzantine faults, including network partitions. An isolated old leader can temporarily still believe it is leader in an earlier term; election safety guarantees one elected leader per term, not one process claiming leadership across all terms. Without the required acknowledgements it cannot commit new log entries. External side effects must remain behind the product’s supported commit and fencing controls. Our recommendation: if the service is built on this kind of majority, do not override it during the incident. For a non-consensus service that cannot merge histories, establish one authorised writer and effective fencing using its own design. Do not invent a majority denominator for a service with no voting protocol.
What running both sites is allowed to mean
NIST's concurrent processing is both sites running, with a purpose and a finish. The purpose is assurance that the recovered system is correct and secure. The finish is a declaration, made with the system owner, that normal operation has resumed, and a decision about whether the system changed enough to need reauthorisation. Data testing checks that recovered data is complete and current to the last available backup. Functionality testing checks that the system actually does what it is supposed to do.
Our recommendation uses that definition narrowly. Both sites may be powered on. Both may be used to read, to compare, and to test. New work that changes state goes to one side only, until the declaration. The other side is the comparison copy, not a second counter. If the design really is the hot-site pattern, both sides already had a rule for how updates meet, and that rule is still operating. A failed link requires the design’s partition policy; do not assume a generic single-writer promotion is automatically safe or supported.
The declaration is the gate that closes the period. Write down which site is live, who decided, and what the other site will do with any changes it took during the test. NIST puts that declaration after the validation, not before it, and not instead of it. Turning the users loose on both addresses and calling it concurrent processing skips the only part of the definition that keeps the two histories from diverging.
The gates, in order
Our recommendation:
- Name the service. Say whether it can tolerate two writers. Identify actual write semantics, transaction boundaries and coordination. Multiple clients can write through one Raft leader; some mailbox or inventory designs deliberately support coordinated multiple writers. A static read-only website creates no application writes, though its supporting systems may. Product labels alone do not settle this question. If you cannot say, treat it as single-writer.
- Write down the current voting configuration, including unreachable voters and any joint-consensus transition. Learners are not votes. Use the product’s actual quorum rules. Servers that are powered off still count in the denominator until a membership change has actually completed. This article does not describe how to change membership. Doing it in the middle of a partition, so that a minority becomes "all of the cluster", manufactures a majority the paper's election rule did not grant.
- For Raft, establish which partition has the required voting quorum, a valid elected leader and ability to commit. A simple node count is necessary evidence, not sufficient proof of application readiness. If no side holds it, both sides refuse new work. Users waiting is the cost the majority rule is paying to avoid two histories.
- If the second site is being used for recovery validation, define that validation role explicitly. It runs so you can test data and function. It does not take a second stream of changes. End the period with the declaration NIST describes, including whether the system now needs a fresh authorisation because it changed.
- Point clients at the declared side after the declaration, not before. Client routing to two sites is safe only if both obey the supported write-coordination design. Before returning traffic to the old site, reconcile writes made since cutover and prove it is current; an intact but stale copy is not a safe rollback.
- When a stopped server returns, it rejoins. It does not resume as a rival. The paper's recovery assumption is that a failed server comes back from stable storage and rejoins the cluster. An operator who lets it keep serving the clients it had during the split is choosing the split brain the election rule had already refused.
Nothing in these gates tells you where to put an extra vote. A deployment that needs a tie-break is a design problem for a day when you are not in the outage. During the outage the membership is the one you already have.
What to choose and why
These are our recommendations. They are not a configuration of any one product.
| Situation | Reasonable choice |
|---|---|
| One side has a majority of the full membership and can communicate | Verify the valid elected leader and commits; a minority cannot commit conflicting entries |
| The cut leaves neither side a majority of the full list | Both sides stop writes. Wait, or repair the path. Do not appoint two leaders |
| An election splits and ends with no leader | Leave it with no leader. A new term is the paper's next step, not a manual writer on each side |
| You want both sites powered on to compare them | Call it concurrent processing. Test. Declare one side. Only that side takes changes |
| The estate was designed as two live sites with continuous synchronisation | Use the documented partition policy and write-coordination rules; do not improvise promotion |
| A returned server still has clients from the split | Stop it serving. Let it rejoin the side that holds the majority |
| Someone proposes dropping the missing servers from the count so a minority can proceed | Do not manufacture quorum by editing the denominator. Use only the supported membership or disaster-recovery procedure, with fencing and explicit loss assessment |
| The failed server may be lying rather than stopped | Raft does not cover that. Treat it as a security incident, and do not use the majority rule as if the server were merely quiet |
Risks and limits
- Raft's availability sentence is about servers that fail by stopping, under non-Byzantine conditions. A compromised server that keeps answering is outside the paper. Do not cite the majority rule as a control against that.
- "A typical cluster of five tolerates two failures" is the paper's example. It is not a measurement of your cluster, and it is not advice to build a five-node cluster in the middle of an incident. Your denominator is your real membership.
- A split vote is described as a situation that ends the term with no leader. The paper then starts a new election. It does not say an operator should force a winner.
- NIST's hot site with two live locations assumes continuous synchronisation. This article does not claim every dual site was built that way. If yours was not, you do not acquire that pattern by restoring the second copy.
- Concurrent processing is optional in NIST’s high-impact template; data and functionality still need validation. They are not permission to skip them because both sites answer a ping.
- No cluster was partitioned, and no election was run. Product-specific behaviour, including any witness or fencing feature, was not opened. This article is the decision. The product manual is still required before anyone changes membership or fencing.
- The paper's membership-change mechanism is acknowledged and not described. Supported reconfiguration can be part of an incident response, but cannot be replaced by arbitrary membership edits. Joint consensus requires both old and new majorities during transition.
Research and verification
Drafted with AI assistance on 11 November 2026 and independently source-reviewed on 13 November 2026. No article commands or recovery operations were executed. This is educational source research, not a lab-validated production procedure.
Sources
Independently reopened 13 November 2026.
- Diego Ongaro and John Ousterhout, In Search of an Understandable Consensus Algorithm (Extended Version): PDF. Majority availability, including the five-server example. Fail-stop and non-Byzantine scope. Election by a majority of the full cluster. Election safety: at most one leader in a term. Split vote ends a term with no leader.
- NIST SP 800-34 Rev. 1, May 2010: PDF. Concurrent processing as running at two locations until the recovered system is assured correct and secure. Declaration after validation. Hot site as two production locations with continuous synchronisation.
This article was originally published on .