Multi-Primary Replication

Multi-primary replication—also called multi-master or active-active replication—allows more than one database node to accept writes.

At first glance, the architecture sounds simple:

                   Clients
              /       |       \
             v        v        v

          Node A    Node B    Node C
          read/write read/write read/write

             \        |        /
              +--- replication +

A client can write to Node A while another client writes to Node B.

But this immediately creates the central problem:

What happens when two nodes accept changes that cannot both be part of one valid database history?

That question is what makes multi-primary replication much harder than ordinary primary–standby replication.

Compare single-primary and multi-primary

Single-primary

                writes
Clients --------------------> Primary
                                 |
                                 | replication
                                 v
                              Standby

There is one obvious serialization point for writes:

Primary

Two conflicting writes ultimately meet there.

Multi-primary

Client 1 ------> Node A

Client 2 ------> Node B

Node A <------> Node B

Now two conflicting decisions may begin on different machines.

The replication system must decide:

Can both commit?

Must one wait?

Must one abort?

Can both commit temporarily and be reconciled later?

Different multi-primary technologies answer those questions very differently.

"Multi-primary" describes topology, not consistency

Consider:

Node A accepts writes
Node B accepts writes

That statement alone tells us almost nothing about correctness.

There are several possible designs.

Asynchronous active-active

A commits locally
B commits locally

changes replicated afterward

Conflicts may be discovered after both sides have already accepted them.

Certification-based synchronous/virtually synchronous

transaction executes locally

before final commit:
    transaction's write set participates
    in global ordering/certification

conflict
    → transaction abort

Galera follows this general model.

Consensus-based distributed database

A client may submit a write to any gateway/node, but the affected partition/range is still coordinated by:

a leader
or
a quorum/consensus group

Systems such as Spanner and CockroachDB are closer to this model.

Therefore:

"Clients can write to any node" does not necessarily mean every node independently commits conflicting writes.

The core multi-primary conflict

Suppose:

products

product_id = 42
stock = 1

Client A reaches Node A.

Client B reaches Node B.

Both try:

UPDATE products
SET stock = stock - 1
WHERE product_id = 42;

If both nodes independently commit without coordination:

Node A:
stock = 0

Node B:
stock = 0

but logically two units may have been sold even though only one existed.

Replication later has to answer:

What state is correct?

Can one write be discarded?

Was an external payment already made?

Did both clients receive SUCCESS?

This is much harder than merely copying rows between servers.

Another conflict: same row, different values

Initial state:

customer.status = "pending"

During a partition or concurrent multi-writer execution:

Node A:
status = "approved"

Node B:
status = "rejected"

Afterward the system encounters:

approved
versus
rejected

Possible strategies include:

prevent both from committing

order them and reject one

let both commit and choose one later

merge them using business logic

Those strategies provide very different guarantees.

Multi-primary architectures fall into different families

The phrase:

multi-master

does not tell you which one is being used.

Why multi-primary can reduce geographic write latency

Suppose users are in:

Berlin
Singapore
California

with a single writable primary in Germany.

A Singapore user must send every write across a long network path:

Singapore ----------------------> Germany primary

A multi-primary design may allow:

Singapore client → Singapore node

instead.

That can reduce the latency between the client and its entry point.

But there is a catch.

If the database requires global coordination before commit:

Singapore node
     |
     +---- coordinate with remote nodes

the long-distance network latency can still appear inside the transaction commit.

Multi-primary therefore does not eliminate the speed of light.

Multi-primary does not automatically provide linear write scaling

This is one of the most important corrections to simplistic notes.

Suppose three Galera nodes exist:

A
B
C

All three can accept writes.

But a committed write set still has to become part of the cluster-wide ordered replicated history and be applied by cluster members.

So adding:

Node D

does not transform:

10,000 writes/sec

into:

20,000 writes/sec

simply because another writable endpoint exists.

In a fully replicated multi-primary cluster:

Every replicated write ultimately creates work on every replica.

MariaDB describes Galera as multi-primary and suitable for distributing write entry points, while also noting that certification conflicts become important under high contention.

For true horizontal write scaling, partitioning/sharding may still be necessary.

Multi-primary and sharding solve different problems

Consider:

3 Galera nodes

each containing:

the entire database

That is replication.

Now compare:

Shard A:
customers 1–1M

Shard B:
customers 1M–2M

That is partitioning.

With replication:

same write eventually affects every copy

With sharding:

different writes may belong to completely different subsets

Therefore sharding can create genuine independent write capacity in a way that adding fully replicated writable nodes may not.

Galera Cluster: the concrete model

MariaDB Galera Cluster is a useful example of certification-based, virtually synchronous multi-primary replication.

Each node can accept transactions:

           MariaDB Galera Cluster

        +---------+
        | Node A  |
        | R / W   |
        +----+----+
             |
      +------+------+
      |             |
      v             v
 +---------+    +---------+
 | Node B  |    | Node C  |
 | R / W   |    | R / W   |
 +---------+    +---------+

But transactions do not independently become unrelated authoritative histories.

At commit time, Galera represents changes as a write set, globally orders the write set, and performs certification to determine whether it can safely commit.

MariaDB describes the process as write-set broadcasting followed by certification and application.

"Synchronous" in Galera needs qualification

Galera is commonly described as:

synchronous multi-master

A more precise description is:

virtually synchronous

Why?

Because these events are not literally simultaneous:

transaction certified

transaction commits on origin

remote node receives write set

remote node applies write set

A remote node can have the write set in its receive queue before its local applier has executed it.

MariaDB explicitly notes that Galera write sets are certified at commit time but do not necessarily apply immediately on every node; they can wait in a receive queue for parallel applier threads.

Therefore:

successful Galera commit

does not mean:

every physical data page on every node changed
at exactly the same instant

Follow one transaction through Galera

Suppose a client connected to Node A executes:

BEGIN;

UPDATE accounts
SET balance = balance - 100
WHERE account_id = 1;

COMMIT;

Conceptually:

1. Transaction executes locally on A.

2. Galera extracts the transaction's write set.

3. The write set enters the cluster-wide ordering protocol.

4. Certification checks whether it conflicts
   with transactions ordered before it.

5. If certification succeeds:
      transaction commits on its origin
      write set is applied on the other nodes.

6. If certification fails:
      transaction is aborted.

This is fundamentally different from asynchronous systems where:

A commits independently

B commits independently

later:
"Oops, these disagree."

What is a write set?

A write set is Galera's representation of the transaction changes relevant to replication and conflict detection.

Conceptually, suppose:

UPDATE customers
SET credit_limit = 5000
WHERE customer_id = 10;

The write set contains information corresponding to the transaction's modifications and the keys required for certification.

Galera replicates that write set through the cluster rather than sending:

"Please run this SQL text."

as ordinary statement-based replication would.

This is why Galera requires row-oriented replication behavior for supported clustered operation.

Certification: the heart of Galera

Suppose two concurrent transactions originate on different nodes.

Transaction T1 on Node A

UPDATE products
SET price = 100
WHERE id = 42;

Transaction T2 on Node B

UPDATE products
SET price = 120
WHERE id = 42;

Both may perform local work optimistically.

Around commit, their write sets enter global ordering.

Suppose ordering places T1 first:

T1 → T2

T1's write set certifies.

When T2 is certified, Galera discovers that its relevant write set conflicts with an earlier transaction in the certification window.

The outcome can be:

T1 commits

T2 aborts

rather than:

A keeps 100
B keeps 120
and someone merges later.

That difference is central to Galera.

The source's "later GTID wins" model is misleading

The source currently describes conflict behavior approximately as:

later GTID wins

with the loser rolled back.

That is not the right mental model.

Galera uses:

global transaction ordering
+
write-set certification

to determine whether a transaction remains valid.

A transaction ordered after a conflicting transaction can fail certification.

The significant concept is not:

largest GTID wins

like last-write-wins conflict reconciliation.

It is:

The write set must certify against the already ordered conflicting transaction history.

MariaDB documents certification as checking incoming write sets against concurrently committed transactions; conflicts cause a transaction to be aborted rather than merged through a timestamp/GTID winner rule.

This is optimistic concurrency

Galera's model is broadly optimistic.

Transactions can execute locally without taking a distributed lock on every remote node first.

Conceptually:

Node A                  Node B

execute T1              execute T2
locally                  locally
   |                        |
   +------- commit ---------+
              |
              v
        global ordering
              |
              v
         certification

If there was no conflict:

commit

If the certification check discovers an incompatible concurrent transaction:

abort/retry

This trades:

less distributed locking during execution

for:

possible transaction abort at commit time

Local execution means commit can still fail

This produces an important application rule.

Suppose application code does:

BEGIN

UPDATE ...
UPDATE ...

COMMIT

All UPDATEs may appear to execute successfully.

Then:

COMMIT

can fail because certification discovers a conflict.

Therefore application code must treat:

COMMIT

as an operation that can fail due to concurrency.

MariaDB's Galera guidance explicitly advises applications to check for errors after COMMIT.

Why retries are necessary

Suppose Node A and Node B repeatedly update the same customer.

One transaction loses certification.

The application can:

rollback
retry complete transaction

But it must rerun the decision from the beginning.

Not:

retry only final UPDATE

because earlier reads may now be outdated.

The familiar safe pattern is:

BEGIN

read current state
make decision
perform writes

COMMIT

On retry:

BEGIN

read current state again
make decision again
perform writes again

COMMIT

Error 1213 can represent Galera conflict behavior

MariaDB exposes some Galera concurrency conflicts using familiar transactional errors, including the deadlock class.

wsrep_retry_autocommit exists specifically to retry autocommit statements automatically for failures including certification failure and high-priority aborts.

Application code should therefore already have a deliberate retry strategy for retryable transactional failures.

However:

every error 1213

should not automatically be interpreted as:

classic local InnoDB lock-cycle deadlock

In a Galera environment, the cluster's certification/concurrency mechanisms can also be involved.

Hot rows are especially expensive in multi-primary Galera

Suppose every checkout performs:

UPDATE global_counter
SET value = value + 1
WHERE id = 1;

Clients send traffic across:

Node A
Node B
Node C

All transactions update the same row.

The cluster continually encounters transactions whose write sets conflict.

Result:

high certification failure rate
+
retries
+
lower throughput

The fact that there are three writable nodes does not help if all writers contend on one logical record.

This is why workload shape matters more than the phrase:

multi-master

Good multi-primary workloads

Galera tends to behave much better when concurrent transactions modify largely independent data.

For example:

Node A clients:
customers 1–1000

Node B clients:
customers 1001–2000

even though both nodes technically accept all writes.

Low contention means fewer overlapping write sets.

A difficult workload is:

every node repeatedly updates
the same inventory row
the same counter
the same account

because certification conflicts become common.

Primary keys matter

Efficient row identification is especially important for certification-based replication.

A good table design includes a stable primary key:

CREATE TABLE orders (
    order_id BIGINT PRIMARY KEY,
    ...
);

rather than relying on rows without useful unique identity.

Current Galera configuration even provides a REQUIRED_PRIMARY_KEY strict mode option for deployments that want to enforce stronger table discipline.

Primary keys are not merely an ORM convention here.

They help the distributed system identify conflicting row changes efficiently.

AUTO_INCREMENT becomes interesting in multi-primary systems

Suppose Node A and Node B both execute:

INSERT INTO orders(...)
VALUES (...);

and both independently generate:

order_id = 100

That would create a key conflict.

Galera provides:

wsrep_auto_increment_control

which dynamically adjusts:

auto_increment_increment
auto_increment_offset

according to cluster membership.

For three nodes, values might conceptually be interleaved:

Node A:
1, 4, 7, 10, ...

Node B:
2, 5, 8, 11, ...

Node C:
3, 6, 9, 12, ...

MariaDB documents this automatic adjustment and enables it by default.

Do not expect contiguous AUTO_INCREMENT values

The consequence is:

AUTO_INCREMENT

should be interpreted as:

generate a convenient unique identifier

not:

generate an uninterrupted sequence

A Galera cluster can produce:

1
4
7
...

from one node because IDs are being coordinated to avoid clashes with values generated elsewhere.

MariaDB explicitly warns that Galera auto-increment sequences can contain gaps.

If business logic depends on:

invoice 100
invoice 101
invoice 102

with no gaps, ordinary AUTO_INCREMENT is the wrong business-sequencing mechanism.

UUIDs do not solve transaction conflicts

Using:

UUID

for identifiers can eliminate certain key-generation collisions.

It does not solve:

two transactions modify customer 42

or:

two transactions reserve the same seat

Those are business-data conflicts, not identifier-generation conflicts.

So:

UUID

can solve:

PK allocation

without solving:

distributed write contention.

Quorum is what prevents split brain in Galera

Suppose a three-node cluster is:

A ----- B ----- C

The cluster relies on quorum.

Now the network partitions:

A       X       B ----- C

The B/C side has:

2 of 3 votes

and forms the majority Primary Component.

A has:

1 of 3

and does not.

The minority side cannot simply continue independently as another writable cluster.

MariaDB documents quorum as requiring more than half of the relevant voting membership; nodes outside the Primary Component stop ordinary query/transaction processing to prevent split brain.

"Primary Component" does not mean primary database node

Terminology is confusing here.

Galera is:

multi-primary

meaning multiple nodes can accept writes.

But it also has a:

Primary Component

The Primary Component is the connected quorum group that is authorized to process normal cluster operations.

It is not:

the one primary writer

A healthy three-node Primary Component can contain:

A
B
C

and all three can remain writable.

A three-node quorum example

Initial cluster:

A ----- B ----- C

wsrep_cluster_size = 3
wsrep_cluster_status = Primary

Node C crashes:

A ----- B      C X

A/B still have:

2 / 3

and retain quorum.

Writes can continue.

Now B also disappears:

A       B X    C X

A has:

1 / 3

and cannot safely form the previous cluster's majority by itself.

The correct behavior is to stop ordinary cluster writes rather than invent an independent authoritative history.

Why two-node clusters are awkward

Suppose the cluster contains only:

A
B

They lose communication:

A       X       B

Each sees:

1 of 2

Neither has a majority.

Allowing both to continue would create split brain.

Therefore odd-sized voting groups such as:

3
5

are often easier to make fault tolerant.

This does not mean:

every deployment needs five database servers

but quorum math must be considered deliberately.

Quorum solves split-brain authority, not conflict hot spots

Quorum answers:

Which connected component is allowed to operate?

Certification answers:

Can this particular transaction coexist
with concurrently ordered transactions?

These are different problems.

You can have:

healthy quorum

and still experience:

many certification conflicts

because applications repeatedly update the same rows.

Node failure in Galera is not ordinary "promotion"

With primary–standby:

Primary fails
→ promote standby

With healthy Galera:

Node A fails

Node B and C already accept writes

There is no database-role promotion equivalent to turning a read-only standby into a writer.

What still has to happen is:

remove failed endpoint from client routing

or have a proxy notice automatically.

For this reason, HA deployments commonly put:

MariaDB MaxScale
HAProxy
ProxySQL
or another routing layer

in front of the database nodes.

MariaDB specifically positions MaxScale alongside Galera for connection routing and HA.

Client routing still matters

Suppose clients connect directly to:

db1.example.com

and Node 1 crashes.

The fact that Nodes 2 and 3 are healthy does not magically rewrite the application's connection string.

A production architecture needs:

load balancer
proxy
service discovery
DNS endpoint
or application-aware node selection

For example:

Application
     |
     v
+----------+
| MaxScale |
+-----+----+
      |
 +----+----+
 |         |
 v         v
Node B    Node C

Multi-primary storage does not remove the need for connection routing.

Read consistency is subtler than "all nodes are synchronous"

Suppose a transaction commits through Node A.

Node B already knows the ordered write set but its local applier has not yet applied it.

A query arrives at B immediately.

Without an explicit causal synchronization requirement, the read can encounter a node that is briefly behind in local apply.

Galera therefore provides:

wsrep_sync_wait

for operations requiring a causality check.

MariaDB documents that wsrep_sync_wait waits until the local node catches up with cluster updates before executing selected operation types, at the cost of additional read latency.

Concrete read-after-write example

Client writes through A; suppose the resulting row is (104, 27, 89.00):

BEGIN;

INSERT INTO orders(id, customer_id, amount)
VALUES (104, 27, 89.00);

COMMIT;

After the write, node A's orders table includes:

Then the client immediately connects to B and reads from the replicated table:

SELECT *
FROM orders
WHERE id = 104;

For a critical causal read, the session can use:

SET SESSION wsrep_sync_wait = 1;

SELECT *
FROM orders
WHERE id = 104;

The read waits for the local node to be synchronized sufficiently before execution.

This is another reason:

"synchronous cluster" does not mean application code can ignore read-consistency semantics.

Why Galera can still have an apply queue

Suppose transactions arrive faster than one node can apply them.

That node develops:

receive queue

write set 101
write set 102
write set 103
write set 104
...

The transactions already belong to the global cluster history.

The problem is local application speed.

Galera uses parallel applier threads to process write sets that can safely be applied concurrently. MariaDB exposes this through Galera applier-thread configuration and receive-queue monitoring.

Flow Control: the slowest node can affect the cluster

Suppose:

Node A:
fast

Node B:
fast

Node C:
slow disk

C's receive queue grows:

C queue:

101
102
103
104
105
106
...

Galera cannot let that queue grow without bound.

When it crosses configured limits, C can trigger Flow Control.

Conceptually:

C:
"I'm falling behind."

        |
        v

cluster pauses/throttles new replication work

        |
        v

C catches up

        |
        v

cluster resumes

MariaDB documents Flow Control as an automatic cluster-wide throttling mechanism triggered when a node's receive queue grows too large.

One slow node can therefore reduce cluster write throughput

This is a crucial performance property.

Suppose:

A can apply 10k writes/sec
B can apply 10k writes/sec
C can apply 2k writes/sec

If the cluster has to prevent C from falling indefinitely behind, C can become a limiting factor.

Therefore:

add a cheap slow third node

can actually hurt cluster performance.

This is very different from simplistic reasoning:

more nodes
=
more throughput

In replicated systems, the slowest required participant can matter greatly.

Monitor Flow Control

Important Galera metrics include:

wsrep_local_recv_queue
wsrep_local_recv_queue_avg
wsrep_flow_control_paused
wsrep_flow_control_sent
wsrep_flow_control_recv

A persistently increasing:

wsrep_local_recv_queue_avg

suggests the node cannot apply write sets as fast as they arrive.

A high:

wsrep_flow_control_paused

means the cluster is spending substantial time throttled by Flow Control.

MariaDB explicitly recommends these metrics for identifying cluster bottlenecks.

State transfer: how does a node catch up?

Suppose Node C has been offline.

During that time:

cluster executes transactions
1000 through 5000

C returns having applied only through:

1000

It must catch up before becoming a normal synchronized member.

Galera has two important mechanisms:

IST
Incremental State Transfer

SST
State Snapshot Transfer

IST: Incremental State Transfer

If a donor still has the missing write sets in its GCache, it can send only the missing history.

Conceptually:

C has through:
1000

Donor GCache contains:
1001 ... 5000

        |
        v

send 1001 ... 5000

C applies them and catches up.

MariaDB describes IST as the preferred, faster mechanism when the donor's GCache contains the write sets required by the returning node.

GCache

Galera's GCache retains recent write sets primarily so nodes that briefly disconnect can catch up through IST.

Conceptually:

current seqno
      |
      v
----------------------------------->
      ^
      |
   GCache keeps
   recent history

A larger useful GCache increases the outage window from which a returning node may recover through IST instead of requiring a full snapshot.

But:

GCache

is not:

a permanent historical backup.

It is recovery/catch-up state.

SST: State Snapshot Transfer

If the required history is no longer available:

C needs transaction 1001

donor GCache begins at 3000

IST is impossible.

The node needs a complete state transfer:

SST

Conceptually:

Donor
  |
  | complete database state
  v
Joiner

After receiving the state, the joiner then processes newer write sets and eventually becomes synchronized.

MariaDB documents SST as a full dataset copy used when incremental catch-up is not sufficient.

IST versus SST

This distinction is much more important operationally than merely saying:

"joining nodes replicate automatically."

SST method matters

The source config uses:

wsrep_sst_method = rsync

and describes:

mariabackup is faster for TB-scale

but that hides a more important operational distinction.

Different SST methods have different:

donor blocking behavior
performance
security characteristics
tooling requirements

MariaDB documents SST methods as separate provisioning mechanisms and notes that donor blocking depends on the chosen method.

For current production systems, choose an SST method from current MariaDB documentation rather than treating rsync as a default recommendation.

The source's wsrep_sst_auth example is also misleading for rsync

The source combines:

wsrep_sst_method = rsync
wsrep_sst_auth = sstuser:...

and then creates an SST account.

Current MariaDB documentation says wsrep_sst_auth is unused when the SST method is rsync; authentication requirements depend on the SST mechanism.

So the correct documentation pattern is:

Pick the SST mechanism first, then follow that mechanism's current authentication and TLS requirements.

Do not teach one generic SST user recipe as universal.

Quorum loss and full-cluster recovery

Suppose all three nodes shut down.

There is no surviving Primary Component.

You must not simply start random nodes with:

--wsrep-new-cluster

until one works.

You first need to determine which node contains the most advanced safe cluster state.

MariaDB's recovery procedure explicitly requires identifying the most advanced node and bootstrapping the new Primary Component from the appropriate state.

Bootstrapping is a cluster-authority operation, not merely a startup convenience.

Why careless bootstrapping is dangerous

Imagine:

Node A applied through seqno 10,000

Node B applied through seqno 9,950

After a full outage, an operator incorrectly bootstraps B as a brand-new authoritative cluster.

The cluster may now start from an older state and effectively discard transactions that existed on A.

The safe question is:

Which node contains the latest recoverable cluster state?

not:

Which server boots fastest?

Monitoring cluster health

A basic health check should include:

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size';
SHOW GLOBAL STATUS LIKE 'wsrep_cluster_status';
SHOW GLOBAL STATUS LIKE 'wsrep_local_state_comment';
SHOW GLOBAL STATUS LIKE 'wsrep_ready';

Healthy values commonly include:

wsrep_cluster_status = Primary

wsrep_local_state_comment = Synced

wsrep_ready = ON

The expected:

wsrep_cluster_size

depends on how many nodes should currently be members.

MariaDB documents these status variables as central quorum and node-health indicators.

Do not monitor only cluster size

Suppose:

wsrep_cluster_size = 3

That tells you three nodes are present in the component.

It does not alone tell you:

Are they Synced?

Is Flow Control constantly active?

Are receive queues growing?

Are certification conflicts exploding?

Is one node repeatedly transferring state?

A healthy cluster needs several categories of metrics.

Practical monitoring table

MariaDB's monitoring documentation specifically recommends receive/send queues, Flow Control metrics, and certification-dependency information for cluster diagnosis.

A corrected minimal MariaDB Galera configuration

For a lab, a basic configuration may look conceptually like:

[mariadb]

bind-address = 0.0.0.0

default_storage_engine = InnoDB
binlog_format = ROW
innodb_autoinc_lock_mode = 2

wsrep_on = ON
wsrep_provider = /path/to/libgalera_smm.so

wsrep_cluster_name = example_cluster
wsrep_cluster_address = gcomm://db1,db2,db3

wsrep_node_name = db1
wsrep_node_address = 10.0.0.20

The node-specific values change on every host.

MariaDB's current Galera deployment documentation continues to identify ROW, InnoDB, wsrep_on, provider configuration, and cluster/node addressing as core configuration.

Do not blindly copy provider paths: they differ across distributions and package layouts.

Current product naming matters

The source describes:

MySQL / MariaDB + Galera

as though this were one interchangeable current deployment.

That should now be qualified.

In 2025 MariaDB acquired Codership/Galera, and in September 2026 MariaDB announced the end of the separately distributed MySQL Galera Cluster line while continuing Galera integration in MariaDB.

So for current notes:

Use MariaDB Galera Cluster as the concrete Galera implementation unless you are deliberately documenting another supported Galera-based product such as a vendor-specific distribution.

Configuration commands should come from the documentation for that exact product/version.

Do not use "rule of thumb = threads equal CPU cores" blindly

The source suggests:

wsrep_slave_threads = 4

with a CPU-based rule.

Parallel apply capacity depends on:

workload dependency structure
CPU
I/O
locking
write-set parallelism

MariaDB exposes:

wsrep_cert_deps_distance

specifically as an indicator of potential parallel application.

So tune applier threads from:

observed receive queues
Flow Control
CPU
wsrep_cert_deps_distance

rather than a universal core-count formula.

wsrep_slave_threads versus wsrep_applier_threads

Terminology differs across product lines and versions.

MariaDB's current Galera documentation still exposes:

wsrep_slave_threads

as the variable controlling parallel write-set appliers.

The MySQL-wsrep branch renamed several legacy slave names, including:

wsrep_slave_threads
→
wsrep_applier_threads

in its own newer releases.

Therefore:

Always use the variable documented by the exact MariaDB/MySQL-wsrep/Percona version being operated.

Do not copy Galera settings between products merely because all of them use Galera technology.

innodb_autoinc_lock_mode = 2

Current MariaDB Galera deployment documentation still includes:

innodb_autoinc_lock_mode = 2

alongside row binary logging and InnoDB.

But it solves a specific database-engine behavior.

It does not replace:

wsrep_auto_increment_control

which addresses cross-node auto-increment allocation.

These settings live at different layers.

Large transactions are costly

Consider:

UPDATE events
SET archived = 1
WHERE event_date < '2020-01-01';

that modifies:

10 million rows

A huge transaction creates a huge distributed write set and long apply/certification consequences.

Effects may include:

large memory/network usage
slow apply
Flow Control
long conflict windows
large rollback cost

MariaDB's Flow Control documentation recommends breaking large write operations into smaller batches when they cause write-set/applier pressure.

DDL is also distributed coordination

Suppose:

ALTER TABLE customers
ADD COLUMN preferred_language VARCHAR(10);

A schema change cannot safely appear on one node while the others continue processing incompatible write sets indefinitely.

Galera therefore has schema-upgrade mechanisms such as:

TOI — Total Order Isolation

RSU — Rolling Schema Upgrade

MariaDB documents TOI as the default mode, ordering DDL consistently with cluster transactions; RSU instead performs the DDL locally while the node is desynchronized and requires careful compatibility handling.

DDL therefore deserves its own deployment plan.

Multi-primary does not mean "no failover planning"

Galera avoids the classic:

promote read-only standby

step while quorum remains healthy.

But applications still need:

health detection
routing
quorum awareness
full-cluster recovery
node rejoin procedures

and operators still need plans for:

one node failure

majority loss

entire cluster shutdown

network partition

state transfer

data-center failure

The failure model changes.

It does not disappear.

Three-node failure table

The critical question is not:

"Is one node still alive?"

It is:

"Does the surviving component have authoritative quorum?"

Geographically distributed Galera needs caution

A tempting topology is:

Berlin
Singapore
Virginia

all in one Galera cluster.

This gives:

local writable endpoints

but cluster coordination must still travel across the WAN.

Effects include:

higher certification/commit latency

higher probability of Flow Control interaction

greater sensitivity to network jitter

more complex quorum placement

MariaDB documentation notes WAN-specific provider considerations, and current Enterprise materials position geographic Galera deployment carefully rather than treating WAN latency as free.

For some applications, another architecture—regional clusters plus asynchronous inter-region replication, for example—may fit better.

Compare Galera with asynchronous active-active replication

Imagine two asynchronous writable sites:

Site A                    Site B

write X                   write Y
commit                     commit

       replicate later

A conflict can exist after both applications already received success.

That requires:

conflict reconciliation

Galera instead tries to maintain a single cluster-wide ordered history:

T1
T2
T3
...

and rejects transactions that fail certification.

So Galera conflict management is closer to:

prevent incompatible committed histories

than:

allow divergent histories
then merge them.

Compare Galera with last-write-wins

Last-write-wins might do:

A: status = approved, timestamp 100

B: status = rejected, timestamp 101

result:
rejected

Both writes may already have happened.

One simply wins reconciliation.

Galera's certification model instead resembles:

T1 ordered first
T1 certifies

T2 conflicts with T1
T2 aborts

The losing application sees a transaction failure and has an opportunity to retry under the new state.

These are fundamentally different semantics.

Compare Galera with consensus-based distributed SQL

Now consider a system such as CockroachDB or Spanner.

A client may send SQL to multiple entry nodes.

But for each replicated range:

writes are ordered through
a Raft/Paxos replica group

The system may still use distributed transaction coordination across several ranges.

That architecture is not normally described as:

every replica independently acts as master

even though clients can write through many database endpoints.

This is why:

multi-primary endpoint

and:

independent master

should not be treated as synonyms.

Conflict avoidance can be better than conflict retry

Suppose customers are geographically assigned:

European customer records
→ normally written through Node A

US customer records
→ normally written through Node B

Both nodes remain capable of writing everything.

But normal routing reduces the probability that two nodes concurrently update the same logical record.

This strategy is sometimes called:

writer affinity

or:

home-region ownership

depending on the architecture.

Multi-primary capability can exist while application routing deliberately reduces multi-writer contention.

External side effects and Galera retries

Suppose:

BEGIN

update order

send payment API request

update inventory

COMMIT

Certification fails at COMMIT.

Database state rolls back.

The payment API does not.

A retry can charge again.

This is why distributed transaction retry safety still requires:

idempotency
outbox patterns
careful side-effect boundaries

Galera's ability to abort a transaction at commit makes this especially important.

Do not confuse replication conflict with business invariant safety

Suppose two transactions modify different rows:

T1:
doctor A off call

T2:
doctor B off call

Business rule:

at least one doctor must remain on call

The write sets may touch different rows.

Simple row-level write-set conflict detection does not automatically mean:

cross-row business invariant is safe

You still need appropriate:

constraints
locking
transaction isolation
application protocol

for such invariants.

Multi-primary replication does not replace transaction-isolation reasoning.

Multi-primary and CAP

Suppose a three-node Galera cluster partitions:

A       X       B ----- C

Could both sides stay writable?

If yes:

A could commit one history

B/C could commit another

and split brain would result.

Galera instead allows the majority Primary Component to continue while the minority cannot perform normal clustered work.

That is a consistency-first response to the partition.

It illustrates CAP concretely:

preserve one authoritative cluster history
→ sacrifice write availability on minority side

Multi-primary therefore does not mean "available everywhere"

This misconception is especially common:

every node is writable
therefore
every node can always keep accepting writes

No.

A healthy multi-primary node that has lost quorum may be intentionally prevented from accepting normal writes.

That is a safety feature.

Availability must always be discussed together with:

network partition
quorum
membership
consistency guarantee

Practical architecture comparison

Do not choose among these using only:

"supports multi-master: yes/no"

The mechanisms and guarantees differ substantially.

What multi-primary replication is good at

It can be a strong fit when you need:

multiple writable connection endpoints

fast failover without promoting a read-only replica

high availability inside a low-latency cluster

maintenance while other writable nodes remain online

geographic/localized write entry where coordination latency is acceptable

and the workload has manageable write contention.

When it becomes difficult

Warning signs include:

very high write contention on the same rows

large transactions

high-latency WAN between cluster nodes

nodes with very different performance

requirements to remain writable on both sides
of a network partition

applications that cannot safely retry transactions

huge state transfers

poor primary-key design

Those are architectural issues, not merely tuning problems.

Common misconceptions

Recommended Galera test plan

Do not validate a cluster only with:

INSERT on A
SELECT on B

Test actual failure and conflict behavior.

Testing the happy path alone does not validate HA.

A safer basic deployment sequence

For a fresh three-node MariaDB Galera lab:

1. Install the same supported MariaDB/Galera version
   on all nodes.

2. Configure:
      InnoDB
      ROW binlog format
      wsrep provider
      cluster name
      cluster member addresses
      node identity

3. Configure network/TLS/firewall rules.

4. Bootstrap exactly one initial cluster node.

5. Verify:
      wsrep_cluster_status = Primary
      wsrep_cluster_size = 1

6. Start Node 2 normally.

7. Let it complete IST/SST as required.

8. Start Node 3 normally.

9. Verify:
      cluster size
      Synced states
      wsrep_ready
      queues and Flow Control

10. Add an HA-aware client routing layer.

11. Test failures before calling the system production-ready.

The current MariaDB quick-start documentation follows this overall bootstrap-one-then-join-others model.

Do not bootstrap ordinary joining nodes

This command:

galera_new_cluster

means:

create a new Primary Component

not:

start MariaDB in Galera mode.

Use it for the deliberate initial bootstrap or appropriate full-cluster recovery case.

Normal joining nodes should start normally and connect to the existing Primary Component.

Careless repeated use can create precisely the split histories quorum is designed to prevent.

Security considerations

A production cluster should secure:

client traffic
cluster replication traffic
IST transfers
SST transfers
administrative access

The source lists ports such as:

3306
4567
4568
4444

which correspond to typical client/Galera/state-transfer purposes, but exact bindings and security rules must follow the deployment.

Current MariaDB Galera documentation also supports encrypted cluster and state-transfer traffic; current 2026 maintenance releases tightened behavior so misconfigured SST encryption fails instead of silently falling back to plaintext in affected methods.

Do not simply expose Galera ports broadly.

Technology summary: what actually happens to a write

The user-visible statement:

"I can send a write to any node"

can therefore hide radically different distributed protocols.

Final mental model

Do not memorize multi-primary replication as:

many masters
+
copy every write everywhere

Use this sequence instead:

CLIENT SENDS WRITE TO ANY WRITABLE NODE
               |
               v
       TRANSACTION EXECUTES
               |
               v
          WRITE SET
               |
               v
       GLOBAL ORDERING
               |
               v
        CERTIFICATION
          /       \
         /         \
   conflict       safe
      |             |
      v             v
   ABORT          COMMIT
                     |
                     v
             REMOTE APPLY QUEUES
                     |
                     v
              OTHER NODES APPLY

Alongside that is a separate membership path:

CLUSTER MEMBERSHIP
       |
       v
     QUORUM
       |
       v
PRIMARY COMPONENT
       |
       +---- majority side may operate
       |
       +---- minority side stops

And a separate recovery path:

NODE REJOINS
    |
    +--> required history in GCache?
            |
       +----+----+
       |         |
      YES       NO
       |         |
      IST       SST

Those three flows explain most of Galera:

transaction correctness
cluster safety
node recovery

The question to ask in any multi-primary system

Do not ask only:

"Can every node accept writes? "

Ask:

"If two nodes receive conflicting writes at the same time, exactly when is the conflict discovered, which operation is allowed to succeed, what does the losing client observe, and what happens if those nodes cannot communicate? "

For Galera, the answer is approximately:

transactions execute optimistically on writable nodes

write sets enter one cluster-wide ordered history

conflicting transactions can fail certification

a majority Primary Component preserves authority
during partitions

lagging/rejoining nodes recover through IST or SST

slow apply can throttle the whole cluster through Flow Control

Once those mechanisms are understood, "multi-master" stops being a marketing label and becomes a concrete concurrency and availability architecture.

References

The source notes provide the original multi-master topology, motivations, conflict-resolution discussion, Galera configuration, SST setup, bootstrapping process, and certification example.

MariaDB's current Galera documentation describes multi-primary write-set replication, global ordering/certification, and the fact that write sets can be queued and applied after certification rather than physically applied simultaneously.

MariaDB's quorum documentation describes the Primary Component, majority requirement, minority-side protection against split brain, and the need to bootstrap from the most advanced state after full quorum loss.

MariaDB's state-transfer documentation distinguishes Incremental State Transfer using retained GCache history from full State Snapshot Transfer when the required history is unavailable.

MariaDB's Flow Control documentation describes cluster throttling when a node's receive queue grows, and its monitoring documentation identifies receive/send queues, Flow Control, and certification dependency metrics as important performance signals.

Current MariaDB system-variable documentation covers wsrep_auto_increment_control, wsrep_retry_autocommit, wsrep_sync_wait, wsrep_slave_threads, SST authentication behavior, and Galera schema-upgrade modes.