Offline-first sync and replication
How offline-first apps sync, animated: CouchDB/PouchDB-style replication between a phone, a laptop and a server that go offline — revision trees, the changes feed, checkpoints, _revs_diff and _bulk_docs, conflicts and the deterministic winning revision, three ways to resolve them, compaction and _revs_limit's false conflicts — against a Firestore-style single-leader cache with pending writes, server timestamps and rolled-back writes.
Added
Works with
- Clocks, CRDTs and gossip Distributed systemsCRDT sync merges concurrent edits that revision trees keep as conflicts, and HLCs fix skewed-clock last-writer-wins
- Partitioning and replication Distributed systemsDynamo-style quorum replication, where concurrent writes come back as siblings like CouchDB's conflicting leaves
How to use it
Two ways to build an app that keeps working without a network, side by side in one page. In the first, every device holds a full database and any two can replicate — the CouchDB and PouchDB model, where concurrent edits become branches of a document’s revision tree. In the second, one server holds the truth and each device keeps a cache with a queue of pending writes — the model of Cloud Firestore and similar mobile backends. Nothing here talks to a real database.
How to use it
The Multi-master replication and Single leader + cache tabs are simulators. Pick a scenario
and press Start. Each step does something — takes a device offline, edits, reconnects, resolves
a conflict — plays until it has happened, and pauses so you can read what happened and why, with
links to the documentation. Next step moves on, Back rewinds (the simulator is deterministic,
so it replays the earlier steps), ⏭ jumps to the end of the step and Step ⇥ advances to the
next single event. Space plays and pauses; → is the next step. Each tab has a playground with no
script. Speed scales everything; Hop time is how long a request takes to cross the diagram at
1× — while one is on its way the simulated clock slows down — and Sim clock turns that off.
Neither changes what happens. ⛶ Full screen (or F) gives the diagram the whole window.
Multi-master replication. A phone, the server and a laptop, side by side (stacked on a phone
screen). Each box shows the replica’s update sequence and checkpoints, what its push and pull
replications are doing, and every document with its winning revision (green), a +n badge for
other live leaves (_conflicts) and its body. Requests are labelled as they travel: GET _local
checkpoints, _changes (a longpoll while waiting), _revs_diff, _bulk_docs and _bulk_get,
and their answers. Tap a replica, or use Do something, to take a device offline or bring it
back, edit, add list items, delete, create documents, resolve a conflict with a chosen strategy,
compact, or drop the next message of a kind. Panels: Revision trees (one document on every
replica, drawn as a tree: winner, conflicting and deleted leaves, stubs without bodies, and …
where _revs_limit forgot older history), Documents (what GET ?conflicts=true answers on each
replica), Changes feeds and checkpoints and Last conflict resolution (the leaves, their
timestamps, the common ancestor, the merged body and the revisions written).
Single leader + cache. The same three boxes, but the server is the authority. Each client shows
what its listener sees — its cache with its own pending writes laid over it, marked pending —
and the server shows each document’s version and update time. Write from either client, take
them offline, reconnect them in either order, stamp a server timestamp, add list items as a
whole-array write or with arrayUnion, or break a rule. Panels: what each client sees next to the
server, the queues of pending writes and rolled-back writes, and a comparison of the two models
with CRDTs.
Revision tree. Write a revision tree as paths, one per line (1-a1 2-b7 3-c4, a line may branch
from a revision named earlier, * marks a deleted leaf), or pick an example. It draws the tree,
ranks the leaves with the reason each beats the next, shows what GET returns with
_conflicts and _deleted_conflicts, and previews what a given _revs_limit would forget.
Scenarios
Multi-master replication:
-
How replication works — revision ids, the push and pull replications, checkpoints in
_localdocuments, the changes feed,_revs_diff,_bulk_docswithnew_edits: false, the longpoll that wakes the other device, and why a replicator’s own changes come back as a no-op. -
Offline edits on two devices — both devices change the same note offline; nothing is rejected on reconnect, the server keeps both revisions as branches, and every replica picks the same winner by the same rule; the same edit made twice gets the same revision id and is no conflict.
-
Resolving conflicts — a field-level three-way merge of edits to different fields; a real clash settled by last-writer-wins, which the phone’s slow clock fools; and a list merged as a set by an application merge on the server.
-
Delete against edit — a deletion with the longer history loses to an edit, so the document comes back; deleting the remaining leaf makes the deletion stick.
-
Interrupted replication resumes — six documents sent two at a time with a checkpoint per batch; a lost
_bulk_docsanswer, a timeout, a retry from the last checkpoint, and_revs_diffskipping what had already arrived. -
Compaction and _revs_limit — compaction keeps revision ids and drops old bodies; with
_revs_limit3 a laptop away for four edits gets a false conflict when it returns. -
Playground.
Single leader + cache:
-
Pending writes and latency compensation — a write shown at once with pending writes, an offline queue, the queue flushed in order on reconnect.
-
Server timestamps — the placeholder and its estimate, and an offline write stamped when it reaches the server.
-
Last write wins (by arrival) — two offline edits: the later arrival wins field by field, nobody is told; whole-list writes lose an item;
arrayUnionkeeps both. -
A rejected write is rolled back — an offline write that breaks a rule is shown until the device reconnects, then refused and taken back.
-
Single-leader playground.
What it models
-
Revisions. Every change to a document makes a new revision
N-hash: N is one more than the parent’s, the hash is derived from the parent revision, the deleted flag and the body — CouchDB computes an MD5 over these and the attachments; this page uses a 128-bit hash built from four seeded 32-bit hashes, which behaves the same way (deterministic, so the same edit to the same parent always gets the same id). A deletion is a revision with_deleted: true, a tombstone. APUTmust name a leaf as its_rev, or it gets 409 Conflict. -
Revision trees and the winner. A document is a tree of revisions; concurrent edits are branches. Every replica chooses the same winning revision from the leaves, without talking to anyone: a leaf that is not deleted beats a deleted one; then the leaf with the longer history (the higher N); then the leaf whose revision hash sorts higher as a string (Replication and conflict model; in CouchDB’s source the leaves are compared by
{not Deleted, RevPos, RevId}). The other live leaves are_conflicts, the deleted ones_deleted_conflicts. When every leaf is deleted the document is deleted. -
Replication follows the CouchDB replication protocol: read the checkpoint (a
_localdocument named after the replication) from both ends and start from it only if both have one; read the source’s changes feed since then, in batches, withstyle=all_docs(each changed document once, at its latest sequence, with all its leaves); ask the target_revs_diffwhich revisions it lacks; fetch those from the source with their_revisionshistory (open_revs/_bulk_get); write them to the target with_bulk_docsandnew_edits: false, which grafts the history onto the target’s tree without making new revisions; save the checkpoint on both ends. Each device runs a push and a pull replication, as PouchDB’ssyncdoes (PouchDB: replication); the pull waits on a longpoll_changesrequest. A request without an answer fails the session, which restarts from the checkpoint. -
Conflict resolution follows the pattern in the CouchDB and PouchDB guides: read the conflicting leaves, compute one body, write it as a new revision on the winning branch and write a deletion on every other live leaf. The strategies: keep the winner; last writer wins by the time each revision was written on its device (an application’s
updatedAtfield — kept beside the body here); a field-level three-way merge against the lowest common ancestor in the tree (two-way when the ancestor’s body is gone or the leaves share no ancestor); and an application merge that also treats lists as sets. -
Compaction and stemming. Compaction removes the bodies of revisions that are not leaves; the ids stay.
_revs_limit(default 1000) keeps only that many revision ids along each branch; a replica whose newest revision has been forgotten everywhere else cannot connect its history and gets a second root — a conflict between a revision and its own descendant. -
The single-leader cache follows Cloud Firestore’s documented behaviour (offline data): writes apply to the local view at once and listeners see them with
hasPendingWrites(listening); offline writes queue and are sent in order; the server applies each write to its current document —updatemerges fields, so the last arrival wins per field;serverTimestamp()is filled in at commit, and until then a listener sees an estimate from the local clock, the previous value or null (adding data);arrayUnionis applied at the server; a write the rules refuse is removed from the queue and the local view rolls back (Security Rules). -
For sync built on CRDTs instead — data types whose replicas merge without conflicts — see Clocks, CRDTs and gossip.
Limits
-
Two documents to start with and three replicas in a star (no device-to-device replication, although CouchDB allows it). Bodies are small flat JSON objects; no attachments, no design documents, no
validate_doc_update, no filtered replication, no_purge. -
Revision hashes are not CouchDB’s MD5, so the ids differ from a real server’s for the same edits (the rules that use them are the same). The checkpoint comparison is simplified to "both ends have one, take the older"; CouchDB compares session histories. Batches are counted in changes; timeouts and retries are fixed and much shorter than a real replicator’s back-off.
-
PouchDB’s own revision ids and defaults may differ from CouchDB’s between versions; this page follows CouchDB.
-
The single-leader model is reduced to what the scenarios need: one document store, two rules standing in for Security Rules, one write in flight per client, no queries, no transactions or batched writes (which need the server), no multi-tab persistence and no cache eviction. Atlas Device Sync (Realm), another single-leader design with its own merge rules, is not modelled.
Linking to an example
Links can be written by hand. Add a tab after #: sim (the default), leader or tree. A
link’s settings replace this page’s saved settings (a button restores yours); a scenario brings its
own settings, and parameters given with it override them.
| Parameter | Meaning | Example |
|---|---|---|
|
A scenario from either list below (add |
|
|
The multi-master scenario, without its own settings |
|
|
Open the multi-master scenario at this step |
|
|
One-way network latency, ms |
|
|
Changes per replication batch |
|
|
|
|
|
The phone’s clock offset in ms (negative = slow), used by last-writer-wins |
|
|
Default resolution: |
|
|
Continuous replication ( |
|
|
The single-leader scenario (with |
|
|
Open the single-leader scenario at this step |
|
|
One-way latency in the single-leader pane, ms |
|
|
The phone’s clock offset in the single-leader pane, ms |
|
|
The rule: most items in a list |
|
|
Pending server timestamps show as |
|
|
A revision tree for the calculator (with |
|
|
Random seed |
|
|
Playback speed: |
|
|
Hop time in seconds ( |
|
Multi-master scenarios:
example= |
Shows |
|---|---|
|
The replication protocol step by step (open) |
|
Offline edits on two devices become a conflict with one winner everywhere (open) |
|
Three-way merge, last-writer-wins with a slow clock, lists merged as sets (open) |
|
An edit beats a longer deletion; making the deletion stick (open) |
|
A lost answer mid-batch; the retry resumes from the checkpoint (open) |
|
Compaction, and the false conflict of a too-small |
|
Three replicas, no script (open) |
Single-leader scenarios:
example= |
Shows |
|---|---|
|
Latency compensation, the offline queue, the flush in order (open) |
|
serverTimestamp() estimated locally and filled in at commit (open) |
|
The later arrival wins per field; whole-list writes against arrayUnion (open) |
|
An offline write refused by the rules and rolled back (open) |
|
A server and two clients, no script (open) |
Examples: last-writer-wins with honest clocks, the same absence with the default _revs_limit — no false conflict, one change per batch, server timestamps shown as null while pending, a tie decided by the hash.
Sources: the CouchDB documentation —
Replication and conflict model,
CouchDB Replication Protocol,
/db/_changes,
/db/_bulk_docs (new_edits),
/db/_revs_limit,
Compaction;
CouchDB source (couch_key_tree, couch_doc) for the leaf ordering;
PouchDB guides on replication and
conflicts;
Cloud Firestore documentation on
offline data,
realtime updates,
adding and updating data,
transactions and
Security Rules.
Feedback is a gift. Something to improve, a feature to add, or a bug? Tell me.
Runs entirely in your browser; nothing you type leaves this page.