MeridiansMeridians

11. Replication model

From the Meridians Wiki · Public · Maintained · joint

Reads — local-first

Every node hydrates from its own IndexedDB and renders locally (Part A); the dashboard list and the open domain come from the local replica. Opening a domain the replica lacks issues a request; the master answers with the canonical doc.

A tunnel client is deliberately a partial replica, but a surface may need several full member Domains without navigating into each one (Constellation Context/Stories/Positions/Streams/Chat, Home Pilot, Usage). Those reads go through loadDomain(id, { atLeastUpdatedAt }): a current projection returns immediately; a missing or entry-stamped-stale projection performs one coalesced, token-authorized request and resolves from the ordinary domain SSE frame. Reads that begin before the EventSource capability or member token arrives stay queued until both are present. Thus lazy replication saves transfer without making aggregate memory thinner than the host's canonical Domains.

Writes — optimistic + push + lock ("1 update at a time")

On a client, outwardDispatch (StoreProvider) classifies each action:

  • VIEW_ONLY (navigation/inspection — SET_ACTIVE_DOMAIN, scene nav, inspector, branch switch, search, …): apply locally, never push. Navigation is independent — a client opening domain B does not move the master or other clients.
  • Document-mutating: apply optimistically to the local reducer, then push the action to the master and lock (a second edit while pending is refused with a toast). The master acks; the client unlocks. ≤1 in-flight edit per client, so the master linearizes everything and there is nothing to merge.
  • Destructive Domain deletion is deliberately not optimistic. The client keeps the document and asset projection until the master confirms canonical deletion with the removed acknowledgement; rejection therefore cannot manufacture an empty local success or trigger early byte cleanup.

Lock release is event-driven, not on a timer. The lock clears when (a) the master's ackPushId matches the pending push (the happy path), (b) the master rejects it, or (c) the SSE connection drops (es.onerror) — the master can't ack on a dead socket, so we unlock immediately and requestReconcile the pending domain. PUSH_ACK_TIMEOUT_MS (15s) is only a last-resort backstop for a connected-but-stuck master that never answers.

ADD_DOMAINcreate push; DELETE_DOMAINdelete push; relayable Domain-document edits → action push carrying {domainId, branchId, action, pushId}. Constellation, onboarding, extraction-job, catalog, game, sync, and view actions stay on their owning document/protocol; action-policy classification prevents them from being mistaken for Domain mutations merely because a Domain happens to be open.

On a Hosted master, canonical /api/local/* writes also require the machine-bound renderer session. An expired or missing capability returns the typed hosted-session-required denial; the shared browser recovery seam re-verifies the current central session, re-mints once, and retries the exact write once. AI and record transports share that single-flight recovery. Other 403s are never retried, and a failed recovery remains a visible record-write failure rather than an optimistic edit that silently disappears.

Master applies the action to the target domain

applyActionToDomain(domain, branchId, action) runs the real reducer against a throwaway AppState whose activeDomain is the target — so the master can fold a client's edit onto any domain/branch without disturbing its own view, with zero reducer duplication. It then persists (through the Part-A write path), adopts (SYNC_ADOPT_DOMAIN), and broadcasts the result + ackPushId. Because edits are deltas applied onto evolving canonical state, concurrent edits to different branches of the same domain both land.

Why actions, not whole documents

Replicating actions (not full-document last-writer-wins) preserves concurrent different-branch editing and keeps payloads small. Analysis / wizard / extension are the exception — they produce a whole new domain, so they ride the create push.

Echo suppression

The master publishes a domain version once. The debounced publish effect dedupes by (id → updatedAt) (lastPublishedRef): a version the host handler already broadcast (a client's accepted push) is never re-published as an echo.

Headless changes reach clients too (MCP / loop / Telegram → tunnel)

The sync broker fans out whatever the master browser publishes — so a change made on the daemon without a browser dispatch (an MCP tool, an autonomous loop, the Telegram bot, a local script committing via /api/local/mutate) has to be relayed down explicitly, or a joined /scenario / member client never sees it. The path: a headless commit → publishRecordEvent (Part A §3) → the master's useRecordStream subscription adopts it (SYNC_ADOPT_DOMAIN) and re-publishes it to the broker (relayHeadlessDomain in store.tsx, mirroring commitGameDomain). This is what makes a Scenario game driven over MCP (or any headless domain edit) propagate to remote players — even for a world the Director isn't actively viewing, where SYNC_ADOPT_DOMAIN alone updates only the dashboard entry, not activeDomain, so the ordinary activeDomain-keyed publish effects would never fire. The relay is deduped against lastPublishedRef and is a no-op when nothing is joined (the broker fans to zero sinks). The same activeDomain-only blind spot that Part A §5 closes for the Director's own IndexedDB is closed here for joined clients — the two fixes are siblings.

12. Asset replication — src/lib/client/cache/asset-manager.ts

Content assets (embeddings/images/audio/texts) are immutable & content-addressed by id prefix (emb_/img_/audio_/text_) and flow both ways:

  • Read-through (client → record): on a local miss, the client fetches the bytes and caches them in its own IndexedDB. Single via /api/sync/asset, bulk via /api/sync/assets. Both routes are SSOT-first: the daemon answers straight from the canonical fs record when it holds the bytes (readAssetById, meta echoed on the same assetMetaHeaders contract as the local asset GET), and only falls back to relaying to the master renderer's cache (requestId-correlated) for bytes not yet mirrored down. The store-wide readable _meta.json is fingerprint-cached as a parsed read projection, so a bulk pull does not reparse the same index per id; an atomic write or direct edit invalidates it. Concurrent identical pulls de-dupe (pullInFlight / batchInFlight).
  • Announce (record → everyone): every fs asset write publishes an id-only notice from the ONE chokepoint (fs-store.writeAssets). Renderers get it on /api/local/stream and absorb the bytes; tunnel clients get the twin asset-notice broker frame (member-gated; the bridge in broker.ts also evicts that id from the broker's bounded asset working set) and just drop any stale resolution — a surface displaying the ref re-resolves and pulls lazily. Every fs→idb import (importAsset) fires meridians:asset-changed, so a cover whose doc frame outran its bytes appears the moment they land instead of after a reload. The aux lane's notices would ride the same bridge if clients ever need them.
  • Upload (client → master): a client that creates an asset uploads it (asset-upload) so the master's record holds it — awaited before the domain referencing it is pushed, so the master never has a dangling ref. The same contract holds on a Local/Hosted master: image, audio, text, and embedding store calls do not return a reference until canonical bytes are acknowledged. A fromCache flag stops a pulled asset from being re-uploaded.
  • Encoding: embeddings travel as compact Float32 binary (no JSON.stringify — that froze the master on large batches); the master encodes in yielding chunks.
  • Mutation discipline: never overwrite an id. Regenerated audio mints a new id and rewrites the scene's ref so the change rides the normal doc sync; useAssetUrl re-resolves on a meridians:asset-changed event. Replacement and clear commit the reference first and reclaim the old bytes last; deletion failure leaks a recoverable orphan rather than producing an unrecoverable dangling ref. (This is the replication face of the content-addressing invariant in Part A.)

13. Reconnect reconciliation (event-driven)

A client catches up continuously, not just on reconnect — the same version-proof discipline as Part A §5, on the tunnel plane. The replica keeps a version ledger (the same record-synced stamps): boot hydration stamps every doc it loads, and every adopt restamps — domain frames, constellation frames, and game-state ticks (a tick advances the domain clock without a full broadcast, so it must move the stamp or live games would trigger whole-world re-pulls). Against that ledger, reconcileReplica (store.tsx) runs on every entries broadcast and re-requests exactly the domains whose master updatedAt moved past the stamped version — planStaleSweep, version-proof, so an in-sync replica costs zero requests per frame. Guards and edges:

  • Loop guard: each (id, version) is requested at most once per connection (a request the master won't honour — not a member of that world — can't storm), cleared on reconnect so a lost request retries.
  • Boot race: entries frames arriving before hydration warms the ledger reconcile nothing (the storm guard); the connect handler replays the reconcile against the buffered latest list once whenHydrationStampsWarm() resolves — deferral, never loss.
  • Open-time convergence: opening a held domain re-requests it when the entries list says the replica copy is behind — the stale copy renders instantly and converges when canon arrives.
  • Folders: constellations have no entries-list twin to version-gate against, so each (re)connect owes the held folders one blind constellation-request refresh (tiny refs-only docs, bounded by folder count), settled on the first reconcile after connect.

Why per-frame matters: direct domain broadcasts only reach connections the master has granted per-domain access (granted per request, wiped on disconnect), so the entries broadcast is the one signal every member always receives — acting on it every frame is what closes the missed-broadcast gap. Local domains are never dropped by a sync; deletions arrive explicitly via removed (which also clears the ledger stamp so a re-created id adopts cleanly).