Persistence — where state lives and how it propagates

# Persistence — where state lives and how it propagates

> **Scope.** This doc owns *how state is stored and kept in agreement* across the two stores and the
> two instance roles. It has two halves:
>
> - **Part A — the store linkage** (§1–§5): the on-disk **record** (canon) and the browser
>   **IndexedDB** (a rebuildable projection), and the exact machinery that keeps them in lockstep on
>   a *single* instance — the write path, the projection echo, the `mirror` flag, catch-up, and boot
>   reconciliation.
> - **Part B — replication** (§6–§19): how *several people* work in one Meridians world hosted from a
>   single **master** and reached by **clients** over an ngrok tunnel — the broker, the command/edit
>   channels, roles, and catch-up.
>
> The *sources-of-truth model* this rides on (record = canon, IndexedDB = projection) is introduced in
> **[ARCHITECTURE.md](../ARCHITECTURE.md) §2–3**; this doc is the implementation deep-dive that
> ARCHITECTURE points to. Living document: when the code and this disagree, fix whichever is wrong.

> **Roles ([LANGUAGE.md](../../vocabulary/index.md)):** the two `SyncRole`s are **master** and **client**. The
> **master** is the authoritative instance that owns the record (the Director's daemon / localhost /
> canonical web app); a **client** joined over a tunnel renders synced state and pushes optimistic
> edits up, holding no credentials. (`clone` is a *retired synonym* for client — code symbols like
> `cloneId` and *duplicating* a domain keep the word, prose does not.)

---

# Part A — the store linkage (record ↔ cache)

# 1. Two stores, one canon

There are exactly two places state lives on a master instance, and they are **not** peers:

| Store | What it is | Where | Role |
|---|---|---|---|
| **The record** | JSON documents + binary assets + aux maps on disk | `MERIDIANS_RECORD_DIR/record/` (`src/lib/server/record/`) | **Canon.** The single source of truth. Versioned, inspectable, owned by the daemon. |
| **IndexedDB** | The browser's local database (`meridians-instance`) | the browser (`src/lib/client/cache/`) | **Projection.** A rebuildable cache. Fast reads; never a co-authority. |

Bulk asset writes are checked at the `fs-store` chokepoint against the runtime disk reserve and,
when configured, `MERIDIANS_RECORD_MAX_BYTES`. Document and aux writes remain ungated: operator-authored
canon and session state must remain writable even when reproducible generated bulk reaches its budget.
The storage order is one strategy: the always-on 500 MB free-space floor applies first, then the one
effective record ceiling, resolved as an explicit positive `MERIDIANS_RECORD_MAX_BYTES`, otherwise the
hosted volume-derived ceiling, otherwise no ceiling on an unconfigured desktop. The admin percentage
uses that same resolved ceiling as its denominator, falling back to true volume occupancy when no
ceiling is known. A future document change appends one pure step to its ordered schema plan, gated by the
body version marker; a document already stamped current is trusted and takes the no-walk path.

Historical formats are an input condition, not a runtime dialect. Product and runtime design target
the current contract only. Every canonical and IndexedDB document read passes through the same
versioned migration boundary, which translates supported historical shapes before features, reducers,
prompts, or analytics receive them. Do not add legacy branches, flags, or fallback vocabularies to
downstream code, and do not preserve an old shape merely because it once existed. A retained transform
must have a real record fixture and an explicit schema step; once supported records no longer need it,
raise the retained baseline and delete it. Compatibility is a short bridge into the future, never a
constraint on the future model. How a transform enters, is rolled through the fleet, and leaves again
— the ledger, the retire condition, the date, the issue — is owned by
[Legacy lifecycle](legacy-lifecycle.md).

The one rule that governs everything below — the **projection model** — is: **the file advances before
the cache.** A document write hits the record first (version bump), then the cache follows via an echo.
If the two ever disagree, the record wins and the cache is re-derived. IndexedDB may hold derived
indexes (embeddings) not in the record; those are rebuildable, so that's fine.

> This corrects an older framing that said "all data lives in the Director's browser IndexedDB." Since
> the daemon (E1), the **record on disk is canon** and IndexedDB is its projection. Prose that says
> otherwise is stale.

### The on-disk layout

```
MERIDIANS_RECORD_DIR/record/
├── domains/         <id>.json + <id>.meta.json   (document + {version, updatedAt, title})
├── constellations/  <id>.json + .meta.json
├── extractions/     <id>.json + .meta.json
├── program-journals/  <domainId>~<ordinal>.json + .meta.json   (segmented Program run/event journal)
├── story-receipts/    <memberId>~<domainId>~active.json + .meta.json   (per-member Story read-receipts)
├── context-manifests/  (same document pattern)
├── assets/
│   ├── embeddings/  emb_<id>.bin   + _meta.json   (content-addressed; _meta maps id → {model,…})
│   ├── audio/       audio_<id>.*   + _meta.json
│   ├── images/      img_<id>.*     + _meta.json
│   └── texts/       text_<id>.*    + _meta.json
└── aux/             meta.json · instanceMembers.json · activity.json · invitations.json ·
                     apiLogs.json · researchCatalog.json · keys.json   (whole-map key/value stores)
    └── api-logs/    YYYY-MM.jsonl   (append-only API-usage journal, monthly segments — the
                     canonical, uncapped log; `apiLogs.json` is its read-only legacy tail.
                     Written by the headless sink + the renderer relay; folded by id on read.
                     See `server/record/api-log-store.ts`.)
```

Three record shapes, three propagation disciplines: **documents** (daemon-first, echoed), **aux maps**
(shadow-mirrored), **assets** (content-addressed, shadow-mirrored + pulled on demand). §3–§4 cover each.

Extraction documents keep their resumable control state inline but bound large payloads through the
canonical `texts/` asset store. A large corpus uses `text_xsrc-<jobId>`; a large per-scene result array
uses deterministic, inspectable JSON shards `text_xres-<jobId>-<shard>`. The document retains the
ordered refs and null result slots, and both record-stream and IndexedDB hydration rebuild the full
in-memory job before the runner or UI sees it. Assets land before refs; a missing or malformed result
shard remains null so resume re-extracts it. Reference-bearing text writes await daemon confirmation
before the document advances, so canon cannot deliberately accept a dangling ref. This is one canonical job split across document + assets,
not a second result store.

Extraction admits at most **500,000 words** per corpus. The runner's windowing, 20-call concurrency
budget, adaptive persistence, result shards, and phase checkpoints make that an operable upper envelope,
not a promise of cheap or equally coherent extraction at every size. Creation, headless extraction, file
composition, imported unstaged sources, and file conversion must reject or disable work above the same
core constant before model calls begin; operators split larger material into coherent volumes.

### The IndexedDB stores

`src/lib/client/cache/db.ts` opens `meridians-instance` with a store per record shape. Out-of-line
document/aux stores are **encrypted at rest** (WebCrypto AES-256-GCM, `encryption.ts`); the in-line
asset stores (`embeddings`, `audio`, `images`, `texts`) are **not** (an embedding is read fully
decrypted on every semantic search — encrypting it would buy nothing). `secrets` is a master-only DEK
mirror, never replicated.

The `meta` store contains two different scopes. Fleet-wide settings (`joinPolicy`, `memberSeeding`,
`aiProfile`, `operationsTimezone`, and `spendCeilings`) mirror to the record and live-sync. Browser
cursors (`activeDomainId` and the active-branch, search, and view-state key families) remain in IndexedDB:
they describe one device, must not clobber another device, and must not create canonical requests during
Hosted authentication bootstrap. The same classification filters boot reconciliation, so it cannot seed,
hydrate, or evict those local keys. Browser-owned conversation snapshots may use the aux record for
durability without joining the live-sync set.

# 2. The single write path (fs is the one writer)

State changes **only** by dispatching a reducer `Action` through one deterministic door. On the server
that door is `applyLocalMutation()` (`src/lib/server/record/mutation.ts`), reached over
`POST /api/local/mutate`:

```
gate → reduce → persist → attribute → echo
 │       │        │          │          └ publishRecordEvent() → every SSE subscriber
 │       │        │          └ audit entry (aux/activity)
 │       │        └ atomic write to the record (temp+rename) + monotonic version bump
 │       └ the pure reducer (src/lib/core/reducer/), clock pinned to `now`
 └ gateAction(): can this actor's role dispatch this action's category?
```

Every canonical action writer is a **caller** of this path, never an alternate reducer:

- an explicitly committed **renderer** edit applies the reducer action optimistically, posts that same
  small action, and waits for the mutate acknowledgement before presenting success. The compact response
  never returns the multi-megabyte Domain; the record stream adopts the committed document and IndexedDB
  stores that document as a projection;
- an ordinary **master** Domain or Constellation dispatch applies optimistically in memory and sends that
  same compact action to the canonical terminus. The passive persistence effect writes only IndexedDB; it
  never reconstructs intent from a whole-document snapshot. UI drafts remain local until commit, so typing
  does not become a filesystem write per keystroke. A joined client relays the same compact action because
  it cannot write the record itself. Cold document creation retains a full-document bootstrap because no
  reducer target exists yet;
- **headless** callers — the MCP server, autonomous loops, the Telegram bot, local scripts — enter the
  same `applyLocalMutation` terminus, so the action gate, reducer, attribution, and disk semantics match.

The full-document writer is a narrow compatibility path for transitions that genuinely own a complete
base and replacement (for example missing-record recovery and legacy orchestration documents), not an observer
of Domain/Constellation state. It submits `base → next`; the server compares a stable hash of `base` to
the canonical body immediately before writing. The coalescer retains the earliest queued base and latest
replacement, and discards dependent successors when a predecessor fails. A surface with an explicit
Save/commit boundary—especially one followed naturally by modal close, navigation, or reload—uses the
awaited small-action store method. A joined client waits for the exact correlated receipt emitted only
after the owner has committed canon; HTTP relay acceptance is not durability. The 2026-09-02 persistence
incident is recorded in
[the incident report](../../../incidents/incident-2026-09-02-canonical-edit-persistence.md).

The compatibility PUT and compact action are not independent lanes. Every renderer-originated write for
one document—replacement, create, delete, or mutation—enters the same per-document canonical queue. Before
an action enters, it waits for an already-draining replacement transition to finish, including its
coalesced latest successor. This prevents a compact action from interposing inside `A→B→D`; content CAS
still refuses a replacement if another writer advanced canon. Different documents remain parallel.

Joined identity follows the same single terminus. The host first verifies the master-minted sync token,
derives the actor's role for the addressed Domain or Constellation, then delegates that constrained actor
through the already owner-admitted local mutation route. Forgeable wire `actor` fields are never used for
authority or attribution. The mutation terminus gates and emits the one durable activity entry. The host
projects that same entry and deterministic uid into its live tail (plus any social toast); its activity relay
therefore folds into the canonical entry instead of minting a second event.

The replicable renderer pattern is deliberately small:

1. Express the change as the existing reducer `Action`; do not construct a replacement Domain.
2. Call the store's awaited `commitToDomain(domainId, action)` (or its Constellation twin) at an explicit
   Save, generated-image, upload, or clear boundary. Plain `dispatch` uses the same compact canonical path
   for persistent Domain/Constellation actions, but its `void` return is not a durability acknowledgement;
   it remains appropriate for local view state and fire-and-report interactions.
   Reusable inline controls accept the returned Promise: they keep their local draft while pending and do
   not leave edit mode when canon rejects the action.
3. Full-document compatibility transitions, explicit actions, and joined same-document commits serialize on one
   renderer-side document lane so concurrent generation composes and no transport can overtake another;
   different documents remain parallel. Reducer intents reserve and apply their optimistic projection before
   waiting in that lane; only canonical transport is serialized. A per-document pending-intent journal replays
   later edits over any rollback or fresh-canon adoption, so an older receipt cannot erase newer visible work.
   Each full transition carries the exact base hash; coalescing preserves
   the first unwritten base and last desired body only across a proven linear chain. A divergent newcomer
   is rejected rather than sharing another snapshot's receipt, and a failed predecessor cancels its derived successors.
   The store pins one clock for optimistic and server reduction and reconciles from fs on denial or version
   drift. A canonical `changed:false` is success only when the local reducer also found no change; otherwise
   it is a refusal and the editor remains open. Joined no-effects are conservative refusals because the
   owner cannot prove the remote optimistic base; a removed entity must never receive a false Save receipt.
   The daemon advances an accepted
   action from `max(requested time, canonical time + 1)`, but reconciliation never infers concurrency from
   timestamps: an unknown prior version or a non-consecutive canonical version forces a fresh pull. This
   also closes the first-save race before the renderer's version ledger has hydrated. The mutation terminus
   itself re-reads the document immediately before
   reduction, so a long-running operation or Exchange call cannot apply its action to the stale copy it
   loaded before external work. This acknowledgement repair uses the explicit fresh document reader,
   bypassing and superseding the boot coalescer's short TTL and any older request still in flight.
4. Every reference-bearing asset store—image, audio, text, and embedding—waits for canonical bytes before
   returning its id (image/audio UI call sites use the explicit `store*ForReference` names). The caller then
   awaits the reducer reference action. Replacement and clear paths do not delete the prior id themselves:
   imported and deduplicated records may share one id across several live fields, so only a record-wide
   reference scan may reclaim it. If record reconciliation or restore has temporarily suppressed mirror
   writes, a reference-safe store waits for that critical section to end; suppression is not success. This
   is the invariant **bytes first → reference second → reference-aware reclamation later**.
5. Bulk work is one logical transaction at its natural boundary, not one giant document replacement and
   not an unbounded storm of tiny fs calls. Use a typed bulk reducer action for one user operation (for
   example `ADD_SOURCE_FILES`), the bounded binary asset-batch transport for many derived payloads, and
   bounded concurrency for genuinely large independent media. Generated images/audio remain individual
   commits because each completed generation is independently useful and already dominates its fs cost.
6. Destructive work is pessimistic at the irreversible edge: commit the canonical reference/document
   removal first, then reclaim orphaned bytes and supporting projections. Cleanup failure may leak an
   orphan, which is recoverable; cleanup before acknowledgement may create a dangling reference or destroy
   the only recoverable copy, which is not. Domain and Constellation delete commands therefore keep their
   visible document until the owner confirms the fs removal; passive persistence effects only evict the
   corresponding IndexedDB projection. Reducers never schedule storage deletion as a side effect.
7. A document that does not exist yet uses the strict `createDomain` / `createConstellation` store
   boundary. It may render an optimistic projection, but callers await the first canonical version before
   closing, navigating, or marking a workflow complete; refusal removes that projection. Profile/avatar
   edits use the analogous strict aux boundary, serialize per member, and treat IndexedDB as a projection
   after canonical success rather than rolling back a saved filesystem value because the cache failed.
8. Invariants that span documents are checked where the complete canonical collection is visible. A
   one-document reducer may not hide an implicit multi-document write. Constellation single-membership,
   for example, rejects a conflicting claim at create/mutation/PUT/operation boundaries; moving a Domain
   is an explicit remove followed by add, each with its own canonical acknowledgement.

No feature should reproduce daemon/session/latch logic or call the mutation route itself. These shared
adapters are the reusable boundary between React interaction and the canonical writer.

Runtime identity is part of failure classification. On loopback, branded Hosted, and raw Fly fallback
hosts, the record is required from the first request: a network throw, 403, or reset failure cannot become
`unavailable` merely because the shared daemon latch has not yet observed a success. Only an ordinary web
build that intentionally has no record route may use the IndexedDB-only fallback.

`daemonFetch` also owns one overall deadline for browser→record requests. It composes that deadline with
the caller's signal and receives each finite response body inside the same scope: caller cancellation
remains an `AbortError`; a deadline before the full response arrives becomes the typed
`408 canonical-record-timeout` response; and another network failure remains “no response.” The distinction
is operationally important: queued mirror writes surface one timeout failure without retrying it as a 5xx
storm, while their existing bounded retry policy still covers short network throws and daemon 5xxs. The
access gate retains its tighter first-screen budget; the shared deadline releases the underlying request
and boot coalescer even after that UI budget has elapsed.

Because the reducer is **pure and clock-injected**, the same action produces a byte-identical result in
the browser (optimistic) and on the server (authoritative) — which is what lets the two stay in lockstep
with no merge. The full constitution is [API_PRINCIPLES.md](../API_PRINCIPLES.md).

`fs-store.ts` owns the disk primitives (`readDocument`/`writeDocument`/`readAsset`/`writeAsset`/`…Aux`).
Five invariants there: **monotonic versions** (read meta → increment, synchronous I/O so no mutex is
needed), a **forward-time guard** (`requireForwardUpdatedAt` refuses a write whose `updatedAt` is
older than the record's logical clock), an **equal-clock identity guard** (the same clock is accepted
only for a deeply identical retry; different bytes receive 409 without a version bump), a **content-CAS
guard** (every full replacement must prove the exact canonical body it derives from; null is
create-if-absent), and a **collection-regression guard** (guarded *domain* PUTs only). Content CAS is the
decisive ancestry proof; the clock and collection checks remain defense in depth and clearer diagnostics.
The regression guard exists because the clock guard alone cannot
catch the worst stale-base failure: a renderer whose replica missed a headless commit (e.g. an
overnight Program run) edits that stale base, stamps a *fresh* `updatedAt`, and its whole-document
write would replace canon — silently deleting every optional collection its base never saw (the
2026-08-06 hosted stream-loss incident). No legitimate reducer action produces key-absence-after-
presence (a slice always writes its collection key back, at worst emptied `{}`), so `writeDocument`
refuses a guarded write that DROPS a populated collection (`streams`, `positions`, `notes`, …) →
**409**, and the client converges to the record's fuller copy. Historical rename lineage
(`projections` ↔ `positions`, `priors` → `notes`, `loop` → `program`) still counts as presence, so a
mixed-version writer cannot trip the guard solely because it uses the other name
(`droppedPopulatedCollections`, pure + tested).

### Versioned document migrations

`DomainState` and `Constellation` carry an optional body-level `schemaVersion`; `DocMeta.version`
remains the optimistic-concurrency counter. `readDocument` and IndexedDB hydration pass these bodies
through the pure plans in `document-migrations.ts`, backed by the generic runner in
`ordered-migrations.ts`. Each plan declares a retained baseline plus the ordered steps after it, and
every step carries its off-ramp (`translates`, `retireWhen`, `retireBy`, `issue`). The Domain plan
currently retains baseline 6 plus two steps (experience vocabulary → v7, scene arrays → v8) because
hosted records still exercise them; the Constellation plan retains only its baseline. The live list is
the plan itself, mirrored in [Legacy lifecycle](legacy-lifecycle.md) — do not restate it elsewhere.

An unmarked, malformed, or older body joins at the retained baseline, receives every registered step,
and is stamped current in memory. A current marker returns the same reference. A future marker remains
unchanged and warns once per process. Reads never write: a stamp reaches the record only on the next
legitimate document write, while `readRawDocument` remains literal.

For the next schema change, append one pure step to the relevant plan; its array position defines the
new version, so the current version cannot drift from the registry. When old steps are no longer needed,
raise the retained baseline and remove them together. Never add a shape-sniff path beside the registry.

Raw Markdown source