Release & deploy [Flow]

# Release & deploy `[Flow]`

One repo, **three independent deploy targets** — the Vercel **gateway**, the **desktop** app, and the Fly **instance image** — plus the running **fleet**. This is how a source change reaches each, and why the fleet only moves on a deliberate roll (no commit silently restarts a user's always-on expert).

```mermaid
flowchart TB
    Dev([source change]) --> Gate["**gate** (pre-push hook + ci.yml on every PR)<br/>lint · typecheck:all · test:run"]
    Gate -->|red| Block["fix before merge"]
    Gate -->|green| Main[(main)]

    Main --> Vercel["**Vercel** builds main<br/>→ gateway @ www.meridians.global<br/>(process.env baked at BUILD time)"]

    Main --> Rel{"ship a target?<br/>(two separate, dev-driven paths)"}
    Rel -->|"npm run release patch/minor/major"| RelPR["release/vX.Y.Z PR<br/>CI → protected main"]
    RelPR -->|"npm run release finalize"| Tag["git tag vX.Y.Z<br/>(the version source of truth)"]
    Tag --> DeskRel["**desktop.yml** on tag<br/>→ publish GitHub Release<br/>(desktop only — no fleet)"]

    Rel -->|"npm run fly-deploy"| Img["**dev machine** (no CI)<br/>build + push :vX.Y.Z<br/>then move :current<br/>(bakes NEXT_PUBLIC_* build args)"]
    Img --> Point["MERIDIANS_TENANT_IMAGE=…:current<br/>pinned ONCE — moving :current IS<br/>the pointer update (no redeploy)"]
    Img -.->|"push succeeded,<br/>promotion failed"| Resume["**fly:promote** immutable tag<br/>resume without rebuild<br/>(--roll is explicit)"]
    Resume --> Point

    Point --> NewInst["NEW instances → new deploy (automatic)"]
    Img -.->|"automatic on success<br/>(unless --no-roll)"| Update["**fleet roll** — dev CLI only<br/>(fly-deploy · fly:fleet-update)<br/>resolve :current → digest · snapshot · recreate<br/>idempotent, running-only"]
    Update --> Fleet["EXISTING hosted users → new release"]

    subgraph provision [Provision a hosted instance — the gateway drives Fly]
        Claim([claim address]) --> Prov["provisionInstance<br/>createApp → setSecrets → createVolume → createMachine"]
        Prov --> Boot["machine boots the IMAGE on :3000<br/>record on /data volume"]
        Boot --> HB["signed heartbeat → gateway"]
        HB --> Ready{readiness?}
        Ready -->|"image·schema·record·scheduler ✓"| Live["ready → Live"]
        Ready -->|keys missing| Keys["needs_keys"]
    end

    Live -.->|"machine destroyed out-of-band"| Gone["reconcileInstanceLiveness<br/>row → destroyed + release domain<br/>→ Director re-claims a NEW domain"]
```

## One image, all tenants — not one image per tenant

A common misconception: that each new build means a new registry entry *per hosted user* whose ref must be threaded into every tenant's env. It doesn't. There is **one shared registry app** (`meridians-daemon`) holding **one repo** of tags — `:vX.Y.Z` are just labels on it, distinct only for rollback + audit. Every tenant machine (`mrd-<slug>`) boots *from* a ref; the image is **app-agnostic and stateless** (identity + record live on the machine's `/data` volume, never in the image).

So a release changes **exactly one pointer**, not N tenant configs: the gateway-level `MERIDIANS_TENANT_IMAGE`, which says "what should new instances boot." The image ref for an *existing* instance lives on its **Fly machine config** (mirrored to `imageVersion` on the instances row) — moved only by a deliberate fleet roll, never by per-tenant env. There are no per-tenant image env vars to keep in sync.

**The pointer is a mutable `:current` channel, pinned once.** Rather than have `MERIDIANS_TENANT_IMAGE` chase a new datestamp/version ref every build (a copy-paste + a Vercel **redeploy**, since env is baked at build time), `fly-deploy` builds the image ONCE under an immutable version tag (`:vX.Y.Z` — for rollback), then **retags `:current` onto those exact manifest bytes by digest** through the registry v2 API (an in-registry copy, no rebuild, local Docker, or temporary auth files). The bytes are hashed before the tag moves and `:current` is read back by digest afterwards. So `:current` and `:vX.Y.Z` are guaranteed the *same* image — never a second build that could diverge on a cache miss. Set `MERIDIANS_TENANT_IMAGE=registry.fly.io/meridians-daemon:current` **once**; a hosted release just moves `:current`, so new provisions track it with no env change and no redeploy. The fleet roll resolves `:current` to its pinned `@sha256:…` digest before comparing, so rolls stay idempotent even though the tag is mutable (Fly stores the digest on each machine — a tag-vs-digest compare would re-roll forever).

## Desktop release and hosted deploy — independent targets

There are **two** shipping commands, deliberately decoupled so one never touches the other's blast radius:

- **`npm run release <version>`** — prepares the desktop release as a version PR. After CI and merge,
  **`npm run release finalize`** tags that protected-main commit; the tag fires **desktop.yml** → the
  three-platform build + GitHub Release. Desktop only; it does **not** touch the hosted image or fleet.
- **`npm run fly-deploy`** — the hosted-image deploy. Builds + pushes the daemon image (`:vX.Y.Z` + `:current`) **from your dev machine, no CI**, then rolls the running fleet on success unless passed `--no-roll`. This is the only thing that moves hosted users.

| Change to… | Reaches via | Fleet affected? |
|---|---|---|
| Gateway code (`(gateway)`, admin, Stripe, provisioning) | `git push` → **Vercel** rebuilds `main` | no |
| Instance/daemon code (engine, reducer, Program, `(instance)`) | **`npm run fly-deploy`** (dev machine — image build + push) | yes by default; `--no-roll` skips it |
| Desktop shell | version PR → `npm run release finalize` (git tag → **desktop.yml** Release) | n/a |

A `git push` updates the gateway automatically; the **fleet is only as current as the last `fly-deploy` you built + rolled**. This is deliberate — the fleet image roll is a separate dev-CLI step (`npm run fly-deploy` / `npm run fly:fleet-update`).

## `fly-deploy` → roll (one command; the roll is the only step that touches live users)

The mental model: `fly-deploy` ships a versioned image, makes it the default for *new* signups when it moves `:current`, then rolls *existing* hosted users onto it. Pass `--no-roll` to ship for new instances only without restarting existing experts.

1. **Build + push** — `npm run fly-deploy [patch|minor|major|X.Y.Z]` builds the image and pushes it under `:vX.Y.Z` **and `:current`**, from the dev machine (no CI in the loop). The image is stateless; the canonical record lives on the mounted `/data` volume, never in the image.
2. **No gateway step** — `MERIDIANS_TENANT_IMAGE` is pinned to `:current` once, so moving the channel in step 1 *is* the pointer update: instances provisioned **after** it boot the new release automatically, with no Vercel change and no redeploy. (First-time setup only: pin `MERIDIANS_TENANT_IMAGE=registry.fly.io/meridians-daemon:current`.)
3. **Roll — ship it to hosted users** — after a successful build, `fly-deploy` rolls the running fleet unless passed `--no-roll`. The roll is idempotent + snapshot-first, resolving `:current` → digest before the roll. Fleet rolls are **dev-CLI only** — there is no UI button and no MCP op; roll without a rebuild via `npm run fly:fleet-update` (also handy for a rollback: `npm run fly:fleet-update -- registry.fly.io/meridians-daemon:v<older>`).

Partial success is resumable at both remote boundaries. If the immutable image exists but promotion failed,
run `npm run fly:promote -- <immutable-tag-or-ref>` to move and verify `:current` without another build;
`--roll` is explicit. If promotion succeeded but one or more tenants failed, rerun `npm run
fly:fleet-update`—already-current and non-running machines are skipped.

## Data survives a roll

A roll recreates the machine on the **same volume** (snapshot first); a migration copies to a new volume keeping the source. Only an audited teardown past its grace window destroys a volume. The image is disposable; the volume is the source of truth.

## Config baked at build time (the gotchas)

- **Vercel** resolves `process.env.*` at **build** time — a var changed in the dashboard is absent until a **redeploy** (`GATEWAY_URL`, the Stripe webhook secret). `npm run vercel:doctor` checks the running build. This is exactly why `MERIDIANS_TENANT_IMAGE` is pinned to the mutable `:current` tag: the value never changes across releases, so a release never needs a Vercel env edit **or** a redeploy — it moves the tag in the registry instead.
- **The instance image** bakes browser-safe `NEXT_PUBLIC_*` (Supabase URL + publishable key, gateway URL) as Docker **build args** — `fly-deploy` sources them from your local `.env` (the gateway env). Managed provider keys + the machine token are injected per-machine as **Fly secrets** at provision, never baked in.
- **The canonical GATEWAY host is `www.meridians.global`** (Vercel) — the apex 307-redirects and a redirect strips `Authorization`.

## Instance address — `<slug>.meridians.global`

Each instance is its own Fly app (`mrd-<slug>`), so a single static wildcard CNAME can't fan out to the right app — the address is wired **per instance**, automatically, right after `createMachine` (best-effort, so a DNS/cert hiccup never fails a provision whose machine is already up):

1. **Fly cert** — `ensureInstanceAddress` → `fly.createCert(app, "<slug>.meridians.global")` (GraphQL `addCertificate`, idempotent). Fly terminates TLS on the app's free shared IP and routes to the machine on `:3000`.
2. **Vercel DNS** — `ensureInstanceCname(slug, "mrd-<slug>.fly.dev")` adds a `CNAME <slug>.meridians.global → mrd-<slug>.fly.dev` via the Vercel DNS API (meridians.global runs on `ns*.vercel-dns.com`). Idempotent; a no-op without `VERCEL_DNS_TOKEN`.

The machine service forces port 80 to HTTPS at Fly's edge. This is part of the authentication boundary: OAuth derives its callback from the browser origin, so a cleartext first visit must never produce an `http://<slug>.meridians.global` callback that falls outside the Supabase redirect allowlist.

One-time gateway config: `VERCEL_DNS_TOKEN` (+ `VERCEL_TEAM_ID` if the zone is on a team). Without it the cert is still added but the CNAME isn't, so the instance stays reachable at `mrd-<slug>.fly.dev` until DNS is configured. The gateway apex/www records (→ Vercel) are untouched — only per-instance subdomains are added.

## Out-of-band teardown self-heals

If a machine is destroyed directly on Fly (CLI/dashboard/reaper), the next status poll runs `reconcileInstanceLiveness`: the row is marked `destroyed` and the Director's `host_domain` claim released, so the deploy flow returns to the claim step and they provision a **new** domain — a stale "active" row can't block re-deploy forever.

Raw Markdown source