19. Outbound research, monitoring & the Sources repo
Source path: knowledge-base/knowledge/architecture/persistence/research-and-scaling.md
# 19. Outbound research, monitoring & the Sources repo
The research/forecast spine adds an **outbound** dimension the collaboration model above doesn't
otherwise touch — the app reaches the open web (via Firecrawl) and the LLM on the Director's keys. How it
fits the single-instance design:
- **The Sources repo** is a **global catalog** (not domain-scoped): Sites · Topics · **Tickers**,
persisted in the `researchCatalog` IndexedDB store (seed merged with the user's own by id —
`lib/engine/research/catalog.ts`). Shared across every domain; contributor+ writes
(`ADD/UPDATE/DELETE_CATALOG_{SITE,TOPIC,TICKER}`, all tagged `contribute`).
- **A Ticker is a source of truth by reference.** `cycle.tickers` / `forecast.monitor.tickers` hold
catalog ticker **ids**, not copies — because a ticker is a *stateful* instrument whose reading history
is one global series (unlike stateless Sites/Topics, which embed). A read scrapes the page +
AI-extracts the typed value and appends to the *catalog* ticker's `history`
(`lib/engine/research/ticker.ts`).
- **Runs are one generalised pipeline.** `ResearchRunDashboard` fans a run's cycles + forecast monitors
out through `/api/ai/firecrawl` (key resolved server-side, never exposed), routes cycle findings into
streams and forecast findings into checks, and reads any attached tickers — deduped, metered per
credit. A **dry-run Test** takes the same path but writes nothing.
- **Cross-domain writes — the honest constraint.** The client reducer's `updateDomain` mutates only the
**active** domain, so a constellation surface can't rewrite a *member's* cycle/forecast cadences with a
plain local dispatch. The capability is *not* missing — the sync layer's `applyActionToDomain(target,
branch, action)` (§11) already folds an action onto **any** domain by id, so constellation-wide
flow-down is a matter of emitting per-member targeted action-pushes, not new plumbing.
Open on GitHubRaw Markdown source