3b. How a research cycle actually spends — the Firecrawl fan-out

# 3b. How a research cycle actually spends — the Firecrawl fan-out

> **Source of truth:** `buildFirecrawlRequest` / `runResearchSource`
> ([`src/lib/engine/research/firecrawl.ts`](../../../../../src/lib/engine/research/firecrawl.ts)),
> [`research/article-novelty.ts`](../../../../../src/lib/engine/research/article-novelty.ts),
> [`research/run-pipeline.ts`](../../../../../src/lib/engine/research/run-pipeline.ts),
> [`research/defaults.ts`](../../../../../src/lib/engine/research/defaults.ts),
> [`core/position/breadth.ts`](../../../../../src/lib/core/position/breadth.ts).
> Modelled in [`constellation-economics.ts`](../../../../../src/lib/core/economics/constellation-economics.ts).

This is the mechanic that underpins both spend and **how much of the outside world we ingest**. It is
counter-intuitive in exactly one way, and getting it wrong misprices everything downstream.

### Three layers of search — only one of them recurs

| Layer | When | Shape | OR-combined? |
|---|---|---|---|
| **Capture research** (onboarding grounding) | once per Public Domain | 3 saturating passes at `limit: 12`, deduped to 30 kept — 48 credits | **No** — depth from many angles is the point |
| **Maintenance research** (loops onboarding proposes) | once, at setup | 3–5 loop drafts, each one question + its term set | n/a |
| **Research loops** | every run, forever | **one OR'd search per cycle** | **Yes** |

Conflating these is why the earlier estimates were wrong: capture is a one-off 48 credits, while a
loop is a call every run for as long as it lives.

### Search terms no longer multiply. Neither do sites.

#### The mechanic, in detail

A cycle issues **one Firecrawl `search` action per search term, per run**. The cycle's trusted sites are
attached to that *same* request as `includeDomains` — a **filter that scopes the query**, not a fan-out.

```
track cycle · 4 search terms · 12 trusted sites  →  4 searches/run   (NOT 48)
```

So **adding sites is free** and only sharpens precision; **adding a search term costs a full search
every run, forever.** That asymmetry is the single most useful thing a Director can know about their
own bill, and it points the right way: prefer a tight term list over a narrow site list.

`monitor` mode inverts it — no search at all, one `scrape` per site, so there **sites are** the
multiplier.

### `articlesPerSource` is the `limit` — and web + news each return it

The breadth net writes `limit`:

| Net | minimal | **focused** | steady | broad | wide | exhaustive |
|---|---|---|---|---|---|---|
| Articles/search/**source type** | 1 | **3 (default)** | 5 | 10 | 20 | 40 |

The default was lowered **5 → 3** once the per-source-type semantics were confirmed: at 5 the everyday
search pulled ~10 articles, which is more than a domain needs to stay current and is paid for twice
(crawl, then the interpret pass that reads every one).

`DEFAULT_SEARCH_SOURCES` is `["web", "news"]`, and Firecrawl's API reference is explicit that `limit`
is *"the maximum number of results to return **(per source type when using multiple sources)**"*.
`runResearchSource` then concatenates both (`[...data.web, ...data.news]`). A `limit: 3` search
therefore returns up to **6** articles, and every one is scraped to markdown, so every one bills a
scrape credit. Modelling `limit` alone **understates both spend and ingest by the source count (2×)**.

> ⚠️ **Two documented caveats on the `news` channel**, which make it a poor default for scoped cycles:
> **`tbs` does not filter news** (*"tbs only applies to web source results"*), so the recency window
> `loopRecency` computes — the contract that a run reads exactly what's new since the last one —
> holds for the **web half only**; news returns unbounded by time every run. Article identity is resolved
> *after* Firecrawl bills but *before* the Article budget, persistence, and interpretation, so repeat hits
> still cost crawl credits but cannot crowd novelty out of downstream model spend or belief movement. And **domain filtering
> against the news index is undocumented**, so a `track` cycle may be paying for a channel that
> returns little. For scoped cycles `["web"]` is likely the better default — same recency guarantee,
> half the spend.

### The arithmetic, per run

```
searches  = cycles                                    ← topics OR into ONE query per cycle
articles  = searches × articlesPerSource × searchSources        ← what we ingest
credits   = searches × ( searchCreditsPer10 × ceil(limit/10)
                       + articlesPerSource × searchSources )     ← what we pay
```

**Search terms no longer multiply calls.** A cycle's topics are OR-joined into a single Firecrawl
query (`"advanced packaging" OR "HBM supply"`), so a 5-term cycle issues **one** search, not five.
Terms still shape *what* is retrieved and therefore how the breadth budget is allocated — a busy
entity takes more of the N slots than a quiet one — but they stop being a per-run charge. Loops mixing
unrelated questions opt out with `topicCombination: "separate"`, and sets wider than `SAFE_OR_TERMS`
fall back automatically, because a long OR chain ranks worse than separate searches.

Cadence multiplies all three by runs/week. Recency (`qdr:d` by default for cycles) bounds *freshness*,
not volume.

**Worked default** — 5 cycles × 4 OR'd terms, focused net (3/source type), web+news, weekday `daily`:
**5 searches/run** (one per cycle) × 6 articles = 30/run → **150 articles/week**, 240 credits/week.
The 20 search terms cost nothing; five cycles is what the bill is made of.

### Sensitivity — double one lever, hold the rest (one domain, `balanced-smart`)

| Lever | Change | Weekly $ | Δ |
|---|---|---|---|
| **Search terms, past 6 (cliff)** | 4 → 7 | $0.38 → $1.84 | **+384%** ⚠️ |
| Research runs / week | 5 → 10 | $0.38 → $0.63 | +65% |
| Cycles | 5 → 10 | $0.38 → $0.62 | +64% |
| Articles / source type (breadth) | 3 → 6 | $0.38 → $0.59 | +55% |
| Tutor firings / week | 1 → 2 | $0.38 → $0.42 | +11% |
| **Search terms, within 6** | 4 → 6 | $0.38 → $0.38 | **free** |
| Position monitors | 1 → 2 | $0.38 → $0.38 | +0.5% |
| Trusted sites | 6 → 12 | $0.38 → $0.38 | **free** |

**Two levers are free, one is nearly free, and one is a cliff.** Terms cost nothing because they OR into
one query, sites because they scope it, and monitors add only a small internal re-pricing call. But the
**7th term in a cycle trips the `SAFE_OR_TERMS` fallback to one search per term and
quadruples that cycle.** The marginal term looks free right up until it isn't; the editor should warn
at six.

The top four are all the same multiplier on `articles/week` and are interchangeable levers on one
number: **how much we pull**. The bottom two are the interesting ones — **continuous monitoring of a
live Position is essentially free** (it re-prices off the internal Signals window, no crawler), and
**sites cost nothing at all.**

---

Raw Markdown source