Authoring specs that survive

Authoring a cairn spec is closer to authoring a contract than to authoring a test. The contract is the durable thing. Steps are repairable hints. This page is the discipline that makes a spec live across months of code churn instead of weeks.

The contract first, the steps second

Start every spec with intent and outcomes, never with steps. intent is one sentence describing what the user wants to happen. outcomes are the typed observables that confirm it did. Until both are written, the spec has no contract; running it just produces a transcript of whatever the steps happened to do.

intent: Search for "spiced chickpeas", filter by category, and see exactly 7 results.
outcomes:
  - id: results-narrowed
    count: { selector: "[data-testid='result-count']", equals: 7 }
  - id: filter-applied
    text: { contains: "category:baking" }
steps:
  - open: { path: "/search?q=spiced+chickpeas" }
  - click: { by: { role: checkbox, name: "Baking" } }

If a colleague can delete every step: in the file and you can still describe what the spec should prove, the contract is right. If removing step: turns the spec into pure prose, the contract is not yet there.

Outcomes are typed observables

The verifier vocabulary is closed: text, notText, url, network, noFailedRequests, console, count, xlsx, file, httpJson, script, process. If you find yourself wanting a new verb, use script with a small JS expression and a .raw.json sidecar — never invent a new verifier type, because then every agent has to learn the new vocabulary.

Each outcome must be enforceable from a single page state. Outcomes are not assertions-in-time; they are assertions-on-state. If you need a sequence ("after step 3 the cart is empty, after step 5 it has 3 items"), write two outcomes and pin them to their step.

outcomes:
  - id: cart-empty-after-clear
    count: { selector: "[data-testid='cart-count']", equals: 0 }
  - id: cart-three-after-add
    count: { selector: "[data-testid='cart-count']", equals: 3 }

Steps are hints, not scripts

A step is an instruction a code generator (you) or a runtime (cairn) might rewrite without changing the contract. Steps must use the typed step vocabulary too (open, wait, click, hover, fill, type, press, scroll, upload, download, transform, request, snapshot, use, batch, eval, monitor). Free-form prose inside an eval: step is allowed, but if you find yourself reaching for it, stop and ask whether one of the typed steps already encodes the intent.

The same locator philosophy that drives playwright should drive cairn: prefer semantic locators (by: role|label|text), fall back to data-testid, and only touch CSS/XPath when nothing else survives.

steps:
  - open: { path: "/settings" }
  - click: { by: { role: tab, name: "API tokens" } }
  - click: { by: { role: button, name: "Rotate token" } }
  - click: { by: { role: button, name: "Yes, rotate" } }

Tags for suite selection

Put stable labels on a spec under metadata.tags. Agents and humans then run a subset of a directory without inventing new folders:

metadata:
  feature: answer-change-temporal
  priority: high
  tags:
    - checkout-regression
    - temporal
    - answer-change

# preview which specs match (no browser)
cairn run flows/ --tag answer-change --select-only --json

# run every matching spec (AND if you pass multiple --tag)
cairn run flows/ --tag answer-change --cold-start --headed
cairn run flows/ --tag temporal --tag checkout-regression --cold-start

Matching is case-insensitive. Multiple --tag flags mean AND (the spec must declare every listed tag). Specs with no metadata.tags never match a tag filter.

Labels, hooks, and A/B stats

Stamp free-form cohort labels on every run in an invocation:

cairn run flows/ --tag answer-change \
  --label path=temporal \
  --label suite=answer-change-ab \
  --before 'tools/set-answer-change-path.sh temporal' \
  --cold-start

• --label key=value (repeatable) is written into each run.json as labels.
• --before <shell> (repeatable) runs after services/secrets and before the first spec — use it for domain setup (path flips, warmers). Failures abort the run.
• --after <shell> (repeatable) runs after all specs and before services teardown; failures are logged, non-fatal.
• services.seed.postCommands always run after seed (even when seed is skipped as fresh) — use for fixture ensure scripts.

Aggregate cohorts:

cairn stats --group-by path --label suite=answer-change-ab --baseline rabbit --format md

Markdown output includes a table, ASCII bar charts (pass rate / duration p50 / optional domain metric), and pairwise deltas. JSON/YAML use schema urn:cairntrace.dev:stats:v1. Domain latency is harvested from outcomes/*.raw.json when fields like processingDurationMS are present.

Keep product-specific scripts (path flip, mongosh fixtures) in the automation project; cairn only orchestrates via hooks + postCommands.

Cold-start is not optional

Every spec must satisfy the cold-start contract: replayable from a fresh browser session. Four supported paths, pick one:

1. imports: [actions/login.yml] + steps: [{ use: login }] — reuse an action file.
2. session: { resume: <checkpoint> } — capture a logged-in state once with cairn checkpoint capture-from-session and resume it.
3. preconditions: { commands: [{ run: "..." }] } — set up state from the shell.
4. coldStart: guest — explicitly acknowledge that a public flow intentionally starts without a session.

There is no path that "just works because my dev session is logged in." A spec that only runs in dev is a spec that does not run. The guest acknowledgement suppresses the setup lint; it does not skip the required cold-start replay.

The contract hash exists for a reason

After editing intent or outcomes, the contract hash changes. If you forget to re-stamp, cairn run refuses the spec — by design. To re-stamp:

cairn spec verify my-spec.yml --stamp

You should be running cairn spec verify --stamp exactly once per contract change, never zero times and never five times. The hash is meant to be noisy when you skip it.

Wait steps are bounded

wait, evaluate, and browser-network calls are all hard-bounded at 30000 ms by default. Override per-step with timeoutMs when the app genuinely needs more time. Do not blanket-timeout the whole spec — make the slow step slow, not the spec slow.

For hydration-sensitive first interactions, prefer:

- open: { path: "/app", waitUntil: networkidle }

over a separate wait: step. Two waits in series are how a fast spec turns into a slow one.

Repairs are first-class

When a run fails, start with agent_context.md and outcomes/*.md. The run also retains events.ndjson, console/, network/, per-step files under diagnostics/, and any snapshots, screenshots, trace, or video enabled by the capture policy. The repair engine reads the structured evidence and proposes step rewrites that preserve the contract hash.

The repair proposal is a suggestion, not an approval. Open the diff, check that the contract is unchanged, and apply only what keeps the behavior intact. If the diff touches intent or outcomes, that is not a repair; that is a contract change, and the hash must be re-stamped.

A checklist before you call a spec done

• cairn explain --format json — surface-level sanity check.
• cairn docs <topic> --format json for focused authoring reminders.
• cairn spec verify my-spec.yml --format json — schema, contract hash, dead links.
• cairn run my-spec.yml --cold-start --format json — single golden run from a fresh browser.
• cairn docs snippets --format md — to lift reusable actions/*.yml files.

If cairn run passes once on cold-start, paste the harness command into your project README. The next agent that touches that flow will thank you.