Baselines

Pin a version as the reference — every change is measured against it.

A baseline is the version you’re comparing everything to. Usually your current published version, but you can pin any version as a baseline for a specific comparison.

The default baseline

By default, comparisons use the currently published version of the Flow as the baseline. Makes sense — you’re asking “is this change worth shipping?”

Custom baselines

Pin a specific version as baseline when:

• Comparing multiple candidates against a common reference — pin one, evaluate the others.
• Testing against a historical high-water mark — “our v12 was our best-ever quality; let’s not regress below that.”
• Isolating the effect of one change — pin an intermediate version to compare “before this change” vs. “after this change”.

Setting a baseline

On any simulation form: Baseline → pick version (or “none” for absolute results).

Or in the CLI:

nora simulate --target draft --baseline version:v14 --dataset support-golden

Reference baselines

Some workspaces pin one version as a permanent reference baseline:

• All CI simulations run against it.
• Regression rate is computed against it.
• New versions are labeled “vs. reference: +/-X%”.

Set from Flow settings → Reference baseline. Only one at a time. Roll forward as you ship confirmed improvements.

Baseline drift

If your reference baseline is old, current published behavior has diverged. Nora shows a baseline drift warning if the reference is more than 90 days out of date. Roll the reference forward to the current published version.

Baseline for cost measurement

Cost baselines are useful too. If your reference baseline shipped at $0.03/run, comparing new changes gives you a running cost trajectory:

• “v18: -$0.005 vs. baseline”
• “v19: +$0.012 vs. baseline”
• “v20: -$0.001 vs. baseline”

Watching this over time keeps cost creep visible.

The “no baseline” mode

Sometimes you want raw absolute numbers — how many pass, how much cost — with no comparison. Set baseline to none. Useful for first-time simulations before you have anything to compare against.

Pinning per-Improvement

Every Improvement candidate has an implicit baseline (the state before the Improvement was applied). Simulation for that candidate uses the implicit baseline by default. Override if you want to test the improvement against something else — e.g., against a hypothetical “no-op improvement” to check whether the change is worth it at all.

Baselines and datasets

Baseline is separate from dataset. A single dataset can be used with many baselines; a single baseline can be used with many datasets.

The full context for a simulation is (target, baseline, dataset). All three are recorded on every run so results stay interpretable months later.