Skip to main content
The scariest signal is one you thought you’d already resolved. Regression detection watches for this specifically — a new signal that looks like something you previously fixed.

​
How regressions are detected

Every resolved Improvement carries a signature — the pattern of signals it addressed. Nora watches new signals against that signature. A signal is flagged as a regression if:
  • It matches the signature of a previously-resolved Improvement.
  • It fired after that Improvement’s deploy time.
  • It’s on the same Flow (regressions within a Flow are much more common than cross-Flow).
Regression flags are additive to normal signal grouping — a regression is still a regular signal in its own cluster.

​
The regression view

Filtered view of Signals: Signals → Regressions. Shows:
  • The regressed signal.
  • The Improvement that supposedly fixed it (with link to the Improvement history).
  • The version(s) the regression appeared in.
  • The gap in days between fix and regression.

​
Why regressions happen

Common causes:
  • Flow change reverted the fix — someone re-edited the same prompt or config.
  • Data change — a document was superseded or removed, disrupting a fix that depended on that data.
  • External change — a tool API changed, model provider updated, upstream source drifted.
  • New adjacent behavior — Agent expanded into new territory that recreated the old pattern in a new context.
  • Fix was too narrow — the Improvement covered specific cases but missed a slightly different variant.
The regression view shows Nora’s best guess for the cause based on what changed between the fix and the regression appearance.

​
What to do

  1. Confirm it’s a real regression (not just a stale signature match).
  2. Look at what changed — flow diffs, data changes, external announcements.
  3. Widen or re-apply the fix — usually by refining the Improvement rather than starting fresh.
If the fix was a prompt change and someone edited the prompt, revert or restructure. If the fix was a data addition and data was removed, restore. If the fix was too narrow, use the regression as a new example and re-run the Improvement flow with a broader pattern.

​
Regression alerts

Regressions are high-priority by default. Turn on Slack/email alerts specifically for regression signals — Signals settings → Regression alerts. Doesn’t add noise (regressions are rare) but you’ll want to hear about them immediately.

​
Preventing regressions

The cleanest prevention: retention tests. When you resolve an Improvement, save the associated failing traces as a retention dataset and run it in every simulation run (see Simulation). Any future change that breaks these tests is caught before it deploys. The Improvement flow suggests this by default when you resolve a large cluster.