nora feedback

Submit thumbs and corrections on traces — from CLI or external apps.

Feedback is how you signal that an answer is good or bad. Every submitted feedback row becomes a Signal, which may join a cluster and eventually seed an Improvement.

The feedback CLI is the surface for external apps and batch submission. In-app thumbs from the chat UI use the same endpoint under the hood.

Commands

Command	Description
feedback describe <flow-slug>	Print URL, auth, and body shape for external POSTs.
feedback list <trace-id>	List existing feedback rows for one trace.
feedback submit <trace-id>	Insert a feedback row from the CLI.

feedback describe

nora feedback describe support-agent

Prints the exact HTTP contract for external feedback POSTs:

• URL — the endpoint to POST to.
• Auth — required headers or signing scheme.
• Body shape — the fields, their types, and which are required.
• Response — what you’ll get back.

Copy-paste this into your external app’s docs and wire up thumbs-up / thumbs-down buttons.

Add --json for a machine-consumable version.

feedback list

nora feedback list tr_abc

Every feedback row for one trace. Includes:

• like or dislike.
• Comment (if any).
• External user info (if the submitting app passed one).
• Metadata payload.
• Structured correction (if any).
• Timestamp.

Useful for aggregating multiple opinions on the same run.

feedback submit

nora feedback submit tr_abc \
  --dislike \
  --comment "Wrong refund policy quoted" \
  --external-user u_alice \
  --external-user-display-name "Alice Smith" \
  --correction '{"expected":"30 days","actual":"14 days"}' \
  --metadata '{"channel":"web-chat","urgency":"high"}'

Flags:

• <trace-id> (positional, required) — the trace to attach feedback to.
• --like or --dislike (exactly one, required).
• --comment <text> — human-readable note.
• --external-user <id> — your app’s user identifier.
• --external-user-display-name <name> — human-readable name (for UI display).
• --correction <json> — structured fix (“what should have happened”).
• --metadata <json> — arbitrary tags/context.

Recipes

Bulk-import feedback from a CSV

Say you have a CSV: trace_id,verdict,comment,corrector_email.

while IFS=, read -r trace verdict comment corrector; do
  flag="--$verdict"   # --like or --dislike
  nora feedback submit "$trace" \
    $flag \
    --comment "$comment" \
    --external-user "$corrector"
done < feedback-batch.csv

Great for importing QA review results from an external system.

Attach a correction from a diff

If your QA process produces a proposed correct output, submit it as --correction:

nora feedback submit tr_abc \
  --dislike \
  --comment "See correction" \
  --correction '{"expected":"'"$(cat expected.txt | jq -Rs)"'"}'

Correction becomes the expected output when this trace is added to a dataset.

Aggregate feedback across many traces

for tr in $(nora traces list --flow support --status completed --json | jq -r '.[].id'); do
  count=$(nora feedback list "$tr" --json | jq 'length')
  [ "$count" -gt "0" ] && echo "$tr: $count feedback"
done

Wire your app’s thumbs UI

# Get the contract
nora feedback describe support-agent

# Your app posts feedback on button press
# curl -X POST <url> -H "Authorization: ..." -d '{...}'

External --external-user values let you attribute feedback across systems without exposing your user IDs to Nora directly.

What happens after submit

• The feedback row is stored on the trace.
• A Signal is created in the queue (see Signals overview).
• The signal may join an existing cluster if similar patterns exist.
• The Improvement flow may propose a fix (see From signal to fix).

You don’t need to do anything else — feedback rolls into the improvement loop automatically.

Editing / deleting feedback

Feedback is immutable via the CLI. For editing (typo fixes, retracting a wrong thumbs), use the app: the reviewer who submitted can retract within 7 days.

Related

• Feedback on a trace — the app-side concept.
• Signals overview — what happens next.
• nora traces — find traces to attach feedback to.