Skip to main content
A Fix is what closes a problem: a fix with proof. From a problem’s evidence, Moda scopes a frozen replay set, routes the change to a target artifact (usually a managed prompt), proposes a candidate revision, and verifies it on held-out production cases before anything is delivered. Open Improve → Fixes to see the ranked queue, or draft one directly from a problem’s Dossier.
Moda never merges a fix and never pushes to your default branch. Delivery is always a draft pull request (or a branch you apply yourself), and a human reviews and merges it.

​
Draft a fix

The Dossier header’s primary action is Draft fix (a split button; Mark as fixed manually… remains available for changes you shipped outside Moda). Drafting kicks off a pipeline whose progress shows directly on the button: Drafting…, Fix proposed — review, Verifying…, then Fix ready or Gate failed.
1

Scope

Moda reads the problem’s verified evidence anchors and builds a replay set from them, split deterministically into repair cases (used to propose the change) and a holdout (reserved for the verdict). Problems with too little usable evidence rest as diagnosis-only fixes rather than guessing.
2

Route and propose

If the evidence concentrates on one managed prompt version, the fix targets that prompt and Moda proposes a candidate revision from the repair cases’ failure evidence. Evidence that points at a tool schema, or at nothing routable, still produces an investigation packet — those fixes are deliverable as diagnosis, not dead ends.
3

Verify

Two replay comparisons run on the frozen set: the gate (baseline vs. candidate) and a control (baseline vs. baseline) that measures replay noise. The verdict reads only the holdout.
4

Deliver

A verified fix can open a draft PR, be applied on a local branch, or be handed to a coding agent as a packet. Merging the PR starts confirmation monitoring.

​
The Fixes page

Improve → Fixes lists every fix, ranked by the underlying problem’s rank. Each row shows the fix type chip, a one-sentence rationale, and a gate badge — plus inline Open PR and Dismiss actions where they apply. The gate badge only appears when the holdout has at least 5 cases; below that the row shows Insufficient evidence instead of a verdict. Dismissing requires a reason, which is recorded as problem feedback and steers discovery.

​
The Fix tab

A problem with a fix gains a Fix tab beside Dossier and Conversations: a vertical stepper of five cards, one per pipeline stage.

​
Lifecycle

The dashboard renders stage chips; each chip covers one or more underlying statuses (the raw status is always visible on the fix detail and in the API).

​
Verification you can trust

The gate is built so a green badge means something:
  • The holdout is the only headline. Repair cases shape the candidate, so their scores are shown as training cases, never as the verdict. The holdout never reaches the proposer.
  • Wins must clear the noise floor. A control run replays baseline against baseline on the same cases; any disagreement there is pure replay noise. The candidate passes only when its holdout wins minus losses exceed max(2, noise floor + 1) and losses stay within the noise floor.
  • Abstentions are counted, not hidden. Holdout cases that cannot be replayed faithfully are excluded from scoring and listed as abstained. More than 30% abstained makes the verdict inconclusive — never a pass.
  • No badge below minimum evidence. A holdout under 5 cases shows Insufficient evidence instead of a verdict.
  • Every number links to its replay run, so you can read the per-case transcripts behind the scoreboard.

​
Delivery channels

​
Confirmation and reopening

Merging a PR whose body or branch carries MODA-FIX-<SHORTREF> confirms the fix — whether Moda or a human authored the PR. The fix moves to Monitoring, and Moda records a mark_fixed on the problem with structured fix provenance (PR, merge commit, shipped prompt version). While the problem sits in fixed-monitoring the fix shows as confirming; if the problem reopens, the fix flips to Regressed. After 14 clean monitoring days it becomes Held.

​
Throttles

  • At most 3 open Moda fix PRs per workspace — merge or close one before submitting another.
  • One open fix per problem and per target file. Drafting again while a fix is active returns the existing fix, and submitting a fix supersedes older drafts for the same problem or file. An already-open sibling PR blocks a second submission.
  • Dismissal always requires a reason, which feeds problem feedback.

​
Next steps