Moda never merges a fix and never pushes to your default branch. Delivery is either a draft pull request (or a branch you apply yourself) that a human reviews and merges, or — for changes that live outside a repo, like tool descriptions and skills — a candidate you apply in your own stack and confirm with Mark applied. No GitHub integration is required for that second path.
Draft a fix
Draft from Improve → Fixes. Drafting kicks off a pipeline whose progress shows on the Fixes queue: Drafting…, Fix proposed — review, Verifying…, then Fix ready or Gate failed.1
Scope
Moda reads the problem’s verified evidence anchors and builds a replay set from them, split deterministically into repair cases (used to propose the change) and a holdout (reserved for the verdict). Problems with too little usable evidence rest as diagnosis-only fixes rather than guessing.
2
Route and propose
Routing is typed by where the evidence points. Evidence that concentrates on one managed prompt version targets that prompt, and Moda proposes a candidate revision from the repair cases’ failure evidence. Failures that concentrate on one tool target that tool’s description: Moda proposes a rewrite grounded in the failing calls’ judge rationale and receipt error payloads (using the registered tool definition when one exists, or reconstructing the current description from receipts otherwise). Agent-behavior problems — and evidence with no dominant prompt or tool — draft a skill: a complete SKILL.md seeded from the problem’s cause and repair evidence. Anything else still produces an investigation packet — those fixes are deliverable as diagnosis, not dead ends.
3
Verify
Two replay comparisons run on the frozen set: the gate (baseline vs. candidate) and a control (baseline vs. baseline) that measures replay noise. The verdict reads only the holdout.
4
Deliver
A verified prompt fix can open a draft PR, be applied on a local branch, or be handed to a coding agent as a packet; merging the PR starts confirmation monitoring. Tool-description and skill fixes are applied in your own stack from the packet and confirmed with Mark applied, which starts the same monitoring.
Fix types
When your workspace has skills synced from your repo, a skill fix checks that library before drafting anything new: the proposer sees the existing skills’ names and descriptions, and when the failure pattern belongs in one of them it proposes a minimal edit to that skill — a current-vs-proposed SKILL.md with a change summary, preserving all unrelated content verbatim — instead of a near-duplicate net-new skill. Skills that were never synced but show up in production receipts are edit candidates too: their current body is recovered from the receipts, and the resulting verdict says so (see Verdict fidelity). Only when no existing skill matches (or the edit attempt is unusable) does the fix draft a net-new SKILL.md.
The Fixes page
Improve → Fixes lists every fix, ranked by the underlying problem’s rank. Each row shows the fix type chip, a one-sentence rationale, and a gate badge — plus inline Open PR and Dismiss actions where they apply. The gate badge only appears when the holdout has at least 5 cases; below that the row shows Insufficient evidence instead of a verdict. Dismissing requires a reason, which is recorded as problem feedback and steers discovery.Campaign mode
One fix at a time is the right cadence for review — not for catching up. Draft fixes for top problems on the Fixes page batch-drafts a fix for each of your top-ranked fixable problems in one click (the same endpoint is available over HTTP and asmoda fixes draft-batch). Problems are taken in rank order; each is skipped — with a reason — when it already has an active fix, its lifecycle is no longer open, or it turns out to be an alias of a problem already in the batch. When the batch returns, the page shows how many fixes were drafted and skipped and refreshes the queue.
Batches cap at 25 fixes per run (default 10). The cap is an LLM spend guard, not a paging nicety: every drafted fix burns real budget once driven — evidence distillation at scoping, candidate generation at propose — so campaigns are deliberately bounded. Run another batch after the first lands.
Drafting is creation-only: nothing advances server-side. Bulk-advance the batch from the terminal with moda fixes drive, which round-robins one advance step per active fix per pass until every fix rests — verified, diagnosis-only, gate-failed, or resting at Proposed with the gate pending — then reports each fix’s landing. (Opening an individual problem’s Fix tab still drives that one fix through the same polling loop.)
The Fix tab
A problem with a fix opens a Fix stepper from Draft fix: five cards, one per pipeline stage.Lifecycle
The dashboard renders stage chips; each chip covers one or more underlying statuses (the raw status is always visible on the fix detail and in the API).Verification you can trust
The gate is built so a green badge means something:- The holdout is the only headline. Repair cases shape the candidate, so their scores are shown as training cases, never as the verdict. The holdout never reaches the proposer.
- Wins must clear the noise floor. A control run replays baseline against baseline on the same cases; any disagreement there is pure replay noise. The candidate passes only when its holdout wins minus losses exceed
max(2, noise floor + 1)and losses stay within the noise floor. - Abstentions are counted, not hidden. Holdout cases that cannot be replayed faithfully are excluded from scoring and listed as abstained. More than 30% abstained makes the verdict inconclusive — never a pass.
- No badge below minimum evidence. A holdout under 5 cases shows Insufficient evidence instead of a verdict.
- Verification results stay visible in the Fix tab, including the holdout score and comparison verdict.
- Artifact gates isolate the artifact. For tool-description and skill fixes, both arms replay the same pinned prompt — the proposed description (or the candidate skill body, registered content-addressed with the skill harness at gate time) is the only difference between arms, so the verdict measures the artifact and nothing else. Skill edits gate current-vs-proposed; the control run’s arms are byte-identical, so it stays a true noise floor.
- Verdicts say what the baseline was. Every verdict carries a fidelity label — see below.
Verdict fidelity
Not every workspace has a managed prompt to pin, and not every amended skill was synced from a repo — the gate runs anyway and labels the verdict with what the baseline arm was made of (gateResult.baselineFidelity):
Fidelity is always shown next to Verified. A generic-prompt or recovered-skill verdict never renders as a plain Verified badge — the qualifier rides the badge on the Fix tab, the Fixes inbox, the CLI summaries, and the packet, so a green check never overstates its baseline.
The counterfactual gate
The comparison gate needs replayable scenario cases; the counterfactual gate proves a fix against the recorded production failures themselves, with zero tenant action. For each anchor — a failure-anchored user turn from a holdout trace the proposer never saw — Moda rebuilds the recorded event stream up to that turn and continues the trace twice: a control arm without the fix and a candidate arm with it. A recurrence judge then asks one question per continuation: did the anchored failure mechanism recur? The verdict reads: prevented N of M replayed production failures — where M counts the informative anchors, those whose control arm still reproduced the failure. A win requires the control arm to recur; regressions (the candidate re-introducing a failure the control avoided) and abstained anchors are counted next to it, never hidden. No informative anchors means inconclusive, never a pass. The Fix tab’s Verification card and the Fixes inbox render this line alongside the comparison verdict, never instead of it, and the fidelity label stays mandatory next to any Verified badge exactly as before. When it upgrades a verdict. A counterfactual pass can lift a fix from Gate failed or Gate inconclusive to Fix ready (VERIFIED) — but only when the comparison gate did not fail with real losses beyond its noise floor. When both gates land, the overall verdict is a pass if either gate passes and neither shows losses above noise; otherwise the stricter verdict stands. The verdict reason always names which gate produced it.
Read the counterfactual line with its three limits in mind:
- Harness continuation agent. The continuations are generated by Moda’s replay harness agent under the same pinned prompt policy as the comparison gate — not by your production agent binary. “Prevented” is evidence about the failure mechanism under a faithful reconstruction, not a bit-exact rerun of your stack.
- Prevention, not regression safety. The gate scores whether the anchored failure recurs. The regressions it counts are re-introductions of that same mechanism where the control arm stayed clean — it does not prove the fix breaks nothing else.
- LLM judge. Recurrence is decided by an LLM judge (three samples, majority vote). Unparseable samples abstain rather than vote, and continuations that never encounter the mechanism count as uninformative rather than recurred — but judge error remains possible, which is why wins must clear the anchor margin instead of counting one-by-one.
Delivery channels
Confirmation and reopening
Merging a PR whose body or branch carriesMODA-FIX-<SHORTREF> confirms the fix — whether Moda or a human authored the PR. The fix moves to Monitoring, and Moda records a mark_fixed on the problem with structured fix provenance (PR, merge commit, shipped prompt version). While the problem sits in fixed-monitoring the fix shows as confirming; if the problem reopens, the fix flips to Regressed. After 14 clean monitoring days it becomes Held.
Mark applied is the same confirmation without a PR: it moves the fix to Monitoring and records the same mark_fixed provenance, with your note in place of a PR and merge commit. The 14-day clock runs from the moment you marked it applied, and a reopen flips the fix to Regressed exactly as it would after a merge. Marking an already-confirmed fix again is a harmless no-op.
Throttles
- At most 3 open Moda fix PRs per workspace — merge or close one before submitting another.
- One open fix per problem and per target file. Drafting again while a fix is active returns the existing fix, and submitting a fix supersedes older drafts for the same problem or file. An already-open sibling PR blocks a second submission.
- Dismissal always requires a reason, which feeds problem feedback.
Next steps
- Problems — the root-cause groups fixes are drafted from.
- Fixes in the Data API — the same objects and pipeline over HTTP.
- CLI reference: Fixes — drive the loop from the terminal, including the fail-to-pass
moda fix verifyloop for coding agents.