Living Harness CI

Overview

moda init maps your repo’s agents, prompts, and tools into a cited harness graph and syncs it to Moda. That map is a snapshot: as your team ships new agents, renames tools, or rewires prompts, the synced graph drifts from the code. The living harness workflow closes that gap. A small GitHub Actions workflow re-runs the hosted harness analysis on every merge to main, approves the refreshed report, and syncs the updated graph to Moda — so the harness Moda serves always matches what is deployed. Each rescan runs three CLI commands:

moda harness analyze --remote --yes   # re-analyze on Moda's servers (safe source snapshot upload)
moda harness approve --yes            # approve the refreshed report non-interactively
moda harness sync --from-report       # sync the cited graph to Moda

Analysis runs server-side on Moda’s infrastructure under a Moda-held LLM key — the workflow needs no LLM credentials of its own, only your Moda API key.

Setup

1. Add the repository secrets

In your repo, add these GitHub Actions secrets:

MODA_API_KEY (required) — a Moda API key (moda_sk_...) from Settings → API Keys
MODA_TENANT_ID (optional) — pins the workspace when the key spans multiple organizations

2. Generate the workflow

The easiest path is moda init, which offers the workflow as an optional setup-plan item, or the explicit flag:

moda init --harness-rescan

To only rescan when agent-relevant paths change, scope it (this implies --harness-rescan):

moda init --harness-rescan-paths='src/agents/**,prompts/**'

Either form writes .github/workflows/moda-harness-rescan.yml, pinned to the CLI version that generated it.

3. Or drop the workflow in by hand

Copy this into .github/workflows/moda-harness-rescan.yml, replacing <version> with the current @moda-ai/cli release (pin a version rather than @latest so CI runs a release you have reviewed):

.github/workflows/moda-harness-rescan.yml

name: Moda Harness Rescan

on:
  push:
    branches: [main]
    # Optional: scope rescans to agent-relevant paths, e.g.
    # paths:
    #   - 'src/agents/**'
  workflow_dispatch:

# Least privilege: this job reads the checked-out repo, uploads a safe
# source snapshot to Moda over the API key, and syncs the refreshed harness
# graph; it never writes back to the repository.
permissions:
  contents: read

# One in-flight rescan at a time; newer pushes supersede older runs.
concurrency:
  group: moda-harness-rescan
  cancel-in-progress: true

jobs:
  moda-harness-rescan:
    runs-on: ubuntu-latest
    env:
      MODA_API_KEY: ${{ secrets.MODA_API_KEY }}
      MODA_TENANT_ID: ${{ secrets.MODA_TENANT_ID }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - name: Re-analyze the harness on Moda
        run: npx -y --package=@moda-ai/cli@<version> moda harness analyze --remote --yes
      - name: Approve the refreshed harness report
        run: npx -y --package=@moda-ai/cli@<version> moda harness approve --yes
      - name: Sync the harness graph to Moda
        run: npx -y --package=@moda-ai/cli@<version> moda harness sync --from-report

Behavior notes

Merge-to-main only. The workflow never runs on pull_request, so unmerged agent changes are never synced to your workspace. Use the workflow_dispatch trigger for an out-of-band rescan.
Path scoping. Add on.push.paths globs so only pushes touching agent-relevant code trigger a rescan. Pushes that touch none of the listed paths skip the workflow entirely.
Superseding runs. The concurrency group cancels an in-flight rescan when a newer push lands; the latest merge always wins.
What leaves your CI. harness analyze --remote uploads a safe source snapshot to Moda for analysis; the snapshot is ephemeral and deleted after the run. LLM credentials stay on Moda’s side.
Failure isolation. The workflow only talks to Moda; it has read-only repo permissions and cannot push commits, so a failed rescan never blocks or mutates your codebase. Re-run it from the Actions tab or wait for the next merge.

Getting Started

Ingestion

Frameworks

Prompt Management

Harness

Data API

Living Harness CI

Overview

Setup

1. Add the repository secrets

2. Generate the workflow

3. Or drop the workflow in by hand

Behavior notes

​Overview

​Setup

​1. Add the repository secrets

​2. Generate the workflow

​3. Or drop the workflow in by hand

​Behavior notes

Overview

Setup

1. Add the repository secrets

2. Generate the workflow

3. Or drop the workflow in by hand

Behavior notes