Multi-Agent Post-Process Proposal #112

Summary

This note proposes a concrete claude-swarm feature: support multiple
independent post-process agents, followed by an optional judge agent.

The core requirement is ref isolation. Multiple post-process agents must not
push to the same agent-work ref.

Concrete Example

We ran a swarm against the latest stable repository pin, e.g.:

120c3c6dac9df8ee4d83f055919bd3488abae4f6

The scan produced eight awaiting-validation tasks. We first ran a GPT post
triage. That run completed and produced a summarized result at, e.g.:

b84490a791fb1d4edee3ea674e8fea6cab86e53a

Then we wanted to rerun post-process from the same pre-triage state using a
different model, Claude Opus, so we could compare independent triage results.
The intended base was:

6fa1fea3bf11406aece19e57c1837f198bb7461e

However, claude-swarm reused:

/tmp/lighthouse-swarm-upstream.git
refs/heads/agent-work

That temp upstream still pointed at the already-triaged GPT result, e.g.:

b84490a791fb1d4edee3ea674e8fea6cab86e53a

As a result, the Opus post-process clone started from the GPT post-triage
output, saw the queue already drained and summarized, and exited without doing
the intended independent triage.

We fixed that run manually by:

1. Creating a fresh branch from the pre-triage base 6fa1fea3.
2. Resetting /tmp/some-repo-swarm-upstream.git agent-work to 6fa1fea3.
3. Removing the stale post-process container.
4. Relaunching SWARM_CONFIG=swarm.json ./swarm/launch.sh post-process.

That proved the deeper issue: the current single agent-work ref conflates
post-process input, output, and run identity. This is fragile for reruns and
would be unsafe for multiple post-process agents.

Motivation

Problem

Today, standalone post-process is effectively single-writer:

/tmp/<project>-upstream.git
refs/heads/agent-work

That is acceptable for one post agent. It is not sufficient for multiple post
agents, especially in parallel.

If two post agents both start from and push to agent-work, several bad states
are possible:

• One agent starts from the other agent's partial or completed triage.
• One agent drains the queue before the other can inspect the original tasks.
• Concurrent pushes race through rebase and retry logic.
• A failed or stale post run can make a later run appear complete.
• The dashboard cannot distinguish "agent did real triage" from "agent saw
  already-processed state".

The bug we hit was the sequential version of the same design problem.

Proposed Ref Model

Keep one temp bare repo if desired, but allocate a per-run ref namespace:

refs/swarm-runs/<run-id>/base
refs/swarm-runs/<run-id>/post/agent-1
refs/swarm-runs/<run-id>/post/agent-2
refs/swarm-runs/<run-id>/judge
refs/heads/agent-work

Rules:

• base is immutable after the post-process phase starts.
• Each post agent starts from base.
• Each post agent pushes only to its own post/<agent> ref.
• No post agent writes to agent-work.
• The judge starts from base.
• The judge writes the final reconciled result to judge.
• Only the launcher promotes judge to agent-work.

Git can safely accept concurrent pushes to distinct refs because each ref has
its own lock. The unsafe part is concurrent writers to the same ref.

Execution Flow

For a two-agent post-process:

1. The launcher records the harvested scan tip as:

   refs/swarm-runs/<run-id>/base
   

2. The launcher starts post agents from that immutable base:

   refs/swarm-runs/<run-id>/post/agent-1
   refs/swarm-runs/<run-id>/post/agent-2
   

3. Each post agent performs full triage independently.

4. The launcher waits for both post agents to exit.

5. The launcher generates a comparison brief, for example:

   Base: refs/swarm-runs/<run-id>/base
   
   Agent outputs:
   - agent-1: refs/swarm-runs/<run-id>/post/agent-1
   - agent-2: refs/swarm-runs/<run-id>/post/agent-2
   
   Diffs:
   - git diff base..post/agent-1
   - git diff base..post/agent-2
   
   Queue decisions:
   - accepted findings by agent
   - rejected findings by agent
   - severity choices by agent
   - unresolved or conflicting task states
   

6. The judge starts from base, reads the comparison brief, inspects both
   refs, and writes the reconciled final result to:

   refs/swarm-runs/<run-id>/judge
   

7. The launcher promotes the judge result:

   refs/swarm-runs/<run-id>/judge -> refs/heads/agent-work
   

8. The existing harvest path can then merge agent-work as it does today.

Judge Behavior

The judge should not blindly merge both branches. It should reconcile them.

For audit triage, the judge should:

• Preserve findings both agents accepted.
• Inspect disagreements before accepting or rejecting.
• Deduplicate overlapping findings.
• Resolve severity differences.
• Preserve rejection rationale for false positives.
• Produce a final summary that explains material disagreements.

The judge should be able to read each post-agent ref directly and should also
receive the launcher-generated comparison brief.

Config Sketch

One backwards-compatible shape:

{
  "post_process": {
    "mode": "multi",
    "agents": [
      {
        "name": "triage-opus",
        "driver": "claude-code",
        "model": "claude-opus-4-8",
        "effort": "max",
        "auth": "oauth",
        "prompt": "tasks/post-triage.md",
        "setup": "tasks/post-setup.sh"
      },
      {
        "name": "triage-gpt",
        "driver": "codex-cli",
        "model": "gpt-5.5",
        "effort": "high",
        "auth": "chatgpt",
        "prompt": "tasks/post-triage.md",
        "setup": "tasks/post-setup.sh"
      }
    ],
    "judge": {
      "name": "judge",
      "driver": "claude-code",
      "model": "claude-opus-4-8",
      "effort": "max",
      "auth": "oauth",
      "prompt": "tasks/post-judge.md",
      "setup": "tasks/post-setup.sh"
    }
  }
}

The existing single-object post_process config should continue to behave as
it does today.

Dashboard Requirements

The dashboard should treat post-process as a phase with multiple workers:

post triage-opus   running
post triage-gpt    exited 0
post judge         pending

Useful controls:

• Show logs for each post agent.
• Stop all post agents for the active run.
• Rerun one failed post agent from base.
• Rerun only the judge after editing the judge prompt.
• Harvest only the judged result.

Cleanup And Stale-State Requirements

The launcher should never implicitly reuse a stale agent-work as the base for
an independent post-process comparison.

It should instead:

• Generate a unique run-id for each post-process run.
• Store the explicit post-process base ref.
• Refuse to start if a run namespace already exists, unless --resume or
  --replace is provided.
• Keep isolated post refs long enough for auditability.
• Provide an explicit cleanup command for old run refs.

This would have prevented the 2026-06-24 Lighthouse rerun issue, because the
Opus post-process would have started from the recorded base 6fa1fea3, not
from the GPT result at b84490a7.