Skip to content

Isolate runtime state and serialize engagement operations - #125

Draft
orsissimo wants to merge 8 commits into
protocol-security:masterfrom
orsissimo:feat/private-runtime
Draft

orsissimo wants to merge 8 commits into
protocol-security:masterfrom
orsissimo:feat/private-runtime

Conversation

@orsissimo

@orsissimo orsissimo commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Step 1 of the split proposed in #115, "Private runtime state, unique identity, and real locking". Based directly on master, with no dependency on driver work.

  • Move state to $XDG_STATE_HOME/swarm-core/<project> (SWARM_RUNTIME_DIR overrides), inside a mode 700 directory; the bare repo stays writable for any container agent UID.
  • Replace sourced swarm.env with mode 600 state.json (swarm-core.state/v1); stop keeps the state file.
  • Record the owning checkout and add swarm-core.project and swarm-core.checkout container labels; refuse collisions between checkouts and accept a separate project id through SWARM_PROJECT.
  • Assign an engagement id when creating the bare repo, generated or supplied through SWARM_ENGAGEMENT_ID, and refuse mismatches in later phases.
  • Lock start, post-process, interactive startup and harvest with flock -n on engagement.lock; fail fast on contention and verify a caller's SWARM_LOCK_FD.
  • Refuse old /tmp/<project>-upstream.git state in start/post-process/interactive/harvest and block commands on unlabeled containers until removed, with recovery instructions.

These are breaking changes for embedders; flock is now required (brew install flock on macOS). Preservation refs, safe replacement, retirement/cleanup and engagement container labels are the next step.

Lifecycle invariants

These invariants cover the whole series: this PR implements 1-5 (cleanup follows in step 2), and later steps implement 6-10; each item names its step.

  1. (step 1) One project id names the runtime directory, image and containers; the state file records the owning checkout, containers carry swarm-core.project and swarm-core.checkout labels, and stop/removal re-checks those labels before acting on container IDs. A second checkout with that id is refused until it sets a distinct SWARM_PROJECT, because an automatic hash suffix would change its id once the first runtime is gone and strand unharvested work.

  2. (step 1) start assigns and saves an engagement id (<UTC timestamp>-<8 hex> unless SWARM_ENGAGEMENT_ID is set); standalone post-process and interactive do the same when they create the bare repo. Later lifecycle phases refuse a supplied SWARM_ENGAGEMENT_ID that differs from the id in state.json.

  3. (step 1; cleanup in step 2) start, post-process, interactive/chat/shell and harvest.sh take engagement.lock with flock -n and fail immediately on contention; a caller holding it passes the descriptor itself through SWARM_LOCK_FD, which the engine verifies is open on the lock file before locking through it, because an inherited "already locked" flag cannot be verified. Interactive releases the engine's copy once the container starts, and start --dashboard before opening the dashboard; stop and read-only commands do not take it, while cleanup will in step 2.

  4. (step 1) The runtime directory must be mode 700, owned by the user and not a symlink, or the engine refuses it. state.json is parsed JSON (swarm-core.state/v1), written with mode 600 and never sourced; the bare repo and mirrors have write permissions for any container agent UID.

  5. (step 1) Legacy /tmp state is never adopted. If the old bare repo is a directory owned by the user and is not a symlink, start, post-process, interactive/chat/shell and harvest.sh refuse to proceed and print commands to stop old containers, fetch their branches and remove the old state.

  6. (step 2) Every destructive path will check the same preservation refs: agent-work, swarm/*, agent-parked/* and any per-run namespace from Multi-Agent Post-Process Proposal #112. A tip the checkout does not contain will block the operation.

  7. (step 2) An existing engine container will make start fail; the engine will never force-replace it or delete state on its own. Removal will require explicit, fail-closed operations for stopped containers and for the whole runtime.

  8. (step 3) The host will resolve a target once to a full commit SHA before containers start, and every phase will use that SHA. A later request for a different commit will fail without re-resolving the target; the target credential will stay on the host.

  9. (step 4) Every initialized submodule, at any depth and including in worktree layouts, will have a host mirror listed in a manifest. Containers will need no network access for submodules.

  10. (step 5) Embedders will depend only on a versioned control interface and container labels. Queries will have no side effects; paths, file formats and container names outside that interface may change.

Changes

  • lib/project.sh: add runtime permission and ownership checks, JSON state helpers, ASCII project ids, engagement checks and descriptor-based locking.
  • launch.sh: use the runtime directory, record engagements, label containers and check ownership before acting on their IDs; build mirror arguments as an array and create transient files with mktemp.
  • dashboard.sh: read state with jq and check checkout and engagement identity before actions; copy container files into a private temporary directory.
  • harvest.sh: check ownership and engagement identity, take the lock and print recovery commands that remove agent-owned files through Docker.
  • costs.sh, progress.sh: use private temporary directories and restrict container reads to the owning checkout.
  • README.md, USAGE.md, CHANGELOG.md: document the runtime contract and upgrade steps, including docker stop -t 60 <names> before fetching work and removing old containers, Alpine removal of agent-owned files, and cleanup by project label.
  • .github/workflows/ci.yml: install flock in the macOS job.
  • tests/test_runtime.sh, tests/test_launch.sh, tests/test_dashboard.sh, tests/test_harvest.sh, tests/test_costs.sh: cover permissions, JSON state, identity collisions, engagement mismatches, locking and temporary-file handling.
  • tests/test.sh: keep the caller's SWARM_PROJECT for smoke runs when another checkout with the same directory name owns the project's containers on the Docker host; document it in the help text and USAGE.md "Testing".
  • tests/_test_env.sh, tests/test.sh, tests/test_ci_workflows.sh, tests/runtime_signal_trap.sh: isolate unit test state from caller settings and update cleanup, dependency and stop-path checks.

The mirror argument, mktemp and ASCII fixes are separate prerequisite commits and can move to another PR if the code owner prefers.

Self-review checklist

  • This PR addresses a single concern (why?). If it covers multiple independent changes, I've split them into separate PRs.

Test plan

  • ./tests/test.sh --unit: 1695 tests in 14 files passed on macOS /bin/bash 3.2.57; 1696 passed on Ubuntu 24.04 Bash 5.2. Both also passed with SWARM_PROJECT and SWARM_ENGAGEMENT_ID exported.
  • ./tests/test.sh --runtime: passed 16/16 on OrbStack and on native Linux (Ubuntu 24.04, host UID 501).
  • Unit and runtime phases pass on every commit; shellcheck -s bash --severity=warning is clean on every changed script of every commit.
  • macOS/OrbStack integration for embedders: two agents ran for 55 minutes with a setup hook, shut down through emergency pushes with the 60-second grace, and produced 28 harvested commits; harvest failed fast during locked post-process.
  • Exercise dashboard h, p, s and start --dashboard through a pseudo-terminal; run and remove an isolated triage runtime.
  • Follow the documented upgrade with old containers running: fetch their commits and remove the old /tmp state.
  • Refuse a second checkout with the same directory name; with separate SWARM_PROJECT values, run both and stop one without stopping the other.
  • Native Linux integration for embedders (Ubuntu 24.04 arm64, Docker 29.1.3, host UID 501, agent UID 1000): two real agents ran for 30 minutes with a setup hook via start --dashboard in a pseudo-terminal; dashboard harvest merged 3 commits mid-run; both agents exited 143 through emergency pushes with the 60-second grace. Post-process held the lock while a harvest attempt failed fast, then its harvest merged the remaining 11 commits and the interactive branch.
  • On that host, launch.sh shell --agent-index 1 ran while the agents were active, with the lock free and container UID 1000, and pushed its branch on exit; start replaced a half-removed bare repo with agent-owned files left behind. Printed recovery commands and documented cleanup remove agent-owned files through Docker.
  • ./tests/test.sh --oauth: passed with a real agent on that host as UID 501 and UID 1000, the second with SWARM_PROJECT while the first checkout's exited container was still there.
  • Alpine Docker-in-Docker with BusyBox flock: start, nested agent push, harvest and export.
  • Phase 2 of ./tests/test.sh --all.

@orsissimo
orsissimo force-pushed the feat/private-runtime branch from ebd572b to 1f77a83 Compare October 31, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant