Skip to content

[Enhancement]: Let a caller retry a runtime start without starting a second runtime #487

Open

Description

@cv

Affected area

Python SDK or binding, Consumer integration

Problem or opportunity

Since #393, the task runtime server starts another runtime for every successful fabric-agent-ctl start; only an uncertain start or a runtime-ID collision blocks later starts. A caller therefore can't safely retry a start whose reply it didn't receive.

NemoClaw runs fabric-agent-ctl start through OpenShell's sandbox exec. If that exec's reply is lost after the server records the runtime, NemoClaw's retry starts a second runtime. The first keeps running under an ID NemoClaw never received, and no command lists a sandbox's runtimes, so NemoClaw can't stop it or detect it.

With v0.5.0, the server refused every start after the first with runtime_exists, so a retry could never create a duplicate. NemoClaw relies on that refusal today.

Proposed enhancement

Either of these would let a caller retry safely:

  • An idempotency key for start: the caller supplies a key, such as a request ID, and a start repeating a key that already started a runtime returns that runtime's handle instead of starting another.
  • A read-only runtime listing: fabric-agent-ctl reports the runtimes the server owns, with their IDs and agent names, so a caller can find a runtime it started before retrying. This overlaps fabric-agent-ctl: read-only status command for the owned runtime #420.

Public contract and compatibility impact

Both are additive. The idempotency key is an optional start input; without it, start behaves as it does now. A listing command adds output but changes no existing command.

Alternatives considered

  • NemoClaw could refuse any start in a sandbox where an earlier start's outcome is unknown, but it records that only in its own state, and replacing the sandbox, the normal recovery, removes the agent first. Its guard would then block that recovery or miss the retry.
  • NemoClaw could write a marker into the sandbox before each start, but it keeps sandboxes free of files it would have to manage, and the marker can't tell whether Fabric started a runtime.

Acceptance criteria

  • A caller can retry fabric-agent-ctl start after a lost reply without leaving a second runtime running, or can find the runtime an earlier start created.
  • The fabric-agent-ctl documentation describes how to retry a start safely.

Additional context

Found while updating NemoClaw to Fabric main at 67acf1cf (NVIDIA/NemoClaw#13698), where NemoClaw's fake runtime server still modeled v0.5.0's single runtime per sandbox.

Duplicate check

  • I searched the existing issues and did not find a duplicate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions