[elezar]

Summary

Run driver conformance through the portable Cargo test suites instead of building and distributing the standalone conformance CLI. Remove the obsolete CLI crate and its unused library APIs now that all supported paths use the library-backed test suites.

Rebased onto main ca187123d on 2026-10-12.

Related Issue

Follow-up cleanup for #2792 and #3864.

Changes

• Run driver conformance from tests/suites/conformance with Cargo.
• Remove standalone conformance-CLI build and setup steps from branch, driver E2E, binary-build, and release workflows.
• Delete crates/openshell-conformance-cli and its workspace lockfile entry.
• Remove the unused scenario registry, aggregate file-transfer wrapper, scenario descriptions, and CLI-only runner methods from the conformance library.
• Resolve relative OPENSHELL_BIN paths before Cargo changes the test working directory.
• Update test, build, policy-advisor, and test-guest documentation for Cargo tests and installed nextest archives.
• Rename the Nix archive binding to reflect that it packages conformance tests rather than a CLI.

Testing

Rebase verification on 2026-10-12:

• nix develop --command cargo test --locked --package openshell-conformance passes: 18 tests.
• All four portable conformance targets compile with cargo test --locked --manifest-path tests/suites/conformance/Cargo.toml --package openshell-test-conformance-cli --no-run.
• Range comparison confirms all three commit patches are unchanged by the rebase.

Earlier verification, before this rebase:

• mise run --jobs 1 pre-commit passes
• cargo test --locked --package openshell-conformance passes (18 tests)
• cargo test --locked --manifest-path tests/suites/conformance/Cargo.toml --package openshell-test-conformance-cli --no-run passes
• Relative binary path remains executable after changing to the conformance package directory
• nix-instantiate --parse tests/artifacts.nix passes
• Agent infrastructure consistency and public skill discovery checks pass
• Live driver E2E jobs were not run locally; the affected GitHub workflows provide that coverage

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-3866.docs.buildwithfern.com/openshell

[krishicks]

Analysis by agent found some cleanup that's worth addressing:

Once the CLI crate is removed, these openshell-conformance items have no remaining callers:

• SCENARIOS, scenarios() and scenario(name)
• FILE_TRANSFER_SCENARIO and run_file_transfer
• the Scenario::description field
• OpenShellRunner::new
• OpenShellRunner::scenario(), which only its own unit test calls

Without the registry's consumer, a scenario added to SCENARIOS does not run unless a matching test file is also added.

[SDAChess]

Two comments still refer to the removed standalone runner:

• e2e/rust/e2e-docker.sh:5: "Run standalone CLI conformance, and optionally a focused Rust e2e test…"
• e2e/rust/e2e-kubernetes.sh:68: "note: skipping standalone CLI conformance in Kubernetes operator workspace mode…"

Could you drop "standalone" from both, since conformance now runs as the Cargo test suite?

#3866 (comment)

[github-actions]

Label test:e2e applied for 702405c. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway, sandbox, and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.