Skip to content

feat(supervisor): route approved egress through appliance VPN - #13

Draft
craig-kindo wants to merge 8 commits into
kindo/advisor-ca-squashedfrom
kindo/vpn-stacked
Draft

feat(supervisor): route approved egress through appliance VPN#13
craig-kindo wants to merge 8 commits into
kindo/advisor-ca-squashedfrom
kindo/vpn-stacked

Conversation

@craig-kindo

@craig-kindo craig-kindo commented Oct 8, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

Add supervisor-terminated, per-sandbox WireGuard tunnels for policy-approved destinations. The tunnel uses userspace tokio_wireguard without a kernel module, and the supervisor remains the policy point for inner traffic.

Stacked on #12 — merge that first; this PR's diff is only the VPN work. Companion mono PR #14658 supplies the via: 'vpn' destination and appliance configuration flow on top of mono #14611.

Related Issue

No accepted issue was provided for this directly requested, multi-repository feature. Repository convention requires an accepted issue for a feature or architecture PR; link one before merge.

Changes

  • Add a per-sandbox TunnelManager and SandboxPolicy.tunnels map with appliance host and UDP port, peer public key, private_key_env_key reference, local address, allowed inner CIDRs, MTU, and keepalive. NetworkEndpoint.tunnel_id selects the tunnel at the upstream dial.
  • Dial the tunnel lazily only after the inner destination passes policy. Missing tunnel configuration, an unreachable appliance, or a failed handshake denies the destination without direct-egress fallback. Inner traffic continues through the existing policy, L7, and credential-injection path.
  • Include the vpn-appliance test example and Linux sidecar cfg-gate fix.
  • Preserve the base PR's single policy-denial builder, pinned CA tests, and public validate_upstream_ca_pem through merge-conflict resolution. The companion mono branch has a post-merge fix that restores its stale_review error kind.

Known state: tunnel reload state and task-local manager selection carry audit-flagged simplification debt to resolve before this draft leaves draft. Key rotation and inner DNS are future phases.

Testing

  • mise run pre-commit passes — not independently rerun during this PR submission; commits are unchanged.
  • Unit tests added/updated — not independently assessed during this PR submission.
  • E2E tests added/updated — the live scenario is recorded in the QA evidence comment.

cargo check -p openshell-supervisor-network -p openshell-server passed on this stacked branch. The companion mono and fork changes passed scenario 5 end to end on digest-verified stacked images; see the doink-qa-evidence comment and local k3d-verify/case-vpn5/STATUS.md report.

Checklist

  • Follows Conventional Commits — the branch includes merge and cherry-pick commits; not independently checked during submission.
  • Commits are signed off (DCO) — not independently checked during submission.
  • Architecture docs updated (if applicable) — not independently assessed during submission.

@craig-kindo
feat(supervisor): trust supplemental upstream CAs per endpoint
a0fb8f1 
Signed-off-by: craig <craig@kindo.ai>
@craig-kindo
feat(supervisor): route policy-approved egress through appliance tunnel
2db194b 
Signed-off-by: craig <craig@kindo.ai>
@craig-kindo
Merge kindo/supervisor-vpn-appliance into kindo/vpn-stacked
37e7b23 
Advisor-ca-squashed review cuts preserved (review_token binding, single
denial builder, presentation-only validation, endpoint CA contracts incl.
pub validate_upstream_ca_pem and the CA-survival tests); VPN appliance
code brought in whole (tunnel_id plumbing, tunnel policy validation,
TunnelRouteConflict merge error, tokio-wireguard vendor, tunnel.rs).
@craig-kindo
fix(sandbox): integrate advisor policy with VPN tunnels
5f8c861 
Signed-off-by: craig <craig@kindo.ai>
(cherry picked from commit 4447bcd89e25787c8d3cf78eceb75677aa3467cc)
@craig-kindo
fix(sandbox): thread tunnel into sidecar entrypoint handler for Linux…
b18a263 
… builds

The VPN integration added tunnel reconciliation to the sidecar entrypoint
handler but referenced `networking`, a binding that only exists in the
outer run_sandbox scope. The handler region is #[cfg(target_os = "linux")],
so macOS-only scoped checks never compiled it and every Linux-target build
of openshell-sandbox failed with E0425 (cannot find value `networking`).
Carry the tunnel manager through SidecarEntrypointHandler instead, populated
from the same RunContext tunnel the policy poll loop uses.

(cherry picked from commit d6719b35c0a223460cfa642094b4407ca1144d30)
@craig-kindo
fix(sandbox): parse IpNet for appliance peer allowed_ips in example
5fd75bb 
The vpn-appliance example built Peer.allowed_ips as Strings via format!
but tokio_wireguard::config::Peer expects Vec<IpNet>, failing the Linux
example build with E0308. Parse the formatted /32 like the file's own
interface address pattern. Verified: cargo check -p openshell-supervisor-network --example vpn-appliance.

(cherry picked from commit 7785c867e86889448f1c5a145fda04c994156027)
@craig-kindo
fix(supervisor-network): address a fleet replica for policy.local gat…
0376edb 
…eway RPCs

policy.local's proposal submit and lookup/wait paths dialed the
configured OPENSHELL_ENDPOINT directly. In a Kubernetes fleet deployment
that endpoint is the headless Service — a discovery address the chart
deliberately omits from the gateway's serving certificate (see the
sandbox chart's openshellServerExtraSans: the per-pod wildcard only), so
the dial fails TLS hostname verification and every wait/lookup returns
gateway_connect_failed. The denial flush hit the same wall and already
addressed a replica via gateway_fleet::unary_endpoint; these two paths
are the outliers.

Lift unary_endpoint (plus endpoint_shape/member_endpoint) into
openshell-core's gateway_fleet so the supervisor-network crate can reach
it without a dependency inversion, delegate the supervisor-process copy
to it, and route both policy.local connect sites through a new
connect_gateway_for_rpc helper with bounded replica rotation.
Single-endpoint deployments are unchanged: unary_endpoint returns the
configured endpoint when no fleet is configured.

(cherry picked from commit 70c88b5d9c3fa4d0f1897e5f06210bfc4b09a4de)
@craig-kindo
fix(driver-kubernetes): resolve claim-bound sandboxes for stop/start
6435c3d 
Stop/start resolved the Sandbox CR by the gateway id-label selector, but
in claim mode the bound Sandbox is pool-generated and carries no gateway
labels — the id label lives on the SandboxClaim, and the claim's
status.sandbox.name names the bound CR. StopSandbox on a claimed sandbox
therefore always failed with 'stop sandbox failed: sandbox not found'.

get_sandbox and delete_sandbox already resolve claims by the same
selector; stop/start now fall back to the same resolution: find the
claim by id label, read status.sandbox.name, and patch the operating
state on the bound CR. Direct-mode sandboxes keep the existing
label-lookup path.

(cherry picked from commit a01b0bb7ce9f77e3137f2c04fd18e356e70b0edd)
@craig-kindo

craig-kindo commented Oct 8, 2026

Copy link
Copy Markdown
Author

E2E Evidence

Cluster: k3d-integration-advisor-vpn (namespace sandbox)
Fork commit: 6435c3dca9dc7b3cab276155b5d321f1a2ef100d
Companion mono commit: d2cc0e682f5ba267c110e1e9e6d5c5fadeac7c01
Verification date: 2026-10-07
Images: freshly built stacked images, digest-verified in the running pods
Artifacts: full local report k3d-verify/case-vpn5/STATUS.md in the mono checkout; round-27 scenario-5-stacked passed

Scenario 5

  • S5.3 ✅ Private inner address 10.80.0.2:8080 became an advisor HITL proposal flagged internal/private.
  • S5.4 ✅ The companion destination flow returned 200 with the WireGuard payload and a per-sandbox supervisorPublicKey.
  • S5.5 ✅ Appliance reported peer_added:true with one peer.
  • S5.6 ✅ The fork's supervisor tunnel carried inner traffic and returned vpn-inner-ok on attempt 1.

The full S5.0–S5.7 scenario ended PASS at 15:29 local time. Fork-side cargo check -p openshell-supervisor-network -p openshell-server also passed on this stacked branch.

@github-actions

github-actions Bot commented Oct 22, 2026

Copy link
Copy Markdown

This pull request has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

state:stale

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@craig-kindo