Skip to content

Local membership status misreports a site with no peers or endpoints #485

Open

Description

@hexfusion

Problem

Under Local membership two status results misreport the state of a site that has no peers and no endpoints yet. Observed on main (c225465) with the operator installed in its defaults (Swim fallback, its own record creation on) and a complete GridNetwork (gridId, Local, poll, SPIFFE, identity and CA refs) applied after it started.

  1. The site's own GridSite stays Pending with reason AwaitingDiscovery and the message "waiting for SWIM discovery to advance to Discovered". Nothing is discovered under Local. What the site is waiting for is its gateway and signals endpoints. The reason comes from the GridSitePhase::Pending fallback in site_phase_under (crates/operator/src/controller/grid_site.rs).
  2. With no peer GridSite declared, PeerSignalsReachable, PeerCatalogsAccepted, PeerPollsComplete, PeerMembership and PeerTimeValid on the GridNetwork all report True. No peer was polled, so the status reads as a healthy mesh that does not exist.

The GridNetwork itself is correct: Pending with GridIdUnset before the network is delivered, then one restart into Local, Active, and Accepted, ResolvedRefs, IdentityValid and Available True.

Scope

  • Under Local, a declared GridSite that cannot leave Pending names the missing endpoint: GatewayAddressMissing when spec.gatewayEndpoint.address is unset, and SignalsAddressMissing when a gateway address is set but no dialable signals address can be derived. AwaitingDiscovery stays for Swim only.
  • With zero declared peers, each peer condition reports Unknown with reason NoDeclaredPeers. It must not be True, and must not be False, since a False peer condition sets Degraded and a single site waiting for its first peer is not degraded.
  • Peer conditions leave Unknown once a peer GridSite is declared and polled.

Acceptance criteria

  • A Local site's own GridSite without endpoints reports GatewayAddressMissing; with a non-dialable gateway address and no signalsEndpoint it reports SignalsAddressMissing; with both declared it moves to Connecting with EndpointsDeclared as today.
  • A Swim site without discovery still reports AwaitingDiscovery.
  • A Local GridNetwork with no declared peers reports the five peer conditions Unknown / NoDeclaredPeers, and is still Active and Available, not Degraded.
  • Declaring a reachable peer moves the peer conditions to their existing True reasons.
  • A kind e2e leg covers a delivered network landing on a running operator and asserts the status before delivery, after delivery, and after a peer is declared.
  • The GridSite reason table and the GridNetwork peer condition tables in docs/architecture/crds.md list the new reasons.

Activity

  1. added
    area/coreCore config, validation, and runtime
    bugInvestigate as a Bug
    on Oct 29, 2026
  2. added this to the v0.3.0 milestone on Oct 29, 2026
  3. hexfusion commented on Oct 30, 2026

    @hexfusion
    CollaboratorAuthor

    Both results reproduce on main 79d3490 on a real cluster: a single-node OpenShift site installed through ACM, with Local membership and poll signals. It has no peers yet, and the provider gateway isn't deployed.

    1. The GridSite site-e has neither spec.gatewayEndpoint nor spec.signalsEndpoint. It stays Pending with Available=False AwaitingDiscovery and the message "waiting for SWIM discovery to advance to Discovered". Under the proposed scope it would report GatewayAddressMissing.
    2. With no peer GridSite declared, the GridNetwork reports PeerPollsComplete, PeerSignalsReachable, PeerCatalogsAccepted, PeerTimeValid and PeerMembership all True, and Available=True.

    On ACM there is a second consequence. The grid-local-endpoints policy that fills in the gateway address waits for the provider gateway's Route. Until that Route exists, the misleading SWIM message is the only status a user sees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions