Skip to content

Validate native RNGD streaming disconnect and concurrent survivor recovery #278

Open

Description

@gspark-etri

Problem

Native RNGD context rejection/recovery is covered by #277, but client disconnect during active generation and concurrent survivor requests remain unverified in the four-model matrix.

Evidence

SDK2026.4 public serving_completions.py installs handle_disconnect and invokes async engine abort. A closed client socket alone does not establish that engine work stopped. Native KONI supports the exact selected model; native registry limitations for the other three models remain recorded in PR275.

Scope

Isolated owned native KONI server, cache off/on, inputs128/8192, concurrency1/4/16. Close one streaming request after actual content is received before its1024-token limit. Capture running/waiting and generation counters, concurrent128-token survivors, and same-server normal recovery. Preserve parity failures and inconclusive cancellation evidence rather than calling every closed socket a successful abort.

Done when

Raw event/counter timing, device/model/FXB/runtime identity, survivor/recovery outputs and cleanup are recorded. Distinguish HTTP disconnect, observed idle drain and evidence of avoided generation. No claim of physical cache reclamation from HTTP alone. Related #201, #277 and PR275.

Owner

Codex supplies isolated measurements and portable client. Production engine fixes belong to the engine maintainer. AI-assisted research.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions