Skip to content

Resolve deployment-owned vLLM serving plans for Slurm #866

Closed

Description

@nabinchha

Priority Level

High

Task Summary

Implement the package-owned vLLM resolver that converts one typed deployment declaration plus planner-supplied placement into process, readiness, backend-endpoint, and logical-endpoint records.

Technical Details & Implementation Plan

  • Dispatch on the deployment server discriminator with vLLM as the only implemented backend.
  • Keep model alias, model source, and optional served model name distinct.
  • Resolve tensor parallelism within a node and pipeline parallelism across nodes in a replica group.
  • Derive replica groups, ranks, lanes, rendezvous inputs, and lane-head backend endpoints from planner-owned placement.
  • Carry per-deployment startup and distributed-initialization deadlines, launch standoff/stagger, queue backpressure, readiness configuration, tokenized extra arguments, and typed environment bindings.
  • Reject extra arguments that override compiler/runtime-owned model identity, ports, topology, rendezvous, launch timing, or middleware behavior.
  • Emit typed logical-endpoint inputs so the runtime can aggregate readiness and load-balance healthy backends while preserving overload responses.
  • Preserve the inspected vLLM runtime version as digest-bound provenance without maintaining a package-version compatibility matrix.

Acceptance criteria

  • A single-node deployment resolves deterministically without importing scheduler or shell modules.
  • A multi-node deployment derives the expected replica groups, ranks, pipeline parallelism, and lane-head endpoints.
  • Two deployments may select different serving images without endpoint or resource identity collisions.
  • Invalid node, GPU, tensor-parallel, replica-group, expert-parallel, image-kind, and resolved-behavior combinations fail before submission.
  • Default and overridden launch-timing and queue-backpressure values serialize in golden records.
  • A rank failure is represented as failure of the coordinated deployment; no per-replica restart contract is introduced.

Out of scope

  • Slurm submission or node allocation.
  • Shell rendering, process supervision, and cleanup.
  • Image building or registry mutation.
  • Dynamo, SGLang, serving plugins, and per-replica recovery.

Investigation / Context

This is a feature lane under #850. Portable deployment intent remains separate from backend-specific process resolution so configuration stays declarative.

Agent Plan / Findings

Build against the reviewed shared configuration, image-inspection, placement, and runtime record contracts. Keep the resolver pure with focused single-node and multi-node golden tests.

Dependencies

Blocked by the shared-contract and fake-infrastructure foundation work tracked by #850.

Activity

  1. self-assigned this
    on Aug 15, 2026
  2. nabinchha commented on Sep 29, 2026

    @nabinchha
    ContributorAuthor

    Closing as completed by merged PR #890. That change implements deterministic single-node and multi-node vLLM serving-plan resolution, topology validation, logical endpoints, launch policy, provenance, golden tests, and wheel verification, covering this issue's acceptance criteria.

  3. added 7 commits that reference this issue on Oct 6, 2026
    767a4bf
    5faf662
    92aaea8
    b59ca7c
    15f4a68
    8377f18
    c00aeb2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

taskInternal development task

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions