Skip to content

Keep DCP watches draining and close abandoned connections - #20972

Merged
David Negstad (danegsta) merged 2 commits into
mainfrom
danegsta-dcp-watcher-investigation
Oct 14, 2026
Merged

David Negstad (danegsta) merged 2 commits into
mainfrom
danegsta-dcp-watcher-investigation

Conversation

@danegsta

@danegsta David Negstad (danegsta) commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Description

Aspire can leave a DCP watch open without an active reader when watch setup is canceled or resource handlers pause event consumption. An unread HTTP/2 watch can exhaust its receive window and block DCP watcher publication and resource mutations.

This change explicitly owns watch responses as soon as headers arrive and closes them on cancellation or periodic restart. An independent reader drains each stream into a lossless queue, so handler latency does not stop network reads. Resource event ordering and existing Aspire lifecycle callback dispatch are preserved.

The reader distinguishes successful and failed Status payloads, surfaces malformed complete records, and recovers from truncated streams and retryable watch errors. The existing five-minute reconnection safeguard remains in place; this does not claim to resolve the separate historical connected-but-silent watch behavior. Lossless buffering means a persistently blocked handler can accumulate queued events rather than backpressure the API server.

Related to #20609 and microsoft/dcp#286. These are client-side fixes, not a replacement for the API server's slow-consumer handling. No dependency updates are required.

Validation

Behavioral regressions cover cancellation, paused consumers, event ordering, periodic reconnection, framing, and error recovery. All KubernetesServiceTests now use HTTPS/HTTP2 with one shared, test-owned certificate; certificate trust validation is disabled only in test clients.

Passed on Windows: 98 tests, 0 failures.

dotnet test --project .\tests\Aspire.Hosting.Tests\Aspire.Hosting.Tests.csproj --no-launch-profile -- --filter-class '*.KubernetesServiceTests' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Own watch responses before reading events and close them on cancellation or
periodic restart. Drain streams independently of resource handlers while
preserving event order and existing lifecycle dispatch. Handle failed Status
payloads and retry interrupted or transiently failed watches.

Add behavioral watch regressions over HTTPS/HTTP2 with a shared test
certificate and retain the five-minute reconnection safeguard.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a4d11434-2c0b-4741-91ea-9eacc92ca29a
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 20972

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 20972"

@aspire-repo-bot
aspire-repo-bot Bot requested a balanced review from Copilot October 13, 2026 19:37
@github-actions github-actions Bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Oct 13, 2026
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Retryable HTTP responses and malformed final records are not handled consistently.

2 open findings
What changed in this PR

This PR prevents abandoned or stalled DCP watches from blocking resource mutations.

Changes:

  • Drains watch streams into an ordered, lossless channel.
  • Adds explicit response disposal, framing, and status handling.
  • Adds HTTPS/HTTP2 regression coverage.
File Description
src/​Aspire.Hosting/​Dcp/​KubernetesService.cs Decouples stream draining from consumers.
src/​Aspire.Hosting/​Dcp/​DcpKubernetesClient.cs Implements watch ownership and parsing.
src/​Aspire.Hosting/​Dcp/​DcpPipelineBuilder.cs Expands retry handling.
tests/​Aspire.Hosting.Tests/​Dcp/​KubernetesServiceTests.cs Adds watch and HTTP2 tests.
tests/​Aspire.Hosting.Tests/​Dcp/​DcpApiServerCertificateFixture.cs Provides the test certificate.

🧠 Review effort: Balanced


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Distinguish incomplete JSON from complete malformed final records so missing
newlines do not turn permanent parsing failures into retries. Apply one watch
status policy to HTTP response failures and in-stream Status failures.

Cover both response forms for every status code without duplicating theory
rows, and add malformed final-record and partial JSON recovery scenarios.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a4d11434-2c0b-4741-91ea-9eacc92ca29a
@github-actions

Copy link
Copy Markdown
Contributor

Tests selector

52 / 99 PR test projects · 4 PR jobs, from 5 changed files.

Selected PR test projects (52 / 99)

Aspire.Cli.EndToEnd.Tests, Aspire.Hosting.Analyzers.Tests, Aspire.Hosting.Azure.EntraId.Tests, Aspire.Hosting.Azure.Kubernetes.Tests, Aspire.Hosting.Azure.Kusto.Tests, Aspire.Hosting.Azure.Provisioning.Tests, Aspire.Hosting.Azure.Tests, Aspire.Hosting.Blazor.Tests, Aspire.Hosting.Browsers.Tests, Aspire.Hosting.CodeGeneration.Go.Tests, Aspire.Hosting.CodeGeneration.Java.Tests, Aspire.Hosting.CodeGeneration.Python.Tests, Aspire.Hosting.CodeGeneration.Rust.Tests, Aspire.Hosting.CodeGeneration.TypeScript.Tests, Aspire.Hosting.Containers.Tests, Aspire.Hosting.DevTunnels.Tests, Aspire.Hosting.Docker.Tests, Aspire.Hosting.Dotnet.Tests, Aspire.Hosting.DotnetTool.Tests, Aspire.Hosting.EntityFrameworkCore.Tests, Aspire.Hosting.Foundry.Tests, Aspire.Hosting.Garnet.Tests, Aspire.Hosting.Go.Tests, Aspire.Hosting.Java.Tests, Aspire.Hosting.JavaScript.Tests, Aspire.Hosting.Kafka.Tests, Aspire.Hosting.Keycloak.Tests, Aspire.Hosting.Kubernetes.Tests, Aspire.Hosting.Maui.Tests, Aspire.Hosting.Milvus.Tests, Aspire.Hosting.MongoDB.Tests, Aspire.Hosting.MySql.Tests, Aspire.Hosting.Nats.Tests, Aspire.Hosting.OpenAI.Tests, Aspire.Hosting.Oracle.Tests, Aspire.Hosting.Orleans.Tests, Aspire.Hosting.PostgreSQL.Tests, Aspire.Hosting.Python.Tests, Aspire.Hosting.Qdrant.Tests, Aspire.Hosting.RabbitMQ.Tests, Aspire.Hosting.Radius.Tests, Aspire.Hosting.Redis.Tests, Aspire.Hosting.RemoteHost.Tests, Aspire.Hosting.Rust.Tests, Aspire.Hosting.Seq.Tests, Aspire.Hosting.SqlServer.Tests, Aspire.Hosting.Testing.Tests, Aspire.Hosting.Tests, Aspire.Hosting.Valkey.Tests, Aspire.Hosting.Yarp.Tests, Aspire.Playground.Tests, Aspire.Templates.Tests

Selected PR jobs (4)

cli-starter-validation, extension-e2e, polyglot, typescript-api-compat


How these were chosen — grouped by what changed

⚠️ 46 of the 52 selected test projects come from a single change — src/Aspire.Hosting/Dcp/DcpKubernetesClient.cs.

🔧 src/Aspire.Hosting/Dcp/DcpKubernetesClient.cs (changed source)
→ 46 via the project graph

show 46

Aspire.Hosting.Analyzers.Tests (2 hops), Aspire.Hosting.Azure.EntraId.Tests (2 hops), Aspire.Hosting.Azure.Kubernetes.Tests (2 hops), Aspire.Hosting.Azure.Kusto.Tests (2 hops), Aspire.Hosting.Azure.Provisioning.Tests (3 hops), Aspire.Hosting.Azure.Tests, Aspire.Hosting.Browsers.Tests (2 hops), Aspire.Hosting.CodeGeneration.Go.Tests, Aspire.Hosting.CodeGeneration.Java.Tests, Aspire.Hosting.CodeGeneration.Python.Tests, Aspire.Hosting.CodeGeneration.Rust.Tests, Aspire.Hosting.CodeGeneration.TypeScript.Tests, Aspire.Hosting.Containers.Tests (2 hops), Aspire.Hosting.DevTunnels.Tests (2 hops), Aspire.Hosting.Docker.Tests (2 hops), Aspire.Hosting.DotnetTool.Tests (2 hops), Aspire.Hosting.EntityFrameworkCore.Tests (2 hops), Aspire.Hosting.Foundry.Tests (2 hops), Aspire.Hosting.Garnet.Tests (2 hops), Aspire.Hosting.Go.Tests (2 hops), Aspire.Hosting.Java.Tests (2 hops), Aspire.Hosting.JavaScript.Tests (2 hops), Aspire.Hosting.Kafka.Tests (2 hops), Aspire.Hosting.Keycloak.Tests (2 hops), Aspire.Hosting.Kubernetes.Tests (2 hops), Aspire.Hosting.Maui.Tests, Aspire.Hosting.Milvus.Tests (2 hops), Aspire.Hosting.MongoDB.Tests (2 hops), Aspire.Hosting.MySql.Tests (2 hops), Aspire.Hosting.Nats.Tests (2 hops), Aspire.Hosting.OpenAI.Tests (2 hops), Aspire.Hosting.Oracle.Tests (2 hops), Aspire.Hosting.Orleans.Tests (2 hops), Aspire.Hosting.PostgreSQL.Tests (2 hops), Aspire.Hosting.Python.Tests (2 hops), Aspire.Hosting.Qdrant.Tests (2 hops), Aspire.Hosting.RabbitMQ.Tests (2 hops), Aspire.Hosting.Redis.Tests (2 hops), Aspire.Hosting.RemoteHost.Tests, Aspire.Hosting.Rust.Tests (2 hops), Aspire.Hosting.Seq.Tests (2 hops), Aspire.Hosting.SqlServer.Tests (2 hops), Aspire.Hosting.Testing.Tests (2 hops), Aspire.Hosting.Valkey.Tests (2 hops), Aspire.Hosting.Yarp.Tests (2 hops), Aspire.Playground.Tests

🧪 tests/Aspire.Hosting.Tests/Dcp/DcpApiServerCertificateFixture.cs (changed test)
→ 1 directly: Aspire.Hosting.Tests
→ 3 via the project graph: Aspire.Hosting.Blazor.Tests, Aspire.Hosting.Dotnet.Tests, Aspire.Hosting.Radius.Tests

📦 affected project Aspire.Hosting.Dotnet
→ 2 tests: Aspire.Cli.EndToEnd.Tests, Aspire.Templates.Tests

📦 affected project Aspire.Hosting
→ 1 test: Aspire.Cli.EndToEnd.Tests

🧪 tests/Aspire.Hosting.Tests/Dcp/KubernetesServiceTests.cs (changed test)
→ 1 directly: Aspire.Hosting.Tests

Job reasons

Job Triggered by
cli-starter-validation affected project Aspire.Hosting.PostgreSQL
extension-e2e • src/Aspire.Hosting/Dcp/DcpKubernetesClient.cs, src/Aspire.Hosting/Dcp/DcpPipelineBuilder.cs, src/Aspire.Hosting/Dcp/KubernetesService.cs
• affected project Aspire.Hosting
polyglot • affected project Aspire.Hosting
• affected project Aspire.Hosting.Azure.Provisioning.KeyVault
typescript-api-compat affected project Aspire.Hosting

Selection computed for commit e92b3d4.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The lifecycle, parsing, retry, and ordering changes are consistently implemented and covered by focused tests.

0 open findings

2 resolved since last review

🧠 Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@github-actions

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

@danegsta

David Negstad (danegsta) commented Oct 13, 2026 •

Copy link
Copy Markdown
Member Author

PR Testing Report

PR Information

  • PR: microsoft/aspire#20972
  • Title: Keep DCP watches draining and close abandoned connections
  • Head commit: e92b3d4bc91ec3452203aac5116a0593593c064c
  • Test date: 2026-10-13
  • Execution: Windows source-unit validation plus isolated Linux dogfood scenarios through the repository container runner.

Artifact Version Verification

  • Expected commit: e92b3d4bc91ec3452203aac5116a0593593c064c
  • Installed CLI: 17.0.0-pr.20972.ge92b3d4b
  • CLI path: /workspace/.aspire/dogfood/pr-20972/bin/aspire
  • PR package hive: /workspace/.aspire/hives/pr-20972/packages
  • Loaded Hosting assembly: 17.0.0-pr.20972.ge92b3d4b, verified from actual startup output for all three AppHosts. Each fixture also rejects a loaded Hosting assembly that does not contain the expected commit suffix.
  • Status: Verified.

The install used the PR's dogfood command inside the unmodified repository container runner:

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 20972

The local source checkout matched the PR head and was clean. The testing did not modify repository files or push commits.

Changes Analyzed

Files Changed

  • src/Aspire.Hosting/Dcp/DcpKubernetesClient.cs: explicit HTTP watch ownership, cancellation, record framing, and Status parsing.
  • src/Aspire.Hosting/Dcp/DcpPipelineBuilder.cs: retries for transient HTTP-response and in-stream watch failures.
  • src/Aspire.Hosting/Dcp/KubernetesService.cs: independent draining into an ordered, unbounded queue with periodic connection rotation.
  • tests/Aspire.Hosting.Tests/Dcp/KubernetesServiceTests.cs: behavioral regressions and HTTPS/HTTP2 fake-server coverage.
  • tests/Aspire.Hosting.Tests/Dcp/DcpApiServerCertificateFixture.cs: class-owned test certificate.

Change Categories

  • Hosting core and runtime orchestration
  • Hosting tests
  • CLI implementation
  • Dashboard implementation
  • Templates
  • Client/components or hosting integrations
  • VS Code extension
  • CI infrastructure

Test Scenarios Executed

Scenario 1: Focused hosting regressions

Objective: Validate the actual changed watch paths and the unchanged executor/restart behavior that consumes them.

Coverage type: Unit, HTTP2 integration, unhappy path, and boundary.

Status: Passed: 367 passed, 0 failed, 1 Linux-only test skipped on Windows.

Command:

$env:MSBUILDTERMINALLOGGER = 'false'
dotnet test --project tests\Aspire.Hosting.Tests\Aspire.Hosting.Tests.csproj --no-build --no-launch-profile -- --filter-method '*.WatchAsync_*' --filter-method '*.WatchResourceRetryPipeline_*' --filter-method '*.DcpExecutorTests.*' --filter-method '*.PeriodicRestartAsyncEnumerableTests.*' --filter-not-trait 'quarantined=true' --filter-not-trait 'outerloop=true'

--no-build reused the current-head test assembly built earlier in this session; no source changes occurred afterward.

Coverage includes cancellation before the first event, response closure while a consumer is paused, draining and FIFO preservation, connection rotation, idle streams surviving API deadlines, UTF-8 and large fragmented records, complete final records without newlines, malformed complete records, truncated/empty streams, successful versus failed Status payloads, and retryable versus permanent HTTP/in-stream statuses. Executor coverage exercises resource state handling and restarts.

Expected unhappy-path outcomes: Cancellation closes the HTTP2 stream; recoverable watch failures reconnect; malformed complete records and permanent failures propagate rather than appearing successful.

Evidence: focused-hosting-tests.log.

Scenario 2: PR-artifact starter orchestration

Objective: Exercise the shipping PR Hosting package with real DCP, project executables, service endpoints, health checks, and resource restart.

Coverage type: Happy path and restart.

Status: Passed.

Project generation:

cli=/workspace/.aspire/dogfood/pr-20972/bin/aspire
hive=/workspace/.aspire/hives/pr-20972/packages
version=$("$cli" --version)
"$cli" new aspire-starter --name WatchStarter --output /workspace/WatchStarter --source "$hive" --version "$version" --test-framework None --use-redis-cache false --localhost-tld false --suppress-agent-init --non-interactive

Execution:

apphost=/workspace/WatchStarter/WatchStarter.AppHost/WatchStarter.AppHost.csproj
"$cli" start --apphost "$apphost" --isolated --non-interactive --format Json
"$cli" wait apiservice --status healthy --timeout 120 --apphost "$apphost" --non-interactive
"$cli" wait webfrontend --status healthy --timeout 120 --apphost "$apphost" --non-interactive
"$cli" describe --apphost "$apphost" --format Json --non-interactive
"$cli" resource apiservice restart --apphost "$apphost" --non-interactive
"$cli" wait apiservice --status healthy --timeout 120 --apphost "$apphost" --non-interactive
"$cli" wait webfrontend --status healthy --timeout 120 --apphost "$apphost" --non-interactive
"$cli" stop --apphost "$apphost" --non-interactive
"$cli" ps --format Json --non-interactive

Observations:

  • API and frontend both reached Running and Healthy.
  • The frontend returned nonempty HTML and the API returned five forecast records with the expected fields.
  • Endpoints were taken from the actual resource snapshot, not hardcoded.
  • Both resources were healthy again after the API restart.
  • Scoped AppHost shutdown succeeded; the subsequent aspire ps result was [].
  • The running AppHost logged the expected Hosting version.

The functional HTTP requests followed HTTPS redirects and bypassed certificate verification only for these isolated, freshly generated development endpoints. Resource readiness used aspire wait, not HTTP polling.

Evidence: scenario-evidence/starter/, container-runtime-scenarios-verified.log, and captured AppHost/CLI logs.

Scenario 3: Concurrent transitions and controlled failure recovery

Objective: Verify that resource mutations and watched state changes continue to make progress during concurrent restart activity.

Coverage type: Happy path, concurrency, and unhappy path.

Status: Passed.

Generated a separate aspire-empty --language csharp app from the same PR hive. Its fixture declares four long-running shell executables and one explicitly started executable that initially exits with code 7.

Execution pattern:

apphost=/workspace/WatchTransitions/apphost.cs
"$cli" start --apphost "$apphost" --isolated --non-interactive --format Json
"$cli" resource worker0 stop --apphost "$apphost" --non-interactive
"$cli" wait worker0 --status down --timeout 60 --apphost "$apphost" --non-interactive
"$cli" resource worker0 start --apphost "$apphost" --non-interactive
"$cli" wait worker0 --status up --timeout 60 --apphost "$apphost" --non-interactive

Each of the four workers completed six stop/start rounds concurrently: 24 completed restart rounds, with successful waits after every transition. All four ended in Running.

The failing executable was then started. Its snapshot reported Finished and exit code 7, as expected. After creating its fixture recovery marker, starting it again brought it to Running; all five executables were running. Scoped shutdown succeeded and aspire ps returned [].

Expected unhappy-path outcome: A child-process failure must be reflected accurately and must not prevent subsequent resource start or state updates.

Evidence: scenario-evidence/transitions/, including one log per worker, the exact failed-resource snapshot, the recovered snapshot, and shutdown output.

Scenario 4: Idle reconnection and shutdown

Objective: Leave a real DCP executable watch idle beyond the normal five-minute connection-rotation interval, then verify subsequent resource transitions and cleanup.

Coverage type: Boundary and recovery.

Status: Passed.

A separate PR-hive app declares one long-running executable. The scenario waited for it to be running, recorded UTC timestamps, executed a 320-second sleep, then stopped and started that resource with explicit down/up waits, checked the final snapshot, and stopped the exact AppHost.

Observations:

  • Recorded idle timestamps: 2026-10-13T22:03:57Z to 2026-10-13T22:09:15Z, or 318 wall-clock seconds, exceeding the actual 300-second reconnection boundary. The script's sleep duration and recorded wall-clock delta are reported separately rather than treating wall clock as a monotonic timer.
  • Both mutations completed successfully after the idle interval, and the final executable snapshot was Running.
  • Scoped shutdown succeeded; the subsequent aspire ps result was [].
  • The running idle AppHost logged the expected Hosting version.

This checks behavior after the normal rotation interval. The focused HTTP2 tests independently verify that a periodic restart actually closes and replaces a connection when consumer progress is paused.

Expected boundary outcome: Resource updates must still arrive after the normal watch-rotation interval, and shutdown must complete without leaving a running test AppHost.

Evidence: scenario-evidence/idle/ and container-transitions-and-idle.log.

Environment and Harness Notes

  • Docker Desktop was responsive and running Linux containers.
  • The repository PowerShell runner conflicts with PowerShell 7's read-only $IsWindows variable. It was run unchanged through Windows PowerShell instead; no runner fix was added to this PR.
  • The runner image lacked .NET. A .NET 10 SDK was installed only into the owned testing state volume.
  • Restore used the downloaded PR hive and the internal NuGet mirror via the RestoreSources environment property.
  • The generated file-based fixtures explicitly set PublishAot=false and SelfContained=false. Their default AOT restore required extra SDK/runtime packages that the template's source mappings did not admit under the internal-feed override. These are managed-runtime watcher scenarios, not AOT publishing validation.
  • An initial snapshot assertion used physical name rather than friendly displayName. The harness was corrected to use the actual CLI JSON contract; both resources had already been healthy. This was not a product failure.
  • Repository automated E2E suites were not run locally. The Linux runs were the explicitly approved, isolated dogfood smoke scenarios.
  • No deployment, cloud service, persistent data volume, or unrelated user AppHost was changed.

Summary

Scenario Status Notes
Focused hosting regressions Passed 367 passed, 1 platform skip, 0 failed
Starter orchestration Passed Real DCP, endpoint responses, health and restart
Concurrent transitions and failure recovery Passed 24 restart rounds; exit 7 observed; recovery succeeded
Idle reconnection and shutdown Passed Updates and shutdown succeeded after the five-minute boundary

Overall Result

PR VERIFIED for the tested scenarios: All four scenarios passed. No watcher regression was found.

The final live PR status read reported 317 passing checks, 13 skipped checks, no pending checks, and no failing checks, with the head still at e92b3d4bc91ec3452203aac5116a0593593c064c. This is separate from the dogfood evidence above.

The validation does not claim to reproduce or conclusively fix the separate historical connected-but-silent watcher failure. It also does not establish bounded memory for an indefinitely blocked consumer; the unbounded, lossless queue is an intentional design tradeoff. AOT publishing and deployment were not tested.

Artifacts and Posting

  • Evidence and report are retained in this session's files/pr-20972-testing/ directory.
  • Scenario sources, command trace, structured snapshots, HTTP responses, timestamps, and AppHost/CLI logs are retained.
  • Screenshots were not needed: this PR does not change dashboard UI, and the runtime assertions use actual resource state and HTTP responses.
  • PR comment: Published with explicit user approval.
  • Processes: All test AppHosts stopped; every post-stop CLI listing was empty, and no container remained mounted to the test-owned state volume.
  • Cleanup: The user chose cleanup. The owned temporary application workspace and its unreferenced Docker state volume were removed. Session report, scenario sources, and complete test evidence remain available. The reusable runner image was retained.

@danegsta David Negstad (danegsta) added area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication area-orchestrator and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Oct 14, 2026
@danegsta
David Negstad (danegsta) merged commit 785cf4c into main Oct 14, 2026
1628 of 1643 checks passed
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 17.0 milestone Oct 14, 2026
@danegsta

Copy link
Copy Markdown
Member Author

/backport to release/13.6

@github-actions

Copy link
Copy Markdown
Contributor

Started backporting to release/13.6 (link to workflow run)

This was referenced Oct 31, 2026
Benjamin Michaelis (BenjaminMichaelis) pushed a commit to BenjaminMichaelis/sheaf that referenced this pull request Nov 1, 2026
Updated [aspire.cli](https://github.com/microsoft/aspire) from 13.5.0 to
13.6.4.

<details>
<summary>Release notes</summary>

_Sourced from [aspire.cli's
releases](https://github.com/microsoft/aspire/releases)._

## 13.6.4

## What's New in Aspire 13.6.4

Patch release for Aspire 13.6 that improves resource startup
reliability, fixes Azure publishing with parameterized enum properties
in TypeScript AppHosts, and tightens CLI diagnostic log permissions on
Unix.

### 🐛 Fixes

- 🖥️ **Resources could fail to start after a DCP create request timed
out** — AppHost now checks whether a resource was created before
retrying a timed-out or HTTP 504 request, avoiding a premature
`FailedToStart` state when DCP completed the operation. API operations
default to a 40-second timeout with an additional 20-second allowance
before the first successful operation, and resource creation has a
two-minute recovery budget. These limits can be configured through
`DcpPublisher:KubernetesApiTimeout`,
`DcpPublisher:KubernetesInitializationAdditionalTimeout`, and
`DcpPublisher:KubernetesCreateRecoveryTimeout`. Established watch and
log streams are not limited by these operation timeouts. Addresses
startup timeouts reported in 13.6. Fixes
[#​20691](microsoft/aspire#20691).
([#​20868](microsoft/aspire#20868), backport of
[#​20752](microsoft/aspire#20752), @​danegsta)

- ☁️ **TypeScript AppHosts could fail to publish parameterized Azure
enum properties** — Assigning a `bicep().parameter(...)` expression to
an enum-valued provisioning property, such as an Azure Storage SKU name,
now works instead of failing with a type-assignment error. Type checking
for literal values and incompatible expression types is preserved. Fixes
a scenario introduced with the Azure provisioning proxy feature in 13.6.
([#​20767](microsoft/aspire#20767), backport of
[#​20557](microsoft/aspire#20557),
@​sebastienros)

- 🔒 **CLI diagnostic logs now use owner-only permissions on Unix** — Log
files are restricted to owner read/write access (`0600`), including
reused files, and the dedicated Aspire logs directory uses owner-only
access (`0700`), repairing existing permissions when needed. Newly
created custom log directories also use `0700`; existing custom parent
directories and Windows permission behavior are unchanged. Diagnostic
logs can contain sensitive data, including dashboard login URLs, so
review them before sharing.
([#​21282](microsoft/aspire#21282), backport of
[#​21276](microsoft/aspire#21276), @​mitchdenny)

---

*Full Changelog:
[v13.6.3...v13.6.4](microsoft/aspire@v13.6.3...v13.6.4)*

*Full commit:
[c89352703f78ae46de90fbf751ed8b52b77b89bf](microsoft/aspire@c89352703f78ae46de90fbf751ed8b52b77b89bf)*


## 13.6.3

## What's New in Aspire 13.6.3

Patch release for Aspire 13.6 that improves resource orchestration
reliability by preventing unread resource watches from stalling resource
updates and operations.

### 🐛 Fixes

- 🖥️ **Resource operations could stall when a watch stopped being read**
— AppHost now closes abandoned watch connections when setup is canceled
and keeps reading resource events even when event handlers are slow.
Resource event ordering, lifecycle callback dispatch, and the existing
periodic reconnection safeguard are preserved.
([#​21002](microsoft/aspire#21002), backport of
[#​20972](microsoft/aspire#20972), @​danegsta)

- 🔄 **Slow watchers could block other resource updates or watcher
shutdown** — The bundled Developer Control Plane (DCP) is updated to
0.26.6, which keeps internal lifecycle watches draining and uses
bounded, asynchronous watch delivery to prevent unread watches from
blocking resource-change publication and unrelated watcher shutdown.
([#​21215](microsoft/aspire#21215), @​danegsta;
includes
[microsoft/dcp#​307](microsoft/dcp#307))

---

*Full Changelog:
[v13.6.2...v13.6.3](microsoft/aspire@v13.6.2...v13.6.3)*

*Full commit:
[38294b1b31552dae42a516cffa0cbc737ec08976](microsoft/aspire@38294b1b31552dae42a516cffa0cbc737ec08976)*


## 13.6.2


*Full commit:
[af07e33f8d1ef733645138b3f8e36067d073148d](microsoft/aspire@af07e33f8d1ef733645138b3f8e36067d073148d)*

## What's Changed
* [release/13.6] Bump version to 13.6.2 by @​danegsta with @​Copilot in
microsoft/aspire#20877
* [release/13.6] Pin MessagePack to 2.5.307 for security fixes by
@​eerhardt in microsoft/aspire#21156


**Full Changelog**:
microsoft/aspire@v13.6.1...v13.6.2

## 13.6.1

## What's New in Aspire 13.6.1

Patch release for Aspire 13.6 that reduces dashboard CPU usage during
sustained metric ingestion, restores scrolling in the Metrics tree,
improves dashboard navigation, fixes container exec watch failures, and
ensures session-scoped containers are cleaned up when stopping .NET
AppHosts on Windows.

### 🐛 Fixes

- 📊 **Dashboard CPU usage could increase during long-running metric
ingestion** — Metric retention repeatedly scanned retained history on
every insertion, causing steadily increasing CPU usage even with a fixed
workload. Cleanup now skips dimensions below capacity and removes only
the oldest surplus points when needed. Existing retention limits and
exemplar behavior are unchanged, and no configuration changes are
required. Regression introduced in 13.6. Fixes
[#​20725](microsoft/aspire#20725).
([#​20747](microsoft/aspire#20747), backport of
[#​20736](microsoft/aspire#20736), @​JamesNK)

- 📈 **The Metrics tree could not be scrolled** — The meter and
instrument tree now scrolls when it exceeds the viewport, allowing you
to reach and select instruments below the fold. Regression introduced in
13.6. Fixes [#​20728](microsoft/aspire#20728).
([#​20762](microsoft/aspire#20762), backport of
[#​20758](microsoft/aspire#20758), @​heyysiri)

- 🧭 **Dashboard navigation could fail after a browser disconnect** —
Pending page-state updates now persist session state before navigating
and skip navigation when the browser has disconnected, avoiding an
unhandled navigation failure. The navigation rail's overflow button also
has consistent sizing, icon alignment, and hover styling in expanded and
collapsed layouts and both light and dark themes.
([#​20545](microsoft/aspire#20545), backport of
[#​20527](microsoft/aspire#20527), @​JamesNK)

- 🖥️ **Container exec watches could terminate shortly after AppHost
startup** — A stalled DCP watch connection could produce a critical
`Watch task over Kubernetes ContainerExec resources terminated
unexpectedly` error and stop subsequent terminal state and log updates
from being observed. Watch connection attempts now retry after timeouts,
and periodic restarts correctly recreate connections without swallowing
unrelated cancellation. Regression introduced in 13.6.0.
([#​20664](microsoft/aspire#20664), backport of
[#​20466](microsoft/aspire#20466), @​radical)

- 🪟 **Session-scoped containers could remain running after stopping a
.NET AppHost on Windows** — AppHosts launched through `dotnet run` or
`dotnet watch` now allow DCP to finish resource cleanup during shutdown,
including existing Ctrl+C and `aspire stop` flows. Persistent resources
remain unaffected, while direct .NET launches and TypeScript/polyglot
AppHosts retain their existing process protection. Fixes
[#​20495](microsoft/aspire#20495).
([#​20514](microsoft/aspire#20514), backport of
[#​20494](microsoft/aspire#20494), @​danegsta)

---

*Full Changelog:
[v13.6.0...v13.6.1](microsoft/aspire@v13.6.0...v13.6.1)*

*Full commit:
[3751e615ad660621f2a7aa4c390ef48e08a0eed8](microsoft/aspire@3751e615ad660621f2a7aa4c390ef48e08a0eed8)*


## 13.6.0

# Aspire 13.6.0

Aspire 13.6 brings persistent application history, a refreshed and more
interactive dashboard, first-party Java and Rust hosting, and new Azure
deployment options. Coordinated .NET builds, portable volume paths,
sharper CLI workflows, and more capable editor tooling make it easier to
build, debug, and deploy applications across languages.

## Highlights

- 🗃️ **Persistent, refreshed dashboard** — SQLite-backed telemetry and
resource snapshots let you revisit up to ten application runs, with
read-only access to completed runs and default retention limits of
100,000 console log messages, structured logs, and traces each. The
dashboard now ships as Native AOT and adopts Fluent UI v5 with a
collapsible navigation rail.
- 🖥️ **AppHost-owned terminals and database REPLs** — Experimental
terminal APIs let AppHosts create interactive sessions in a dashboard
dock, dialog, or separate window. Opt-in `WithRepl()` commands open
bundled clients for PostgreSQL, MySQL, MongoDB, SQL Server, Redis, and
Valkey, while `aspire terminal ps` and `aspire terminal tape play`
support discovery and repeatable terminal interactions.
- 🌐 **First-party Java and Rust hosting** — New `Aspire.Hosting.Java`
and `Aspire.Hosting.Rust` packages bring Maven, Gradle, Spring Boot,
Quarkus, and Cargo applications into the app model, with generated
container builds and VS Code debugging. These integrations build on work
that originated in the Aspire Community Toolkit.
- 🛠️ **More flexible AppHosts** — The prerelease `Aspire.Hosting.Dotnet`
integration coordinates compatible projects into shared restore and
build groups and supports .NET SDK container publishing. Portable
volume-path environment variables work across local execution and
deployment, while TypeScript AppHosts gain standard `appsettings.json`
configuration and Deno runtime support.
- ⌨️ **Sharper CLI workflows** — Select launch profiles with `aspire
run` and `aspire start`, update repository-local CLI manifests with
`aspire update`, create file-based C# AppHosts with `aspire init
--language csharp --file-based`, and export TypeScript API data with
`aspire sdk export`. Terminal commands no longer need a feature flag,
Linux certificate trust includes Firefox NSS databases, and `aspire stop
--force --volumes` adds explicit cleanup of Aspire-owned volumes.
- 💻 **More capable VS Code tooling** — Coding agents can start and stop
AppHosts through the extension, and the Aspire pane exposes deploy,
publish, and pipeline actions. Multi-root discovery, worktree-scoped
lifecycle operations, preserved launch arguments, clearer debug logs,
and missing-debugger guidance make complex workspaces more predictable.
- ☁️ **New Azure options in preview** — Azure Connector Namespace models
external-service connections and managed MCP server configurations.
Azure Container Apps Sandboxes adds isolated sandbox deployments with
configurable resource tiers and lifecycle policies, while Azure
Container Apps Express offers a simplified environment option for rapid
provisioning.
- ☸️ **More expressive, reliable deployments** — Experimental Azure
Provisioning SDK proxies let polyglot AppHosts customize infrastructure.
Deployment state is isolated under `ASPIRE_HOME`, Kubernetes preserves
inherited hostnames and embedded parameter values, and AKS gains
persistent-volume provisioning and more reliable cleanup.
- 🔌 **Expanded integrations** — Experimental Deno hosting and MongoDB
replica sets join Foundry Toolboxes, remote Foundry Local endpoints,
Blazor WebAssembly debugging, and configurable Dev Tunnel expiration.
Azure Cosmos DB and AI Inference client integrations gain health checks,
and the vNext Cosmos DB emulator sends its own telemetry to the
dashboard.

## ⚠️ Breaking changes

Notable changes include automatic TLS for local MongoDB servers when a
certificate is available, the Linux-based vNext Cosmos DB emulator
becoming the default, new Azure Front Door origin names that can require
cleanup of existing origins, portable connection-string
environment-variable aliases on stricter deployment targets, and
experimental terminal types moving from `Aspire.Hosting.Terminals` to
`Aspire.Hosting.ApplicationModel`.

See the full list and migration guidance in the [Aspire 13.6 breaking
changes](https://aspire.dev/whats-new/aspire-13-6/#breaking-changes).

## 📖 Learn more

For complete details, examples, migration guidance, and everything new
in this release, read [What's new in Aspire
13.6](https://aspire.dev/whats-new/aspire-13-6/).

Thank you to all the community contributors who helped make Aspire 13.6
possible! 💜

---

_Full Changelog:
[v13.5.4...v13.6.0](microsoft/aspire@v13.5.4...v13.6.0)_

_Full commit:
[56f3e9c0d216c0c7069dabb49dd0464e4827744f](microsoft/aspire@56f3e9c0d216c0c7069dabb49dd0464e4827744f)_


## 13.5.4

## What's New in Aspire 13.5.4

Patch release for Aspire 13.5 that fixes Kafka health-check resource
leaks, DevTunnel errors with automatically selected regions, misleading
Azure emulator dashboard entries, and unintended changes to generated
starter apps, plus Homebrew compatibility and Radius API diagnostic
updates.

### 🐛 Fixes

- 📨 **Kafka health checks leaked producers and polling threads** — Each
AppHost health-check execution created a new Kafka producer without
disposing it, accumulating background threads over time. Health checks
now reuse a producer per Kafka resource and dispose it with the AppHost,
while keeping multiple Kafka resources independently configured. Fixes
[#​20091](microsoft/aspire#20091).
([#​20094](microsoft/aspire#20094), backport of
[#​20092](microsoft/aspire#20092), @​davidfowl)

- 🌐 **DevTunnels could fail when the region was selected automatically**
— Tunnel setup and health checks now use the cluster-qualified tunnel ID
returned by the DevTunnel CLI for port operations and access queries.
This fixes failures when a bare tunnel ID cannot be resolved for those
operations. Regression introduced in 13.3. Fixes
[#​18790](microsoft/aspire#18790).
([#​19853](microsoft/aspire#19853), backport of
[#​19230](microsoft/aspire#19230), @​Vladipz)

- ☁️ **Emulator-only AppHosts showed an unused Azure environment** — The
dashboard now hides the `azure-environment` resource when no Azure
resources require cloud provisioning, instead of leaving it visible in
**Not started**. It remains visible for apps that combine local
emulators with resources requiring Azure provisioning. No AppHost
changes are needed. Fixes
[#​19617](microsoft/aspire#19617).
([#​19998](microsoft/aspire#19998), backport of
[#​19843](microsoft/aspire#19843), @​eerhardt)

- 🧩 **Starter app generation could alter unrelated JavaScript values** —
Dynamic port replacement could also replace matching numeric literals in
bundled JavaScript, including Bootstrap timing values. Port
substitutions are now restricted to `localhost:` URLs, preserving the
original library files while still configuring the requested ports.
Fixes [#​20030](microsoft/aspire#20030).
([#​20110](microsoft/aspire#20110), backport of
[#​20031](microsoft/aspire#20031),
@​bart-vmware, @​JamesNK)

- 🍎 **Updated the Aspire Homebrew cask for Homebrew 6.x** — Replaced
deprecated cask URL and post-install syntax with the supported
equivalents, resolving compatibility issues with current Homebrew while
preserving install-channel metadata.
([#​20119](microsoft/aspire#20119), backport of
[#​19965](microsoft/aspire#19965), @​askpt,
@​joperezr)

- 🧪 **Radius cloud-provider callback interfaces now carry the
experimental diagnostic** — `IAwsRadiusProviderBuilder` and
`IAzureRadiusProviderBuilder` are now marked with `ASPIRERADIUS003`,
matching the existing `WithAwsProvider` and `WithAzureProvider` methods.
Code referencing these interfaces directly must now acknowledge the same
experimental API diagnostic.
([#​19874](microsoft/aspire#19874),
@​sebastienros)

---

*Full Changelog:
[v13.5.3...v13.5.4](microsoft/aspire@v13.5.3...v13.5.4)*

*Full commit:
[9c1b401dd67746739044f68959cbf4d3d7af93a6](microsoft/aspire@9c1b401dd67746739044f68959cbf4d3d7af93a6)*

## 13.5.3

## What's New in Aspire 13.5.3

Patch release for Aspire 13.5 that fixes Dashboard Graph view crashes
for resources with multi-path icons and restores missing public URLs for
DevTunnel resources.

### 🐛 Fixes

- 📊 **Dashboard Graph view could crash for Azure Blob resources** —
Resources such as those created with `AddBlobs` use icons containing
multiple SVG paths, which caused an XML parsing exception and broke the
dashboard circuit. The graph now combines multi-path icons correctly.
Regression introduced in 13.5. Fixes
[#​19489](microsoft/aspire#19489).
([#​19585](microsoft/aspire#19585), backport of
[#​19579](microsoft/aspire#19579),
`@​sebastienros`)

- 🌐 **DevTunnel public URLs were missing from the Dashboard and MCP
snapshots** — DevTunnel port resources could report `Running` and
`Healthy` while showing no public URLs. Proxyless port allocation is now
limited to compute and container resources, allowing DevTunnels to
publish their actual public endpoints. Regression introduced in 13.5.
Fixes [#​19496](microsoft/aspire#19496).
([#​19625](microsoft/aspire#19625), backport of
[#​19590](microsoft/aspire#19590),
`@​karolz-ms`, `@​danegsta`)

### 🏷️ Housekeeping

- 🚀 Bumped branding to 13.5.3

---

*Full Changelog:
[v13.5.2...v13.5.3](microsoft/aspire@v13.5.2...v13.5.3)*

*Full commit:
[b5f143315ffb6968ea939a9978797a5b20e4c688](microsoft/aspire@b5f143315ffb6968ea939a9978797a5b20e4c688)*

## 13.5.2

## What's New in Aspire 13.5.2


Patch release for Aspire 13.5 that removes an unused native helper
binary from the Windows CLI archives so 13.5 servicing releases stay
publishable to WinGet.


### 🐛 Fixes



- 🪟 **Windows CLI archives shipped an unused ~4.9 MB `hex1bpty.exe`** —
The Windows CLI archives (`aspire-cli-win-{x64,arm64}-*.zip`) bundled
Hex1b's out-of-process PTY host, which Aspire never executes (DCP owns
every pseudo-terminal Aspire surfaces). Besides the wasted download, the
extra unexplained executable stalled the WinGet publish, since every
binary in the archive goes through executable and malware validation. A
build-only MSBuild target now drops the file from the CLI publish
output; Unix native assets are unaffected. Regression new in 13.5.
([[#​19557](https://github.com/microsoft/aspire/pull/19557)](https://github.com/microsoft/aspire/pull/19557),
backport of #​19554, `@​mitchdenny`)



### 🏷️ Housekeeping



- 🚀 Bumped branding to 13.5.2




*Full Changelog:
[[v13.5.1...v13.5.2](https://github.com/microsoft/aspire/compare/v13.5.1...v13.5.2)](https://github.com/microsoft/aspire/compare/v13.5.1...v13.5.2)*


*Full commit:
[[a22cec24d76e764b3681977e314ab4a0aeed0240](https://github.com/microsoft/aspire/commit/a22cec24d76e764b3681977e314ab4a0aeed0240)](https://github.com/microsoft/aspire/commit/a22cec24d76e764b3681977e314ab4a0aeed0240)*


## 13.5.1

## What's New in Aspire 13.5.1


Patch release for Aspire 13.5 fixing a TypeScript/Java polyglot AppHost
compatibility regression when running the 13.5 SDK under an older
(13.4.x) CLI, plus a DCP update and release-pipeline housekeeping.

### 🐛 Fixes


- 🍎 **Polyglot AppHosts could crash on startup on macOS** — On macOS,
polyglot (TypeScript/Python/Java/Go/Rust) AppHosts could crash during
startup due to an interaction between how DCP's Go runtime forks
detached processes and how .NET Native AOT installs its signal handlers.
Updated DCP (Developer Control Plane) to 0.25.13 to resolve the crash.
([[#​19528](https://github.com/microsoft/aspire/pull/19528)](https://github.com/microsoft/aspire/pull/19528))

- 🔗 **Polyglot AppHosts on the 13.5 SDK crashed under an older CLI with
`MissingMethodException`** — A TypeScript or Java AppHost built with the
13.5 SDK failed to start when launched by an older (13.4.x) CLI, because
the newer codegen called `Aspire.TypeSystem` members that don't exist in
the CLI's older contract. Code generation now probes for these additive
capabilities before using them, so older CLIs skip only the unsupported
feature and startup succeeds. Regression introduced in 13.5 by #​19365.
Fixes #​19503.
([[#​19524](https://github.com/microsoft/aspire/pull/19524)](https://github.com/microsoft/aspire/pull/19524),
backport of #​19506, `@​adamint`)


### 🏷️ Housekeeping



- 📦 Updated DCP (Developer Control Plane) to 0.25.13
([[#​19528](https://github.com/microsoft/aspire/pull/19528)](https://github.com/microsoft/aspire/pull/19528))

- 🔧 Restored WinGet publication using .NET 9 `wingetcreate`
([[#​19509](https://github.com/microsoft/aspire/pull/19509)](https://github.com/microsoft/aspire/pull/19509))

- 🧹 Removed the pipeline-scoped `Publish-Build-Assets` group from the
release pipeline
([[#​19523](https://github.com/microsoft/aspire/pull/19523)](https://github.com/microsoft/aspire/pull/19523),
[[#​19163](https://github.com/microsoft/aspire/pull/19163)](https://github.com/microsoft/aspire/pull/19163))

- 🚀 Bumped branding to 13.5.1
([[#​19531](https://github.com/microsoft/aspire/pull/19531)](https://github.com/microsoft/aspire/pull/19531))




*Full Changelog:
[[v13.5.0...v13.5.1](https://github.com/microsoft/aspire/compare/v13.5.0...v13.5.1)](https://github.com/microsoft/aspire/compare/v13.5.0...v13.5.1)*


*Full commit:
[[69db530a4816698cf1d5fa4557933e0ac4f127c6](https://github.com/microsoft/aspire/commit/69db530a4816698cf1d5fa4557933e0ac4f127c6)](https://github.com/microsoft/aspire/commit/69db530a4816698cf1d5fa4557933e0ac4f127c6)*

Commits viewable in [compare
view](microsoft/aspire@v13.5.0...v13.6.4).
</details>

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aspire.cli&package-manager=nuget&previous-version=13.5.0&new-version=13.6.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication area-orchestrator

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants