Skip to content

[SDK Sentinel] Fix time-skipping external cancel test race - #2027

Open
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/python/40606438889-1-01e89cf44f19
Open

sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/python/40606438889-1-01e89cf44f19

Conversation

@sdk-sentinel-bot

Copy link
Copy Markdown
Contributor

Caution

This PR contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.

Summary

Prevent the external-workflow cancellation test from racing its target timer. The failure appeared in the [macOS ARM time-skipping V2 job](https://github.com/temporalio/sdk-python/actions/runs/40522868526/job/122254383729). The [preceding main CI](https://github.com/temporalio/sdk-python/actions/runs/40519296785) passed, while the [first captured bad CI](https://github.com/temporalio/sdk-python/actions/runs/40522868526) failed with unchanged relevant code.

Root cause

The target used a 1000-second workflow timer. Time-skipping V2 can advance that timer before the separate cancellation workflow sends its request, letting the target complete normally instead of raising cancellation.

Fix

Use the existing timer-free wait-forever workflow as the target. External cancellation is now its only completion path, removing the nondeterministic ordering.

Validation

The failure did not reproduce in 50 pre-change stress iterations. With the fix, 100 time-skipping V2 iterations and one normal-environment run passed. The fixed repository validator and unchanged baseline passed. macOS ARM CI remains required.

Validation status: validated-without-reproduction

  • Flake confidence: high
  • Fix confidence: high
  • Value: medium
  • Patch scope: standard — 1 changed file; 1.4 KiB
  • Local reproduction: not-reproduced — Fifty pre-change Linux time-skipping V2 stress iterations passed; the captured scheduling race did not occur locally.
  • Regression coverage: pass-after-only — The existing focused test passed 100 time-skipping V2 stress iterations and one normal-environment iteration after the candidate.
  • Unchanged baseline (Poe lint, bridge lint, and editable build) — passed
  • Independent candidate (Poe lint, bridge lint, and editable build) — passed

Investigator-run checks

  • poe test -s --workflow-environment time-skipping-v2 tests/worker/test_workflow.py::test_workflow_cancel_external --flake-finder --flake-runs 50 — passed; 50 attempt(s). Before the edit, all 50 iterations passed, so natural local reproduction was not obtained.
  • poe test -s --workflow-environment time-skipping-v2 tests/worker/test_workflow.py::test_workflow_cancel_external --flake-finder --flake-runs 100 — passed; 100 attempt(s). All 100 iterations passed with the final candidate on Linux.
  • poe test -s --workflow-environment local tests/worker/test_workflow.py::test_workflow_cancel_external — passed; 1 attempt(s). The timer-free target preserved normal local-environment cancellation behavior.
  • .ci-flake-runtime/input/validate.sh — passed; 1 attempt(s). The fixed validator passed formatting, imports, type checks, docstyle, Rust clippy, and editable bridge build. The trusted unchanged baseline also passed.

Required target CI

  • Continuous Integration / build-lint-test (3.10, macos-arm), time-skipping-v2 step

Residual risks

  • The natural failure was not reproduced locally.
  • The captured failure originated on macOS ARM, while local validation used Linux.

Automation provenance

If this finding should not be fixed, apply the sdk-sentinel:false-positive label and close the PR. Sentinel will suppress the finding until its affected source changes.

SDK Sentinel recurrence history

  • Unique failed CI run attempts: 1
  • First occurrence: 2026-10-27 14:55 UTC
  • Latest occurrence: 2026-10-27 14:55 UTC
  • Recent occurrences:

@sdk-sentinel-bot
sdk-sentinel-bot requested a review from a team as a code owner October 28, 2026 00:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant