fix(desktop-e2e): remove root causes of four retry-masked desktop flakes by waleedlatif1 · Pull Request #8940 · simstudioai/sim

waleedlatif1 · 2026-10-12T20:17:57Z

Summary

Desktop Playwright retries once in CI, so flakes never show up as failures. In 143 recent runs, 8 green runs in gating jobs passed only on the retry. This PR fixes four of those flakes at their root and adds diagnostics for the other three.

Fixed

packaged-smoke:91 (also gates releases) and the same race at :46. Cleanup deleted the profile (userData) folder while the app's leftover helper processes, such as Crashpad's handler, could still write to it, which caused ENOTEMPTY. The app now launches in its own process group. Cleanup waits, with a bounded poll, until no process in that group or naming the unique temp profile path is left, then deletes the folder.
background-executor:253. The consent dialog closes its own window before Playwright gets the click acknowledgement, so the click fails even though the answer landed. A new answerShellDialog helper waits for the window to close instead. Each caller still asserts the recorded outcome.
background-executor:83. This checked a 1.5s wall-clock limit on a shared runner, while the product's real pickup deadline is 15s (repository.ts:58, constants.ts:79). The test now asserts that every call is claimed within 15s of being issued, holds the first command so lease renewal is checked, and releases that hold in a finally. Fast pickup is still asserted in test F.
browser-tools:609, a real product bug. When a click makes a page close its own tab (for example a popup's Allow button), Electron can reject the final mouseReleased after the tab is gone. The driver then reported "Native click dispatch failed" for a click that had landed. cdp.clickAt now raises a typed TabClosedOnReleaseError only when the final release went to a live tab and that tab is destroyed by the time the release is rejected. The driver treats only that error as a delivered click, returning tabChanged and doNotRetry. Any failure before input, or on the move, press or hold, keeps its existing error.

Diagnostics only (browser-page-dialogs:46, browser-focus:32): failures now print the tool error and the dialog id, so the next flake names its cause. smoke:85, a hang in app.close(), needs a DEBUG=pw:browser run.

Playwright still retries once. failOnFlakyTests gets enabled once the remaining three are fixed.

Test plan

desktop-live / desktop-e2e green, with no "flaky" lines for the four fixed specs

- packaged-smoke: remove the profile only after every process that can write it (the app's process group, Crashpad's handler) is gone; SIGKILLing main left helpers creating files during rmSync (ENOTEMPTY) - background-executor: answer shell dialogs by waiting for their window to close, since the dialog destroys itself before the click is acknowledged - background-executor A: replace the 1.5 s p95 wall-clock threshold with a gated check that every call is claimed while chat B's first command still holds its queue, within Sim's 15 s pickup window, and that held leases renew - driver: a click whose tab closes itself mid-dispatch (popup Allow -> window.close()) is a dispatched click, not a failed one, whichever of the close and the input acknowledgement lands first - page-dialogs / browser-focus / browser-tools: print the tool error and the shown dialog id when they fail, to root-cause the remaining flakes

…hat then closed - cdp.clickAt raises TabClosedOnReleaseError only when the final mouseReleased was sent to a live tab and the tab is destroyed when it rejects; the driver rescues only that error (no string match, no pre-input/press/hold cases), with a note that does not claim the page closed itself and doNotRetry - background-executor A: assert each call is claimed within Sim's 15 s pickup window measured from issue, and always release the gated commands

vercel · 2026-10-12T20:18:03Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 12, 2026 8:40pm UTC

cubic-dev-ai

All reported issues were addressed across 8 files

_{Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic}

greptile-apps · 2026-10-12T20:23:16Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

This PR reduces desktop test flakes and handles clicks whose tab closes during the final release.

Clicks that close their tab are reported as unconfirmed, not failed.
Packaged smoke tests wait for profile writers before removing test data.
The executor test checks pickup across Sim's full 15-second window.
Consent tests treat a closed prompt as proof the answer landed.
Browser test failures now include the tool error or dialog ID.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Send final click release] --> B{Release rejected?}
  B -->|No| C[Normal click result]
  B -->|Yes| D{Tab was live and is now closed?}
  D -->|No| E[Keep original error]
  D -->|Yes| F[TabClosedOnReleaseError]
  F --> G[Report unknown click effect]
  G --> H[Do not retry; inspect current state]

_{Reviews (2) · Last reviewed commit: "fix(desktop): report a click whose tab c..." · Reviewed by Greptile}

The tab being destroyed proves teardown, not that Chromium processed the release, so the rescued result no longer claims effectObserved and its note says the outcome could not be confirmed. Adds clickAt tests for each TabClosedOnReleaseError guard and a driver test for the rescued shape.

waleedlatif1 · 2026-10-12T20:40:46Z

@greptile

waleedlatif1 · 2026-10-12T20:40:48Z

@cubic-dev-ai review this PR

cubic-dev-ai · 2026-10-12T20:40:55Z

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

No issues found across 10 files

Confidence score: 5/5

Automated review surfaced no issues in the provided summaries.
No files require special attention.

_{Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic}

waleedlatif1 added 2 commits October 12, 2026 13:10

cubic-dev-ai Bot reviewed Oct 12, 2026

View reviewed changes

Comment thread apps/desktop/src/main/browser-agent/cdp.ts

greptile-apps Bot reviewed Oct 12, 2026

View reviewed changes

Comment thread apps/desktop/src/main/browser-agent/cdp.ts

vercel Bot temporarily deployed to Preview October 12, 2026 20:40 Inactive

cubic-dev-ai Bot reviewed Oct 12, 2026

View reviewed changes

waleedlatif1 merged commit 3bfd224 into staging Oct 12, 2026
41 checks passed

waleedlatif1 deleted the fix/desktop-e2e-flakes branch October 12, 2026 20:49

TheodoreSpeaks mentioned this pull request Oct 13, 2026

v0.9.20: project membership enforcement, db and perf improvements, realtime and auth hardening, CI and test fixes #8963

Merged

Navigation Menu

Repository navigation

fix(desktop-e2e): remove root causes of four retry-masked desktop flakes - #8940

fix(desktop-e2e): remove root causes of four retry-masked desktop flakes#8940
waleedlatif1 merged 3 commits into
stagingfrom
fix/desktop-e2e-flakes

waleedlatif1 commented Oct 12, 2026

Uh oh!

vercel Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 12, 2026 •

edited

Loading

Diagram

Uh oh!

Uh oh!

waleedlatif1 commented Oct 12, 2026

Uh oh!

waleedlatif1 commented Oct 12, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 12, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

waleedlatif1 commented Oct 12, 2026

Summary

Test plan

Uh oh!

vercel Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

waleedlatif1 commented Oct 12, 2026

Uh oh!

waleedlatif1 commented Oct 12, 2026

Uh oh!

cubic-dev-ai Bot commented Oct 12, 2026

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 12, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 12, 2026 •

edited

Loading