Repository navigation
Conversation
The suite fails 386 tests on master, and the failure bodies carry 2403 screenshot-mismatch mentions against 471 timeout mentions, so the reference images have drifted rather than the editor having broken. That wall of red hides whether a change under review broke anything. --ignore-snapshots turns screenshot assertions into no-ops and leaves every functional assertion live. It is a stopgap: the fix is to regenerate the references on chromium-linux. The revert is already open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four critical advisories are dev dependencies and they fail the job on every branch, master included. Reporting them keeps the signal without hiding the build and test results behind a red check. The revert is already open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two temporary CI changes so a phase 4 review can tell whether a change under review broke anything. The revert is already open and stacked on this: #122
Why
Measured from the last complete master Playwright run (18 September, run 35380101859):
Classifying the failure bodies: 2403 screenshot-mismatch mentions against 471 timeout mentions, and effectively no structural errors. The reference images have drifted; the editor has not broken.
Separately,
npm audit --all --audit-level=criticalfails on every branch including master, on four critical advisories that are all dev dependencies (handlebars,next,shell-quote,websocket-driver).What this changes
--ignore-snapshotson the Playwright command. Screenshot assertions become no-ops; every functional assertion stays live.continue-on-error: trueon the audit step. It still reports.Quarantining by file would have disabled a third of the suite including its passing tests. Quarantining 386 test titles would conflict at every upstream merge. Disabling the two failure classes is smaller and reverts in one diff.
What this does not fix
The references still need regenerating on chromium-linux, and the timeouts still need investigating. This hides drift; it does not resolve it.
What did I (author) test?
--ignore-snapshotsis documented in the pinned Playwright 1.44.1: "Ignore screenshot and snapshot expectations".