test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0

`test.fail()` in after hooks relabels an already-executed test and can make a failing run exit 0

Found while qualifying Playwright's modifier handling for 9l (9lives), as a sibling of #43676 (fixed by #43713). 9l assesses Playwright suites and consumes their execution results, so a modifier that changes a completed test's expectedStatus after execution directly corrupts execution evidence.

Reproduction (verified on 1.61.1, 1.64.1 and 1.65.0-alpha-2026-10-28)

import {test, expect} from '@playwright/test';

test.afterEach(() => {
  test.fail();
});

test('genuinely failing', () => {
  expect(1).toBe(2);
});

$ npx playwright test a.spec.ts

  ✘  1 a.spec.ts:7:5 › genuinely failing (3ms)

  1 passed (0.3s)

$ echo $?
0

The test visibly fails, yet the summary says 1 passed and the process exit code is 0.

What happens

test.fail() called in an afterEach (also afterAll, also an auto-fixture teardown) runs after the test body finished.
_modifier sets testInfo.expectedStatus = 'failed' directly — no after-hook guard, and unlike skip/fixme it does not throw.
A failing test: status = failed now equals expectedStatus = failed → outcome expected → green exit code. A real failure is indistinguishable from a declared expected failure.
A passing test: relabeled unexpected → the run fails retroactively.

Scenario	Result
failing body + `test.fail()` in `afterEach`	`expected`, exit 0
failing body + `test.fail()` in `afterAll`	`expected`, exit 0
failing body + `test.fail()` in fixture teardown	`expected`, exit 0
same with `--retries=1`	exit 0
passing body + `test.fail()` in `afterEach`	`unexpected`, exit 1
control: declared `test.fail('title', fn)` that fails	`expected`, exit 0 (correct)

Why #43713 does not cover it

#43713 sets testInfo._allowSkips = false before after hooks, which only affects the TestSkipError path (skip/fixme). I verified this by applying the same one-line patch locally: test.skip(true, ...) in afterEach is then correctly rejected (exit 1), while test.fail() in afterEach still relabels and exits 0.

The fail branch of _modifier needs the same treatment: after the test has finished executing, it must not mutate expectedStatus — it should be rejected (reported as a failure), as #43713 does for skips.

Impact for downstream tooling

Any consumer of Playwright results (reporters, CI gates, evidence tools) reads expected as "intentional failure". With one stray test.fail() in a shared after hook — or any code reaching it through a fixture teardown — genuine failures pass CI silently. The relabeled annotation is also attached to the finished test, leaving no trace that the expectation was set after execution.

I'd like to investigate this and contribute a fix. I'll start by reproducing the incorrect exit status with a focused regression test, then ensure test.fail() cannot change a test's expected status after execution. I'll keep the change focused and account for the related test.skip() work in #43713.

I've reproduced the issue and prepared a focused fix locally.

The change prevents test.fail() from modifying a test's expected status during after hooks or fixture teardown.

Validation completed:

29/29 test-modifier tests passed, including three new regression tests.
ESLint passed.
TypeScript compilation passed.
git diff --check passed.

I'm also aware of the related work in #43713 and can coordinate the changes to avoid duplication.

Would a maintainer be willing to confirm whether this issue is approved for community contribution? The fix is ready for review once approved.

Hi, I'm the Playwright bot and I reproduced this on Linux.

Reproduced on 1.65.0-next (f33f0b751be0dd559d86c1ed27ca8baa5f9cb2b2). test.fail() changes expectedStatus during cleanup, so a failed test can become an expected failure and make the run exit 0.

The proposed fix rejects active test.fail() and testInfo.fail() calls after the test body finishes, before changing the expected status or adding an annotation. False conditions remain no-ops.

Verification

The existing modifier test now checks a failing body followed by calls from afterEach, test-scoped fixture teardown and afterAll. Before the fix, the inline run exits 0 and counts the test as passed. With the fix, it exits 1, retains the original failure and reports each invalid modifier call.

The modifier, hook and fixture suites pass: 114 tests. npm run build and npm run flint also pass. This is browser-independent runner behavior; no browser is launched.

Draft fix: #43753

_{Triaged on Linux by the Playwright bot - agent run}

I don't think this is a problem. One should be able to mark tests as "expected to fail" in a fixture teardown based on some conditions. This is very rare, but not unreasoanble.

Report results Playwright relabeled after the test body ran; flag aft…

Navigation Menu

Repository navigation

test.fail() in after hooks relabels an already-executed test and can make a failing run exit 0 #43738