Skip to content

GH-4505: let a deduplication claim ride the Marten transaction - #4568

Merged
jeremydmiller merged 2 commits into
mainfrom
gh-4505-transactional-deduplication
Sep 23, 2026
Merged

jeremydmiller merged 2 commits into
mainfrom
gh-4505-transactional-deduplication

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #4505 for Marten. Polecat and Fisher are #4564; gRPC is #4566.

What changed

A logical deduplication claim was written up front on its own connection and given back in a finally
when the chain failed — a try/catch/finally around the whole chain, a status-code test, and a
ReleaseAsync that swallows its own failures, at a cost of one round trip to claim and another to
release. It was also a story the docs told and the code never implemented: nothing rode the business
transaction, because every IDeduplicationStore claims through DbDataSource.CreateCommand().

On a single-database Marten store the claim is now queued onto the document session's unit of work, the
same way Wolverine has always queued its own inbox rows. It commits with the handler's events and
documents and rolls back with them:

  • a chain that throws never reaches SaveChangesAsync, so it never claimed;
  • a chain that stops with a ProblemDetails 404 or a FluentValidation 400 returns before it, so it never
    claimed either.

#4501 is fixed by construction rather than by noticing a status code, and there is no window for a prompt
retry to race, because there is nothing to undo.

The "has this been claimed?" query is an IBatchableFrame, so MartenBatchingPolicy folds it into
whatever IBatchedQuery the chain was already running. On the composition #4501 was reported against —
[Deduplicated] on a [WriteAggregate] endpoint — the check and the FetchForWriting are one round trip,
against two on the happy path and three on the failure path today.

A chain whose id is optional opts out of the batch, because batch enlistment is unconditional code and
unkeyed traffic on a mixed stream is supposed to cost no round trip at all.

The one thing that is genuinely new

Two concurrent callers can both read "not claimed" — neither claim is committed, so neither is visible to
the other. The deduplication table's primary key arbitrates at commit, and the loser is converted into the
same refusal the check produces (the endpoint's configured 409, or a discard on a handler) rather than
escaping as a raw Postgres unique violation.

Worth saying plainly, and it is in the docs: the loser does its work and throws it away at commit,
where claim-and-release refuses it before doing any. That is the normal cost of the optimistic pattern and
it is the rare path, but it is a real behaviour change for anyone whose handler has side effects outside
the transaction.

This relies on an ordering fact: SaveChangesAsync is a postprocessor and the response writer is appended
to Postprocessors after it, so the commit fails while Response.HasStarted is still false and a clean
problem document can still go out.

Two deviations from the issue text

#4561 is not deleted. The issue's "what to delete" list conflicts with its own Scope section
("non-transactional chains keep the current path"). logical_deduplication_on_http_endpoints builds its
host with PersistMessagesWithPostgresql, so every endpoint in it — including
TransactionalDeduplicatedEndpoint — takes claim-and-release and still needs #4547's ordering fix.
Removing writeBeforeTry reintroduces that race for every non-Marten HTTP endpoint. All 10 tests in that
file pass unchanged. Explained in a comment on #4505.

There is still a try/catch, just a much smaller one. The finally, the compensating ReleaseAsync
and the Response.StatusCode >= 400 test are gone. What remains is a catch with an exception filter that
sets a bool; the refusal is the chain's own existing BuildDeduplicationStopCondition, so there is no
new per-chain-type code.

The guard is not CanApply

Worth calling out for review because getting it wrong is silent and total. CanApply answers "could this
provider own the chain's transaction?", which is true for any chain that merely takes an
IDocumentSession. A claim queued onto a unit of work nothing saves is never written — so the endpoint
reports itself as deduplicated while every replay runs, with no exception and nothing logged, and every
source-level assertion still passes.

The guard requires the CreateDocumentSessionFrame and the DocumentSessionSaveChanges postprocessor to
actually be on the chain. deduplication_without_a_commit_keeps_the_release pins it with an endpoint that
calls SaveChangesAsync itself under a host with no AutoApplyTransactions.

Scope

Marten only, main and ancillary stores. Database-per-tenant falls through to claim-and-release, because
that composition builds the message store from a different database than the session commits to —
detected at codegen time via Tenancy.Cardinality.

Tests

  • MartenTests/deduplication_rides_the_marten_transaction.cs (8) — generated-source assertions that the
    claim is queued and nothing is released, the batch is shared with the aggregate load, an optional id
    stays standalone, a throwing handler never claimed, a claim-only unit of work still commits, and the
    concurrent race.
  • MartenTests/AncillaryStores/deduplication_rides_an_ancillary_marten_transaction.cs (2) — the claim
    lands in the ancillary schema, not the main one, and an ancillary rollback takes it.
  • Wolverine.Http.Tests/Marten/deduplication_on_a_marten_transaction.cs (5) — the [Deduplicated] HTTP: a dedupe claim survives a non-2xx chain stop (ProblemDetails / 404), turning retries into false duplicates #4501 composition over
    HTTP, plus the no-commit fallback.

The concurrency test uses a barrier inside the handler to hold both executions past their check before
either commits. Two plain concurrent sends pass against a build with no commit-race handling at all,
because the optimistic check refuses the second one whenever the first has already committed. Verified as a
negative control: with the wrapper removed the loser escapes as a raw MartenCommandException.

Verification

Clean pinned dotnet build wolverine.slnx -c Release -f net9.0 --no-incremental — 222 projects, 0
errors, 0 warnings
.

CoreTests 3032/3034 (2 skipped) · MartenTests 697/697 · PostgresqlTests logical_message_deduplication
11/11 · Wolverine.Http.Tests logical_deduplication_on_http_endpoints 10/10 unchanged.

Also in this commit

Two code comments claimed "Oracle supplies its own" deduplication store. Oracle supplies none —
OracleMessageStore does not derive from MessageDatabase<T> and keeps NullDeduplicationStore — so a
[Deduplicated] chain on Oracle throws at the first message, which is the intended outcome for an
unsupported store but is not what the comments said. Corrected. The two stale XML doc comments #4505 listed
as a loose end are corrected too, and docs/guide/durability/idempotency.md now describes both paths.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDUrBeB4tTnKj4AExCS1nj

jeremydmiller and others added 2 commits September 23, 2026 08:41
A logical deduplication claim was written up front on its own connection and
given back in a finally when the chain failed. That is three moving parts that
all have to stay right -- a try/catch/finally around the whole chain, a
status-code test, and a ReleaseAsync that swallows its own failures -- and it
costs a round trip to claim plus a round trip to release. It was also a story
the docs told and the code never implemented: nothing rode the business
transaction, because every IDeduplicationStore claims through
DbDataSource.CreateCommand() on a fresh connection.

On a single-database Marten store the claim can be queued onto the document
session's unit of work instead, exactly as Wolverine has always queued its own
inbox rows. It then commits with the handler's events and documents, and rolls
back with them. A chain that throws never commits, so it never claimed; a chain
that stops with a ProblemDetails 404 or a FluentValidation 400 returns before
SaveChangesAsync, so it never claimed either. GH-4501 is fixed by construction
rather than by noticing a status code, and there is no window for a prompt retry
to race because there is nothing to undo.

The "has this been claimed?" query is an IBatchableFrame, so MartenBatchingPolicy
folds it into whatever IBatchedQuery the chain was already running -- the check
and a [WriteAggregate] FetchForWriting land in one round trip. A chain whose id
is optional opts out of the batch, because batch enlistment is unconditional code
and unkeyed traffic is supposed to cost nothing.

Two concurrent callers can still both read "not claimed", since neither claim is
committed and so neither is visible to the other. The deduplication table's
primary key arbitrates at commit, and the loser is converted into the same
refusal the check produces -- the endpoint's configured 409, or a discard on a
handler -- rather than escaping as a raw Postgres error. That relies on
SaveChangesAsync being a postprocessor with the response writer appended after
it, so the commit fails while Response.HasStarted is still false.

The guard is deliberately not CanApply. That answers "could this provider own the
chain's transaction?", which is true for any chain that merely takes an
IDocumentSession -- and a claim queued onto a unit of work nothing saves is never
written, so the endpoint reports itself as deduplicated while every replay runs,
silently. It requires the CreateDocumentSessionFrame and the SaveChanges
postprocessor to actually be present.

Scope is Marten, main and ancillary stores. Database-per-tenant falls through to
claim-and-release, because that composition builds the message store from a
different database than the session commits to. Polecat and Fisher are #4564:
their message store is a separate connection pool, and on Fisher an off-session
write to the same SQLite file is a self-deadlock, so the mechanism there is
ITransactionParticipant rather than QueueOperation. gRPC is #4566.

The GH-4547 interim fix stays. This issue's delete list conflicts with its own
scope section: non-transactional chains keep claim-and-release, and for them the
release still has to be registered before the failure response is flushed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDUrBeB4tTnKj4AExCS1nj
…n fixture

The new HTTP fixture counts MartenDedupDocument rows to prove the handler ran
exactly once. ResetResourceState clears Wolverine's own tables -- the
deduplication table included -- but leaves the application's Marten documents
alone, so the counts accumulated across runs: the class passed on a virgin
database and failed on every rerun.

That is the worst shape a test can have. CI provisions a fresh database per run,
so it would have stayed green there indefinitely while failing for whoever next
ran the suite locally -- which is exactly how it surfaced here, on the second
full-suite pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDUrBeB4tTnKj4AExCS1nj
@jeremydmiller
jeremydmiller merged commit 6d58cd2 into main Sep 23, 2026
44 checks passed
jeremydmiller added a commit that referenced this pull request Sep 24, 2026
The Fisher half of GH-4505, after Marten (#4568) and Polecat (#4573). Same guarantee:
the claim commits with the handler's work and rolls back with it, so a chain that throws
or answers a ProblemDetails never claimed at all and there is no compensating release to
order against the response.

On SQLite the mechanism is a correctness requirement rather than a preference. Wolverine's
message store here is built from new WolverineSqliteDataSource(store.Options.ConnectionString)
-- the same FILE, a different pooled connection. Two connections to one file are two writers,
and the second blocks on the first from inside the first one's transaction, which hangs rather
than failing. So the claim is enlisted as an ITransactionParticipant, handed the live
connection and transaction, and even the optimistic existence check goes through the session's
own AdvancedSql rather than the store's data source.

Fisher has no database-per-tenant composition, so unlike the Marten and Polecat twins there is
no tenancy cardinality to guard.

The commit-race classifier is scoped to the deduplication table, sharing the table-name parsing
GH-4565 added for the inbox: a bare SQLITE_CONSTRAINT test would match a duplicate key raised by
the application's own documents in the same transaction and report it to the caller as "already
handled". It is exposed through FisherDeduplicationFailures rather than off SqliteMessageStore
directly, because the frame renders the declaring type into generated source and that type is
internal -- InternalsVisibleTo does not reach generated code.

The commit race IS reachable on SQLite, which #4571 left open. One writer means the two commits
serialize on the file lock rather than racing inside a transaction, but both executions still
hold a claim the other cannot see, so the loser still trips the primary key. The single-writer
model changes when it finds out, not whether it does.


Claude-Session: https://claude.ai/code/session_01VDUrBeB4tTnKj4AExCS1nj

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deduplication should ride the native transaction on Marten, Polecat and Fisher — batched optimistic check, no compensating release

1 participant