Skip to content

GH-5108 Logical deduplication claim rides the EF Core transaction - #5111

Merged
jeremydmiller merged 1 commit into
mainfrom
gh-5108-efcore-transactional-dedup
Oct 21, 2026
Merged

jeremydmiller merged 1 commit into
mainfrom
gh-5108-efcore-transactional-dedup

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #5108

Problem

#4505 / #4568 made the logical deduplication claim ride the Marten (and later Polecat/Fisher) transaction. EF Core chains still used claim-and-release: IMessageDeduplicator.TryClaimAsync committed the claim on its own connection before BeginTransactionAsync, and the claim was given back only when the chain threw. A process killed between the claim and the EF Core commit left the claim committed and the work undone; inbox recovery then replayed the envelope with the same DeduplicationId and it was discarded as a duplicate. The message was lost, and the log said it was deduplicated.

Reproduction

src/Persistence/EfCoreTests/deduplication_rides_the_ef_core_transaction_5108.cs, run on PostgreSQL and SQL Server. A process kill can't be staged deterministically, but what decides whether a kill loses the message can: the handler reads the claim table from a separate connection while its transaction is still open (READPAST on SQL Server). Anything visible there survives a kill.

Before the fix, 4 of 14 fail on both providers:

no_observer_outside_the_transaction_can_see_the_claim_until_the_handler_commits
  Handler5108.ClaimsVisibleDuringHandler should be 0 but was 1
the_claim_is_written_on_the_dbcontext_transaction_and_nothing_is_released
  (generated code calls _messageDeduplicator.TryClaimAsync before BeginTransactionAsync)

After the fix, all 15 pass, including the new Lightweight fallback test.

Fix

EFCorePersistenceFrameProvider now implements IPersistenceFrameProvider.TryBuildTransactionalDeduplication, as the issue proposes:

  • It applies only when the chain has EnrollDbContextInTransaction and CommitEfCoreEnvelopeTransaction (Eager mode, a single non-tenanted DbContext) and the store for that DbContext (main, or its ancillary store) is an IMessageDatabase whose deduplication store implements the new ITransactionalDeduplicationStore. This rests on the same assumption the EF Core outbox already makes when it writes envelope rows through the DbContext connection.
  • The new ClaimDeduplicationIdInEfCoreTransactionFrame calls EfCoreEnvelopeTransaction.TryClaimDeduplicationIdAsync. That runs the claim INSERT on the DbContext's connection and current transaction. Because the frame depends on the EfCoreEnvelopeTransaction variable, it lands inside the enroll frame's try block, after BeginTransactionAsync and before the handler.
  • No release frame is emitted. A throw, a rollback or a process kill takes the uncommitted claim with it.
  • Concurrent copies: the second INSERT blocks on the first copy's uncommitted key. It is refused once that copy commits, or goes through if that copy rolls back. Unlike Marten there's no commit-time race, so no CommitRaceWrapper is needed.

Supporting changes in Wolverine.RDBMS:

  • New public ITransactionalDeduplicationStore, implemented by RdbmsDeduplicationStore (PostgreSQL, SQL Server, MySQL, SQLite).
  • New protected virtual string? DeduplicationClaimConflictClause on MessageDatabase. PostgreSQL overrides it with on conflict do nothing, so a duplicate is reported as zero rows instead of an error that would abort the caller's transaction. Other engines keep the existing unique-violation classification; a duplicate-key error leaves their transaction usable.

There are no breaking API changes; everything is additive.

Configurations

Configuration Claim
Eager, single DbContext, relational message store (incl. ancillary store per DbContext) rides the EF Core transaction (new)
TransactionMiddlewareMode.Lightweight claim-and-release (unchanged)
Database-per-tenant DbContext or MultiTenantedMessageStore claim-and-release (unchanged)
EF Core sagas (transaction applied after dedup is woven) claim-and-release (unchanged)
Oracle / any store without an ITransactionalDeduplicationStore claim-and-release (unchanged)

Tests run locally

  • EfCoreTests, the new 5108 classes: 15/15 on PostgreSQL and SQL Server. Full EfCoreTests: 432 tests. The first run's failures were RabbitMQ not running locally. After starting RabbitMQ, EfCoreTests.Bugs was 62/63; the one failure, Bug_2075, has no dedup or EF involvement and passes when run alone.
  • DeduplicationStoreCompliance, with 6 new in-transaction facts (rollback leaves no claim, commit refuses the next claim, a refused claim leaves the transaction committable) under both modes: PostgreSQL 38/38, SQL Server 38/38, SQLite 38/38. MySQL was not run locally.
  • Wolverine.Http.Tests dedup filter: 81/81. CoreTests dedup: 36/36. MartenTests dedup: 16/16.
  • dotnet build wolverine.slnx -c Release -f net9.0: clean.

Docs: one paragraph in docs/guide/durability/idempotency.md saying when EF Core chains ride the transaction.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DCggXgxnG3EBXoR9rrq37U

On an EF Core chain in Eager mode the deduplication claim was committed on its
own connection before BeginTransactionAsync and only released when the chain
threw. A process killed between the claim and the EF Core commit left the claim
behind and the work undone, so inbox recovery discarded the replay as a
duplicate and the message was lost.

EFCorePersistenceFrameProvider now implements TryBuildTransactionalDeduplication
for Eager mode with a single, non-tenanted DbContext against a relational message
database: the claim is an INSERT on the DbContext's connection and transaction,
written after the transaction begins and before the handler runs, with no
release frame. A concurrent copy blocks on the uncommitted claim and is refused
once it commits. Lightweight mode, database-per-tenant and EF Core sagas keep
claim-and-release.

New ITransactionalDeduplicationStore (Wolverine.RDBMS), implemented by
RdbmsDeduplicationStore; PostgreSQL appends "on conflict do nothing" so a
duplicate does not abort the caller's transaction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCggXgxnG3EBXoR9rrq37U
@jeremydmiller
jeremydmiller merged commit dfd6f20 into main Oct 21, 2026
47 checks passed
This was referenced Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

# Logical deduplication on an EF Core transactional chain is not crash-safe: the claim is committed before the EF Core transaction (follow-up to #4505)

1 participant