[Thom Chiovoloni (thomcc-work)]

Summary

This is part of an effort to make duroxide-pg-opt redundant, although it stands on its own and has its own benefits, so I'm PRing it separately. It's a draft until I finish AI review.

Adds notification-driven long-polling for orchestration and activity work, adapted from duroxide-pg-opt. Existing constructors continue to use short polling by default. The goal is to reduce empty-queue queries and polling-induced latency without changing queue eligibility, locking, or acknowledgment semantics.

let mut config = ProviderConfig::url(database_url);
config.long_poll.enabled = true;
let provider = PostgresProvider::new_with_config(config).await?;

The same configuration is supported for Entra-authenticated connections.

Implementation

• Register notification waiters before checking the queue to avoid lost wakeups.
• Track upcoming work in both queues, including delayed inserts and abandoned/released work.
• Rescan periodically to recover missed notifications, and resubscribe after detected connection failures.
• Bound timer storage and coalesce catch-up scans and repeated unsuccessful wake-triggered fetches to a 100 ms cadence. The first notification wake remains immediate.
• Preserve zero-timeout fetch behavior. The polling deadline bounds notification waiting; it never cancels a potentially mutating database fetch.
• Own listener/timer task lifetimes and safely discard cancelled refresh connections.
• Use table-identity-based channel names, avoiding PostgreSQL's notification-channel length limit.
• Require SQLx 0.8.3 for explicit listener reconnect handling; native-TLS configuration is unchanged.

Migration and activation

Migration 0027 installs disabled notification triggers. Notifications are activated schema-wide only when a provider is constructed with both MigrationPolicy::ApplyAll and long-polling enabled.

VerifyOnly performs no DDL and requires prior activation. Activation is atomic, follows acknowledgment lock ordering, and retries bounded lock contention. Disabling an individual provider's listener does not disable shared triggers.

Long-polling reserves one connection within the configured pool, so at least two connections are required. Client-only providers can leave it disabled.

This remains a normal forward migration: older binaries reject unknown migration versions even when notifications are disabled. It does not convert databases using the incompatible duroxide-pg-opt migration history.

Performance

Local release-mode measurements used PostgreSQL 17.11 without assertions or memory wiping, normal durability settings, and the same Duroxide 0.1.32 runtime configuration.

Measurement	Short polling	Long-polling
Idle provider SQL calls/s	39.6	0.308
Sparse one-activity workflow median latency	165 ms	19 ms
Loaded one-activity workflows/s	130	177
Loaded five-activity workflows/s	27	53

The loaded figures are medians across three repetitions. These are local engine-level measurements, not production capacity estimates. An ignored idle-query benchmark is included for reproduction.

Operational considerations

• Activated queue-writing transactions cannot use PREPARE TRANSACTION. Schemas that have not opted in retain their previous behavior.
• PostgreSQL serializes notification-emitting commits through a server-wide lock. WAL or synchronous-replication stalls can delay other notification writers, including in other databases. Loaded-write performance should be evaluated before activation.
• Detected disconnects recover automatically, but silent network partitions are not actively probed. Polling fallbacks remain available; SQLx listener cleanup can still wait indefinitely on a wedged socket. Proactive detection and bounded cleanup are deferred.
• Providers must be recreated if their queue tables are dropped and recreated.

[Copilot]

🔵 Needs a closer look

Unresolved wakeup gaps and schema-wide listener recovery require fixes and PostgreSQL-backed human validation.

2 open issues

• Notify orchestration waiters when releasing instance locks · New
• Wake consumers for timers maturing during catch-up scans · New

What changed in this PR

Adds opt-in notification-driven long polling to the PostgreSQL provider while preserving short polling by default.

Changes:

• Adds listener recovery, bounded timer tracking, and long-poll configuration.
• Installs disabled notification triggers with explicit schema-wide activation.
• Adds integration coverage, CI validation, and operational documentation.

File	Description
tests/​postgres_provider_test.rs	Enables validation in either polling mode.
tests/​long_poll_tests.rs	Adds integration tests and an idle-query benchmark.
src/​provider.rs	Integrates configuration and long-poll fetches.
src/​notifier.rs	Implements listeners, timers, recovery, and waiting.
src/​migrations.rs	Adds notification activation and verification.
src/​lib.rs	Exports and documents long-poll configuration.
README.md	Documents usage and operational limitations.
migrations/​README.md	Documents activation and test commands.
migrations/​0027_diff.md	Records migration changes.
migrations/​0027_add_long_poll_notifications.sql	Installs disabled triggers and updates cleanup.
docs/​migration-policy.md	Explains activation and compatibility.
CHANGELOG.md	Records the unreleased feature.
Cargo.toml	Raises the SQLx minimum to 0.8.3.
.github/​workflows/​test.yml	Runs validation with long polling enabled.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Schema-wide trigger activation and concurrency-sensitive listener, timer, and cancellation behavior warrant human review backed by PostgreSQL integration results.

No open issues 🎉

2 resolved since last review

• Wake consumers for timers maturing during catch-up scans
• Notify orchestration waiters when releasing instance locks

_{Review effort: Balanced}