[bgentry]

The Ruby scheduler opens one savepoint for each due job, up to 1,000 per pass. It uses the savepoint to catch a unique key conflict, roll back that row, and mark the row discarded. Go's scheduler finds conflicts without savepoints, and since #1420 Go's transaction helpers join a caller's open transaction instead of opening a savepoint. This change does the same in Ruby.

Savepoints have a cost in Postgres. A savepoint gets its own transaction ID when it first writes or locks a row. Each backend caches at most 64 of these IDs. A released savepoint keeps its entry until the top-level transaction ends. A savepoint rolled back after a conflict frees its entry. A scheduler pass that makes more than 64 jobs available therefore overflows the cache. Until that transaction ends, snapshots that any session takes are marked as overflowed. Their visibility checks on rows that recent transactions wrote must then look up pg_subtrans. This can slow queries across the server.

Go finds conflicts before it writes. Its JobSchedule query locks the due rows, numbers the rows that share a unique key (jobs_with_rownum), finds keys that another row already holds (unique_conflicts), and sets each row to available or discarded, all in one statement. Ruby now uses an equivalent query with the same row locks. A locked due row now makes the scheduler wait instead of skipping the row, as in Go. On SQLite, Ruby follows Go's SQLite driver and checks each unique job for a conflict before it writes. The outcome matches Go: the same rows become available, the same rows are discarded, and discarded rows get unique_key_conflict: scheduler_discarded metadata. The pass now also notifies a queue whose only changed job is discarded, as Go does.

The driver transaction helpers also open a savepoint inside an application transaction: Sequel with savepoint: true and Active Record with requires_new: true. An insert opens two. Both helpers now join the open transaction, like Go's WithTxV. If a River call raises, the application must roll back, or wrap the call in its own savepoint to recover and continue. A rollback_exception raised in Driver#transaction now reaches the caller's enclosing transaction block, as a rollback raised in the caller's own code would. Active Record swallows an ActiveRecord::Rollback raised in a joined block, so the Active Record helper yields directly and the rollback reaches the transaction's owner.

River still follows the application's own transaction settings. A non-joinable Active Record transaction (such as a Rails transactional test fixture) or a Sequel transaction with auto_savepoint still gets a savepoint, as any nested block would. RIVER_USE_LEGACY_SUBTRANSACTIONS=1 (or true) restores the old savepoints, matching Go's temporary fallback. The README states only the contract: River calls join an open transaction, and the application owns any savepoint. The changelog notes the fallback.

New specs run a scheduler pass with both kinds of conflict inside a caller's transaction, for Sequel and Active Record on Postgres and SQLite. They assert the outcome that Go produces and that no savepoint opens. On Postgres, they also check that each changed row has the caller's transaction ID as its xmin. Another spec runs the other transactional operations in a caller's transaction and asserts that none opens a savepoint.

[brandur]

Nice. Yeah it's key to get rid of these with great prejudice.