Bring dead letter jobs to River UI by brandur · Pull Request #711 · riverqueue/riverui

brandur · 2026-10-24T17:53:57Z

We got a request in #706 that dead letter jobs should be in the UI, and
that is indeed very true. It's often quite convenient to pull up a UI
and retry a job from there instead of trying to write a one-off script
to do it programmatically.

Here, dead letters are added as a new entry on the left sidebar under
"Queues". I also thought about it integrating it directly into the
"Jobs" pane, but we'd need some new view in there as well, and I don't
think it's any better of a fit.

Like jobs, dead letters get a "filter" textbox at the top that lets you
select a variety of attributes to filter the view underneath in pursuit
of finding specific job(s).

The new pane is shown only for the Pro binary: in OSS, navigation is
hidden, access to the pane is blocked, and the API endpoints aren't
mounted.

Fixes #706.

brandur · 2026-10-24T17:59:10Z

@codex review

chatgpt-codex-connector · 2026-10-24T17:59:24Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-24T18:08:01.616719Z	`5a4f019`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

We got a request in #706 that dead letter jobs should be in the UI, and that is indeed very true. It's often quite convenient to pull up a UI and retry a job from there instead of trying to write a one-off script to do it programmatically. Here, dead letters are added as a new entry on the left sidebar under "Queues". I also thought about it integrating it directly into the "Jobs" pane, but we'd need some new view in there as well, and I don't think it's any better of a fit. Like jobs, dead letters get a "filter" textbox at the top that lets you select a variety of attributes to filter the view underneath in pursuit of finding specific job(s). The new pane is shown only for the Pro binary: in OSS, navigation is hidden, access to the pane is blocked, and the API endpoints aren't mounted. Fixes #706.

chatgpt-codex-connector · 2026-10-24T18:02:50Z

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 7f77265833

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

brandur · 2026-10-24T18:04:13Z

@codex review

chatgpt-codex-connector · 2026-10-24T18:08:00Z

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 5a4f01972c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

brandur · 2026-10-24T18:38:25Z

@bgentry Want to do a pass on this one? It'd be cool if we optimized to maybe get this one in, so maybe if you have major UI stuff you want to change you could just take the PR over and have Opus do it on your end.

bgentry

🤖 This review is automated, sent at Blake's request. I ran the branch locally against Postgres with seeded dead letters, including a 200k-row table for query plans.

The feature flag is wired correctly. dead_letter_jobs goes from Extensions() to /api/features, then to the sidebar and the route guard. Retry works end to end.

Two problems block the merge:

In light mode, most text in the new pane is white on white.
The list shows only the 200 oldest dead letters. Newer ones cannot be reached without a filter.

#712 is stacked on this branch and fixes most of what follows, one commit per fix. Each inline comment starts with its severity (Blocking, Major, Minor, or Nit) and ends with its status in #712.

Four things need your decision:

Do you want a pagination cursor? #712 only changes the sort to newest first, which matches the Jobs list.
Do you want a delete action here? The Pro driver already has JobDeadLetterDeleteByID.
The kind, queue, and tag filters need indexes that this repository cannot add.
Should the dead letter list and autocomplete queries move to the Pro driver? See the comment on the driver interface.

bgentry · 2026-10-24T19:02:50Z

+              <table className="w-full divide-y divide-slate-300 text-left text-sm dark:divide-slate-700">
+                <thead>
+                  <tr>
+                    <th className="px-3 py-3" scope="col">


🤖 Blocking: this text is invisible in light mode.
src/index.css:77 sets the default text color to near-white. These th and td elements set no color. In the light theme the browser computes rgba(255, 255, 255, 0.87) on a white background. Dark mode is correct. Loading… on line 58 has the same problem.
Fixed in #712: the list now uses the Jobs row layout, which sets colors for both themes.

bgentry · 2026-10-24T19:02:50Z

+        </Button>
+        <div className="flex flex-wrap items-center justify-between gap-4">
+          <div>
+            <h2 className="text-2xl font-semibold">{job.kind}</h2>


🤖 Blocking: same light mode problem. The kind heading and every dt and dd (lines 43 to 79) have no text color. Fixed in #712.

bgentry · 2026-10-24T19:02:50Z

+      <>
+        <TopNavTitleOnly title="Dead Letter Job" />
+        <div className="space-y-4 p-8">
+          <p>{error.message}</p>


🤖 Blocking: same light mode problem. This error message and Loading… on line 86 have no text color. Fixed in #712.

bgentry · 2026-10-24T19:02:51Z

+      AND (NOT jsonb_exists(@filters::jsonb, 'priorities') OR priority IN (SELECT value::smallint FROM jsonb_array_elements_text(@filters::jsonb -> 'priorities')))
+      AND (NOT jsonb_exists(@filters::jsonb, 'queues') OR queue IN (SELECT value::text FROM jsonb_array_elements_text(@filters::jsonb -> 'queues')))
+      AND (NOT jsonb_exists(@filters::jsonb, 'tags') OR tags && ARRAY(SELECT jsonb_array_elements_text(@filters::jsonb -> 'tags'))::varchar[])
+    ORDER BY id


🤖 Blocking: the list shows only the 200 oldest dead letters.
The query sorts by id ascending, the limit stops at 200, and there is no cursor. With more than 200 rows, an operator cannot see the newest failures without a filter. The Jobs list sorts finalized states newest first.
#712 sorts by id DESC in both drivers. It does not add a cursor. That is your decision.

bgentry · 2026-10-24T19:02:51Z

+FROM (
+    SELECT id, attempt, finalized_at, kind, max_attempts, queue
+    FROM /* TEMPLATE: schema */river_job_dead_letter
+    WHERE (NOT jsonb_exists(@filters::jsonb, 'ids') OR id IN (SELECT value::bigint FROM jsonb_array_elements_text(@filters::jsonb -> 'ids')))


🤖 Major: filtered queries read the full table, and the list polls every 2 seconds.
The table has only a primary key and a workflow index. If a filter matches fewer rows than the limit, Postgres reads every row. I ran EXPLAIN ANALYZE with 200k rows: an exact id lookup removed 200,043 rows by filter, because the OR stops Postgres from using the primary key. A kind filter did the same.
#712 fixes the ID lookup with a separate query that searches the primary key, on both drivers. The kind, queue, and tag filters still read the table. They need indexes from River Pro, or a slower refresh while filters are active.

bgentry · 2026-10-24T19:02:52Z

+	Priorities []int16  `json:"-" validate:"omitempty,max=4,dive,min=1,max=4"`
+	Queues     []string `json:"-" validate:"omitempty,max=100,dive,required,max=1024"`
+	Tags       []string `json:"-" validate:"omitempty,max=100,dive,required,max=1024"`
+	Limit      *int     `json:"-" validate:"omitempty,min=1,max=200"`


🤖 Nit: the limits 200, 20, and 100 and the facet strings repeat in the handler and both drivers. #712 names them in riveruidriver.

bgentry · 2026-10-24T19:02:52Z

+}
+
+// IDs are strings so JavaScript can preserve the full int64 range.
+type deadLetterJobListItem struct {


🤖 Nit: the JSON tags on the driver struct define the API response. #712 declares a separate response struct. The wire format does not change.

bgentry · 2026-10-24T19:02:52Z

+      toastSuccess({ message: "Job enqueued for retry", duration: 2000 });
+      await Promise.all([
+        queryClient.invalidateQueries({ queryKey: listDeadLetterJobsKey() }),
+        queryClient.invalidateQueries({ queryKey: ["listJobs"] }),


🤖 Nit: ["listJobs"] is a raw literal. #712 exports a prefix key from @services/jobs.

bgentry · 2026-10-24T19:02:52Z

+          search: inJobsPane
+            ? {}
+            : {
+                id: undefined,


🤖 Nit: this key list duplicates retainSearchParams in the Jobs route. #712 derives both from one list.

bgentry · 2026-10-24T19:02:52Z

 	JobCountEstimate(ctx context.Context, params *JobCountEstimateParams) (map[rivertype.JobState]JobCountEstimateResult, error)
+	// JobDeadLetterList returns at most Max jobs, ordered by ID ascending.
+	// Only Pro endpoints call this query; it requires the dead letter table.
+	JobDeadLetterList(ctx context.Context, params *JobDeadLetterListParams) ([]*JobDeadLetterListResult, error)


🤖 Major (design): these Pro-only queries are in the OSS driver.
JobDeadLetterList and JobDeadLetterAutocomplete are on the OSS Executor interface, with a stub schema for a table that River Pro owns. Only Pro endpoints call them. That is why the Pro handler must select an OSS UI driver for itself.
The Pro driver already has JobDeadLetterGetByID and JobDeadLetterGetAll. A filtered, limited list and the autocomplete query fit there. The Pro handler can then call a.DB as it does for get and retry, and the OSS interface loses both methods and the stub table.
This needs a River Pro release first. If you keep the queries here for now, choose the UI driver once when the handler is built and put it on APIBundle, so that no endpoint selects its own.

bgentry

@brandur on a separate note, the amount of queries being added here directly in riverui concerns me.

Shouldn't basic dead letter queue list APIs and filters be added to riverpro directly, rather than being built into a separate driver layer in riverui? Presumably anybody using Pro in any language might want these for i.e. a CLI, their own tooling etc.

Also note that some of these queries appear to be dangerously inefficient, and the riverpro repo is a better place to perf test them or add indexes via migrations if needed.

Reuse the Jobs layouts for consistent dead letter list and detail pages, with resilient filters, refresh errors, and not-found states. Show newest dead letters first, use primary-key ID lookups, and align query validation and autocomplete across Postgres and SQLite. Cancel superseded search suggestions and share common UI components.

brandur · 2026-10-24T19:18:34Z

@bgentry Definitely not against these functions being available in River Pro. It's a good idea even.

But I think you might be understating the amount of time and effort it is to get every changed query baked into River Pro before it can be used in River UI. You add a minimum of four and up to five additional PRs to review for every change (1 River Pro for change, 1 River Pro for release, 1 River Pro for CLI, 1 homepage, 1 River UI Pro version upgrade), necessitate a new River Pro release (which takes a minimum of an hour just by itself — the mandatory 10 minute wait thanks to the proxy is brutal, requiring constant trial/error, then you have Godoc + changelog), and then need a River Pro upgrade in this repo.

IMO we gotta be able to move a bit faster on this stuff. The query perf testing is just as easy from this repo as anywhere else if needed.

bgentry

Ok, I opened a riverpro PR to give us the option to add these APIs there now and #713 shows how that would clean up this PR. This doesn't need to block on it. However I do think we should think carefully about whether it's reckless to ship this UI as an auto-updating page given that it could be extremely slow for larger customers (hundreds of milliseconds, lots of DB load). A separate open PR adds indexes to make those concerns vanish, at the cost of another pro migration.

Up to you on how to proceed!

brandur · 2026-10-24T20:45:30Z

@bgentry K sounds good. Just on that last point though — remember that bringing back any query optimizations from Pro to this project would take an LLM about a minute (>60x faster than the Pro update and release process at minimum).

And if we do really get another Pro release cut very soon here, then it's pretty fast to update River UI and bump it with an updated PR too.

brandur · 2026-10-24T20:46:20Z

@bgentry Do you want to merge the stack when you're satisified with the new UI (or is it good now?)?

bgentry · 2026-10-24T20:51:27Z

I already merged my fixes into here so you should be good to go. I'm not sure if there were query perf fixes in the Pro APIs I added, but it did say it fixed some driver compatibility issues that were caught by Pro's larger test matrix. The main perf issue requires indexes to fix which is why I was saying we might want to make this one not auto refresh until a fix for that is shipped.

There's some concern around query performance for dead letter jobs. Just so that we can get this shipped out without too much ado, here we disable automatic refreshes for dead letter jobs. Automatic refresh for normal jobs continues as normal. Users can enable dead letter refresh with the toggle (it's just off by default).

brandur · 2026-10-24T23:43:20Z

Cool! Alright I just pushed one more patch that turns dead letter job automatic refresh off by default. Normal job auto-refresh stays on, and users can enable automatic refresh for dead letter jobs by opting in with the control.

One of the PRs that came into #711 (dead letter jobs) added a bunch of "fixed" entries in the changelog for what it was doing, but this doesn't make sense because of all #711 is a brand new feature so it didn't exist before and there was nothing to fix. Here, strip those back out again. I've also amended the release [1]. [1] https://github.com/riverqueue/riverui/releases/tag/v0.22.0

brandur force-pushed the brandur-dead-letters-in-ui branch from ed2a325 to 7f77265 Compare October 24, 2026 17:57

brandur force-pushed the brandur-dead-letters-in-ui branch from 7f77265 to 5a4f019 Compare October 24, 2026 18:02

brandur requested a review from bgentry October 24, 2026 18:37

brandur mentioned this pull request Oct 24, 2026

Add Dead letter queue support #706

Closed

bgentry mentioned this pull request Oct 24, 2026

Fix dead letters pane issues found in review #712

Merged

bgentry reviewed Oct 24, 2026

View reviewed changes

bgentry mentioned this pull request Oct 24, 2026

Use Pro APIs for dead letter jobs #713

Draft

bgentry approved these changes Oct 24, 2026

View reviewed changes

brandur merged commit bbf43bb into master Oct 24, 2026
24 checks passed

brandur deleted the brandur-dead-letters-in-ui branch October 24, 2026 23:54

brandur mentioned this pull request Oct 25, 2026

Remove "fixed" entries in changelog related to dead letter #716

Merged

Conversation

brandur commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Oct 24, 2026

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 24, 2026

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

bgentry left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bgentry left a comment

Choose a reason for hiding this comment

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

bgentry left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

bgentry commented Oct 24, 2026

Uh oh!

brandur commented Oct 24, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

brandur commented Oct 24, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 24, 2026 •

edited

Loading

bgentry left a comment •

edited

Loading