Skip to content

improvement(db): disable embedding FTS fastupdate and index delete-action foreign keys - #8943

Merged
waleedlatif1 merged 1 commit into
stagingfrom
perf/gin-fastupdate-fk-indexes
Oct 12, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
perf/gin-fastupdate-fk-indexes

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • emb_content_fts_idx: SET (fastupdate = off), then a bounded gin_clean_pending_list drain. This is the 0364 / fix(file-search): avoid foreground GIN cleanup stalls #7995 pattern. With fastupdate on, the insert that pushes the shared pending list past gin_pending_list_limit merges the whole list into the index in the foreground (ginInsertCleanup with work_mem). It does this while holding the document FOR UPDATE row lock. A 2-row chunk insert took 8.6s and the slowest insert this week took 53s, with no lock waits logged for those statements.
  • Add concurrent indexes for the referential actions on user and workspace deletion. Without them, every deletion runs a sequential scan inside the deleting transaction:
    • document.uploaded_by (SET NULL, partial IS NOT NULL): p50 24s per deletion
    • execution_large_values.workspace_id (CASCADE): p50 21s. Its cleanup indexes lead with workspace_id but are partial on deleted_at, so workspace_id = $1 cannot use them.
    • user_table_rows.created_by (SET NULL, partial IS NOT NULL): p50 10s
    • user_table_definitions.created_by (CASCADE): about 100ms
    • Other writers wait behind the deleting transaction's row locks. One upsert waited 14.6s.
  • schema.ts mirrors all five changes. drizzle-kit generate produces nothing new after 0410.

Side effects

  • Insert latency: each chunk now inserts its lexemes (about 120 on average) straight into the index. This is a bounded cost per row instead of an occasional merge of every writer's pending entries, so the median goes up a little and the multi-second stalls go away. Slow large batches whose time scales with row count come from the synchronous search projection, not this index, and are out of scope here.
  • WAL: with fastupdate on, every entry is written to WAL twice: once into the pending list and again when merged. Direct insertion writes it once. The one-time drain is the same as one routine flush. The index builds only scan the heap (read-only) and write their own small btree pages: two are partial on mostly-null columns, and the other two dedup on low-cardinality keys.
  • Locks (PG17): ALTER INDEX ... SET (fastupdate) takes AccessExclusiveLock on the index only (checked in pg_locks; the table is not locked). That still blocks writes to embedding and planning of queries on it. It is bounded by the 5s lock_timeout plus the time-budgeted retry in migrate.ts. gin_clean_pending_list takes RowExclusiveLock and does not conflict with traffic. CREATE INDEX CONCURRENTLY takes ShareUpdateExclusiveLock and waits for older transactions to finish.
  • Replay: everything after COMMIT; is idempotent. A second run is a no-op ALTER, an empty drain, and drop-and-recreate of only the new indexes (the 0407/0408 convention, which recovers an INVALID build).
  • Compatibility: purely additive. The deployed app issues no queries that depend on these. Search semantics are unchanged, because fastupdate only changes when entries reach the main tree. HOT is unaffected, because the app never updates the newly indexed columns.
  • Excluded: workspace_file_search_segment_workspace_content_trgm_idx. Nothing in the app inserts into that table anymore, so its write path has no tail to fix.

Type of Change

  • Improvement (performance)

Testing

  • Ran on a throwaway PG17 cluster:
    • Applied the 0410 statements twice in a row; both runs succeeded. The first drain flushed pending pages and the second was a no-op.
    • Confirmed the ALTER INDEX lock level.
    • Used auto_explain with nested statements, including with plan_cache_mode = force_generic_plan, to confirm that all four RI action queries (... WHERE $1 = col) use the new indexes, the partial ones included.
  • bunx drizzle-kit generate: no schema changes
  • bun run check:migrations origin/staging: passes
  • check:schema-mock and check:comment-hygiene: pass
  • biome on schema.ts: clean
  • Full lint, audits, type-check and tests are left to CI.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…tion foreign keys

Inserts into embedding could stall for seconds merging the shared GIN pending
list in the foreground, while holding the document row lock. Turn fastupdate
off (the 0364 pattern) and drain the existing list once.

User and workspace deletion ran sequential scans for four referential actions
inside the deleting transaction, holding row locks other writers queued behind.
Add concurrent indexes backing document.uploaded_by,
user_table_rows.created_by, user_table_definitions.created_by and
execution_large_values.workspace_id.
@vercel

vercel Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 12, 2026 9:26pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Critical impact] The PR appears safe to merge; no actionable issues were found.

Summary

Disables fastupdate on emb_content_fts_idx, drains pending entries with a five-minute limit, and adds four indexes for user and workspace deletion.

  • Embedding text-search writes add entries directly to the GIN index.
  • User and workspace deletes find related rows through indexes.

Intentional tradeoffs acknowledged by waleedlatif1: slightly higher median insert cost, waits for older transactions during concurrent builds, and rebuilding the new indexes on replay. These were not reported as defects.

Reviews (1) · Last reviewed commit: "improvement(db): disable embedding FTS f..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit bf285f0 into staging Oct 12, 2026
48 checks passed
@waleedlatif1
waleedlatif1 deleted the perf/gin-fastupdate-fk-indexes branch October 13, 2026 01:59

This branch was successfully deployed

1 active deployment
Preview — 97558cd7 Deployed Oct 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant