improvement(knowledge): finish Confluence passes past unreadable attachments, rank unauthenticated keyword search in one statement

[waleedlatif1]

Summary

Confluence: a 403/404 on one parent's attachments no longer fails the listing

• listConfluenceAttachments used to record attachment_access_denied (403/404 on /pages/{id}/attachments) as a listing failure. Any pass with one permanently restricted page therefore stayed incomplete: the sync was partial, lastSyncAt never advanced, and every slice recounted the whole connector.
• That parent is now skipped and logged as a warning. The page still returns reconciliationSafe: false and sets syncContext.reconciliationUnsafe. Because checkpoint.unsafe stays set for the rest of the pass, reconciliation never deletes attachments it did not list. This is the same deletion behavior as before.
• Missing scope (401 attachment_scope_mismatch) and server failures (attachment_listing_unavailable, plus the 3-in-a-row abort) are still listing failures. A failure listing the pages themselves still throws.

Connector document count

• The member engine now uses the same gate the content engine has had since improvement(db): cut hot-path connector count and rate-limit lock costs #8944. Once a pass has finished, an unfinished slice keeps the recorded count instead of scanning the whole connector. Until a pass finishes, every slice still counts, so the first-pass fix from ed2e401 still holds.
• countLiveConnectorDocuments and the gate (keepsRecordedDocCount) moved to sync-primitives so both engines share them.
• With the Confluence fix, a connector stuck on restricted attachments now finishes passes, so the existing gate applies to it.

Keyword search for callers with no person behind them (workflow, API, Copilot)

• These callers now rank through keywordCandidateRankingQuery, the materialized match → readable documents → rank statement signed-in readers already use. The old ORDER BY ts_rank_cd over a per-chunk embedding ⋈ document join (with a per-base fan-out for many bases) is gone. Over 7 days in prod the old shape ran p50 ~400 ms / p99 ~9.2 s; the CTE shape ran p50 ~27 ms / p99 ~1.2 s.
• The ACL predicate is the same read.rankCondition, and hydration is unchanged. The ordering is still a global ts_rank_cd DESC over every readable match. Ties are now broken by chunk id instead of arbitrarily. The candidate side also requires document.knowledge_base_id to be in the searched bases, which only makes the filter stricter.
• The search budget is deliberately unchanged: these callers still run unbudgeted. Adding a deadline would turn slow searches into partial results for API callers, which is a behavior change.

Not changed (deliberately)

• I did not add a "skip the count when a slice changed no membership" gate for connectors that have never finished a pass. I could not prove it exact. Restores (deletedAt: null on relisted tombstones) and resurrecting updates are not counted separately. Out-of-band excludes, deletes and restores never touch lastSyncDocCount. Without a recorded count timestamp there is also no bounded-staleness variant.

Test plan

• attachments.test.ts: a 403/404 parent is skipped, the listing stays complete, no listingFailures, and it is still reconciliation-unsafe. Fails on the old code.
• member-sync-engine-content-credential.test.ts: an unfinished slice after a finished pass writes no lastSyncDocCount. Fails on the old code.
• Focused suites pass: sync-engine, member-sync-engine (both files), confluence, search queries, application search.
• CI: type-check, audits, integration (filtered-search / scale cover keyword search end to end).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
docs	Skipped		Oct 13, 2026 6:40pm UTC

[cubic-dev-ai]

No issues found across 7 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic</sub>

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues were found.

Summary

This PR lets Confluence syncs finish despite unreadable attachments, shares the document-count gate between sync engines, and ranks keyword matches in one statement for callers without a signed-in reader.

• Confluence sync skips parents whose attachments return 403 or 404.
• Unfinished sync slices reuse a saved document count.
• Keyword search ranks readable matches across searched bases in one query.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Search query] --> B[Find matching chunks]
  B --> C[Keep readable documents]
  C --> D[Rank keyword matches]
  D --> E[Load selected chunks]
  E --> F[Combine search results]
  G[Old ranking query] --> H[Compare scores and tied groups]
  F --> H

Loading

_{Reviews (2) · Last reviewed commit: "test(knowledge): compare single-statemen..." · Reviewed by Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

No issues found across 8 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic</sub>