Repository navigation
Conversation
This was referenced Oct 22, 2026
min_idx() chose the k nearest candidates of a chunk by scanning the whole chunk once per result, k times n comparisons. With the default chunk size of 1024, a KNN query with k of 1024 or more spent about a million comparisons per chunk: on an in-memory 10,000-row float[768] table, k = 1000 and k = 4096 took 17.9 and 18.1 ms against 9.0 ms for k = 1. Upstream has the fix behind a build option: asg017/sqlite-vec 0de765f added SQLITE_VEC_EXPERIMENTAL_MIN_IDX, a min_idx() that keeps the k best candidates in a binary heap in the output array, whose root is the one that ranks last, so a chunk costs n log k comparisons, then sorts the heap in place. That heap compares distances alone, so tied rows come back in another order than the scan's: built with the option, 779 of 1,000 tie-heavy queries on asg017's main return other rows or another order. This makes that heap the only min_idx() and gives it one total order, min_idx_after(): by distance, between equal distances the larger index first, as the scan's <= chose, and a NaN distance after every number instead of wherever its position put it. vec0 refuses NaN and infinite elements and zero vectors under cosine distance, but a table written before v0.2.6-alpha, which began refusing NaN, can still hold one, and a static blob reads the application's memory as it is. The bitmap of taken slots the callers allocated for the scan goes with it, from the KNN loop, the static blob filter, and the declarations in tests/sqlite-vec-internal.h and tests/unittest.rs. merge_sorted_lists(), which merges each chunk's nearest rows into the ones kept from the chunks before it, ranks a NaN after every number too, as mceachen's photostructure/sqlite-vec@a791836 does in that fork: it took a later chunk's NaN ahead of the numbers still unmerged. Measured on that table (medians of 5 interleaved runs of 7 queries), k = 1000 and k = 4096 take 9.6 and 9.4 ms, k = 100 8.9 ms instead of 9.9, and k = 1 9.3 ms against 9.0. Over 4,000 random KNN queries on tables built to tie heavily (vectors on a 5 by 3 grid, chunk sizes 8 to 64, deletes, metadata filters, k from 1 to 1000), every result matches the previous selection row for row; with the tie rule reversed, 827 of 1,000 such queries differ. tests/test-knn-topk.py checks the k nearest against a brute-force order for k below, at and above the chunk size, with and without a filter, pins the order of tied rows, checks that a static blob vector with a NaN element comes back last (the scan returned it first, from slot 0), and, with mceachen's test from a791836, that a NaN patched into a stored chunk comes back behind the earlier chunk's rows, which fails on main.
MayCXC
force-pushed
the
knn-topk-heap-vlasky
branch
from
October 23, 2026 16:20
8931f4f to
22791f5
Compare
This was referenced Oct 24, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
min_idx()chose the k nearest candidates of a chunk by scanning the whole chunk once per result, k times n comparisons. With the default chunk size of 1024, a KNN query with k of 1024 or more spends about a million comparisons per chunk. On an in-memory 10,000-rowfloat[768]table (medians of 5 interleaved runs, each the median of 7 queries):asg017/sqlite-vec has the fix behind a build option: asg017/sqlite-vec@0de765f added
SQLITE_VEC_EXPERIMENTAL_MIN_IDX, amin_idx()that keeps the k best candidates in a binary heap in the output array, whose root is the one that ranks last, so a chunk costs n log k comparisons, then sorts the heap in place. That heap compares distances alone, so tied rows come back in another order than the scan's: built with the option, 779 of 1,000 tie-heavy queries on asg017's main return other rows or another order than its default build.This ports that heap as the only
min_idx()and gives it one total order,min_idx_after(): by distance, between equal distances the larger index first, as the scan's<=chose, and a NaN distance after every number instead of wherever its position put it. Avec0table refuses NaN and infinite elements and zero vectors under cosine distance, but one written before v0.2.6-alpha, which began refusing NaN, can still hold a NaN, and a static blob reads the application's memory as it is. The bitmap of taken slots the callers allocated for the scan goes with it, from the KNN loop, the static blob filter, and the declarations intests/sqlite-vec-internal.handtests/unittest.rs.merge_sorted_lists(), which merges each chunk's nearest rows into the ones kept from the chunks before it, ranks a NaN after every number too, as photostructure/sqlite-vec@a791836 by mceachen does in that fork. It took a later chunk's NaN ahead of the numbers still unmerged; mceachen's test from that commit, which patches a NaN into a stored chunk to stand in for an older table, fails on main and on this PR's earlier head.To check the order is unchanged, I ran 4,000 random KNN queries against main and this branch on tables built to tie heavily (vectors on a 5 by 3 grid, chunk sizes 8 to 64, deletes, metadata filters, k from 1 to 1000), each build in its own process: every result matched row for row. With the tie rule reversed, 827 of 1,000 such queries differ.
tests/test-knn-topk.pychecks the k nearest against a brute-force order for k below, at and above the chunk size, with and without a filter, pins the order of tied rows, checks that a static blob vector with a NaN element comes back last (on main the scan returns it first, from slot 0), and that a stored NaN in the second chunk comes back behind the first chunk's rows. The suite passes (147 passed, 4 skipped), andtest-knn-topk.pypasses against an AddressSanitizer and UndefinedBehaviorSanitizer build; before the merge change the C unit tests, the Rust harness (cargo test --manifest-path tests/Cargo.toml) andtest-loadable.pyunder AddressSanitizer passed too, and the change touches none of what they test.It merges cleanly with #9 through #14 and with #17 and #19. The same change, with the merge fix as mceachen's a791836, is on photostructure/sqlite-vec's main (from photostructure#2), and it is open upstream as asg017#337, which makes the build option's heap the default.