You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b02677e
Browse filesBrowse the repository at this point in the historyBrowse files
fix(sqlite-vec.c): keep a static blob KNN inside its candidate bitmap
vec_static_blob_entriesFilter() rounds the blob's vector count up to a
multiple of 8 (bsize) for its candidate bitmap, then clears the bits
past the last vector with a loop that starts at bsize itself, one bit
past the bitmap's end. When the vector count fills the bitmap's bytes
exactly, as 64 vectors do, SQLite's allocator has no padding to absorb
the write, and AddressSanitizer reports a heap-buffer-overflow in
bitmap_set() from the filter.
The loop now clears the bits from the vector count up to bsize. The
memory test's static blob runs a KNN query over its 64 vectors; under
`make test-asan` it reports the overflow without the fix and passes
with it. test-loadable.py checks the same query's rows; against the
AddressSanitizer build it also reports the overflow without the fix.
Copy file name to clipboardExpand all lines: CHANGELOG.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,6 +12,7 @@
12
12
13
13
- A KNN query that takes its query vector, `k`, or `mmr_lambda` from another table of a join now uses it. `select v.rowid from p join v on v.embedding match ? and v.k = p.k` failed with `A LIMIT or 'k = ?' constraint is required`; the same join with `v.k = 3 and v.mmr_lambda = p.lambda` returned no rows; and `select v.rowid from v join p on p.id = v.tag where v.embedding match p.vector and v.k = 2`, with `p.id` an integer primary key, could return no rows. These joins now run one KNN per row of the other table, so a filter on the `vec0` table whose value comes from that table applies before the k nearest rows are chosen. Ported in part from the unmerged [vlasky#14](https://github.com/vlasky/sqlite-vec/pull/14) ([asg017#331](https://github.com/asg017/sqlite-vec/pull/331)), which also applies filters whose value comes from a joined table before the k nearest rows are chosen; that part is not ported, and ARCHITECTURE.md's idxStr section says why.
14
14
- Fixed a memory leak: closing a connection freed the `vec_static_blobs` registry but not the name of each blob registered in it.
15
+
- Fixed a write one byte past the end of a heap allocation in KNN queries on a `vec_static_blob_entries` table. SQLite's allocator rounds the allocation up, which hides the write unless the blob's vector count rounds up to a multiple of 64, as 64 vectors do.
0 commit comments