[MayCXC]

Three memory errors in the static blob modules, one commit each. test_vec_static_blob_entries registers blobs and runs a KNN query, but over a 5-vector blob, where the overflow lands in the allocator's padding, and no run of the suite checks for leaks.

Leak of the registered names. Inserting a row into vec_static_blobs copies its name with sqlite3_mprintf() into the connection's registry, and the module freed the registry with sqlite3_free() alone when the connection closed, so every registered name leaked. The module's destructor now frees each registered name before the registry.

Overflow. vec_static_blob_entriesFilter() rounds the blob's vector count up to a multiple of 8 (bsize) for its candidate bitmap, then clears the bits past the last vector with a loop that starts at bsize itself, one bit past the bitmap's end, in every KNN query on a static blob. SQLite's allocator rounds the request up only in builds that keep memory statistics (the default), which hides the write there for small blobs; with SQLITE_DEFAULT_MEMSTATUS=0 it lands outside the allocation for any vector count, as mceachen found. At 64 vectors the bitmap is exactly 8 bytes, and AddressSanitizer reports a heap-buffer-overflow in bitmap_set() from the filter:

insert into temp.vec_static_blobs(name, data)
  select 'w', vec_static_blob_from_raw(?1, '<f4', 4, 64);  -- ?1: address of 64 float[4] vectors
create virtual table w using vec_static_blob_entries(w);
select rowid from w where vector match ? and k = 3 order by distance;

The loop now clears the bits from the vector count up to bsize.

Leak of a KNN's results. vec_static_blob_entriesFilter() allocates a KNN query's rowids and distances into the cursor's knn_data, and vec_static_blob_entriesClose() freed knn_data alone, so every KNN query on a vec_static_blob_entries table leaked both arrays. The close now calls sbe_query_knn_data_clear(), which was defined but not called, as photostructure/sqlite-vec@f56fdeb does.

A C program that registers a 64-vector blob, runs that query and closes the connection, built with -fsanitize=address against this tree:

• on main, AddressSanitizer stops at the heap-buffer-overflow in bitmap_set();
• with only the overflow fixed, LeakSanitizer reports 304 bytes in 3 allocations: the name from vec_static_blobsUpdate(), and the KNN's rowids and distances from vec_static_blob_entriesFilter();
• with all three commits it reports nothing and prints rows 32, 33 and 31.

test_vec_static_blob_entries_knn_with_whole_candidate_bytes runs the query from Python and checks its rows; against an AddressSanitizer build (LD_PRELOAD of libasan) it reports the overflow on main and passes here. The full suite passes (144 passed, 4 skipped, 259 snapshots), as do the C unit tests, and test-loadable.py passes against the AddressSanitizer build.

It merges cleanly with #9 through #14, #17, #18 and #20 through #24. The first two fixes landed on photostructure/sqlite-vec from photostructure#3, as photostructure/sqlite-vec@01c827e and photostructure/sqlite-vec@b02677e; that fork already had the third. asg017/sqlite-vec has removed the static blob modules, so these have no upstream twin.

[vlasky]

Thanks, all three fixes check out here: the off-by-one reproduces on main under AddressSanitizer as a one-byte read past the candidate bitmap for a 64-vector blob, the destructor is the right hook for the names, and sqlite3_memory_used() after close drops to the same constant as a connection that never touched the module.

One leak in the same family remains, and since this PR is titled "two leaks in static blobs" I would rather close it here than in a follow-up. vec_static_blob_entriesFilter() assigns a fresh knn_data without freeing the one the cursor already holds, and SQLite filters a cursor repeatedly without closing it: once per outer row when the table is the inner loop of a join, and once per value for k IN (1, 3) (sqlite3.h documents the latter at the sqlite3_vtab_in section). vec0Filter() handles this by calling vec0_cursor_clear() first; the static blob filter has no equivalent, and the new close frees only the last set. Measured on this branch: select q.id, w.rowid from q cross join w where w.vector match q.v and k = 2 with a three-row q leaves 608 bytes allocated after sqlite3_close().

I cannot push to your branch (your repository is a fork of asg017/sqlite-vec rather than of this one, so GitHub never grants the maintainer-edit permission here), so here is the change, for the top of vec_static_blob_entriesFilter() right after p is assigned, before the idxNum branch. With it the join scenario drops to zero bytes after close and the single-query, k = 0, 63, 64 and 65-vector cases are unchanged:

  // A cursor can be filtered again without being closed, as the inner loop
  // of a join or once per value of k IN (...): free the previous results.
  if (pCur->knn_data) {
    sbe_query_knn_data_clear(pCur->knn_data);
    sqlite3_free(pCur->knn_data);
    pCur->knn_data = NULL;
  }

If you would rather, I can put it on a branch here for you to pull, with a test that runs the k IN (1, 3) form.

Noted but not asked for: min_idx() still reads the uninitialised padding floats for a non-multiple-of-eight blob (masked by the bitmap check, so no visible effect). #18's heap replacement tests the bitmap first, which removes that read, so I am leaving it to #18.