[MayCXC]

A KNN query whose k comes from another table fails, and one whose query vector comes from it can return no rows:

create virtual table v using vec0(embedding float[2] distance_metric=l2, tag integer);
insert into v(rowid, embedding, tag) values
  (1, '[1,0]', 1), (2, '[2,0]', 1), (3, '[3,0]', 1),
  (4, '[4,0]', 2), (5, '[5,0]', 2), (6, '[6,0]', 2);
create table categories(id integer primary key, q text, n integer);
insert into categories values (1, '[0,0]', 2), (2, '[0,0]', 2);

select c.id, v.rowid from v join categories c on c.id = v.tag
where v.embedding match '[0,0]' and v.k = c.n;
-- A LIMIT or 'k = ?' constraint is required on vec0 knn queries.
select c.id, v.rowid from v join categories c on c.id = v.tag
where v.embedding match c.q and v.k = 2;
-- no rows

vec0BestIndex skips an unusable constraint, and when a KNN argument reads another table of the query, SQLite also weighs plans that scan vec0 before that table, in which the argument is unusable. A k taken from a one-row table fails the same way in either FROM order.

When a MATCH on a vector column is present, vec0BestIndex now returns SQLITE_CONSTRAINT for a plan in which the MATCH, k or any hidden parameter column declared after k is unusable, the way json_each refuses a plan that cannot supply its argument, so the planner takes one that gives vec0 every argument. Both queries above then return the 2 nearest in each category, (1, 1), (1, 2), (2, 4), (2, 5): with categories read first, v.tag = c.id is usable too.

Filters on rowid, partition key, metadata and distance columns are left to the plan, as in @mceachen's port, photostructure/sqlite-vec@64db37c. vec0 cannot tell a lookup join from a filter join, and refusing the plans without such a filter turns his lookup from the 3 nearest rows into 3 rows per category. ARCHITECTURE.md's idxStr section says when a filter applies before the k all the same: the other table supplies a KNN argument, a CROSS JOIN names it first, vec0 is on the right of a LEFT JOIN with the filter in ON, or the KNN is a correlated subquery whose filter reads the outer query's columns.

tests/test-knn-joined-constraints.py takes the query vector and k from another table in both FROM orders, checks that a table that supplies one of them filters before the k, and that a lookup join returns k rows in either FROM order. 6 of its 12 cases fail on main and all pass here. The full suite passes (251 passed, 66 skipped, 204 snapshots).

The same change for vlasky/sqlite-vec is vlasky#14.

[mceachen]

Ported only the argument checks from vlasky#14
in photostructure@64db37c.
The filter checks change a category-name lookup from 3 nearest rows overall
to 9 (3 per category) in our SQLite 3.54.1 test. Reproducer: vlasky#14 (comment)

[MayCXC]

This now carries only the argument checks, as in your port (141abec): vec0BestIndex refuses a plan when the MATCH or k is unusable and leaves filters to the plan, so your category lookup returns 3 rows in either FROM order here (SQLite 3.46.1). The tests take the query vector and k from another table, also through a primary-key lookup on a metadata and a partition key column, and run your lookup in both FROM orders on both columns; 6 of the 12 cases fail on 04d28bd.