Repository navigation
Fix signedness in NEON int8 L1 distance - #322
Open
ariannamethod wants to merge 1 commit into
Open
ariannamethod wants to merge 1 commit into
ariannamethod wants to merge 1 commit into
Conversation
This was referenced Oct 23, 2026
MayCXC
added a commit
to MayCXC/sqlite-vec
that referenced
this pull request
Oct 23, 2026
The Makefile enabled NEON only on macOS arm64, so a Linux arm64 build, the linux-aarch64 release binary among them, ran every distance's scalar loop. NEON is part of the arm64 baseline: GCC and Clang define __ARM_NEON for aarch64 with no -m flag, so the Linux block needs only the define, beside its AVX check and inside its guard against Android cross-compiles. OMIT_SIMD=1 still builds without it. The define follows the compiler's target, `$(CC) -dumpmachine`, not uname, as mceachen's photostructure/sqlite-vec@350c67a has it: 32-bit Raspberry Pi OS runs a 64-bit kernel, so uname says aarch64 while gcc targets 32-bit ARM, which lacks the AArch64-only intrinsics the NEON kernels use. Under an aarch64 kernel, `make loadable CC=arm-linux-gnueabihf-gcc` fails when uname decides (implicit declaration of vaddvq_f32, unknown type name float64x2_t) and builds a 32-bit ARM extension when the compiler does; for gcc and clang targeting aarch64 the build commands are the same either way. GCC cannot build the NEON code on main, because l1_int8_neon() relies on implicit signed/unsigned vector conversions; this builds on asg017#322, which makes them explicit. Built with GCC 14 at -O3 on Linux arm64, a KNN query on a 100,000-row float[768] table takes 19.6 ms instead of 59.2 ms for k = 10 and 24.2 ms instead of 65.8 ms for k = 100 with the L2 metric, and 17.5 ms instead of 85.3 ms and 23.3 ms instead of 90.6 ms with cosine (medians of 20 queries, each build in its own process: the extension exports its internal functions, so a second build loaded into the same process runs the first one's KNN code). L1 and Hamming distances were bit-identical to the scalar loops' on 2,000 random pairs at each of six lengths from 8 to 1,536, though the float32 L1 kernel adds in another order than the scalar loop, both in double, so the two can differ in the last digit on other inputs. Float32 L2 and cosine distances differ in the last digits, since the NEON kernels split their sums across 16 partial sums where the scalar loops keep one; for L2 the NEON result is closer to the exact value, a mean relative error of 3.0e-8 instead of 1.7e-7 at 768 dimensions. So a KNN query can order rows with nearly equal distances differently, as the macOS arm64 build already does; 4 of the 2,000 ranks of the k = 100 L2 queries swapped such rows. Int8 cosine distances can differ in the last digit as well, since cosine_int8_neon() finishes in float and the scalar loop in double. The Test workflow's build-linux-aarch64-extension job already runs the suite on an arm64 runner, so it now tests the NEON build. Locally the suite passes on it (478 passed, 132 skipped, 204 snapshots).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #299.
Reinterpret absolute int8 differences as unsigned before widening, and use unsigned NEON accumulators and reduction. Preserves differences up to 255. Adds boundary tests for vector blocks and scalar leftovers.
Testing: