Skip to content

Make memvfs shareable between threads behind a threadsafe feature - #204

Open
LucaCappelletti94 wants to merge 2 commits into
Spxg:masterfrom
LucaCappelletti94:feat/memvfs-threadsafe
Open

LucaCappelletti94 wants to merge 2 commits into
Spxg:masterfrom
LucaCappelletti94:feat/memvfs-threadsafe

Conversation

@LucaCappelletti94

@LucaCappelletti94 LucaCappelletti94 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Part of #202. Stacked on #205, so only 2172557 is added here.

memvfs keeps real SQLite lock levels per file, following the unix VFS, so connections sharing a database get SQLITE_BUSY on conflict. Two connections to one memvfs database used to share pages without any locking, which the docs called unsupported. This is the only change for builds without threadsafe.

With threadsafe, which now also turns on rsqlite-vfs/threadsafe, the file map and every file are guarded by SQLITE_MUTEX_FAST mutexes from SQLite's mutex API and shared through Arc, so those connections may live on different workers. install then requires the OsCallback to be Send + Sync, expressed by a new MemVfsOs trait that every OsCallback + 'static satisfies without the feature. The last VFS error becomes per thread, since xGetLastError names no connection, which pulls in std.

@Spxg

Spxg commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Thanks for the PR. But how does it work correctly on wasm? With SQLITE_THREADSAFE=0, I don't think the mutex-related symbols would be exported.

@Spxg

Spxg commented Oct 13, 2026

Copy link
Copy Markdown
Owner

The feature needs a SQLite built with SQLITE_THREADSAFE above 0, so sqlite-wasm-rs turns it on only in the follow-up that adds its own threadsafe feature.

Oh, I see. However, I'm curious whether it's possible to have a proper mutex implementation. Did you use futex-based mutexes or no-op mutexes in your tests?

@LucaCappelletti94

Copy link
Copy Markdown
Contributor Author

Oh, I see. However, I'm curious whether it's possible to have a proper mutex implementation. Did you use futex-based mutexes or no-op mutexes in your tests?

Futex-based, never no-op. The native tests here use libsqlite3-sys's bundled SQLite, built with SQLITE_THREADSAFE=1 and pthread mutexes.

For Wasm, the follow-up PR adds a futex-style mutex on memory.atomic.wait32 and makes it SQLite's default. Its worker tests fail with the no-op mutexes. It is hand-written because std and parking_lot sleep in memory.atomic.wait32 under contention without checking the thread, which traps on Chrome's main thread, and SQLite needs a recursive mutex that it enters and leaves in separate C calls.

I will open the stacked PR shortly so you can have a look, I have been trying to find either some seams or possible cleanups to make it smaller as for now it is ~1k lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants