You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add a threadsafe feature for SQLite across workers sharing one memory - #205
Part of #202. Adds a threadsafe feature that compiles SQLite with SQLITE_THREADSAFE=1 and gives it a mutex that works across workers sharing one memory. The memory VFS stays single-threaded until #204.
SQLITE_OS_OTHER builds otherwise get no-op mutexes, so shim/threadsafe.h redirects SQLite's sqlite3DefaultMutex() to the mutex in src/mutex.rs. It spins on Wasm atomics, then sleeps in memory.atomic.wait32 where the thread may block. It is hand-written because std and parking_lot sleep in memory.atomic.wait32 under contention without checking the thread, which traps on Chrome's main thread, and SQLite needs a recursive mutex that it enters and leaves in separate C calls.
Whether a thread may block depends on the running module, since --shared-memory needs +atomics and +atomics alone still links an unshared memory. A sixth host hook, rust_sqlite_wasm_host_can_block, answers it, and the wasm-bindgen adapter tries a zero-timeout Atomics.wait on the module's own memory, which throws on unshared memory and on threads that may not block.
Thanks for your PR! I think the mutex implementation could be split into a separate PR, then VFS.
Also, we should take a closer look at the relationship between threadsafe, atomics, and --shared-memory. That's actually why I submitted #207, since it relies on --shared-memory rather than atomics.
I split #205 as suggested. It now holds only the threadsafe feature and the mutex, rebased on #207, and #204 sits on top with the memvfs changes and the worker tests that share memvfs files. The worker tests here use :memory:, which never touches memvfs's files.
My understanding is that wasm-ld refuses --shared-memory unless the Rust objects are built with +atomics, and +atomics without --shared-memory links an unshared memory. So atomics is a precondition for threads, and whether memory is shared is known only to the running module.
can_block used to wait on a private SharedArrayBuffer, so a worker in an +atomics build without --shared-memory was told it could wait and would trap in memory.atomic.wait32 under contention. It now waits on the module's own memory, threadsafe_can_block checks that a worker may block exactly when memory is shared, and CI also runs it on an +atomics build without --shared-memory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #202. Adds a
threadsafefeature that compiles SQLite withSQLITE_THREADSAFE=1and gives it a mutex that works across workers sharing one memory. The memory VFS stays single-threaded until #204.SQLITE_OS_OTHERbuilds otherwise get no-op mutexes, soshim/threadsafe.hredirects SQLite'ssqlite3DefaultMutex()to the mutex insrc/mutex.rs. It spins on Wasm atomics, then sleeps inmemory.atomic.wait32where the thread may block. It is hand-written becausestdandparking_lotsleep inmemory.atomic.wait32under contention without checking the thread, which traps on Chrome's main thread, and SQLite needs a recursive mutex that it enters and leaves in separate C calls.Whether a thread may block depends on the running module, since
--shared-memoryneeds+atomicsand+atomicsalone still links an unshared memory. A sixth host hook,rust_sqlite_wasm_host_can_block, answers it, and thewasm-bindgenadapter tries a zero-timeoutAtomics.waiton the module's own memory, which throws on unshared memory and on threads that may not block.