Skip to content

druntime getTLSRange uses static %fs:0 offset for Bionic shared libraries (x86/x86_64), so the GC never scans TLS #5311

Open
Open
druntime getTLSRange uses static %fs:0 offset for Bionic shared libraries (x86/x86_64), so the GC never scans TLS#5311

Description

@PetarKirov
PetarKirov
opened on Oct 23, 2026

Summary

With LDC 1.42.0, a D shared library for Android (Bionic) built with
-shared -link-defaultlib-shared=false gets a GC that never scans the
library's real TLS block on x86/x86_64. GC allocations referenced only
from thread-local variables are collected while still in use.

Root cause

In druntime rt/sections_elf_shared.d (as of v1.42.0), getTLSRange
takes a static-offset shortcut:

version (Shared) {} else version (linux) version (X86_Any)
    version = Static_Linux_X86_Any;
// ...
version (Static_Linux_X86_Any)
{
    // tcb = %fs:0 (or %gs:0)
    void* start = tcb - sz;
    return start[0 .. sz];
}
else
{
    auto ti = tls_index(mod, 0);
    return (__tls_get_addr(&ti) - TLS_DTV_OFFSET)[0 .. sz];
}

The tcb - sz computation assumes initial-exec TLS laid out directly
before the thread control block. That holds for an executable, but not
for a dlopen'd shared library under Bionic, whose TLS blocks are
dynamic (allocated through the DTV, addressed via __tls_get_addr).
A static druntime linked into an Android .so is exactly the
non-Shared, Linux, x86_64 configuration that selects the shortcut —
so the GC scans sz bytes before the TCB instead of the library's
TLS, and every GC root that lives only in that TLS is invisible.

aarch64 is unaffected: Static_Linux_X86_Any is x86-only, so it takes
the __tls_get_addr(&ti) branch, with mod from dlpi_tls_modid.

Observed effect

In sparkles, a
thread-local cache of GC-allocated scratch arrays
(private CellWrapPool cellWrapPool;) had its buffers swept between
calls; the memory was handed out again by smallAlloc while the pool
still sliced it, ending in a SIGSEGV on an x86_64 emulator
(emulator-5554, API 36). Moving the cache's backing storage off the
GC heap (malloc) fixed the crash without touching the runtime.

Suggested fix

Exclude CRuntime_Bionic from Static_Linux_X86_Any (so Android uses
the __tls_get_addr branch like aarch64), or more generally avoid the
static-offset shortcut whenever the druntime is linked into a shared
library rather than the main executable. The stated reason for the
shortcut — not depending on __tls_get_addr in non-Shared builds —
does not apply on Bionic, which has exported __tls_get_addr since
API 29.

Workaround until then, for anyone hitting this: do not keep GC roots
whose only reference is a thread-local variable in an Android shared
library.

Activity

  1. kinke commented on Oct 24, 2026

    @kinke
    kinke
    on Oct 24, 2026
    Member

    Hi Petar, thx for the detailed analysis! - Restricting that code branch to version (CRuntime_Glibc) is probably more robust indeed.

  2. PetarKirov
    added 18 commits that reference this issue on Oct 24, 2026

    druntime: scan TLS through __tls_get_addr on Bionic x86/x86_64

    edc4ad3

    test(base): check that the collector scans this thread's TLS

    2ddd949

    docs(specs/terminal): put the Android TLS fix in druntime

    04a2499

    test(base): check that the collector scans this thread's TLS

    24953b6

    docs(specs/terminal): put the Android TLS fix in druntime

    4301b3a

    test(base): check that the collector scans this thread's TLS

    d9b3c31

    docs(specs/terminal): put the Android TLS fix in druntime

    2ecf51e

    test(base): check that the collector scans this thread's TLS

    ff646a8

    docs(specs/terminal): put the Android TLS fix in druntime

    fa49c5d

    test(base): check that the collector scans this thread's TLS

    30a7cd7

    docs(specs/terminal): put the Android TLS fix in druntime

    8d7b620

    test(base): check that the collector scans this thread's TLS

    6d785fc

    docs(specs/terminal): put the Android TLS fix in druntime

    be8abb5

    test(base): check that the collector scans this thread's TLS

    8bb1c8e

    docs(specs/terminal): put the Android TLS fix in druntime

    0129802

    test(base): check that the collector scans this thread's TLS

    c9fe28f

    docs(specs/terminal): put the Android TLS fix in druntime

    6fd18e4

    druntime: scan TLS through __tls_get_addr on Bionic x86/x86_64

    f1609f5

  3. trout314 commented on Oct 31, 2026

    @trout314
    trout314
    on Oct 31, 2026

    Minimal example (glibc). A D library holds the only references to 64
    GC objects in a thread-local array, and a C host loads it with dlopen:

    // ldc2 -shared -link-defaultlib-shared=false lib.d -of=libtlsbug.so
import core.memory : GC;
import core.runtime : Runtime;

__gshared int finalized;

class Node
{
    ~this() { ++finalized; }   // runs only when the GC frees the object
}

Node[64] tlsNodes;   // module-level, so thread-local: the only references

extern (C) int lib_init()
{
    return Runtime.initialize() ? 1 : 0;
}

extern (C) void fill()
{
    foreach (ref n; tlsNodes)
        n = new Node;
}

// Every node is still referenced from tlsNodes, so this should return 0.
extern (C) int collect_and_count()
{
    GC.collect();
    return finalized;
}
    // cc host.c -ldl -o host && ./host ./libtlsbug.so
#include <dlfcn.h>
#include <stdio.h>

int main(int argc, char **argv)
{
    void *lib = dlopen(argc > 1 ? argv[1] : "./libtlsbug.so", RTLD_NOW);
    if (!lib) { fprintf(stderr, "%s\n", dlerror()); return 2; }
    int (*lib_init)(void) = (int (*)(void)) dlsym(lib, "lib_init");
    void (*fill)(void) = (void (*)(void)) dlsym(lib, "fill");
    int (*collect_and_count)(void) = (int (*)(void)) dlsym(lib, "collect_and_count");

    lib_init();
    fill();
    int n = collect_and_count();
    printf("%d of 64 objects referenced from TLS were finalized\n", n);
    return n != 0;
}
    ldc2 -shared -link-defaultlib-shared=false lib.d -of=libtlsbug.so
ldc2 -shared -link-defaultlib-shared lib.d -of=libtlsctl.so    # control
cc host.c -ldl -o host
./host ./libtlsbug.so
LD_LIBRARY_PATH=<ldc>/lib ./host ./libtlsctl.so

    On GitHub's ubuntu-22.04 runner (glibc 2.35), 10 fresh processes each:

    druntime LDC 1.40.0 LDC 1.43.0
    static (-link-defaultlib-shared=false) 63 of 64 finalized, every run 64 of 64 finalized, every run
    shared (control) 0 of 64, every run 0 of 64, every run

    With static druntime the collector frees objects that are still referenced
    from the library's TLS, because it scans [fs:0 - memsz, fs:0) instead of
    that TLS. (Stack scanning is conservative, so a stale pointer can keep one
    object alive.)

    Where we hit the crash. LDC 1.40.0, Linux x86_64 (built in an Ubuntu 20.04 container,
    glibc 2.31; run on Ubuntu 22.04, glibc 2.35). The D library is built with
    -shared -link-defaultlib-shared=false and loaded by CPython through ctypes
    (dlopen). Worker threads are attached with thread_attachThis. This is the
    non-Shared, linux, X86_64 configuration, so getTLSRange returns
    [fs:0 - sz, fs:0) for every thread, with sz = the library's PT_TLS
    memsz (3681 here).

    What goes wrong. For a dlopen'd library that range isn't the library's
    TLS block. On glibc the module gets static-TLS surplus or a dynamic block, and
    either way it isn't directly below the TCB. So the real TLS is never scanned
    (the same effect as in this issue), and an unrelated 3.6 KB is scanned
    instead:

    • Pool threads: the TCB is at the top of the thread's stack mapping, so
      fs - sz is always mapped, just unrelated.
    • The thread that loaded the library, here the process's main thread: glibc
      places its TCB inside the small block ld.so allocates at startup for the
      static TLS. In our processes fs sat 0xc40 bytes into an 8 KB anonymous
      mapping, so fs - 3681 starts 545 bytes into the page below it. Whatever
      happens to be mapped there varies from process to process: sometimes the
      locale archive, sometimes another library's data, and sometimes nothing.
      When nothing is mapped, the first collection segfaults in the TLS scan:
    #0-3  (faulthandler re-raise)
#4    <signal handler called>
#5    core.internal.gc.impl.conservative.gc.Gcx.collectRoots(void*, void*)
#6    rt.sections_elf_shared.scanTLSRanges(Array!(void[])*, ...)
#7    core.thread.threadbase.scanAllTypeImpl(...)
#8    core.thread.osthread.callWithStackShell(...)
#9    thread_scanAll
#10   core.internal.gc.impl.conservative.gc.Gcx.markParallel()
#11   core.internal.gc.impl.conservative.gc.Gcx.fullcollect(bool, bool)
#12   core.internal.gc.impl.conservative.gc.Gcx.bigAlloc(...)
...
rsi = r14 = 0x7f0c6d3b1dd0   (pbot)  == fs - 3681, aligned down to 16
fs  = 0x7f0c6d3b2c40          mapping 0x7f0c6d3b2000-0x7f0c6d3b4000 rw-p [anon]
page 0x7f0c6d3b1000           unmapped

    Measured. In 1921 fresh processes, each process checked
    /proc/self/maps for whether [fs - memsz, fs) was fully mapped for the
    main thread, then ran one large threaded solve:

    computed range crashed survived
    reaches unmapped mem 36 2
    fully mapped 0 1883

    That's about 2% of processes, deterministic per process. It shows up on
    glibc 2.35 with CPython 3.10 through 3.13.

    Suggestion. The static-offset shortcut is only valid when druntime lives
    in the main executable. Rather than keying on the C runtime, take the
    shortcut only for the executable, and for a shared object use
    dl_iterate_phdr's dlpi_tls_data (glibc and musl provide it; check which
    Bionic API level added it). It gives the calling thread's block for that module without
    depending on __tls_get_addr, so it keeps the shortcut's goal of supporting
    static libc links. For example:

    // in the dl_iterate_phdr callback for this DSO, on the thread in question:
if (size >= dl_phdr_info.dlpi_tls_data.offsetof + (void*).sizeof
    && info.dlpi_tls_data !is null)
    range = info.dlpi_tls_data[0 .. tlsMemsz];

    Workaround we use. After rt_init, and after each thread_attachThis
    (while collections are still blocked), we look up our library's
    dlpi_tls_data and overwrite the thread's single entry in
    rt.sections_elf_shared.initTLSRanges(). That fixes both the crash and the
    missed roots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions