[zaniebot]

Standard-library discovery beside a Unix runtime library misses the usual prefix/lib/libpython layout. Search parent directories for the standard-library landmarks while retaining adjacent framework and Windows layouts. On Linux, use the mapped runtime filename to handle relative loader paths and executables linked with a static libpython.

Addresses the library-resolution change tracked in astral-sh/python-build-standalone#909.

[zaniebot]

Draft upstream issue title: Relocated Unix runtimes can fail to find the standard library

Draft upstream issue body:

On Linux, an embedded Python can fail to find the standard library beside its executable if PyConfig.executable points to an unavailable launcher. Static builds don't populate the runtime-library path, so startup can fall back to the compile-time prefix and can fail to import encodings.

This can be reproduced with a static debug build of CPython at 69d0f9688af48403f6eaea20ef30f85139ed8196, a host at prefix/bin/host, and the standard library at prefix/lib/python3.16:

#include <Python.h>
int main(void) {
    PyConfig config;
    PyConfig_InitPythonConfig(&config);
    config.use_environment = 0;
    config.site_import = 0;
    PyStatus status = PyConfig_SetString(&config, &config.executable, L"/nonexistent/launcher");
    if (!PyStatus_Exception(status)) status = Py_InitializeFromConfig(&config);
    PyConfig_Clear(&config);
    if (PyStatus_Exception(status)) Py_ExitStatusException(status);
    int result = PyRun_SimpleString("import sys; print(sys.base_prefix)");
    Py_Finalize();
    return result;
}

The compile-time prefix must not contain another usable standard library. Build the host against the matching static library and run it from outside the installation. It fails to initialize on the unmodified build; locating the mapped executable and searching its parents finds the relocated prefix.

There's a related issue with shared libraries: for prefix/lib/libpython, the fallback checks below prefix/lib without also searching its parents for the standard-library landmarks.

We carry a patch for this in python-build-standalone; I'd like to upstream the runtime-path lookup and parent-directory search. See astral-sh/python-build-standalone#909 and the broader relocation discussion in python/cpython#119696.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux (Ubuntu 24.04, x86-64)