Repository navigation
Linker error for no_std binary using musl #58163
Description
I tried to build a simple no_std "Hello World!" program using the x86_64-unknown-linux-musl target. The source code (in src/main.rs) is here:
#![no_std]
#![no_main]
use libc::{c_char, c_int, exit, puts};
#[no_mangle]
pub extern "C" fn main(_: c_int, _: *const *const c_char) -> c_int {
unsafe {
puts(b"Hello World!\0" as *const u8 as *const i8);
}
0
}
#[panic_handler]
fn panic_handler(_: &core::panic::PanicInfo) -> ! {
unsafe {
exit(1)
}
}When using my default x86_64-unknown-linux-gnu target, the program compiles and runs like it should, but with x86_64-unknown-linux-musl, I get this linking error:
error: linking with `cc` failed: exit code: 1
|
= note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-Wl,--eh-frame-hdr" "-m64" "-nostdlib" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crt1.o" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crti.o" "-L" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "/home/johannes/hello/target/x86_64-unknown-linux-musl/release/deps/hello-8e567d3b97d6bd11.hello.c72za6mx-cgu.0.rcgu.o" "-o" "/home/johannes/hello/target/x86_64-unknown-linux-musl/release/deps/hello-8e567d3b97d6bd11" "-Wl,--gc-sections" "-no-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/johannes/hello/target/x86_64-unknown-linux-musl/release/deps" "-L" "/home/johannes/hello/target/release/deps" "-L" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/home/johannes/hello/target/x86_64-unknown-linux-musl/release/deps/liblibc-8c2b3cf08263e000.rlib" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-879310dc3b96af61.rlib" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-d0572f7a936161bf.rlib" "-static" "-Wl,-Bdynamic" "/home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crtn.o"
= note: /usr/bin/ld: /home/johannes/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crt1.o: in function `_start_c':
/build/musl-1.1.20/crt/crt1.c:17: undefined reference to `__libc_start_main'
/usr/bin/ld: /home/johannes/hello/target/x86_64-unknown-linux-musl/release/deps/hello-8e567d3b97d6bd11.hello.c72za6mx-cgu.0.rcgu.o: in function `main':
hello.c72za6mx-cgu.0:(.text.main+0xa): undefined reference to `puts'
collect2: error: ld returned 1 exit status
I can build a normal (std-using) "Hello World!" program for musl without problems, only this no_std version doesn't work.
I am also able to work around this problem by instead moving the source code to src/lib.rs and compiling it as a library with crate-type = ["staticlib"], then linking manually with musl-gcc. The error only occurs when trying to build a binary directly.
My Cargo.toml. Uncomment the two lines and move main.rs to lib.rs for the workaround.
[package]
name = "hello"
version = "0.1.0"
authors = ["me"]
edition = "2018"
#[lib]
#crate-type = ["staticlib"]
[dependencies]
libc = { version = "0.2", default-features = false }
[profile.dev]
panic = "abort"
[profile.release]
panic = "abort"Reacted by Yota Toyama
Activity
mati865 commentedon Feb 5, 2019
Which Rust version are you using
rustc -vV?On latest nightly your example fails with:
Without
panic_handler(_)it builds for bothgnuandmusl.jgriebler commentedon Feb 5, 2019
Ah, I forgot to mention that
libcneedsdefault-features = falseso that it doesn't linkstdimplicitly. I'll attach myCargo.tomlto the issue description.I tried it with both 1.32.0 and the latest nightly.
mati865 commentedon Feb 5, 2019
When using native no_std gnu target required symbols come from
"-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"linker arguments.When cross compiling to musl Rust provides those symbols built in
liblibcto make it easier but with no_stdliblibcis not included anymore resulting in undefined symbols error.Manually adding
~/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-ad64cf40491cbcc1.rlibto the linker made it compile.So the question to Rust (libs?) team is whether
liblibcshould be given to the linker forno_stdbuilds or something else would be preferred.Gist with linker args for different builds: https://gist.github.com/mati865/210dbf7f27b15ffa96bf9e0b79f5d0b5
gnzlbg commentedon Aug 7, 2019
This is expected behavior. This program is not linking
muslat all, to do that, one currently needs to enable the privaterustc-dep-of-stdlibc cargo feature which requires other hacks like linking therustc-hackcrates from https://github.com/rust-lang/rust/tree/master/src/tools (not from crates.io) . It is probably easier to just manually patch libc to always link musl (EDIT: see below for a possibly easier workaround).If you want to statically link musl to your binary, the easiest way is to not make it a
#![no_std]binary, such that libstd links it properly for you.Sadly, solving this issue isn't trivial. I suppose we could expose a cargo feature from libc that allows users to force the linking of the C library such that you could use that here, but there are other things to consider here, like how this interacts with
-C target-feature=-crt-static.gnzlbg commentedon Aug 7, 2019
An alternative workaround that might be easier is to try adding this to your binary:
jgriebler commentedon Aug 9, 2019
I'm not sure if this is the right place to ask this, but is that available on stable? For me this only works when using
#![feature(link_cfg)], even though the tracking issue linked in the error has been closed for a long time.But even with that, the workaround isn't working for me. On the
gnutarget it works fine (as before), but onmuslI now get this error:Details
I don't know enough about how linking is done to try to get this to work, but maybe someone else knows what to do with this error.
(Also, just to clarify: I don't actually need this for anything, I just stumbled across this behaviour when toying around with
#[panic_handler]when it was stabilised.)gnzlbg commentedon Aug 9, 2019
Nope, but neither are
#![no_std]binaries IIRC, right ? (that is, you'd need nightly anyways to be able to reach this issue)I see
x86_64-pc-linux-gnuin those error messages. Is that normal? How are you compiling the#![no_std]binary ? E.g. if you want to compile withcargo build --target=x86_64-unknown-linux-muslI think you need to make sure that the proper C toolchain is used.Makes sense - this is kind of a known issue, but is good to have a proper bug report for it, so thank you for trying so many things here!
jgriebler commentedon Aug 9, 2019
It's available on stable since 1.30, when
#[panic_handler]was stabilised, though afaik only withpanic=abort.Ah, that's interesting. I tried it again with
-C linker=musl-gccand now your workaround works (though my original example still doesn't). Does this mean that#[link(...)]will always use the systemcceven on themusltarget?gnzlbg commentedon Aug 9, 2019
The problem is that we can't statically link musl to a binary twice, and libstd for musl always links musl statically.
The only thing that the
stdfeature oflibccontrols is whetherlibcitself haslibstdas a dependency. If it doesn't, that does not mean thatlibstdwon't be linked to the final binary (e.g. if the final binary isn't#![no_std], or if it is#![no_std]but contains anextern crate std, thenlibstdwill be linked). There is no way forlibcto know whetherlibstdwill be linked, so it has to assume it will, and never link musl, since iflibstdis actually linked, andlibclinks musl,muslwill be statically linked twice, and the build will fail.So when you have a
#![no_std]binary that links libc, and you disable the libcstdfeature, that does not pull in musl, because you could writeextern crate std;and that would break that use case. If you enable thestdfeature,libcshould pulllibstdfor you, so that you get a valid musl, but this is probably not what you want if you are building a#![no_std]binary. So the only thing you can do is link musl yourself, which is what that the workaround here does.This is bad, super hard to discover, etc and there should be a better way to do this.
The simplest thing to do would be adding a new cargo feature to the
libccrate, that forces linking the C library no matter what. That way, you could addlibcwith aforce_link_c_libsfeature as a dependency, and your example would "just work". If you then, by accident, pull in libstd, e.g., by doingextern crate std;, or adding a dependency that does that (e.g. by not using#![no_std]), then your binary would fail to link, and the error message would be an obscure linking error, and there is no easy fix for that.So I think we probably want a more elaborated way of solving all these issues, that has nice error messages, and always works in the obvious way.
I think, that if you add:
to your example, and remove the libc dependency from your Cargo.toml, your code will also work, because it will use the
libcversion that the standard library uses, but without pulling inlibstd. Thatlibcversion is special and different from the one in crates.io, and it will statically link musl for you.Exploring a solution in this direction might be worth it. That is, you want a #![no_std] binary with a libc that does link the C standard library. It shouldn't be necessary for you to have to pull in
libcfrom crates.io to do that, although it should be possible.gnzlbg commentedon Aug 9, 2019
@mati865 maybe we could move the parts of the
libccrate that actually link stuff into a separate crate, e.g.,libc_linkage, that always links the stuff. We can then make that crate "unique", by using the Cargolinks = "..."key, preventing it from being linked twice into a dependency graph.We could then make
libcvialibstdalways link it, and just allow configuring forlibcfrom crates.io whether it should link it or not. If a user messes this up, cargo will error saying that there are twolibc_linkcrates in the dependency graph., which is nice. A problem is that cargo will error even iflibc_linkdynamically links stuff, but I think this might be fine ?This allows you to pull in different versions of the bindings, by pulling different versions of the libc crate, into a dependency graph, but ensuring that stuff is linked only once.
jgriebler commentedon Aug 9, 2019
Thanks for the explanation, that does make some sense. I've tried your suggestion with
rustc_private, but I couldn't get it to work right now. Maybe I still did something wrong.I've found an even simpler alternative to your previous workaround though: Just passing
-lctorustc(along with the right linker) seems to be enough. That one even works on stable, which probably makes this whole thing mostly a non-issue. I guess it's still warranted to leave this issue open, since having to do that seems rather unintuitive.mati865 commentedon Aug 9, 2019
@gnzlbg I never played with
linksfield but from what I've read that could work.To make sure I understand
libc_linkagewould be native library that links tolibc,libm,libpthreadand whatever else either as static or dynamic libs. Thenlibccrate would uselinks = "libc_link".There are targets that do not support creating dylibs, could that be issue?
gnzlbg commentedon Aug 9, 2019
No, i think this just doesn't work. The
libcused byrustcis not the same as the one used by the std library :(I think it can be an
rlib, it just needs to contain the#[link ...] extern { ... }blocks to force rustc to pass the appropriate-lcetc. flags.This isn't what I meant. The
libc_linkcrate would have alinks = "...unique..."key.libcwould havelibc_linkas a dependency.I haven't worked out the rest. For example, if
libc_linkis optional, we could build libstd with it enabled, and if a user tries to enable it while compiling libc from crates.io that will only work if libstd is not linked. However, if a user does not include libstd, and forgets to enable it, the error message will be obscure. This is because this dependency isn't really optional forlibc.So maybe we could publish
libc_linkalong withlibcore,liballoc, etc. and have it as a required dependency oflibc. Crates can use whatever libc version they want, but they all link to the exact same libc_links.libstdwould link against it as well.That means that just including
libcas a dependency would work for#![no_std]binaries, even iflibstdis not available. The "con" of this approach, is that changes tolibc_linkshave to go through the release process. Maybe we could offer a way to override thelibc_linkcrate used, e.g., via a-C ...argument or something. I've wanted to be able to select a different implementation oflibmin the past, so this might be a step towards that direction.