Repository navigation
Building on i686-pc-windows-gnu #9
Description
It is important that we can build on 32-bit targets, I've tried to follow the instructions in the README.md,
but substituting with the 32-bit variants.
This doesn't work for some reason. And here's the latest error-message/hurdle:
[redacted] MINGW32 /c/Users/tpb398/Documents/GitHub/libR-sys
$ cargo build --target=i686-pc-windows-gnu
Compiling libR-sys v0.1.9 (C:\Users\tpb398\Documents\GitHub\libR-sys)
error: failed to run custom build command for `libR-sys v0.1.9 (C:\Users\tpb398\Documents\GitHub\libR-sys)`
Caused by:
process didn't exit successfully: `C:/cargo_target_dir\debug\build\libR-sys-46304186790a078b\build-script-build` (exit code: 101)
--- stdout
cargo:rustc-env=R_HOME=C:/R/R-4.0.2
cargo:rustc-env=R_HOME=C:/R/R-4.0.2/bin/i386
cargo:r_home=C:/R/R-4.0.2
cargo:rustc-link-search=C:/R/R-4.0.2/bin/i386
cargo:rustc-link-lib=dylib=R
cargo:rerun-if-changed=build.rs
cargo:rerun-if-changed=wrapper.h
--- stderr
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'clang.dll\', \'libclang.dll\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [(C:\\Users\\tpb398\\scoop\\apps\\msys2\\current\\mingw32\\bin\\libclang.dll: invalid DLL (32-bit)), (C:\\Users\\tpb398\\scoop\\apps\\msys2\\current\\mingw32\\bin\\libclang.dll: invalid DLL (32-bit))])"', C:\Users\tpb398\.cargo\registry\src\github.com-1ecc6299db9ec823\bindgen-0.53.3\src/lib.rs:1956:31
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I've searched for clang.dll in the mingw folders, and there is nothing named that.
It is the libclang.dll that is problem.
Here's a few more diagnostic things that I see mentioned when trying to troubleshoot this:
Details
$ clang -v
clang version 11.0.0 (https://github.com/msys2/MINGW-packages 20f70d38fae51f89ce3c06d054a77139606df472)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: C:\Users\tpb398\scoop\apps\msys2\current\mingw32\bin
Activity
Ilia-Kosenkov commentedon Nov 28, 2020
Hi @CGMossa,
Did you have any success building against
i686?There is one issue with the building process - the library relies heavily on the
Rexecutable (present inPATH) to determineR-specific indlude directories. On64-bit systemPATHusually containsx64folder ofR, meaninglibR-sysis linking against64-bit version ofR.dll, and ultimately fails.Another thing I have no explanation for - my path contains both
mingw32andmingw64, and by default a64-bitclangis invoked. If I leave onlymingw32, I get the exactly same error as you have posted.However, with
mingw64andi386Ron the path, the library can be built and tests can be run, but two of them fail (looks like some native integer/pointer size mismatch). I have no experience inbindgenand cannot comment on that, but I attach a detailed stacktrace.I do not understand why the pipeline is able to build a
32-bit app and link agains32-bit version ofR.dllwhile using64-bitclangand libraries, this is beyond me.Below is the result of
$(echo $env:RUST_BACKTRACE ; cargo test --release --target=i686-pc-windows-gnu ; clang -v ; Rscript.exe -e '.Platform$r_arch') *>&1 > dump.txtdump.txt
clauswilke commentedon Nov 28, 2020
Using the R executable to find
R_HOMEis almost certainly the wrong approach and I already have a PR pending to fix this: KyleMayes/clang-sys#12It's possible that we need the build script to detect whether it's trying to build a 32bit or a 64bit target and modify the paths accordingly. Could you post the typical path layout on Windows when both 32bit and 64bit versions are installed? Do they use two different
R_HOMEvalues or are they nested under one singleR_HOME? Where are the include files (e.g.Rinternals.h)? Where are the library files (something likelibR.dll)?Ilia-Kosenkov commentedon Nov 28, 2020
@clauswilke,
So the layout is typically like this:
There is a default
Program Files\Rfolder, to where all versions go (unless the user modified it during installation). In my case, it can look like this:Details
Within
R-4.0.3Details
\includeDetails
and platform-specific
\binDetails
Here,
R.exeis a64-bit app and I guess thisbinfolder is added to thePATH.Now, both
x64andi386contain a full set of.exeapps and.dll, which are platform-specific, e.g.Details
So, to build a
32- or64- bit project, a correctR.dllshould be linked. The current approach invokes64-bitRand gets an....\bin\x64\R.dllas include, while the correct one fori686is....\bin\i386\R.dll.There is yet another way. Default
Rinstallation can write values to Registry. Depending on whether the installer was executed as admin (elevated) or not, values can be written to different places, but they can (probably) be tested during the build process. Examples:ls HKLM:\SOFTWARE\R-core\givesand
ls HKLM:\SOFTWARE\R-core\Rproduces all currently active installation ofR(32-bit).So, for instance (using powershell for the sake of example), a query of the form
"$(Get-ItemProperty HKLM:\SOFTWARE\R-core\R64 | select -expand "InstallPath")\bin\x64"results in
C:\Program Files\R\R-4.0.3\bin\x64and
"$(Get-ItemProperty HKLM:\SOFTWARE\R-core\R | select -expand "InstallPath")\bin\i386"results in
C:\Program Files\R\R-4.0.3\bin\i386,which should give correct
64- and32-bit paths no matter howRis installed.clauswilke commentedon Nov 28, 2020
So we have the same
Rinternals.hbut differentR.dll. I think when compiling on Windows we need to set the correct library path depending on the target we're compiling for. That should be possible.clauswilke commentedon Nov 28, 2020
I have amended my PR KyleMayes/clang-sys#12 to set different library paths depending on the target architecture. But I can't test it since I don't have windows. Could you guys test it? Note that you'll have to set the
R_HOMEenvironment variable for this code branch to be executed.clauswilke commentedon Dec 8, 2020
I have merged KyleMayes/clang-sys#12. Does this close this issue?
Note also the problem described in KyleMayes/clang-rs#82. @CGMossa, do you have this problem?
yutannihilation commentedon Dec 12, 2020
I still see the same error on my MSYS2. The error @Ilia-Kosenkov described in #9 (comment) is solved, but the original one stays the same.
This error
comes from clang-sys crate's validation:
I really don't understand why
target_pointer_widthis64when we specify the target of 32bit...CGMossa commentedon Dec 12, 2020
yutannihilation commentedon Dec 12, 2020
Here's printed-out
bindgen_builder.command_line_flags(). Is this the same on your environment?CGMossa commentedon Dec 12, 2020
yutannihilation commentedon Dec 12, 2020
Yeah, I think we are stuck in front of the same problem...
This seems because
cargolooks for the default toolchain even when we specifytarget. If I switch the default toolchain to 32-bit,I can pass the validation, but I still get this error (the message says "The specified procedure could not be found." Sorry I don't know how to show English messages...):
clauswilke commentedon Dec 12, 2020
I think what is happening is that
cargouses the default toolchain to compilebuild.rs, and then runsbuild.rswith the specific compile target.If you switch the default toolchain, you also need a 32-bit clang. Did you install that? The github action does not.
yutannihilation commentedon Dec 12, 2020
Yes, I just followed https://github.com/extendr/libR-sys#windows-specific-instructions
Btw,
it seems GitHub Actions runner already has it installed. I tried to install
mingw32/mingw-w64-i686-clangon GitHub Actions CI runner, but it says it's up to date.(from my test branch: https://github.com/yutannihilation/libR-sys/runs/1542066028?check_suite_focus=true#step:3:33)
clauswilke commentedon Dec 12, 2020
Side note: I find it so confusing to figure out what is and isn't already installed in the GitHub Actions runner. And if you once install something and then remove the install line it still remains in the cache. I wish there was an easy way to just clear the cache.
24 remaining items
Ilia-Kosenkov commentedon Dec 16, 2020
@clauswilke,
--releasehas no effect - still fails both ways.However, I put an example code using
c++, in particular,msvc.Strangely, it works fine for both
x64andx86, meaning I dynamically loaddll, find two functions, invoke them, free resources.The only difference so far is that I use simple
LoadLibraryW, whilerustfails with error atLoadLibraryExW, which has additional flags. I will try playing withLoadLibraryExWto see if it fails under certain conditions.I can share a repo which can be reviewed and tested -- if needed.
Building from source is another step, I think there still maybe some
rust-related issue there.Ilia-Kosenkov commentedon Dec 16, 2020
This is getting weirder and weirder with every step.
In short,
cargo runfails on a sample project, however ifcargo buildand then just execute the*.exe-- everything works fine.If anyone is testing my
rust_clang_run, could you please invoke the output.exemanually?clauswilke commentedon Dec 16, 2020
Interesting! Can you reproduce this in the GH action? Run
cargo build, run the*.exe, and then run it again withcargo run?Ilia-Kosenkov commentedon Dec 16, 2020
Did not try running it with
cargo runafterwards, just created separate jobs.Here is the last report https://github.com/Ilia-Kosenkov/rust_clang_run/actions/runs/426337151.
Jobs that are called
Run i686do build and then run executable, other jobs docargo runonly.clauswilke commentedon Dec 16, 2020
I have removed this issue from the milestone for the next libR-sys release. It isn't immediately show-stopping, as I've now a set of manually fixed-up bindings that work for now. We should still try to get the i686 libclang issue resolved, though.
Ilia-Kosenkov commentedon Dec 16, 2020
It feels like we have enough to go and ask 'rust' people about this issue. Clearly it is neither 'msys'/'mingw' nor 'clang' (I called the latter from x86 C++ app with no problem).
clauswilke commentedon Dec 16, 2020
Agreed. We can file an issue with cargo: https://github.com/rust-lang/cargo
Do you want to do this or do you prefer if I do so?
Ilia-Kosenkov commentedon Dec 17, 2020
@clauswilke,
If you can do that, please do.
I will participate and provide clarifications of what I found if needed.
clauswilke commentedon Dec 17, 2020
Issue filed with cargo: rust-lang/cargo#8990
Ilia-Kosenkov commentedon Dec 18, 2020
Following suggestions from
rust/cargopeople, I was able to buildi686and test*libR-syswithout pre-built bindings.* because of
PATHresolution issue (see KyleMayes/clang-rs#91), ifPATHis updated to explicitly include onlyR_HOME\bin\i386, thenalso works.
mati865 commentedon Dec 19, 2020
I think cross-compiling from any Windows toolchain, even
x86_64-gnuwould work here.Ilia-Kosenkov commentedon Dec 22, 2020
With the answers we got from other teams & the successful deployment of our own GitHub CI, should this issue be closed?
README contains all the necessary information on how to build on windows, including
i686.clauswilke commentedon Dec 22, 2020
Yes, agreed. If new problems crop up we can open a new issue.