Skip to content

Testing a cross-compiled toolchain #108914

Open
Open
Testing a cross-compiled toolchain#108914
Labels
A-testsuiteArea: The testsuite used to check the correctness of rustcP-lowLow priorityT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Description

@jfgoog
jfgoog
opened on Mar 8, 2023

We (Android platform) build a custom version of rustc. Basically, we take the source distro, apply a few patches, and run x.py with a custom config.toml.

I am working on 2 things, which are related:

  • We want to build a Rust toolchain that runs on Windows (MinGW). For various reasons, we'd prefer to do this by cross-compiling (i.e. build a Windows rustc on a Linux machine).

  • We want to test our toolchain in our CI pipeline. For example, run x.py test tests/ui

So, I had the following "clever" idea:

  1. Cross-compile a windows toolchain on Linux.
  2. Copy it to a Windows machine.
  3. In Windows, x.py --stage 0 test tests/ui with the stage 0 / bootstrap configured to use my new toolchain.

I got a very helpful warning that I probably shouldn't be doing this, and inviting me to file this issue. When I proceeded anyway, with COMPILETEST_FORCE_STAGE0=1, I got a number of test failures (~200 out of ~13,000). (For comparison, when I ran x.py test tests/ui the "right" way, the tests all passed, so I'm pretty confident the failures weren't due to our patches and custom configuration.)

Is there a correct, supported way to do this? That is, run tests "after-the-fact" on a toolchain you just built, possibly on a different machine? If not, I am willing to add support for it, if there is interest.

I found several things that seemed related, but not quite right:

  • According to the docs, ./x.py test distcheck applies to a source tarball, not a compiled toolchain.
  • The remote test server seems to be for testing different targets, not for when the toolchain itself is cross-compiled. It also seems like it would be very difficult to use in our CI pipeline. It also seems like it poses security issues in a production environment (opening a port allowing arbitrary code execution).

Activity

  1. jfgoog commented on Mar 8, 2023

    @jfgoog
    jfgoog
    on Mar 8, 2023
    ContributorAuthor

    I took a look at some of the test failures, and they all seem to be related to the verbosity of error messages. For example, src/test/ui/borrowck/borrowck-move-out-of-overloaded-auto-deref.rs reports the following diff:

    9          |
10      note: this function takes ownership of the receiver `self`, which moves value
11        --> $SRC_DIR/core/src/iter/traits/collect.rs:LL:COL
-          |
-       LL |     fn into_iter(self) -> Self::IntoIter;
-          |                  ^^^^
15
16      error: aborting due to previous error
17

    From digging into the code, I wonder if -Zui-testing should be forcing the extra context to be printed, but sometimes does not.

  2. the8472 commented on Mar 8, 2023

    @the8472
    the8472
    on Mar 8, 2023
    Member

    Maybe copy over the output toolchain into the appropriate build/<target>/stage1* directories and then run tests with --keep-stage? Low confidence in that it'll work since I haven't tried that.

  3. jfgoog commented on Mar 8, 2023

    @jfgoog
    jfgoog
    on Mar 8, 2023
    ContributorAuthor

    I tried your suggestion, and it doesn't work. Or, at least, it's not straightforward. The contents of the stageX directories don't neatly correspond to a packaged toolchain. For example, I have stage1, stage1-std, and stage1-rustc directories, so I would probably have to do some surgery on the packaged toolchain and figure out where each bit came from, and where to restore it. It seems like that is likely to be brittle.

    It also did a bunch of unnecessary work, including building LLVM, before finally failing with:

    Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Assembling stage1 compiler (x86_64-unknown-linux-gnu)
Warning: Using a potentially old libstd. This may not behave well.
Copying stage1 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
thread 'main' panicked at 'fs::read(stamp) failed with No such file or directory (os error 2) ("/usr/local/google/home/jamesfarrell/src/rust-toolchain/out/rustc/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/.libstd.stamp")', lib.rs:1421:24
  4. workingjubilee
    added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
    on Mar 8, 2023

  5. jyn514 commented on Mar 9, 2023

    @jyn514
    jyn514
    on Mar 9, 2023
    Member

    I took a look at some of the test failures, and they all seem to be related to the verbosity of error messages. For example, src/test/ui/borrowck/borrowck-move-out-of-overloaded-auto-deref.rs reports the following diff:

    This is about missing the source for libstd, not the verbosity. Don't know why that would vary between machines; maybe the absolute path to the repository is getting embedded into rustc at build time somehow?

    In general, I don't want to spend very much time at all trying to support this; bootstrap already tries to support far too many use cases and we have no way to test "a toolchain built on one machine but run on another" in CI.

  6. jyn514
    added
    P-lowLow priority
    on Mar 9, 2023

  7. jfgoog commented on Mar 9, 2023

    @jfgoog
    jfgoog
    on Mar 9, 2023
    ContributorAuthor

    This is helpful. I will see if I can figure out why we can't find the libstd sources.

    I sympathize with your observation about testability. It would improve our testing, but make it harder for you.

  8. jfgoog commented on Mar 13, 2023

    @jfgoog
    jfgoog
    on Mar 13, 2023
    ContributorAuthor

    Just to close the loop, since I didn't mention it earlier: It is possible for us to build natively on Windows, just more awkward. We will probably try that route, which has the advantage of being what the official version of x86_64-pc-windows-gnu does.

  9. RodBurman commented on Apr 1, 2026

    @RodBurman
    RodBurman
    on Apr 1, 2026

    It has been 3 years since the last moment on this issue, so is it still an issue? If not we should close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-testsuiteArea: The testsuite used to check the correctness of rustcP-lowLow priorityT-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions