Skip to content

fix: rusi rustc_private probes for 1.100+, use-tree imports, and #129 review findings - #130

Merged
prabhu merged 6 commits into
mainfrom
fix/rusi-rustc-probes-and-review-findings
Oct 19, 2026
Merged

prabhu merged 6 commits into
mainfrom
fix/rusi-rustc-probes-and-review-findings

Conversation

@prabhu

@prabhu prabhu commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #129 from an adversarial review of it. The headline: the rusi compiler backend does not build on any Rust 1.100 toolchain — today's beta, and stable from mid-November — and on 1.100+ it drops every leaf of use a::{b, c} from its imports. Both are fixed here, along with the review's other findings.

What was wrong, and what changed

1. The rustc_private version gate could not work (rusi, high). #129 selected the wrapper's API shape with one cfg set when the toolchain reports ≥ 1.101. Nightlies carry the version of the release they lead to, and the three forks landed on different days:

Change Upstream First nightly
ProjectionElem::PhantomDeref rust-lang/rust#159103 2026-08-29 (1.100)
HIR UseTree / UseKind::Nested rust-lang/rust#161349 2026-09-26 (1.100)
run_compiler → compiler_entrypoint rust-lang/rust#163647 2026-10-03 (1.101)

On main, rustc 1.100.0-beta.7 and nightly-2026-09-29 fail either way: E0023/E0004 without the cfg, E0432 with it. The wrapper now has a build.rs that compiles a tiny probe per fork against the RUSTC cargo provides and sets one cfg each. The driver no longer injects RUSTFLAGS, which also stops it overriding cargo-config build.rustflags/target.*.rustflags on new toolchains. Standalone builds need no flags. The build script also adds <sysroot>/lib to the link path when the LLVM library lives only there, so the macOS -L workaround the README asked for is gone.

2. use a::{b, c} lost its leaves on 1.100+ (rusi, high). The new HIR keeps a whole use tree in one item with relative nested prefixes. The wrapper read only the top prefix, so it reported stems like std and std::collections as imports and dropped the rest. It now walks the tree, records each leaf at its own span, and folds self. The old shape also stops emitting empty-path ListStem imports. A new use-tree-app fixture pins identical leaf imports and positions on both shapes.

3. CI never exercised the forked code (CI, medium). The compiler-backend job now has three legs: the pinned nightly (Linux and Windows), beta (the leg that would have caught 1.100), and a rolling-nightly canary that is allowed to fail. Beta and the canary are named through RUSI_TEST_TOOLCHAIN, which the driver's ignored tests now honor as the CLI tests already did. The new import test refuses to pass on a stub fallback when a toolchain is pinned that way.

4. The runtime-symbol scan was easy to evade (rusi, medium). Each case below was checked against rustc 1.99.1:

  • #[no_mangle] #[export_name = "exit"] now reports exit; export_name wins wherever it appears, as in rustc.
  • #[export_name = concat!("mem", "set")] is now evaluated.
  • Exports under cfg_attr(..) are now seen, including the common cfg_attr(not(test), no_mangle).
  • #[no_mangle] associated functions are now reported. rustc doesn't lint them, but they still interpose.
  • macro_rules! bodies that spell out an export are now read from their tokens, with medium confidence.
  • POSIX symbols are no longer flagged in Windows-only code. rustc lints them only on unix-like targets; the six core memory symbols are still flagged everywhere.

5. Relative cache roots (rusi and cdxrs, low). rusi compiles and runs the materialized wrapper tree, but it accepted relative RUSI_CACHE_DIR, XDG_CACHE_HOME and HOME/LOCALAPPDATA, so a relative value could place that tree inside the repository being analyzed. A relative RUSI_CACHE_DIR is now an error; the other relative values are ignored. cdxrs now applies the same rule to HOME, LOCALAPPDATA and USERPROFILE, judged by the target platform's rules.

6. Unknown schema formats pass silently (cdxrs, medium). #129's idn-email regression came from jsonschema treating a format it doesn't know as valid. Validators are now built with should_ignore_unknown_formats(false). To check it, I built without the idna feature: the suite fails with Unknown format: 'idn-email' instead of validating less. get_validator also compiles each schema once per process; before, it recompiled on every call even though its doc comment said it cached.

7. gradlew.bat was always dirty (low). *.bat text eol=crlf means LF in the index and CRLF on checkout, but the blob was stored as CRLF, so every fresh clone showed the file as modified. It is renormalized; checkouts still get CRLF.

Looked at and left unchanged

  • PhantomDeref → Deref. Deref and Opaque are equivalent to the flow engine (both only feed is_indirect), and treating it as a deref keeps the flow, which is the conservative choice.
  • cdxui lockfile adds stacker/psm/cc. They aren't in the resolved build graph for any target (cargo tree -i psm --target all is empty); they are lockfile entries for an inactive optional pest dependency. All 8 zigbuild release targets of cdxui and cdxrs build with zig 0.17.0.

Verification

  • 8 toolchains on macOS: 1.98.1, nightly-2026-08-21, nightly-2026-09-01, 1.100.0-beta.7, nightly-2026-09-29, nightly-2026-10-05, 1.99.1 and nightly-2026-10-15. On each:
    • wrapper clippy -D warnings and tests pass;
    • the probes select the expected shape;
    • rusi analyze --backend compiler on use-tree-app produces byte-identical imports.
  • Emulated CI legs: the pinned nightly, beta and canary legs each pass 12 CLI and 6 driver ignored tests with the real embedded backend.
  • Stable suites: rusi 335 tests and cdxrs 145 tests pass; clippy and fmt are clean.
  • Cross-builds: cdxui and cdxrs zigbuild release builds pass for all 8 targets.
  • Windows 11 ARM64 VM (MSVC):
    • cdxrs, cdxui and rusi stable tests, clippy and fmt pass.
    • Wrapper clippy and tests pass on 1.99.1, nightly-2026-08-21, 1.100.0-beta.7 and nightly-2026-10-18, with the expected probes.
    • The pinned-nightly, beta and rolling-nightly ignored suites pass: 12 CLI and 6 driver tests each, with the import test in strict mode.
    • A fresh worktree checks out clean, gradlew.bat included.
  • CI: Rust 1.99.1 toolchain + crates to latest; wrapper through nightly 1.101; Windows CI legs #129 recorded no GitHub Actions runs, so this PR's run is the first time the Windows legs and the new beta and canary legs execute on hosted runners.

Not in this repo: cdxgen's JS validator (ajv-formats with strict: false) still accepts any idn-email, so cdxgen and cdxrs now disagree on contact emails. cdxgen's resolveCacheDirFor has the same relative-root gap. Both belong in a cdxgen PR.

… release

The wrapper picked its rustc_private API shape from the resolved
toolchain's release number (>= 1.101 set `rusi_rustc_ge_1_101` via
RUSTFLAGS). Nightlies carry the version of the release they lead to,
not of the change, and the three forks landed on different days:

- ProjectionElem::PhantomDeref   rust-lang/rust#159103, nightly 2026-08-29 (1.100)
- HIR UseTree / UseKind::Nested  rust-lang/rust#161349, nightly 2026-09-26 (1.100)
- compiler_entrypoint rename     rust-lang/rust#163647, nightly 2026-10-03 (1.101)

So no setting compiled on the 1.100 beta (and 1.100 stable, due
mid-November), nor on nightlies 2026-08-29..2026-10-02: the old shape
fails with E0023/E0004, the new one with E0432 (reproduced on
1.100.0-beta.7 and nightly-2026-09-29).

The wrapper's new build.rs compiles one tiny probe per fork against the
RUSTC cargo hands it and sets one cfg each (rusi_rustc_mir_phantom_deref,
rusi_rustc_hir_use_tree, rusi_rustc_compiler_entrypoint). The driver no
longer touches RUSTFLAGS — which also stops it overriding cargo-config
`build.rustflags`/`target.*.rustflags` on >= 1.101 — and standalone
builds need no flags. build.rs also points the linker at <sysroot>/lib
when the LLVM library lives only there (macOS nightlies and betas
without llvm-tools), replacing the manual -L workaround.

The 1.100+ `use` handling dropped every leaf of `use a::{b, c}`: the
new HIR keeps the tree in one item with relative nested prefixes, and
the wrapper read only the top prefix, reporting stems such as `std` as
imports. It now walks the tree, records each leaf at its own span with
the parent prefix applied and `self` folded, and skips the braces; the
older shape no longer emits empty-path `ListStem` imports either. A new
use-tree-app fixture and ignored test pin identical leaf imports and
positions on both shapes.

Verified on 1.98.1, nightly-2026-08-21, nightly-2026-09-01, 1.100 beta,
nightly-2026-09-29, nightly-2026-10-05, 1.99.1 and nightly-2026-10-15:
wrapper clippy -D warnings and tests pass, the probes pick the right
shape, and `rusi analyze --backend compiler` reports identical imports.

CI's compiler-backend job gains a beta leg (named through
RUSI_TEST_TOOLCHAIN, which the driver tests now honor too) and a rolling
nightly canary allowed to fail; the new import test refuses to pass on a
stub fallback when a toolchain is pinned that way.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
The materialized wrapper sources are compiled and executed, which is why
there is no temp-dir fallback. Yet a relative RUSI_CACHE_DIR,
XDG_CACHE_HOME or HOME/LOCALAPPDATA resolved against the working
directory — often the repository being analyzed — where a pre-planted
tree (a `.cargo/config.toml` beside current sources is not detected as
stale) would be built and run.

A relative RUSI_CACHE_DIR is now an error naming the variable; relative
XDG_CACHE_HOME and home roots are ignored, as the XDG specification
requires and as cdxrs already does for its cache. The resolution is a
pure function with its own unit test.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
The syn-level scan that mirrors rustc's runtime-symbol lints returned on
the first export attribute it met and could not read computed names, so
it missed exports rustc itself reports (checked against rustc 1.99.1):

- `#[no_mangle] #[export_name = "exit"] fn innocuous` exports `exit`:
  export_name wins wherever it appears, as in rustc.
- `#[export_name = concat!("mem", "set")]` exports `memset`: concat!
  over literals (and nested concat!) is now evaluated.
- `#[cfg_attr(not(test), no_mangle)]`, the common no_std spelling, and
  any export nested under cfg_attr or unsafe(..) is now seen; the
  predicate is not evaluated, since the shipped build exports.

It also scans two places rustc does not lint but where a symbol is
interposed all the same: `#[no_mangle]` associated functions in impl
blocks (with a native-export signal too), and macro bodies, whose
`#[no_mangle] .. fn read` / `#[export_name = "memcpy"] .. fn $name`
items are read from the tokens with medium confidence because the stable
backend does not expand macros.

The POSIX symbols (open/read/write/close/exit, the malloc family) are
linted by rustc only for unix-like targets, so a definition gated to
Windows by its own cfg, an enclosing inline module, or a file-level
`#![cfg(windows)]` is no longer flagged; the six core memory symbols
still are on every target.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
jsonschema accepts an unknown `format` silently, which is how idn-email
validation vanished when 0.52 moved it behind the `idna` feature: the
format became unknown, and contact addresses went unchecked. #129 pinned
that one format with a regression test; the class of bug stayed open.
Validators are now built with should_ignore_unknown_formats(false), so a
future feature reshuffle fails schema compilation (a
schema.internal-error finding, and failing tests) for every format the
vendored schemas assert — date, date-time, idn-email, iri-reference.
Checked by building without `idna`: the suite fails with
"Unknown format: 'idn-email'".

get_validator was documented as compiling and caching validators per
spec version but recompiled the 1.6/1.7 schema, with all its referenced
schemas, on every call. It now compiles each once per process.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
#129 made the Linux XDG_CACHE_HOME check host-independent, but HOME,
LOCALAPPDATA and USERPROFILE were taken as-is, so a relative value put
the cache under the current working directory — the case the XDG check
exists to prevent. Each environment-derived root must now be absolute by
the target platform's rules (`/`-rooted on Linux and macOS; drive- or
UNC-rooted on Windows, where `C:cache` and `\cache` are relative), and a
relative LOCALAPPDATA falls back to USERPROFILE. CDXGEN_CACHE_DIR stays
an explicit override.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
#129's `*.bat text eol=crlf` means "LF in the index, CRLF on checkout",
but the blob was committed with CRLF, so every fresh clone showed the
file as modified (an EOL-only diff) until someone renormalized it. This
is that renormalization; checkouts still get CRLF.

Signed-off-by: Prabhu Subramanian <prabhu@appthreat.com>
@prabhu
prabhu enabled auto-merge October 19, 2026 11:19
@prabhu
prabhu added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 9e01538 Oct 19, 2026
@prabhu
prabhu deleted the fix/rusi-rustc-probes-and-review-findings branch October 19, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant