Repository navigation
Add harness field to cargo metadata output #11846
Description
Problem
For third party rust compilation tools, we would like to follow the behavior of cargo. This requires using cargo metadata and its respective crate for parsing Cargo.toml files. However, the harness value is missing from the metadata output. Thus cannot follow the correct behavior for things like [[bench]] which tend to use harness = false resulting in --test. See here.
Proposed Solution
Simple add a harness value to the cargo metadata output.
Notes
No response
Reacted by Vlad Beskrovny
Activity
epage commentedon Mar 15, 2023
This part stood out to me. I'm assuming you'll be needing more than just
harnessfor doing this. Would it be possible to further enumerate your needs so we can talk about them as a whole rather than piecemeal?Out of curiosity, are you able to say what this tool is?
Also, as an FYI, we have a Zulip stream dedicated to build system integrators](https://rust-lang.zulipchat.com/#narrow/stream/334885-t-cargo.2Fbuild-integration), in case that becomes useful in the future
jordanisaacs commentedon Mar 15, 2023
It is for nix's build system. Specifially the buildRustCrate build function from nixpkgs and crate2nix. The
buildRustCratenix function invokes rustc in a very nix specific way, but attempts to follow some of the standards set by cargo such as file layout & build target kinds. While crate2nix is a codegen tool for takingCargo.tomlandCargo.lock(relying on cargo metadata) and generating a nix file that compiles the crate and its dependencies using invocations ofbuildRustCratefunction.crate2nix/buildRustCrate currently has limited support for tests, examples, benches, etc. I am working on better support but ran into the fact that
harnessis not outputted by cargo metadata. This is important because many benches useharness = falseand use criterion. Note:buildRustCratedoes not rely on any cargo metadata functionality. But,crate2nixrelies on it to generate the correct codegen. An WIP PR forbuildRustCratehas each target take a harness paraemeter in its nix function. In a futurecrate2nixPR I would like to use cargo metadata to output the correct harness parameter in the invokation ofbuildRustCrate.It's still very much WIP but I think the only other piece that is missing from the serialization is
benchfor knowing whether by default it should be compiled in bench mode. These seem reasonable as other pieces such astestanddocare already serialized.I hope this made some sense as a quick overview of my needs & what crate2nix & buildRustCrate as they are pretty complicated. Also thanks for the zulipchat link, I did not know about it.
epage commentedon Mar 20, 2023
Thanks for the explanation!
[-]Add harness to cargo metadata output[/-][+]Add harness field to cargo metadata output[/+]qwandor commentedon Nov 15, 2023
I'm running into the same issue with
cargo_embargo, a tool we're using in Android to generate Soong build files for third-party crates.vlad20012 commentedon May 15, 2026
RustRover needs this field as well to support running harnessless tests like Criterion. We'll send a PR. It should be similar to #6953 by me adding
doctestfield into the same placeepage commentedon May 15, 2026
Note this is not tagged as
S-accepted, so it is not ready for a PR.I'm also curious how RustRover expects to use this.
vlad20012 commentedon May 15, 2026
@epage We use
cargo test -- -Z unstable-options --format json --show-outputto show test results as a tree view instead of a raw terminal, but that does not work with tests without the harness, so we need the field to choose on whether to show test tree using these flags or not.epage commentedon May 16, 2026
Thanks for sharing details on the use case! I'm off two minds on this.
It isn't guaranteed that alternative harnesses will reimplement the same interface though many try. In particular in this case, the unstable json output is more unique to the built-in harness. Being able to differentiate it can help. As part of my thought process for stabilizing json output includes exploring how to declare what a harness supports which would also be a big help for you with this.
On the other hand, adding stable features that better enable the use of unstable features is a little concerning. The current json output will be broken. If you are curious with where it is heading, my current thoughts are tracked at https://github.com/assert-rs/libtest2, particularly https://github.com/assert-rs/libtest2/blob/main/DESIGN.md, https://github.com/assert-rs/libtest2/blob/main/crates/libtest-json/src/event.rs or https://github.com/assert-rs/libtest2/blob/main/crates/libtest-json/event.schema.json.
I am curious what the rest of the team thinks on this (it is already marked as
S-needs-team-input)epage commentedon May 21, 2026
We had a non-binding conversation at the All Hands where we had a mixture of
Seems like we could go ahead with this. The change is small. We will still have an FCP to get the full team's input if not talked about sooner in the next cargo team meeting.
feat: add `harness` field to cargo metadata output (rust-lang#11846)
feat: add `harness` field to cargo metadata output (rust-lang#11846)
epage commentedon Jun 9, 2026
In looking at the PR, I'm reminded that for third party test harnesses opting into the first party contract, we'd expand
harness. This has the issue of changing a fields shape which can break people due to how people naively implement things in serde.@oli-obk any thoughts from your end on dealing with changes to the shape of a type? Is it fine? Should we wait / make this unstable?
oli-obk commentedon Jun 9, 2026
cargo_metadata doesn't care. We'll adjust our parsing and most of the ecosystem won't notice.
As for ui_test and similar custom harnesses, we do our best to emulate the default harness, and it would be sad if we had worse IDE support just because we're not the default harness
feat: add `harness` field to cargo metadata output (rust-lang#11846)
feat: add `harness` field to cargo metadata output (rust-lang#11846)
feat: add `harness` field to cargo metadata output (rust-lang#11846)