Repository navigation
Log output of parallel runs separately and output sequentially #2458
Description
Since rustc is run in parallel sometimes two outputs might write over each other with quite annoying results.
Would be great if the output of each command invocation is logged separately and reported at execution end in some fixed order (e.g. alphabetical).
Activity
alexcrichton commentedon Mar 9, 2016
This is unfortunately pretty difficult to do, for a number of reasons:
The good news is that almost all compiler invocations are output-free unless you're likely working on the top-level crate (where errors will show up). To me though it seems that the current situation is worth it due to the colorization + streaming. If we could preserve both of those but synchronize the output I'd love to do so!
lu-zero commentedon Mar 9, 2016
Passing
--color alwaysto rustc would work just fine for that purpose of coloring, while for gcc and clang-fcolor-diagnosticsshould do.I can reliably reproduce the problem by having any kind of error in the code and the issue
cargo testbtw.alexcrichton commentedon Mar 9, 2016
Unfortunately that wouldn't work for Windows because colors work differently there (they're a property of the console, not of the output stream).
lu-zero commentedon Mar 11, 2016
I have not enough experience about windows to tell alternatives =/
[-]Make the rustc output stable[/-][+]Log output of parallel runs separately and output sequentially[/+]U007D commentedon Jan 23, 2018
Yesterday on IRC, I posted
A discussion followed, focused on buffering the output from multiple processes as the compilation ran, and this issue was referenced as the open issue relevant to my question. Apologies if this is not the correct issue--please let me know and I will move this comment. (This issue's title seems to prescribe an implementation to an implied problem, rather than being a problem/mission statement like "Stop outputting duplicate errors from parallel builds", so perhaps that is where my confusion comes from.)
I and another participant in the discussion wondered why each compilation thread wouldn't consult a centralized dictionary during compilation, something like so:
Process x:
Process y:
The above synchronization strategy could be upgraded to a lock-free algorithm, but the actual synchronization mechanism isn't really the key point here--it's that each process can still live-stream its output without any chance of duplicate error reporting; the delay from buffering, and the color output issues are both side-stepped by retaining the live streaming behavior.
I assume an approach like this has already been discussed and discarded, but on the off chance it hasn't, I wanted to suggest it in hopes it might help?
alexcrichton commentedon Jan 23, 2018
@U007D seems plausible to me! That'd be an issue though for rust-lang/rust rather than Cargo as it's the one actually emitting errors.
U007D commentedon Jan 23, 2018
Oh, so this is the wrong issue, then! :) I'll open the issue/add the comment there. Great, thanks, @alexcrichton.
ishitatsuyuki commentedon Jun 23, 2018
I think the color part may be actually easier than you think. ANSI is considered universal by most tools: they use ANSI in Windows pipes where the special terminal APIs are not available, and on POSIX ANSI is the standard.
So what we should do is: pass
--color=alwaysunconditionally, buffer them, and parse the output (this is the part that needs time to implement). Then, we feed the parsed output into termcolor again, so it's now translated to the native colors, or discarded if colors are not available.Auto merge of #7450 - ehuss:stabilize-cache-messages, r=alexcrichton