Tracking Issue for Parallel Rustc Front-end #113349
Description
This is a tracking issue for Parallel Rustc Front-end
This feature will improve compilation speed by parallelizing the process of rustc front end (before the codegen which has been parallelized).
The feature was previously called parallel queries (and its tracking issue can be found here), developed by the former parallel rustc working group. Thanks to their work, developers can currently use parallel rustc by setting parallel_compiler=true in config.toml when building the compiler.
The parallel rustc working group has rebooted (here's the associated MCP), which will make parallel front-end the default option in rustc. Here's the (draft) landing strategy for this feature.
Below is the task list.
Solve the problem of single-thread performance reduction
- Introduce
DynSendandDynSyncauto trait for parallel compiler #107586 - Use only one shard with a single thread #111755
- Make
Shardedan enum and specialize it for the single thread case #114860 - Use conditional synchronization for Lock #111713
- Remove conditional use of
Shardedfrom query state #114894 - Add optimized lock methods for
Shardedand refactorLock#115388 - make query caches use
Shardedonly for multiple threads #115162
Optimize multi-threading performance
Enable parallel compilation by default in nightly rustc
- test for parallel compiler by default #115235
- enable parallel rustc front end in nightly builds #117435
Bugs or test failures of parallel rustc
- get rid of
tcxin deadlock handler when parallel compilation #98570 - Check that a query has not completed and is not executing before starting it #108845
- Deadlock when printing query stack -- fixed by Avoid calling queries during query stack printing #112708
- Exponential (?) time complexity in
evaluate_trait_predicate_recursivelyin rustdoc when proving Send/Sync #106930 - ICE: parallel compiler:
attempted to read from stolen value: rustc_middle::thir::Thir:-Zthir-unsafeck=yes#111520 - Fix waiting on a query that panicked #115198
- Add a
CurrentGcxtype to let the deadlock handler accessTyCtxt#115220 - Add
ParallelGuardtype to handle unwinding in parallel sections #115144 - ICE: parallel compiler:
deadlock detected#111521 - ICE: parallel compiler:
no ImplicitCtxt stored in tls#111522 - ICE: parallel compiler:
value must be in cache after waiting#111528 - UI test failures with parallel-compiler=true #75760
- parallel compiler: unconditional panic
Resource temporarily unavailablewhen limiting max memory #115021 - parallel compiler:
assertion failed: found_cycle#115223 - other issues here
Testing
- Add parallel front-end test suite to compiletest #118698
- Enhance rustc-perf options to support multi-threading
Documentation
- Adjust documentation (doc-guide)
About tracking issues
Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Activity
the8472 commentedon Jul 10, 2023
The
corecrate might be a good benchmark for this. You can immediately build it after rustc and it spends about 10% of its walltime in tokenization/parsing/macro-expansion and ~80% in the query parts. The fraction of codegen is small in comparison.tigerfsh commentedon Nov 7, 2024
It's been a long time since there was any update. Has all work been paused？
SparrowLii commentedon Nov 7, 2024
Them main work now is to fix deadlock and ice bugs in the issue list
I recently submitted a PR for an enhanced compiler test to reproduce the issue and successfully located three deadlock bugs. Unfortunately, it takes timeto figure out the cause of these bugs, which is what is making the updates slow.
I expect it will take another 1 to 2 months to fix the remaining bugs, after which the stabilization will begin.
Auto merge of rust-lang#132282 - Noratrieb:it-is-the-end-of-serial, r…
safinaskar commentedon Feb 3, 2025
@SparrowLii
As well as I understand there is no such option anymore. Please, update post
Auto merge of rust-lang#136471 - safinaskar:parallel, r=<try>
Auto merge of rust-lang#136471 - safinaskar:parallel, r=SparrowLii
Auto merge of #136471 - safinaskar:parallel, r=SparrowLii
Auto merge of rust-lang#136772 - safinaskar:parallel-2025-02-08-11-55…
Auto merge of rust-lang#136772 - safinaskar:parallel-2025-02-08-11-55…
Auto merge of #136471 - safinaskar:parallel, r=SparrowLii
14 remaining items
Auto merge of #143035 - ywxt:less-work-steal, r=<try>
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #144722 - ywxt:parallel-reproducibile, r=oli-obk
Auto merge of #144722 - ywxt:parallel-reproducibile, r=oli-obk
Auto merge of #144722 - ywxt:parallel-reproducibile, r=oli-obk
Auto merge of #144722 - ywxt:parallel-reproducibile, r=SparrowLii
Auto merge of #144722 - ywxt:parallel-reproducibile, r=SparrowLii
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
Auto merge of #143035 - ywxt:less-work-steal, r=oli-obk
petrochenkov commentedon Jul 8, 2026