Repository navigation
benchmark: disable concurrent recompilation+sweeping? #11683
Description
I've noticed that many benchmarks have lots of variance unless you pass --noconcurrent_sweeping --noconcurrent_recompilation which forces V8 to run the mark-sweep GC phase and the optimizing compiler on the main thread.
Does it make sense to run with those flags by default? Considerations:
- It makes the benchmarks more reliable.
- It's not representative of how node is normally run.
- Micro-benchmarks aren't representative of normal use either.
- Is the variance indicative of problems with V8's thread pool?
(I'd say the answer to 4. is no, variable latency is practically a given with thread pools, but I haven't extensively tested what impact different values for --v8-pool-size=... have.)
cc @nodejs/benchmarking
Activity
joyeecheung commentedon Mar 4, 2017
I am a bit worried that this could make the microbenchmarks even more micro and results in optimizations landing when they can cause slowdowns in real-word usage, or prevent optimizations from landing even they benefit real-world usage..
kunalspathak commentedon Mar 8, 2017
Having the flag will also make it difficult for other VMs to get comparable results with
node-v8which was one of the thing that #9615 addressed.hashseed commentedon Mar 14, 2017
V8 has a --predictable flag to reduce performance flakiness at the cost of running somewhat slower, by turning off a set of concurrency features.
Trott commentedon Jul 30, 2017
This issue has been inactive for sufficiently long that it seems like perhaps it should be closed. Feel free to re-open (or leave a comment requesting that it be re-opened) if you disagree. I'm just tidying up and not acting on a super-strong opinion or anything like that.
Add nodejs flags to reduce variance