Repository navigation
Ignition + TurboFan: Node.js benchmarks #11851
Description
- Version: 8.0
- Platform: Windows 7 x64
- Subsystem: v8, benchmark
This commit in the v8/node/vee-eight-lkgr has turned the new Ignition+TurboFan pipeline on by default. To see the performance impact, I've built vee-eight-lkgr before and after this commit and have started common Node.js benchmark suites with them.
Both builds have these same versions:
{ http_parser: '2.7.0',
node: '8.0.0-pre',
v8: '5.9.0 (candidate)',
uv: '1.11.0',
zlib: '1.2.11',
ares: '1.10.1-DEV',
modules: '56',
openssl: '1.0.2k',
icu: '58.2',
unicode: '9.0',
cldr: '30.0.3',
tz: '2016j' }As my machine is not a fast one, I will post the results piecemeal.
UPD. All suites are complete.
TOC:
- arrays (+ fixed benchmarks)
- assert
- buffers (+
buffer-swap.jswith--set n=1e6) - child_process
- crypto
- dgram
- domain
- es (+ new
foreach-bench.js) - events
- fs
- http
- misc
- module
- net (except
tcp-raw-pipe.js) - os
- path
- process
- querystring
- streams (+ on Windows)
- string_decoder
- timers (+ on Windows)
- tls
- url
- util
- vm (+ on Windows)
Activity
vsemozhetbyt commentedon Mar 14, 2017
arrays:
vsemozhetbyt commentedon Mar 14, 2017
assert:
mscdex commentedon Mar 14, 2017
Yikes... those performance hits... especially for Uint8Array.
hashseed commentedon Mar 14, 2017
There is still improvements being made in the area of TypedArrays: https://bugs.chromium.org/p/v8/issues/detail?id=5977
Fishrock123 commentedon Mar 14, 2017
Hmm yeah I think we should definitely keep an eye on that before merging. We use
UInt8Arraya lot.hashseed commentedon Mar 14, 2017
@bmeurer @psmarshall could we keep an eye on this?
bmeurer commentedon Mar 14, 2017
Yes, this is mostly known. We know that there's some work to be done, not only for TypedArrays.
vsemozhetbyt commentedon Mar 15, 2017
I had to abort
buffer-swap.jsbenchmark after 2 runs because 1 run lasts ~ 2.5 hours, so default 60 runs would take 6 days. Maybe somebody with a free fast machine will be able to run a wholebuffer-swap.jscycle later.buffers:
A gist with the same data.
mscdex commentedon Mar 15, 2017
@vsemozhetbyt You can reduce the iteration count
n(and/or the number of runs if the results don't seem to vary too wildly) if a single run takes too long. I would make sure a single run executes for at least 5-10 seconds or so to give V8 plenty of time to properly optimize code and whatnot.vsemozhetbyt commentedon Mar 16, 2017
@mscdex If I get it right, the problem is that a nominal 'one run', as the progress bar and the
compare.jshave it, includes many real runs with different config variants, from 2 up to more than 100. These real runs can last a very different amount of time with the samenparameter, from a second up to several minutes. If we try to keep all these real runs near at least 5-10 seconds, the real runs with maximum time span get a huge time span.For example, I've reduced
nforbuffer-swap.jsfrom5e7to1e6. The whole set of 60 nominal runs has lasted ~ 3 hours, with one nominal run ~ 3 min. This benchmark has 66 config variants, some of them have lasted near 1 second, some of them have lasted near 15-20 seconds. To balance these cases, thenshould be changed separately and conditionally for each config variant. Unfortunately, this can be achieved either by editing the code or by many complicated CLI arguments sets and separate runs.vsemozhetbyt commentedon Mar 16, 2017
buffer-swap.js with --set n=1e6:
vsemozhetbyt commentedon Mar 16, 2017
child_process:
mscdex commentedon Mar 16, 2017
@vsemozhetbyt Yep, benchmarking isn't always easy :-) Explicitly passing a lower
--runsvalue may help as previously noted, especially for benchmarks that don't involve I/O and thus are usually less volatile.47 remaining items
YurySolovyov commentedon Apr 13, 2017
I just startted the PC and went working.
Here is
arrays assert child_process crypto dgram domainand somees.I've selected node 6 vs node 8 cause this is probably what most people would migrate from.
Results of 12h of benchmarking:
Maybe someone can suggest a number of iterations so that all benches can run in ~8h ?
I know net ones can be fuzzy and buffer just takes way too long.
mrmeswani commentedon Apr 20, 2017
Is there an easy way to turn off and use the older v8 compiler? I am trying to test this in my server setup and i would like to run some tests with acmeair benchmark. I am not quite sure of how I can get an earlier branch of v8/node/vee-eight-lkgr before the commit.
hashseed commentedon Apr 21, 2017
You can pass
--no-turbo. Or you could check out this commit.vsemozhetbyt commentedon Apr 24, 2017
Today the Node.js canary branch have been updated up to v8 6.0.
I've built it on Windows and run this simple benchmark script with a real life case, comparing canary and nightly:
This is a significant progress since almost the same v8 5.9 test (the eslint versions are different + doc linting is added, but I think this hardly distorts the comparison).
mhdawson commentedon Apr 25, 2017
Nightly benchmark run on latest canary: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-footprint-canary/1/console. Will try to pull out comparison summary against master tomorrow.
mhdawson commentedon Apr 26, 2017
Summary comparison between master and canary for nightly benchmarks. Note: just a single run of each so need more runs to validate:
MylesBorins commentedon Jun 20, 2017
Could we get an update on these stats based on the current 6.0 branch cut?
sathvikl commentedon Jun 21, 2017
@MylesBorins could you point to the 6.0 branch cut you are referring to .. ?
gibfahn commentedon Jun 23, 2017
I think @MylesBorins is talking about the nightly v8-canary build (which is node master + V8 lkgr), and available here: https://nodejs.org/download/v8-canary/
vsemozhetbyt commentedon Jun 23, 2017
FWIW, v8-canary has V8 6.1 now.
targos commentedon Jun 23, 2017
If someone wants to build and test Node with V8 6.0, I have this branch: https://github.com/targos/node/tree/v8-6.0
targos commentedon Aug 20, 2017
should this stay open?
vsemozhetbyt commentedon Aug 20, 2017
@targos I think we can close it now. If anybody objects, feel free to reopen.