Repository navigation
gcc fails to compile with ASAN enabled #43370
Description
Version
16.14.2
Platform
Linux simon-linuxdesktop 5.6.14-050614-generic #202005200733 SMP Wed May 20 07:38:05 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Subsystem
No response
What steps will reproduce the bug?
Reproduce: attempt to build node with debug and asan enabled: ./configure --debug --enable-asan && make -j4
Same issue on ubuntu 22.04 and 20.04, and with gcc/g++ 9 and 11.
How often does it reproduce? Is there a required condition?
Every time
What is the expected behavior?
No response
What do you see instead?
...
g++ -o /home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/default-worker-threads-task-runner.o ../deps/v8/src/libplatform/default-worker-threads-task-runner.cc '-D_GLIBCXX_USE_CXX11_ABI=1' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64' '-DLEAK_SANITIZER' '-DV8_USE_ADDRESS_SANITIZER' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_HAVE_TARGET_OS' '-DV8_TARGET_OS_LINUX' '-DV8_EMBEDDER_STRING="-node.20"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DENABLE_MINOR_MC' '-DOBJECT_PRINT' '-DV8_INTL_SUPPORT' '-DV8_ATOMIC_OBJECT_FIELD_WRITES' '-DV8_ATOMIC_MARKING_STATE' '-DV8_ENABLE_LAZY_SOURCE_POSITIONS' '-DV8_USE_SIPHASH' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_WIN64_UNWINDING_INFO' '-DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH' '-DV8_SNAPSHOT_COMPRESSION' '-DV8_ENABLE_WEBASSEMBLY' '-DV8_ALLOCATION_FOLDING' '-DV8_ALLOCATION_SITE_TRACKING' '-DV8_ADVANCED_BIGINT_ALGORITHMS' -I../deps/v8 -I../deps/v8/include -fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope -pthread -Wno-unused-parameter -m64 -Wno-return-type -fno-strict-aliasing -m64 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O2 -fno-rtti -fno-exceptions -std=gnu++14 -MMD -MF /home/simon/dev/vendor/node/out/Release/.deps//home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/default-worker-threads-task-runner.o.d.raw -c
In file included from ../deps/v8/src/libplatform/default-platform.cc:14:
../deps/v8/src/base/platform/platform.h: In static member function ‘static v8::base::Stack::StackSlot v8::base::Stack::GetRealStackAddressForSlot(v8::base::Stack::StackSlot)’:
../deps/v8/src/base/platform/platform.h:477:16: error: operands to ?: have different types ‘char*’ and ‘v8::base::Stack::StackSlot’
476 | return real_frame
| ~~~~~~~~~~
477 | ? (static_cast<char*>(real_frame) + kAsanRealFrameOffsetBytes)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
478 | : slot;
| ~~~~~~
../deps/v8/src/base/platform/platform.h:478:18: error: conversion from ‘v8::base::Stack::StackSlot’ to ‘char*’ is ambiguous
478 | : slot;
| ^~~~
../deps/v8/src/base/platform/platform.h:450:5: note: candidate: ‘v8::base::Stack::StackSlot::operator void*() const’ <near match>
450 | operator void*() const { return reinterpret_cast<void*>(value); }
| ^~~~~~~~
../deps/v8/src/base/platform/platform.h:450:5: note: no known conversion from ‘void*’ to ‘char*’
../deps/v8/src/base/platform/platform.h:451:5: note: candidate: ‘v8::base::Stack::StackSlot::operator uintptr_t() const’ <near match>
451 | operator uintptr_t() const { return value; } // NOLINT
| ^~~~~~~~
../deps/v8/src/base/platform/platform.h:451:5: note: no known conversion from ‘uintptr_t’ {aka ‘long unsigned int’} to ‘char*’
In file included from ../deps/v8/src/libplatform/default-worker-threads-task-runner.h:14,
from ../deps/v8/src/libplatform/default-worker-threads-task-runner.cc:5:
../deps/v8/src/base/platform/platform.h: In static member function ‘static v8::base::Stack::StackSlot v8::base::Stack::GetRealStackAddressForSlot(v8::base::Stack::StackSlot)’:
../deps/v8/src/base/platform/platform.h:477:16: error: operands to ?: have different types ‘char*’ and ‘v8::base::Stack::StackSlot’
476 | return real_frame
| ~~~~~~~~~~
477 | ? (static_cast<char*>(real_frame) + kAsanRealFrameOffsetBytes)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
478 | : slot;
| ~~~~~~
../deps/v8/src/base/platform/platform.h:478:18: error: conversion from ‘v8::base::Stack::StackSlot’ to ‘char*’ is ambiguous
478 | : slot;
| ^~~~
../deps/v8/src/base/platform/platform.h:450:5: note: candidate: ‘v8::base::Stack::StackSlot::operator void*() const’ <near match>
450 | operator void*() const { return reinterpret_cast<void*>(value); }
| ^~~~~~~~
../deps/v8/src/base/platform/platform.h:450:5: note: no known conversion from ‘void*’ to ‘char*’
../deps/v8/src/base/platform/platform.h:451:5: note: candidate: ‘v8::base::Stack::StackSlot::operator uintptr_t() const’ <near match>
451 | operator uintptr_t() const { return value; } // NOLINT
| ^~~~~~~~
../deps/v8/src/base/platform/platform.h:451:5: note: no known conversion from ‘uintptr_t’ {aka ‘long unsigned int’} to ‘char*’
g++ -o /home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/delayed-task-queue.o ../deps/v8/src/libplatform/delayed-task-queue.cc '-D_GLIBCXX_USE_CXX11_ABI=1' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64' '-DLEAK_SANITIZER' '-DV8_USE_ADDRESS_SANITIZER' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_HAVE_TARGET_OS' '-DV8_TARGET_OS_LINUX' '-DV8_EMBEDDER_STRING="-node.20"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DENABLE_MINOR_MC' '-DOBJECT_PRINT' '-DV8_INTL_SUPPORT' '-DV8_ATOMIC_OBJECT_FIELD_WRITES' '-DV8_ATOMIC_MARKING_STATE' '-DV8_ENABLE_LAZY_SOURCE_POSITIONS' '-DV8_USE_SIPHASH' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_WIN64_UNWINDING_INFO' '-DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH' '-DV8_SNAPSHOT_COMPRESSION' '-DV8_ENABLE_WEBASSEMBLY' '-DV8_ALLOCATION_FOLDING' '-DV8_ALLOCATION_SITE_TRACKING' '-DV8_ADVANCED_BIGINT_ALGORITHMS' -I../deps/v8 -I../deps/v8/include -fno-omit-frame-pointer -fsanitize=address -fsanitize-address-use-after-scope -pthread -Wno-unused-parameter -m64 -Wno-return-type -fno-strict-aliasing -m64 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O2 -fno-rtti -fno-exceptions -std=gnu++14 -MMD -MF /home/simon/dev/vendor/node/out/Release/.deps//home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/delayed-task-queue.o.d.raw -c
make[1]: *** [tools/v8_gypfiles/v8_libplatform.target.mk:158: /home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/default-worker-threads-task-runner.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [tools/v8_gypfiles/v8_libplatform.target.mk:158: /home/simon/dev/vendor/node/out/Release/obj.target/v8_libplatform/deps/v8/src/libplatform/default-platform.o] Error 1
rm c811dc5233a4a706afbbaa1b86deb16c3131dc34.intermediate
make: *** [Makefile:113: node] Error 2
Additional information
No response
Activity
aduh95 commentedon Jun 10, 2022
Can you try with Node.js 16.15.1? This GitHub Actions that compiles and test Node.js with ASAN enabled is successful: https://github.com/nodejs/node/runs/6699505955?check_suite_focus=true.
SimonWoolf commentedon Jun 10, 2022
Unfortunately I get the same result with node 16.15.1.
I notice that that github action is using clang wheras I'm gcc, is it possible asan builds have never worked with gcc? I'll try forcing it to build with clang and report back.
Edit: confirmed
F3n67u commentedon Jun 11, 2022
I can reproduce when running
./configure --debug --enable-asan && make -j16.Error happened at:
node/deps/v8/src/base/platform/platform.h
Lines 623 to 639 in cb7e854
After switching to clang/clang++, then build successfully.
Env Info
Node source lastest commit
[-]Node 16 doesn't compile with ASAN enabled[/-][+]gcc fails to compile with ASAN enabled[/+]aduh95 commentedon Jun 11, 2022
It looks like cliang was introduced on the GitHub Actions CI on #32776, although at this time it seems that gcc was able to compile an ASAN build, although it was taking a long time.
@nodejs/build do we want to document that Clang is required for ASAN builds or can we fix the underlying issue?
bnoordhuis commentedon Jun 11, 2022
This is an upstream V8 issue but there's probably an easy fix:
I speculate clang applies an implicit conversion and gcc does not.
F3n67u commentedon Jun 11, 2022
I tried to apply @bnoordhuis's suggest fix and rerun the ASAN build using GCC, but it was never successful because of running out of memory even if the total memory is greater than 32GB. The running out of memory problem when using GCC to run an ASAN build is also mentioned at #32776.
Maybe we should document that Clang is required for ASAN builds just like @aduh95 suggest?
bnoordhuis commentedon Jun 11, 2022
Does that out of memory error happen when linking? If yes, this might help:
Replace
lldwithgoldto use the gold linker.F3n67u commentedon Jun 11, 2022
@bnoordhuis Thank you for your help! It's late night in my timezone, I will try your solution tomorrow and report back.
F3n67u commentedon Jun 13, 2022
@bnoordhuis I failed to make GCC ASAN build work, It fails either GCC being killed or disk space not enough. It seems don't happen during link stage, the command is still
g++ -owhen error happened. (I am not sure, I don't know how to differentiate when is link, I am not familiar with the c++ toolchain)I tried the asan build in the Gitpod WebIDE container, it's less powerful than I thought( I thought it give me 64G memory and 16Core, but it seems it is shared by many instances). I thought I will not help to make any progress to forward this issue, I don't have a powerful Linux computer to verify those things. It will be great if anyone who has a powerful Linux computer could chime in
I found the following points:
outdirectory could be 25Gbnoordhuis commentedon Jun 14, 2022
Thanks for the update, @F3n67u. I agree it makes sense to recommend/require clang if gcc is that problematic.
j1elo commentedon Dec 20, 2022
Hi, I'll chime in to comment my experience with this. I don't see any reason to require clang, or an alternative linker, given that the default tools on Linux (gcc and ld) work properly, provided that the machine has enough resources (and the code fix from #43370 (comment) is applied).
I've been following the instructions from BUILDING.md, which doesn't even mention clang, lld, or gold, so I didn't think of using them. Maybe clang or an alternative linker would require much less space, I don't know... but with GCC and ld, a complete Debug build of Node.js (with
./configure --debug) will take:The second point is the most problematic. Given parallel execution (with either of
./configure --ninjaormake -j$(nproc)), this means that linking 4 executables at the same time (for a 4 core CPU) will require upwards of 32GB of memory. My machine has 16GB, so the kernel Out Of Memory (OOM) killer would terminate theldprocess. Here, the options are to re-run withmake -j1, which only requires around 8GB of free memory; or growing the swap partition to ensure enough memory is available. I tested both options and they worked fine (albeit obviously it's always better to avoid using swap).Both of above points happen because several (unnecessary?) binaries are compiled, and they are multi-GB each:
For a dev trying to report an issue and wanting to run a Debug build of Node.js, only the last one is useful (and it's also the only one installed with
make install).Finally, yes, I can confirm that the Address Sanitizer (
./configure --enable-asan) build does need the patch mentioned in comment #43370 (comment) (tested with and without, with GCC 9.4.0; only with the patch it compiled successfully).To wrap, some questions to advance on this:
nodebinary, thus greatly reducing disk and memory requirements? (but still make a complete debug build, aka. V8; otherwise I already know about./configure --debug-node)-j$(nproc)and the link steps with-j1.lldorgold, for someone who is using those instead ofld?bnoordhuis commentedon Dec 20, 2022
No. You can instruct
maketo omit cctest and embedtest but the mksnapshot binaries are prerequisites.make node -C out BUILDTYPE=Release, IIRC.BUILDTYPE=Debug+ ASAN may be too slow to be usable.It needs to be upstreamed first. We don't, as a rule, float patches that haven't been accepted upstream.
Feel free to upstream it, no attribution needed.
bnoordhuis commentedon Dec 29, 2022
I'm going to close this. The OOM errors can perhaps be avoided by serializing the link step, see #45949 (comment).
deps: V8: cherry-pick cb30b8e17429