Repository navigation
GC crash in Environment::NearHeapLimitCallback with latest V8 #218
Description
After the CLs linked in this crbug: https://bugs.chromium.org/p/v8/issues/detail?id=12503
In test/parallel/test-heapsnapshot-near-heap-limit-worker.js
crashes with the following stacktrace:
stacktrace
#
# Fatal error in ../../v8/src/heap/heap.cc, line 1818
# Debug check failed: AllowGarbageCollection::IsAllowed().
#
#
#
#FailureMessage Object: 0x70000c510630
1: 0x1282c3032 node::NodePlatform::GetStackTracePrinter()::$_3::__invoke() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
2: 0x124ac23a6 V8_Fatal(char const*, int, char const*, ...) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
3: 0x124ac1e45 v8::base::(anonymous namespace)::DefaultDcheckHandler(char const*, int, char const*) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
4: 0x12066086d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
5: 0x120660b05 v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
6: 0x120c18663 v8::internal::HeapSnapshotGenerator::GenerateSnapshot() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
7: 0x120c053a3 v8::internal::HeapProfiler::TakeSnapshot(v8::ActivityControl*, v8::HeapProfiler::ObjectNameResolver*, bool, bool) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
8: 0x1281e680d node::heap::WriteSnapshot(v8::Isolate*, char const*) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
9: 0x1281d97be node::Environment::NearHeapLimitCallback(void*, unsigned long, unsigned long) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
10: 0x120661919 v8::internal::Heap::InvokeNearHeapLimitCallback() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
11: 0x120660682 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
12: 0x12066ba74 v8::internal::Heap::FinalizeIncrementalMarkingIfComplete(v8::internal::GarbageCollectionReason) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
13: 0x1206868db v8::internal::IncrementalMarkingJob::Task::RunInternal() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
14: 0x1282c1976 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task> >) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
15: 0x1282c0586 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
16: 0x11e8b023b uv__async_io [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
17: 0x11e8c265d uv__io_poll [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
18: 0x11e8b0671 uv_run [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
19: 0x12829221f node::SpinEventLoop(node::Environment*) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
20: 0x12830376f node::worker::Worker::Run() [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
21: 0x128306d03 node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::$_3::__invoke(void*) [/Users/codebytere/Developer/electron-gn/src/out/Testing/Electron.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]
22: 0x7ff80cdd44f4 _pthread_start [/usr/lib/system/libsystem_pthread.dylib]
23: 0x7ff80cdd000f thread_start [/usr/lib/system/libsystem_pthread.dylib]
node:assert:400
throw err;
^
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert(stderr.includes('ERR_WORKER_OUT_OF_MEMORY'))
I haven't yet had time to dig into the root cause but it seems that as of now the call to heap::WriteSnapshot(env->isolate(), filename.c_str()); in Environment::NearHeapLimitCallback expected GC to be allowed but at the time it's invoked it's not. We're hitting this in our latest upgrade roll, and i'm happy to work on a PR for this if I could potentially get some guidance for what might be the best approach!
cc @joyeecheung (i believe you added the functionality initially) and @targos
Activity
test: disable test-heapsnapshot-near-heap-limit-worker.js
test: disable test-heapsnapshot-near-heap-limit-worker.js
codebytere commentedon Feb 25, 2022
@targos any potential thoughts on the above?
targos commentedon Feb 25, 2022
I'm not able to reproduce with the latest canary.
chore: bump chromium to 100.0.4894.0 (main) (#32852)
joyeecheung commentedon Feb 26, 2022
The V8 changes moved the invocation of NearHeapLimitCallback (maybe only in some scenarios, and that's why it's not reproducible with canary) to a DisallowGarbageCollection scope, and since taking a heap snapshot incurs GC this naturally breaks the --heapsnapshot-near-heap-limit flag. Maybe the invocation of the NearHeapLimitCallback should be placed in a phase that allows GC instead - I replied in https://bugs.chromium.org/p/v8/issues/detail?id=12503 to ask about this.
chore: bump chromium to 100.0.4894.0 (main) (#32852)
chore: bump chromium to 100.0.4894.0 (18-x-y) (#33093)
joyeecheung commentedon Apr 6, 2022
FYI I opened a separate bug in V8 for this https://bugs.chromium.org/p/v8/issues/detail?id=12777
joyeecheung commentedon Apr 12, 2022
Should be fixed by https://chromium-review.googlesource.com/c/v8/v8/+/3575468
chore: bump chromium to 100.0.4894.0 (main) (electron#32852)