Skip to content

npm security updates time out in VulnerabilityAuditor on a large workspace monorepo (buildDependencyChains path explosion) #16451

Open

Description

@rcdailey

Is there an existing issue for this?

  • I have searched the existing issues

Package ecosystem

npm

Package manager version

npm 11.17.0 (as reported by the updater)

Language version

Unknown for the updater image. My local reproduction used Node.js 22.20.0 with @npmcli/arborist 10.0.3.

Manifest location and content before the Dependabot update

/package.json and /package-lock.json in a private repo, so I can't link them. Shape of the project:

  • npm workspaces: apps/* and libs/*
  • lockfileVersion: 3
  • 3,180 entries under packages in the lockfile
  • A large root overrides block

dependabot.yml content

version: 2
updates:
  - package-ecosystem: npm
    directory: /
    schedule:
      interval: weekly
    open-pull-requests-limit: 0
    groups:
      react:
        applies-to: security-updates
        patterns:
          - react
          - react-dom
      tamagui:
        applies-to: security-updates
        patterns:
          - tamagui
          - '@tamagui/*'
      storybook:
        applies-to: security-updates
        patterns:
          - storybook
          - '@storybook/*'
      typescript-eslint:
        applies-to: security-updates
        patterns:
          - '@typescript-eslint/*'
          - typescript-eslint

Updated dependency

Transitive dependencies with open alerts, for example fast-uri 3.1.6, undici 6.28.0, ip-address 10.3.1.

What you expected to see, versus what you actually saw

I expected each security update job to finish its audit in a few seconds and either open a PR or report that no fix is possible.

Instead, node /opt/npm_and_yarn/dist/run.js (the vulnerability auditor) goes silent for most transitive dependencies and gets killed after 900 seconds of inactivity. A job with several alerts runs for 30+ minutes, and one run was canceled by a runner shutdown before it finished. This has happened on every security update run since security updates were turned on for the repo (4 runs over 4 days).

I reproduced it outside of Dependabot and I believe the time is spent in buildDependencyChains in npm_and_yarn/helpers/lib/npm/vulnerability-auditor.ts, not in npm or the network:

Dependency arb.audit() buildDependencyChains walk
fast-uri (<3.1.7) 0.6s still running when I stopped it at 120s (775M recursive calls, 2 chains found)
image-size (<2.0.3) 3.4s finished in 1.8s (14.9M calls, 44,328 chains)

That matches the job logs: image-size is the only dependency whose audit completes, and the logs show no registry traffic during the 900 seconds before each timeout, so the process is CPU bound.

My read on the cause: helper only tracks visited per path (a new Set for each branch), so it enumerates every distinct path from the root to the vulnerable package. On a densely connected tree that grows exponentially. I suspect #14754 made this much more likely for workspace repos, since following Link targets means the walk now descends into every workspace's full dependency tree instead of stopping at the link. I haven't confirmed that by testing the version before #14754.

Native package manager behavior

npm update fast-uri --force --ignore-scripts --package-lock-only (the same command the updater runs after the audit fails) completes in about 9 seconds locally.

Images of the diff or a link to the PR, issue, or logs

The repo is private, so here are excerpts from a job log (updater image ghcr.io/dependabot/dependabot-updater-npm:a194169ca2c503ed0c53cf1194723f819c5ce28e):

22:32:40 INFO <job_1598794700> Checking if fast-uri 3.1.6 needs updating
22:32:40 INFO <job_1598794700> Latest version is 4.2.1
22:32:40 INFO <job_1598794700> VulnerabilityAuditor: starting audit
22:32:40 INFO <job_1598794700> Started process PID: 1248 with command: node /opt/npm_and_yarn/dist/run.js
22:32:47 proxy GET https://registry.npmjs.org:443/fast-uri
22:32:47 proxy GET https://registry.npmjs.org:443/ajv
22:47:47 WARN <job_1598794700> Process PID: 1248 timed out after 900s. Terminating...
22:47:48 ERROR <job_1598794700> Process PID: 1248 failed due to timeout: Timed out due to inactivity after 900 seconds
22:47:48 INFO <job_1598794700> VulnerabilityAuditor: npm:vulnerabilityAuditor failed after 907.37s while auditing fast-uri: http fetch POST 200 http://127.0.0.1:37807/-/npm/v1/security/advisories/bulk 42ms

The same pattern repeats for undici (1002s) and ip-address (903s) in that job, and again in the other runs.

This is the script I used to time the two phases. The walk mirrors buildDependencyChains but only counts results:

import Arborist from "@npmcli/arborist";
import http from "http";

const [dir, name, range] = process.argv.slice(2);
const body = JSON.stringify({ [name]: [{ id: 1, vulnerable_versions: range }] });
const server = http.createServer((req, res) => {
  res.writeHead(200, { "Content-Type": "application/json" });
  res.end(body);
});
await new Promise((r) => server.listen(0, "127.0.0.1", r));
const opts = {
  path: dir,
  auditRegistry: `http://127.0.0.1:${server.address().port}`,
  force: true,
  dryRun: true,
  ignoreScripts: true,
};

let t = Date.now();
const report = await new Arborist(opts).audit();
console.log(`audit(): ${(Date.now() - t) / 1000}s`);

const CAP_MS = 120_000;
let calls = 0;
let chains = 0;
t = Date.now();
const helper = (node, visited) => {
  if (!node) return;
  if (++calls % 1_000_000 === 0 && Date.now() - t > CAP_MS) throw new Error("cap");
  const n = node.target ?? node;
  if (visited.has(n.name)) return;
  if (report.has(n.name)) {
    if (report.get(n.name).isVulnerable(n)) return void chains++;
    if (n.name === name) return;
  }
  if (!n.edgesOut.size) return;
  const next = n.isProjectRoot ? visited : new Set([n.name, ...visited]);
  for (const e of n.edgesOut.values()) helper(e.to, next);
};
try {
  helper(report.tree, new Set());
  console.log(`walk: ${(Date.now() - t) / 1000}s, calls=${calls}, chains=${chains}`);
} catch {
  console.log(`walk: capped at ${CAP_MS / 1000}s, calls=${calls}, chains=${chains}`);
}
server.close();
node probe.mjs <project-dir> fast-uri "<3.1.7"

Smallest manifest that reproduces the issue

I don't have a minimal manifest yet. The trigger seems to be tree size and connectivity rather than any one dependency, so I'm not sure a small manifest would show it. I'm happy to run a patched helper against the real project if that would help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions