Bound concurrent filesystem reads during asset discovery and source ZIP generation #278

Summary

The Devvit CLI currently performs some filesystem reads with unbounded concurrency. For projects containing large numbers of assets or source files, this can result in hundreds or thousands of simultaneous readFile() operations.

While this is functionally correct, it can unnecessarily increase memory usage, file descriptor pressure, and disk contention.

Current behavior

Two areas currently read files with effectively unbounded concurrency:

• packages/cli/src/util/AssetUploader.ts

  • queryAssets() uses Promise.all() to read every asset simultaneously.

• packages/cli/src/util/getAppSourceZip.ts

  • Source files are added to the ZIP by launching all file reads at once before awaiting them.

Proposed solution

Reuse the existing mapAsyncWithMaxConcurrency() helper to limit concurrent filesystem reads to a reasonable maximum (for example, 16).

This keeps the implementation consistent with the rest of the repository while:

• reducing peak memory usage
• reducing file descriptor pressure
• improving behavior on slower disks and CI runners
• preserving parallelism

Importantly, this proposal only limits filesystem reads.

It does not change:

• upload concurrency
• ZIP contents
• hashing behavior
• ignore rules
• asset ordering
• output format

Benefits

• Lower peak resource usage
• Better scalability for repositories with many files
• More predictable performance across operating systems
• Minimal implementation risk

Implementation notes

A small, self-contained change would:

• replace the Promise.all() asset reads in AssetUploader.ts
• replace the unbounded ZIP source file reads in getAppSourceZip.ts
• reuse the existing mapAsyncWithMaxConcurrency() utility
• keep directory traversal behavior unchanged

This should be a low-risk performance improvement with no observable behavioral changes.