[jeremydmiller]

Closes #5820

What it is

src/TenantRebuildStress: a console harness for the JasperFx/jasperfx#1113 and #1127 scenario. It uses per-tenant event partitioning, many tenants, and a composite projection rebuilt with one agent per tenant. It reports the process's peak physical footprint, sampled once a second with macOS top -stats mem,cmprs (never RSS).

• Scenario: stage 1 is a multi-stream projection in which every event touches --fanout of --buckets large documents (--payload-kb), so each page builds thousands of big upserts. Stage 2 is a small single-stream projection.
• Rebuild: a store-wide RebuildProjectionAsync(name, tenantId: null, ...), with MaxConcurrentRebuildsPerDatabase = --concurrency so every tenant rebuilds at once.
• Queue setting: --serialize on|off sets DaemonSettings.SerializeTenantProjectionBatches by reflection, so the same harness runs on a JasperFx from before the setting existed (--serialize unset).
• Modes: --mode seed|rebuild|both. Seed once, then rebuild in fresh processes so runs share one data set. Each run prints a RESULT|version|serialize|seconds|peakMemMB|peakMem+CmprsMB|buckets|touches line and the projected counts, to check against the events.

First numbers (MacBook, Postgres 17 in Docker)

24 tenants × 1,000 events, 4,000 buckets of 8 KB per tenant, 40 buckets per event, BatchSize 500, 24 concurrent cells. Every run projected the same 95,997 buckets and 955,190 touches, which matches a count taken straight from mt_events.

JasperFx	SerializeTenantProjectionBatches	GC	Peak MEM	Rebuild
2.98.0	—	Server	5,562 MB	4.9 s
2.100.0-local.1	off	Server	5,766 MB	4.6 s
2.100.0-local.1	on	Server	4,992 MB	13.8 s
2.98.0	—	Workstation	8,283 MB	6.9 s
2.100.0-local.1	off	Workstation	8,230 MB	7.4 s
2.100.0-local.1	on	Workstation	355 MB	16.4 s

The batch queue bounds the memory that built batches hold: 8.2 GB to 355 MB under workstation GC, with the rebuild about twice as slow. Under server GC on this many-core box, the GC's per-heap allocation budgets set the footprint (about 5 GB) whatever the queue does, so the queue's effect barely shows there. A container with a memory limit constrains those budgets, so the warm-up node in #1113 is the measurement that settles it. This harness can be run with DOTNET_GCHeapHardLimit to approximate one.

The 2.100 rows need the JasperFx 2.100 pin and #5829, which are on the jasperfx-2.100-local branch. On master, a second per-tenant composite rebuild on the same schema projects nothing (#5829). The harness itself only references Marten, so it builds on master's pin.

🤖 Generated with Claude Code