Make the extension size limit configurable, with namespace and extension overrides #2129

Upstream implementation issue for the requirements proposed in EclipseFdn/open-vsx.org#7864 (see the proposal comment): an admin-configurable default upload size limit, plus overrides for verified namespaces and individual extensions, with extension > namespace > default precedence.

This issue records what that needs in this repository. Line references are against main at the time of writing.

The constraint that shapes the design

The limit is enforced while the request body is still streaming, and the namespace is not known until after the upload has finished.

LocalRegistryService.java:841:

try (var content = new DrainOnCloseInputStream(rawContent, publishingConfig.getMaxContentSize())) {
    var tempFile = extensions.createExtensionFile(content);   // limit enforced here
    ...
    try (var processor = new ExtensionProcessor(tempFile)) {  // namespace known only here

ExtensionService.createExtensionFile caps the stream at maxContentSize + 1 and rejects with 413. Neither publish endpoint (RegistryAPI.java:1433 and :1490) carries a namespace — the package body is the only source of identity.

A per-namespace limit therefore cannot be applied up front. The workable shape is two-stage:

1. Stream against a ceiling of max(default, highest configured override). Cheap to compute and cacheable; with no overrides configured it equals the default, so current behaviour is unchanged.
2. Once ExtensionProcessor has read the manifest, resolve the applicable limit (extension, then namespace, then default) and reject with 413 if the package exceeds it.

The trade-off is worth stating plainly: an oversized upload from a namespace with no override is accepted up to the ceiling before being rejected. That is the cost of the namespace not being in the request. The alternative is a namespace-qualified publish endpoint, which is a much larger and client-breaking change.

What already exists and should not be rebuilt

• The configurable default has a home. The generic Setting key/value store with SettingsService + SettingsCache (Redis-invalidated across pods) currently holds only read-only. A max-extension-size key belongs there, with ovsx.publishing.max-content-size (PublishingConfig, default 512 MB) as the bootstrap fallback.
• The audit trail exists. LogService.logAction(admin, ResultJson) writes a PersistedLog row from the result's success message, and every admin mutation already calls it. The "previous and new values" requirement means composing them into that message, consistent with the other admin actions.
• The admin CRUD page has a template. webui/src/pages/admin-dashboard/tiers/ (tiers.tsx, tier-form-dialog.tsx, delete-tier-dialog.tsx, use-tiers.ts) is the shape this page needs. Tiers themselves are rate-limit buckets on customers and are not a home for size limits.
• Verification has one definition. RepositoryService.isVerified(namespace) is hasMemberships(namespace, ROLE_OWNER).

New: one entity, one migration

A single table rather than two, scoped the way personal_access_token already does it, so the pattern stays consistent:

extension_size_override
  id
  scope_namespace_id  FK namespace,  ON DELETE CASCADE
  scope_extension_id  FK extension,  ON DELETE CASCADE, nullable
  max_size            bigint not null
  created_by_id       FK user_data
  created_at, updated_at
  unique (scope_namespace_id, scope_extension_id)  -- NULLS NOT DISTINCT

Cascades from the start. Resolution is one query returning both candidate rows for the namespace, then picking the most specific. The resolved limits and the ceiling should be cached and invalidated through the same Redis channel SettingsService already uses.

Decisions the requirements do not settle

What happens when a namespace stops being verified? The requirements forbid granting an override to an unverified namespace but say nothing about verification lapsing, which happens when the last owner is removed. Suggestion: keep the override applying, since silently shrinking a publisher's limit mid-release is worse, and surface "namespace no longer verified" on the admin page for review. Worth deciding explicitly, because the opposite choice was made for trusted publishers (those are revoked when namespace ownership lapses, #2114). Either is defensible; they should not diverge by accident.

Is there a hard ceiling on what an admin can set? Nothing would stop an override of 100 GB. Since storage and bandwidth are the stated concern, an absolute maximum that both the admin UI and the API validate against seems worth having.

Retroactivity. The requirements say an extension override applies to future versions. Since the limit is only checked at publish time and nothing re-validates stored versions, that falls out for free — but it is worth confirming nobody expects existing oversized versions to be pruned.

Consequence for the CLI

cli/src/publish.ts reads the single global maxExtensionSize from /api/version and rejects locally before uploading (added in #1953). With overrides in place that value is wrong for a namespace with a higher limit, and the CLI would refuse to upload a package the server would accept — a false negative worse than having no check.

/api/version cannot answer this, as it has no extension context. Either the local check becomes advisory (warn, upload, let the server decide), or the real limit is resolved per extension through a small endpoint. Advisory looks preferable, since the requirements already call for the 413 to state the applicable limit and point at the documentation, which covers the user-facing need.

Prerequisite outside this repository

As noted in the downstream thread, the application has allowed 512 MB for years while the infrastructure caps uploads at 256 MB. None of the above is observable until the deployment limit is raised: the reverse-proxy body size, and on the application side Tomcat's maxSwallowSize, which governs how much of a rejected body is drained (what DrainOnCloseInputStream works around). Those need to move together, and the ceiling should not exceed what the infrastructure permits.

Suggested order

1. Infrastructure limit raised, and the app-side ceiling plumbing verified against it.
2. Default limit into Setting, with settings API and UI. Delivers the 1 GB starting point on its own.
3. Entity, migration, resolution and caching, two-stage enforcement, 413 message carrying the applicable limit and a documentation link.
4. Admin page on the tiers template, with LogService audit on each mutation.
5. CLI pre-check made advisory.
6. Public documentation.

Steps 1 and 2 resolve the original request on their own; 3 to 5 deliver the targeted override proposal.