[aljoscha]

The cluster controller realizes configurations accepted by DDL. Checking the owner's current size allow-list again can strand refresh schedules and hydration bursts, and block allowed resizes in the same decision batch.

Concretize controller replicas without a size allow-list. Keep allocation lookup, disabled-size checks, availability-zone configuration, and transactional resource limits unchanged. DDL remains responsible for authorizing requested sizes.

Extend cluster-controller testdrive coverage for refresh wake-up and hydration bursts after the allow-list shrinks, including DDL rejection and an allowed resize while the retained burst is active.

Fixes SQL-564.

[def-]

QA LLM Review

1. MEDIUM -- Two ALTER CLUSTER paths never authorize HYDRATION SIZE, so the size allow-list is now bypassable

src/adapter/src/coord/cluster_controller.rs:990

Dropping the controller's allow-list check assumes that DDL has already authorized every size the controller realizes. That is false for the graceful reshape path and for the unmanaged-to-managed conversion, which both commit an AUTO SCALING STRATEGY without checking its HYDRATION SIZE. Before this PR the controller refused those burst replicas. Now any cluster owner can get replicas at a size that allowed_cluster_replica_sizes forbids, and CREATE CLUSTER would reject the same policy.

Details

reshape_alter_cluster_managed (src/adapter/src/coord/sequencer/inner/cluster.rs:527-534) validates only target.size. It then commits new_config with the new auto_scaling_strategy applied immediately (:630-684). sequence_alter_cluster_unmanaged_to_managed (:1589-1720) calls ensure_valid_replica_size nowhere. The planner (plan_auto_scaling_strategy, validate_auto_scaling_strategy in src/sql/src/plan/statement/ddl.rs) does not check sizes either. Only sequence_create_managed_cluster (:887-899) and the direct managed-to-managed path (:1486-1499) check it. The flags enable_auto_scaling_strategy and enable_hydration_burst both default on.

Repro, with allowed_cluster_replica_sizes = 'scale=1,workers=1', 'scale=1,workers=2', run as an ordinary role:

CREATE CLUSTER c (SIZE 'scale=1,workers=1');
-- SIZE change routes to reshape; the strategy is committed unchecked
ALTER CLUSTER c SET (SIZE 'scale=1,workers=2', AUTO SCALING STRATEGY = (ON HYDRATION (HYDRATION SIZE = 'scale=2,workers=2')));

CREATE CLUSTER u REPLICAS (r1 (SIZE 'scale=1,workers=1'));
ALTER CLUSTER u SET (MANAGED, AUTO SCALING STRATEGY = (ON HYDRATION (HYDRATION SIZE = 'scale=2,workers=2')));

When a burst arms on either cluster, build_create_replica_op now creates a scale=2,workers=2 replica. The conversion path also leaves an explicit SIZE on an empty cluster unchecked. For example, ALTER CLUSTER e SET (MANAGED, SIZE = '<forbidden>', SCHEDULE = ON REFRESH (...)) then gets its refresh-window replica realized at that size. Only max_credit_consumption_rate still limits the cost. The new testdrive only covers rejection through CREATE CLUSTER.

A fix is to authorize these sizes in the two DDL paths before relying on DDL-only authorization. Check HYDRATION SIZE whenever the strategy changes in reshape_alter_cluster_managed, and check both the explicit SIZE and HYDRATION SIZE in sequence_alter_cluster_unmanaged_to_managed, through one shared helper.

[aljoscha]

one issue was already addressed by my concurrent #40447, pushed a fix for the other issue

[ggevay]

LGTM. One inline suggestion and three notes.

• Description: it becomes the commit message, but it doesn't mention the conversion change. SET (MANAGED, SIZE = ...) on an empty cluster now rejects sizes that are off the allow-list, unknown or disabled. Main stores all three.

• SQL-564: Fixes closes the issue, but its second fix, distinguishable errors for missing, off-list and disabled sizes, isn't in this PR. The new tests still expect "unknown cluster replica size" for all three. Should this PR say Part of instead, or should we move that item to its own issue?

• Question, not blocking: should an unchanged SIZE skip the allow-list too, here or in a follow-up? DDL still re-validates it, so once SIZE has left the list, even SET (REPLICATION FACTOR 0) fails with "unknown cluster replica size 100cc", unless the statement also moves SIZE to an allowed value. Main does this too, but with this PR the controller keeps provisioning replicas for such a cluster. It's your #39557 point about retired sizes.

  I tried it on this head (commit, no new tests, in case you want it): such ALTERs work, a new off-list size is still rejected, and the tests that exercise the allow-list pass. The catch is that REPLICATION FACTOR can then grow at an off-list size, as #39557 allows for disabled sizes.

[ggevay]

The list isn't per owner: it's one system variable, with only mz_system exempt. "DDL authorizes" also misses sizes that older releases stored through the conversion this PR fixes. Maybe:

Suggested change

	// DDL authorizes sizes before committing the configuration. Reconcile
	// must still realize it if the owner's size allow-list later shrinks.
	// The allow-list is a DDL-time check on the role writing a size.
	// Reconcile does not repeat it, so a size the current list rejects
	// still gets its replica, for example after the list shrinks.