Skip to content

feat(computing-unit): add admin cluster capacity endpoint - #8986

Open
kunwp1 wants to merge 2 commits into
apache:mainfrom
kunwp1:feat/admin-cu-cluster-capacity
Open

kunwp1 wants to merge 2 commits into
apache:mainfrom
kunwp1:feat/admin-cu-cluster-capacity

Conversation

@kunwp1

@kunwp1 kunwp1 commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Problem. When a computing unit fails to start because the cluster is full, an admin cannot see from the app how much of the cluster is left (#6476), and the cluster capacity panel (#6482) has nothing to show. The backend only lists pods in the computing-unit pool namespace and never reads node capacity.

Fix. Adds GET /api/computing-unit/admin/cluster-capacity (ADMIN only), which returns total, allocated and available for CPU (millicores), memory (bytes) and GPU (devices). total is the allocatable of every Ready, uncordoned node, and allocated is the requests of the pods running on them in every namespace, with init containers and runtime-class overhead counted the way kube-scheduler counts them. That is the figure kubectl describe node reports as "Allocated resources", so available bounds what a new unit can claim. Live usage from metrics-server is not used because it reads below what is committed and has no GPU metric. The pod list is narrowed on the API server to pods that have not finished, so a cluster that keeps many finished pods does not ship them all. On a deployment without Kubernetes computing units it answers 404, and a Kubernetes API failure comes back with the cluster's own message.

Nodes are cluster-scoped, so the chart adds a ClusterRole that can only list nodes and pods, bound to the computing unit manager's service account. This is the one new permission: the manager can now list pods in every namespace, where it could previously see only its own pool.

Before:  admin sees a unit stuck Pending -> kubectl describe nodes -> adds up the requests by hand
After:   admin sees a unit stuck Pending -> GET /computing-unit/admin/cluster-capacity -> available CPU, memory, GPU

Not included. The panel that shows it (#6482) and telling the owner on terminate (#6483). available is an upper bound, not a go or no-go: it sums across nodes, so capacity spread thinly over many nodes can still refuse a unit that needs one node's worth, and it ignores the pool's ResourceQuota, node placement (taints, node selectors, runtime-class scheduling), pod-level resources and native sidecars. The ClusterRole is unconditional, with no chart value to turn it off.

Any related issues, documentation, discussions?

Closes #6481. Part of #6476.

Feeds the panel in #6482 and follows #8888, which added the page.

How was this PR tested?

Tests were added for the aggregation, the client wrapper, the guard and the endpoint. They cover node and pod filtering (cordoned, not Ready, no Ready condition, no status, unbound pods, finished pods), the effective-request rule (container sum, init containers, overhead, containers with no resources), decimal and fractional quantities, a custom GPU resource key, available floored at zero, the wrapper reading nodes and the unfinished pods of every namespace, the 404 on a non-Kubernetes deployment, a Kubernetes failure keeping its message, and the JSON field names the panel will read. The tests were written first and watched fail on a stub, apart from the JSON field-name test, which pins the response shape. Breaking the aggregation on purpose (finished pods counted, init containers summed instead of maxed, no floor at zero, a not-Ready node counted, a cordoned node counted, overhead dropped) turns at least one test red each time.

sbt "ComputingUnitManagingService/testOnly org.apache.texera.service.util.ClusterCapacitySpec org.apache.texera.service.util.KubernetesClientSpec org.apache.texera.service.util.ComputingUnitHelpersSpec org.apache.texera.service.resource.AdminComputingUnitResourceSpec"
sbt "ComputingUnitManagingService/test"

The first command passes 107 tests and the second passes 289. scalafmtCheckAll and scalafixAll --check ran clean. The ClusterRole was checked by rendering its template alone with helm template against the chart's values.yaml (rendering the whole chart needs its subcharts downloaded) and asserting it grants exactly list on nodes and pods and is bound to the manager's service account, and bin/k8s/tests/test_helm_values.sh passes. Not exercised: a live Kubernetes cluster, since the local stack has none, so the Kubernetes calls are only checked against a stubbed client, and the endpoint over HTTP.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code, Claude Sonnet 5.5

@kunwp1 kunwp1 self-assigned this Oct 14, 2026
@kunwp1

kunwp1 commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @aicam

@github-actions github-actions Bot added feature infra platform Non-amber Scala service paths labels Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @eugenegujing, @tanishqgandhi1908, @dzueck
    You can notify them by mentioning @eugenegujing, @tanishqgandhi1908, @dzueck in a comment.

@github-actions
github-actions Bot requested a review from aicam October 14, 2026 19:55
@codecov-commenter

codecov-commenter commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.83673% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.62%. Comparing base (a182a23) to head (ad6bd0d).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...g/apache/texera/service/util/ClusterCapacity.scala 94.28% 0 Missing and 2 partials ⚠️
.../service/resource/AdminComputingUnitResource.scala 0.00% 1 Missing ⚠️
...che/texera/service/util/ComputingUnitHelpers.scala 83.33% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8986      +/-   ##
============================================
- Coverage     92.62%   92.62%   -0.01%     
- Complexity     5060     5062       +2     
============================================
  Files          1258     1259       +1     
  Lines         53791    53830      +39     
  Branches       6707     6709       +2     
============================================
+ Hits          49824    49859      +35     
- Misses         2313     2315       +2     
- Partials       1654     1656       +2     
Flag Coverage Δ *Carryforward flag
access-control-service 77.38% <ø> (ø)
agent-service 99.16% <ø> (ø) Carriedforward from a182a23
amber 88.11% <ø> (ø) Carriedforward from a182a23
computing-unit-managing-service 61.81% <91.83%> (+1.32%) ⬆️
config-service 87.37% <ø> (ø)
file-service 82.00% <ø> (ø)
frontend 96.60% <ø> (ø) Carriedforward from a182a23
notebook-migration-service 83.73% <ø> (ø)
pyamber 98.58% <ø> (ø) Carriedforward from a182a23
workflow-compiling-service 72.43% <ø> (-1.67%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature infra platform Non-amber Scala service paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Admin CUs: backend endpoint for cluster resource capacity

2 participants