Skip to content

fix: bound incomplete path GET storage - #136

Closed
LIghtJUNction wants to merge 1 commit into
mainfrom
codex/fix-chunk-uploads-to-prevent-disk-exhaustion
Closed

LIghtJUNction wants to merge 1 commit into
mainfrom
codex/fix-chunk-uploads-to-prevent-disk-exhaustion

Conversation

@LIghtJUNction

Copy link
Copy Markdown
Contributor

Motivation

  • Prevent unauthenticated resumable GET /g/v1/chunk/... uploads from growing persistent SQLite storage unbounded and from indefinitely preventing TTL-based cleanup by replaying a single chunk.
  • Enforce global aggregate limits so chunk storage participates in the same eviction/accounting rules as posts and attachments.

Description

  • Add a new configurable limit path_max_transfers and expose it in the configuration, example config, rendered schema, and help text (src/msgd/config.py, deploy/etc/msg-lmm-best/msg.conf, src/msgd/render.py, src/msgd/server.py).
  • Count persisted chunk bytes in the global storage accounting by including SUM(LENGTH(data)) FROM path_get_chunks in _storage_bytes() and surface path_chunk_transfers/path_chunk_bytes in statistics (src/msgd/store.py).
  • Enforce a transactionally-checked global transfer-count and aggregate-byte bound inside put_path_get_chunk while holding the store lock, returning appropriate StoreError conditions when limits are reached (src/msgd/store.py).
  • Make exact chunk replays idempotent but prevent them from extending transfer TTLs by updating created only when progress is made, and thread the new max_storage_bytes/max_transfers parameters through the server chunk handler (src/msgd/store.py, src/msgd/server.py).
  • Add regression tests that assert global bounded accounting, the transfer-count limit, storage-cap enforcement, and that exact replays do not refresh TTLs (tests/test_path_get.py).

Testing

  • Ran linting with uv run ruff check . and it passed.
  • Ran the focused path-get tests with uv run python -m unittest tests.test_path_get -v and observed all tests in that file pass (9 tests passed).
  • Ran the full test suite with uv run python -m unittest discover -s tests -v and observed 147 tests passed and 1 skipped (the skip is due to an unset VALKEY_TEST_URL).
  • Verified repository checks with git diff --check and local status; no outstanding style or diff-check failures remained.

Codex Task

Copilot AI lite review requested due to automatic review settings September 28, 2026 11:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-09-28T11:51:36.075584Z e66f001 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@LIghtJUNction

Copy link
Copy Markdown
Contributor Author

已被 #143 取代,关闭。旧补丁打在已删除的 src/msgd 上;现行 src/msg 的对应容量上限在分支 fix/capacity-recovery-20260928(head e77b53b),该分支已包含合并后的 #142。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants