Repository navigation
Conversation
Add a configurable parent for remote log downloads, matching the Java client capability while preserving the Rust OS temporary-directory default. Each scanner owns an isolated temporary subdirectory and retains the existing cleanup lifecycle. Expose the option in Python and C++ and document it in both API trees. Cover directory creation, isolation, cleanup, invalid paths, Unix permission errors, and binding configuration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Fixes #4602.
Allow users to configure the parent directory for remote log segment downloads in the Rust client and its Python and C++ bindings.
The default parent directory is the OS temporary directory plus
/fluss, matching the Java client'sclient.scanner.io.tmpdirdefault. This change aligns remote log download directory configuration with Java and covers remote log segments, including changelog scanning for KV tables. KV snapshot downloads are outside its scope.Brief change log
scanner_io_tmp_dirto the Rust configuration and expose it in Python and C++.remote-logs-<UUID>subdirectory for each scanner and create directories when an actual download starts.Tests
Tests cover default directory selection, lazy creation, paths containing spaces, directory isolation and cleanup, invalid paths, Unix permission errors, and download retry/error propagation.
A live OSS or Fluss-cluster download was not exercised. Subsequent edits only simplified documentation and changed test path strings from
/datato/tmp; the full suite was not rerun after those edits.API and Format
Adds the following configuration entry points:
Config::scanner_io_tmp_dirConfig.scanner_io_tmp_dirand thescanner.io.tmpdirconfiguration keyConfiguration::scanner_io_tmp_dirRust and Python use
Nonefor the default; C++ uses an empty string. No storage format, wire protocol, or RPC changes.Documentation
Updated the Rust README configuration table and the Rust, Python, and C++ API references in both documentation trees.
Generative AI disclosure
Yes — Codex was used to prepare the implementation, tests, and documentation.