[rust] Add Fluss 1.x protocol support to the admin client by gnuhpc · Pull Request #631 · apache/fluss-rust

gnuhpc · 2026-06-19T01:10:54Z

What

Adds 27 new admin methods to FlussAdmin, consuming the message wrappers from #629/#630:

Database/table extensions: list_database_summaries, alter_database, alter_table, get_table_stats
KV snapshot operations: get_latest_kv_snapshots, get_kv_snapshot_metadata, create_kv_snapshot_lease, get_lake_snapshot
ACL management: create_acls, list_acls, drop_acls
Cluster configuration: describe_cluster_configs, alter_cluster_configs
Server management: add_server_tag, remove_server_tag, rebalance, list_rebalance_progress, cancel_rebalance
Producer offsets: register_producer_offsets, get_producer_offsets, delete_producer_offsets
Monitoring: get_cluster_health, list_remote_log_manifests
KV snapshots: list_kv_snapshots, release_kv_snapshot_lease, drop_kv_snapshot_lease

(The proto-compat fix for scanner.rs / admin.rs lives in #628 so that PR leaves the tree building.)

Stack

Part 4/6, stacked on #630 → #629 → #628. All target main.

🤖 Generated with Claude Code

gnuhpc · 2026-06-19T19:05:34Z

Rebased onto updated #630 and applied the same proto::*/Default cleanup the maintainer requested on #629 (#629 (review)) to every signature this PR adds. All 21 admin methods that previously leaked proto::Pb* in args or returns now take/return domain types. use crate::proto; is gone from admin.rs; only the pre-existing use crate::proto::GetTableInfoResponse; (from main) remains.

Response-side domain types added under crates/fluss/src/metadata/:

cluster_health.rs — ClusterHealth
kv_snapshot.rs — KvSnapshot, LatestKvSnapshots, RemotePathAndLocalFile, KvSnapshotMetadata, AcquireKvSnapshotLeaseResult, ActiveKvSnapshots
lake_snapshot.rs — LakeBucketSnapshot, LakeSnapshotInfo (distinct from the existing LakeSnapshot used by get_latest_lake_snapshot)
rebalance.rs — RebalanceProgress, TableRebalanceProgress, BucketRebalanceProgress, BucketRebalancePlan
remote_log.rs — RemoteLogManifestEntry
Extended config.rs (DescribeConfig), database.rs (DatabaseSummary), table_stats.rs (BucketStats/TableStats/BucketStatsError), acl.rs (CreateAclResult/DropAclsFilterResult/DropAclMatchingAcl), producer_offsets.rs (ProducerOffsets)

For trivially-shaped responses (single field), methods now return the primitive directly instead of wrapping — e.g. rebalance returns String (the rebalance id), register_producer_offsets returns Option<i32>, list_acls returns Vec<AclInfo>, list_remote_log_manifests returns Vec<RemoteLogManifestEntry>.

alter_table parameter fix (same drop-silently-features class as target_columns in #629): previously took only config_changes + add_columns and hardcoded vec![],vec![],vec![] for drop/rename/modify columns. Now takes an AlterTableChanges bundle struct (avoids clippy::too_many_arguments) exposing all 5 change types, plus ignore_if_not_exists is now a parameter instead of hardcoded false.

#[derive(Default)] not present anywhere in this PR (admin.rs has none); the wrappers consumed are already cleaned up on #629/#630.

cargo build + clippy (-D warnings) + 556 unit tests pass locally. Pushed 29a4d0b.

gnuhpc · 2026-06-20T17:12:53Z

Rebased on top of updated #630. Same pub(crate) visibility pattern applied. All admin signatures use domain types (only the pre-existing use crate::proto::GetTableInfoResponse; import remains). alter_table now exposes drop/rename/modify column changes via an AlterTableChanges bundle struct.

Verified against real Java Fluss 0.9.1-incubating (68/68 integration tests pass) and Fluss 1.0-SNAPSHOT-dev (73/73 with fluss_v1 feature).

fresh-borzoni

@gnuhpc thank you for the PR, left some comments, PTAL

fresh-borzoni · 2026-06-21T13:48:36Z

+        let response = self
+            .admin_gateway()
+            .await?
+            .request(GetLakeSnapshotRequest::new(table_path, snapshot_id, None))


why do we hardcode readable to None?

Done. Exposed as a parameter in get_lake_snapshot(table_path, snapshot_id, readable).

fresh-borzoni · 2026-06-21T13:52:36Z

+    pub in_sync_replicas: i32,
+    pub num_leader_replicas: i32,
+    pub active_leader_replicas: i32,
+    pub status: i32,


Done. Added ClusterHealthStatus enum (Green/Yellow/Red/Unknown). from_pb now returns Result since the conversion can fail on unknown values.

fresh-borzoni · 2026-06-21T13:53:08Z

+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct BucketRebalanceProgress {
+    pub rebalance_plan: BucketRebalancePlan,
+    pub rebalance_status: i32,


Done. Added RebalanceStatus enum (NotStarted/Rebalancing/Failed/Completed/Canceled/Timeout). from_pb returns Result with collect::<Result<Vec<_>>>()? for the nested bucket progress.

fresh-borzoni · 2026-06-21T13:53:19Z

+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct RebalanceProgress {
+    pub rebalance_id: Option<String>,
+    pub rebalance_status: Option<i32>,


Done. Same enum applied here as Option<RebalanceStatus>.

fresh-borzoni · 2026-06-21T13:59:56Z

+/// Per-bucket KV snapshot info.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct KvSnapshot {
+    pub bucket_id: i32,


these use raw i64/i32 for table_id/partition_id/bucket_id, but the crate has TableId/PartitionId/BucketId aliases (lib.rs) that the rest of the code use. Let's stay consistent

Done. Applied TableId/PartitionId/BucketId across all metadata types: kv_snapshot, lake_snapshot, producer_offsets, rebalance, table_stats, and kv_snapshot_lease.

…dable - Add ClusterHealthStatus enum (Green/Yellow/Red/Unknown) to cluster_health.rs - Add RebalanceStatus enum (NotStarted..Timeout) to rebalance.rs - Replace raw i32/i64 with BucketId/TableId/PartitionId aliases in kv_snapshot.rs, lake_snapshot.rs, rebalance.rs, producer_offsets.rs - Expose readable parameter in admin.get_lake_snapshot() instead of hardcoding None Addresses reviewer feedback on PR apache#631. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fresh-borzoni · 2026-06-24T13:23:29Z

@gnuhpc Thank you, can you, please, rebase/cherry-pick/resolve conflicts, so it's comfortable to continue review?

Add 27 new admin methods to FlussAdmin: - Database/table extensions: list_database_summaries, alter_database, alter_table, get_table_stats - KV snapshot operations: get_latest_kv_snapshots, get_kv_snapshot_metadata, create_kv_snapshot_lease, get_lake_snapshot - ACL management: create_acls, list_acls, drop_acls - Cluster configuration: describe_cluster_configs, alter_cluster_configs - Server management: add_server_tag, remove_server_tag, rebalance, list_rebalance_progress, cancel_rebalance - Producer offsets: register_producer_offsets, get_producer_offsets, delete_producer_offsets - Monitoring: get_cluster_health, list_remote_log_manifests - KV snapshots: list_kv_snapshots, release_kv_snapshot_lease, drop_kv_snapshot_lease

…dable - Add ClusterHealthStatus enum (Green/Yellow/Red/Unknown) to cluster_health.rs - Add RebalanceStatus enum (NotStarted..Timeout) to rebalance.rs - Replace raw i32/i64 with BucketId/TableId/PartitionId aliases in kv_snapshot.rs, lake_snapshot.rs, rebalance.rs, producer_offsets.rs - Expose readable parameter in admin.get_lake_snapshot() instead of hardcoding None Addresses reviewer feedback on PR apache#631. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…hot_lease) Extends the BucketId/TableId/PartitionId alias consistency fix to table_stats.rs (BucketStatsRequest, BucketStats) and kv_snapshot_lease.rs (KvSnapshotLeaseForBucket, KvSnapshotLeaseForTable). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Now that pr/3 provides GoalType and ServerTag enums in the RPC wrappers, update the admin client methods to use them in their public signatures too. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Update the new admin method signatures introduced by this PR to use the i64/i32 type aliases from lib.rs instead of raw primitives, matching the underlying RPC message wrappers. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

gnuhpc · 2026-06-24T23:35:29Z

@fresh-borzoni Rebased onto main, conflicts resolved. Stack is clean now.

fresh-borzoni

@gnuhpc Thank you for the changes, left some comments, PTAL

fresh-borzoni · 2026-06-25T12:19:18Z

+        producer_id: &str,
+        table_offsets: Vec<ProducerTableOffsets>,
+        ttl_ms: Option<i64>,
+    ) -> Result<Option<i32>> {


Java's registerProducerOffsets returns a RegisterResult enum (fromCode, throws on unknown).
Could we port that as an enum + try_from_i32 like RebalanceStatus/ClusterHealthStatus do

Done — added RegisterProducerResult { Created, AlreadyExists } in metadata/, with to_i32/try_from_i32 matching the other enums. register_producer_offsets now returns Option<RegisterProducerResult> (Option kept because the proto field is optional).

fresh-borzoni · 2026-06-25T12:22:05Z

+    pub async fn get_table_stats(
+        &self,
+        table_id: TableId,
+        buckets_req: Vec<crate::metadata::BucketStatsRequest>,


why do we have FQ paths here and in some other parts(AlterConfig, PbDatabaseSummary) as well?

Done — BucketStatsRequest added to the import block here; also cleaned up the FQ for PbDatabaseSummary in metadata/database.rs and the self-referencing crate::metadata::AlterConfig in metadata/table_change.rs.

fresh-borzoni · 2026-06-25T12:40:24Z

+    }
+
+    /// Create ACLs. Returns one result per submitted ACL (success or per-ACL error).
+    pub async fn create_acls(&self, acl: Vec<AclInfo>) -> Result<Vec<CreateAclResult>> {


nit: why acl, if we pass vector, should it be plural?

Done — renamed to acls.

fresh-borzoni · 2026-06-25T12:40:35Z

+    }
+
+    /// Drop ACLs matching filters. Returns one result per submitted filter.
+    pub async fn drop_acls(&self, acl_filter: Vec<AclFilter>) -> Result<Vec<DropAclsFilterResult>> {


ditto: plural

Done — renamed to acl_filters.

fresh-borzoni

@gnuhpc Also this

fresh-borzoni · 2026-06-25T12:53:27Z

+        producer_id: &str,
+        table_offsets: Vec<ProducerTableOffsets>,
+        ttl_ms: Option<i64>,
+    ) -> Result<Option<i32>> {


enum as well?

Done — same enum applies; see reply on the other thread.

fresh-borzoni · 2026-06-25T12:54:17Z

+    }
+
+    /// Alter a database's configuration.
+    pub async fn alter_database(


Why did we drop comment param?

Good catch — restored. Added comment: Option<&str> to both alter_database and the AlterDatabaseRequest wrapper, passing through to the proto field that was being hardcoded to None.

- Add `RegisterProducerResult` enum mirroring Java's `RegisterResult` (Created=0, AlreadyExists=1); `register_producer_offsets` now returns `Option<RegisterProducerResult>` instead of raw `Option<i32>`. - Expose `comment: Option<&str>` on `alter_database` and the `AlterDatabaseRequest` wrapper (was hardcoded to `None`). - Replace fully-qualified paths with imports: `BucketStatsRequest` in admin.rs, `PbDatabaseSummary` in metadata/database.rs, and `AlterConfig` self-reference in metadata/table_change.rs. - Rename `create_acls(acl)` to `create_acls(acls)` and `drop_acls(acl_filter)` to `drop_acls(acl_filters)` for plural consistency with the `Vec` argument type. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

gnuhpc · 2026-06-26T01:31:47Z

@fresh-borzoni All 5 round-2 items addressed in 828f9fe — new RegisterProducerResult enum, comment exposed on alter_database, FQ paths cleaned in 3 spots, plural ACL param names. pr/5/6 rebased to follow.

fresh-borzoni

@gnuhpc Thank you for the changes, LGTM 👍

This was referenced Jun 19, 2026

[rust] Add version-aware integration test harness (0.9.x / 1.x) #632

Open

[rust] Add RPC message wrappers for core DB/table operations #629

Merged

gnuhpc force-pushed the pr/4-admin-methods branch from e4a4715 to 29a4d0b Compare June 19, 2026 19:05

gnuhpc force-pushed the pr/4-admin-methods branch from 29a4d0b to 9fbe919 Compare June 20, 2026 17:12

fresh-borzoni reviewed Jun 21, 2026

View reviewed changes

fresh-borzoni mentioned this pull request Jun 21, 2026

[rust] Add RPC message wrappers for extended operations #630

Merged

gnuhpc force-pushed the pr/4-admin-methods branch from 9fbe919 to 5e7f29c Compare June 22, 2026 02:50

gnuhpc force-pushed the pr/4-admin-methods branch from 5e7f29c to 3537c7d Compare June 24, 2026 01:55

warmbupt and others added 5 commits June 25, 2026 07:31

Update admin.rs to use GoalType/ServerTag enums after rebase

4a9cdf6

Now that pr/3 provides GoalType and ServerTag enums in the RPC wrappers, update the admin client methods to use them in their public signatures too. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

gnuhpc force-pushed the pr/4-admin-methods branch from 3537c7d to 6c52a1f Compare June 24, 2026 23:34

fresh-borzoni reviewed Jun 25, 2026

View reviewed changes

fresh-borzoni approved these changes Jun 26, 2026

View reviewed changes

fresh-borzoni merged commit 0dae316 into apache:main Jun 26, 2026
11 checks passed

Uh oh!

Conversation

gnuhpc commented Jun 19, 2026

What

Stack

Uh oh!

gnuhpc commented Jun 19, 2026

Uh oh!

gnuhpc commented Jun 20, 2026

Uh oh!

fresh-borzoni left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fresh-borzoni commented Jun 24, 2026

Uh oh!

gnuhpc commented Jun 24, 2026

Uh oh!

fresh-borzoni left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fresh-borzoni Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fresh-borzoni left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gnuhpc commented Jun 26, 2026

Uh oh!

fresh-borzoni left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

fresh-borzoni Jun 25, 2026 •

edited

Loading