[DO-NOT-MERGE] Change MinIO to RustFS for integration testing by MonkeyCanCode · Pull Request #3663 · apache/polaris

MonkeyCanCode · 2026-02-04T04:48:21Z

I am not sure if we want to take this PR as it is changing existed integration testing on S3-compatible storage from MinIO to RustFS instead of creating a new test-containers. But here are the high level changes from this PR:

Refactored MinIO test-containers to RustFS test-containers
Use fixed port for RustFS test-containers due to it needs :<service_port> when using virtual host style access instead of only domain (details in virtual host mode caused rustfs mistakenly consider other instance's domain in cluster as bucket name rustfs/rustfs#1593)
Changed S3 exception message as RustFS returns it without period at the end (change in integration-tests/src/main/java/org/apache/polaris/service/it/test/CatalogFederationIntegrationTest.java)
Removed custom cmd used by MinIO as RustFS image has those configured already
Changed healthcheck port from 9000 (used by MinIO) to 9001 (used by RustFS)

ML: https://lists.apache.org/thread/8o31ly7cd8ov70opjbtg630qlhrfl5yh

Checklist

🛡️ Don't disclose security issues! (contact security@apache.org)
🔗 Clearly explained why the changes are needed, or linked related issues: Fixes #
🧪 Added/updated tests with good coverage, or manually tested (and explained how)
💡 Added comments for complex logic
🧾 Updated CHANGELOG.md (if needed)
📚 Updated documentation in site/content/in-dev/unreleased (if needed)

adutra

Thanks for the PR @MonkeyCanCode , it's really interesting that you were able to migrate so smoothly!

Indeed, I wonder if it wouldn't be better to have both MinIO and RustFS tests, at least for some time.

Also, the need for a fixed port is a minor annoyance, but we can live with that I guess.

adutra · 2026-02-04T10:21:35Z

tools/rustfs-testcontainer/src/main/java/org/apache/polaris/test/rustfs/RustfsContainer.java

-    addExposedPort(DEFAULT_PORT);
+    withNetworkAliases(randomString("rustfs"));
+    withLogConsumer(new Slf4jLogConsumer(LoggerFactory.getLogger(RustfsContainer.class)));
+    addFixedExposedPort(S3_API_PORT, S3_API_PORT);


Do we really need fixed ports?

Hmm I see, it's explained here:

https://lists.apache.org/thread/5cvgk6m83hnsoly9gq4bxdchcc69f3l0

That's unfortunate, because it could cause test failures especially for devs (not so much for CI imho).

Could we have a code comment for this too (for ease of recollection later :) )?

Yeah. that is the unfortunate pieces I found so far. I will add a comment for this.

adutra · 2026-02-04T10:23:12Z

tools/rustfs-testcontainer/src/main/java/org/apache/polaris/test/rustfs/RustfsContainer.java

-  private static final String MINIO_DOMAIN = "MINIO_DOMAIN";
+  private static final String RUSTFS_ACCESS_KEY = "RUSTFS_ACCESS_KEY";
+  private static final String RUSTFS_SECRET_KEY = "RUSTFS_SECRET_KEY";
+  private static final String RUSTFS_SERVER_DOMAINS = "RUSTFS_SERVER_DOMAINS";


nit: why "DOMAINS" in the plural?

This is because RUSTFS_SERVER_DOMAIN is not valid but RUSTFS_SERVER_DOMAINS instead (https://github.com/rustfs/rustfs/blob/e30781654d26006cda575fa0746775eb3d29b8d3/rustfs/src/config/mod.rs#L73). But I think you are ref to the variable name. I am changing this to RUSTFS_DOMAIN in the new PR as we are using one domain only.

adutra · 2026-02-04T14:57:18Z

Heads up: you need to rebase this PR for it to pass CI.

dimas-b · 2026-02-04T15:15:57Z

Indeed, I wonder if it wouldn't be better to have both MinIO and RustFS tests, at least for some time.

+1 to running MinIO tests and RustFS tests in parallel for a while. If all is well in CI we can reduce load by removing MinIO tests later.

dimas-b · 2026-02-04T15:19:58Z

We could probably start with adapting these tests to RustFS (by copy initially): RestCatalogMinIOSpecialIT, PolarisRestCatalogMinIOIT and keep Federation tests on MinIO initially... just for the same of simplicity... WDYT?

dimas-b · 2026-02-04T15:26:59Z

CHANGELOG.md

 - Changed from Poetry to UV for Python package management.
 - Exclude KMS policies when KMS is not being used for S3.
 - Improved default KMS permission handling to better distinguish read-only and read-write access.
+- Replaced MinIO with RustFS for integration testing.


nit: TBH, I doubt users are interested in this and developers can view GH PRs 😉

dimas-b · 2026-02-04T15:29:04Z

...tests/src/main/java/org/apache/polaris/service/it/test/CatalogFederationIntegrationTest.java

    assertThatThrownBy(() -> spark.sql("INSERT INTO ns1.test_table VALUES (3, 'Charlie')"))
        .hasMessageContaining(
-            "software.amazon.awssdk.services.s3.model.S3Exception: Access Denied. (Service: S3, Status Code: 403,");
+            "software.amazon.awssdk.services.s3.model.S3Exception: Access Denied (Service: S3, Status Code: 403,");


Suggestion: use RegEx for account for both cases.

Sure thing. let me update it in a diff PR.

dimas-b · 2026-02-04T15:31:16Z

Actually, this PR looks good to me as it is. Perhaps, as discussed above, we could keep RestCatalogMinIOSpecialIT, PolarisRestCatalogMinIOIT on MinIO for a while. Migrating NoSQL and Federation tests to RustFS is fine.

MonkeyCanCode · 2026-02-05T02:30:52Z

Thanks so much for the quick review @dimas-b and @adutra . I think we all think it is better to have both minio and rustfs test-containers co-existed and copied over the existed s3-compatible storage IT over with the needed changes in this PR. I will keep this one and mark it as DO-NOT-MERGE in case if we have more people who would like to review. In the mean time, I will create a different PR for the new test-containers.

MonkeyCanCode · 2026-02-05T07:08:53Z

@adutra @dimas-b I have created #3679 to address the feedback and have both test-containers co-existed as well as migrated certain ITs to RustFS. Please let me know if anything should be change. I will close this one for now.

MonkeyCanCode added 2 commits February 3, 2026 22:40

Change MinIO to RustFS for integration testing

f5e16d7

Change MinIO to RustFS for integration testing

174a39c

github-project-automation bot added this to Basic Kanban Board Feb 4, 2026

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Feb 4, 2026

MonkeyCanCode requested review from dimas-b, flyrain and snazy and removed request for dimas-b February 4, 2026 04:48

Change MinIO to RustFS for integration testing

70ec59e

adutra reviewed Feb 4, 2026

View reviewed changes

dimas-b reviewed Feb 4, 2026

View reviewed changes

MonkeyCanCode changed the title ~~Change MinIO to RustFS for integration testing~~ [DO-NOT-MERGE] Change MinIO to RustFS for integration testing Feb 5, 2026

MonkeyCanCode requested a review from jbonofre February 5, 2026 02:31

MonkeyCanCode mentioned this pull request Feb 5, 2026

Add support for rustfs test-containers #3679

Merged

6 tasks

MonkeyCanCode closed this Feb 5, 2026

github-project-automation bot moved this from PRs In Progress to Done in Basic Kanban Board Feb 5, 2026

Conversation

MonkeyCanCode commented Feb 4, 2026

Checklist

Uh oh!

adutra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MonkeyCanCode Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adutra commented Feb 4, 2026

Uh oh!

dimas-b commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dimas-b commented Feb 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimas-b commented Feb 4, 2026

Uh oh!

MonkeyCanCode commented Feb 5, 2026

Uh oh!

MonkeyCanCode commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MonkeyCanCode Feb 5, 2026 •

edited

Loading

dimas-b commented Feb 4, 2026 •

edited

Loading

dimas-b Feb 4, 2026 •

edited

Loading