Skip to content

Fix ozone getting started example #3574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MonkeyCanCode merged 2 commits into from
Jan 27, 2026
Merged

Fix ozone getting started example #3574

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a4ebec3
Fix Ozone getting started example
MonkeyCanCode Jan 27, 2026
c2c6803
Fix Ozone getting started example
MonkeyCanCode Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions getting-started/ozone/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,16 @@ bin/spark-sql \
--conf spark.sql.catalog.polaris.warehouse=quickstart_catalog \
--conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \
--conf spark.sql.catalog.polaris.credential=root:s3cr3t \
--conf spark.sql.catalog.polaris.client.region=irrelevant
--conf spark.sql.catalog.polaris.client.region=us-west-2 \
--conf spark.sql.catalog.polaris.s3.access-key-id=polaris_root \
--conf spark.sql.catalog.polaris.s3.secret-access-key=polaris_pass
```

Note: `s3cr3t` is defined as the password for the `root` user in the `docker-compose.yml` file.

Note: The `client.region` configuration is required for the AWS S3 client to work, but it is not used in
this example since Ozone does not require a specific region.
Note: The `client.region`, `s3.access-key-id`, and `s3.secret-access-key` configuration are required
for the AWS S3 client to work, but they are not used in this example since Ozone does not require them
when securing S3 is not enabled.

## Running Queries

Expand Down Expand Up @@ -87,11 +90,8 @@ The lack of STS API is represented in the Catalog storage configuration by the

## S3 Credentials

In this example Ozone does not require credentials for accessing its S3 API. Therefore, neither
Polaris, not Spark use any S3 access keys.

If Ozone were configured to require credentials, Spark and Polaris would have to their own separate
S3 access key / secret properties because credential vending is not possible with Ozone 2.0.0.
In this example Ozone does not have securing S3 enabled for accessing its S3 API. Therefore, use
any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY will work for accessing S3 API.

## S3 Endpoints

Expand Down
7 changes: 4 additions & 3 deletions getting-started/ozone/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ services:
OZONE-SITE.XML_ozone.scm.client.address: "ozone-scm"
OZONE-SITE.XML_ozone.scm.datanode.id.dir: "/data/metadata"
OZONE-SITE.XML_ozone.scm.names: "ozone-scm"
OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1"
no_proxy: "ozone-om,ozone-recon,ozone-scm,ozone-s3g,localhost,127.0.0.1"
ozone-om:
image: *ozone-image
Expand Down Expand Up @@ -83,8 +84,8 @@ services:
JAVA_DEBUG: true
JAVA_DEBUG_PORT: "*:5005"
AWS_REGION: us-west-2
AWS_ACCESS_KEY_ID: minio_root
AWS_SECRET_ACCESS_KEY: m1n1opwd
AWS_ACCESS_KEY_ID: polaris_root
AWS_SECRET_ACCESS_KEY: polaris_pass
POLARIS_BOOTSTRAP_CREDENTIALS: POLARIS,root,s3cr3t
polaris.realm-context.realms: POLARIS
quarkus.otel.sdk.disabled: "true"
Expand Down Expand Up @@ -112,7 +113,7 @@ services:
/assets/cloud_providers/await-s3.sh http://ozone-s3g:9878/ ;
source /assets/polaris/obtain-token.sh;
echo Creating bucket...;
curl -X PUT --user "invalidKey:secret" --aws-sigv4 "aws:amz:us-west-1:s3" \
curl -X PUT --user "polaris_root:polaris_pass" --aws-sigv4 "aws:amz:us-west-2:s3" \
http://ozone-s3g:9878/bucket123 ;
echo Creating catalog...;
export STORAGE_CONFIG_INFO='{"storageType":"S3",
Expand Down