Ozone guide: add missing X-Iceberg-Access-Delegation header#3591
Ozone guide: add missing X-Iceberg-Access-Delegation header#3591snazy wants to merge 1 commit intoapache:mainfrom
X-Iceberg-Access-Delegation header#3591Conversation
This aligns the Ozone guide with the other object storages guides.
| --conf spark.sql.catalog.polaris.token-refresh-enabled=false \ | ||
| --conf spark.sql.catalog.polaris.warehouse=quickstart_catalog \ | ||
| --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ | ||
| --conf spark.sql.catalog.polaris.header.X-Iceberg-Access-Delegation="" \ |
There was a problem hiding this comment.
Why do we need an empty value here?
There was a problem hiding this comment.
So that Polaris does not vend any credentials.
See the ceph guide.
There was a problem hiding this comment.
BTW: The Polaris behavior might have changed in the meantime, since the Ozone guide went initially in.
There was a problem hiding this comment.
Absent header should mean no credential vending 🤔
Does it not work without the empty header?
There was a problem hiding this comment.
@snazy so I ran ur branch for the CI testing but with my local image (compiled from latest main from public repo) and it worked after removed the vended-credentials: https://gist.github.com/MonkeyCanCode/ff5dc40fca97cc122c503cd130b400c8
As you can see in the above:
...
::endgroup::
::group::Execution of Write SQL file for code block at line 53
::endgroup::
::group::Dump of Spark SQL shell code block at line 53
${SPARK_SQL_BIN} \
--packages org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.iceberg:iceberg-aws-bundle:1.9.0 \
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.polaris.type=rest \
--conf spark.sql.catalog.polaris.uri=http://localhost:8181/api/catalog \
--conf spark.sql.catalog.polaris.token-refresh-enabled=false \
--conf spark.sql.catalog.polaris.warehouse=quickstart_catalog \
--conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \
--conf spark.sql.catalog.polaris.credential=root:s3cr3t \
--conf spark.sql.catalog.polaris.client.region=us-west-2 \
--conf spark.sql.catalog.polaris.s3.access-key-id=polaris_root \
--conf spark.sql.catalog.polaris.s3.secret-access-key=polaris_pass \
-f "${BUILD_TESTS_DIR}/.current.sql"
rm -f "${BUILD_TESTS_DIR}/.current.sql"
...
Test summary:
✅ Test passed for it/../../site/content/guides/ozone/index.md (47.89s)
There was a problem hiding this comment.
Even with an explicit Polaris server image build from "latest main", it is still failing with the same error message.
But it does not fail with apache/polaris:latest (1.3.0) pulled from Docker Hub.
So there is definitely something odd.
There was a problem hiding this comment.
Hmm, that is odd as I built from latest last week. Let me take another look if there are any other differences later tonight.
There was a problem hiding this comment.
So I built the image from your repo's main and it still work. Please see detail commands and output in https://gist.github.com/MonkeyCanCode/8c670106e1f3111f31a2982b512460e0.
There was a problem hiding this comment.
So this is weird: I accidentally purged literally all container images (didn't read the warning emitted by docker image prune -a 🤦) and after that it worked for me as well.
This aligns the Ozone guide with the other object storages guides.