Guides: fix setup scripts to yield correct exit code by snazy · Pull Request #3612 · apache/polaris

snazy · 2026-01-29T14:31:18Z

The statements in the shell scripts for the setup services are often concatenated using ;, which means that a previous' command exit code is not propagated and the service, although it failed, is determined to be successful.

This change updates those scripts to use && for the statement concatenation.

"Final" setup services (aka "polaris-setup") now have a final sleep 120. This is due to the behavior of docker compose up --detach --wait, which considers any service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (#3553). That sleep 120 in "polaris-setup" services does not cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services".

To avoid merge conflicts, this change also:

updates affected curl invocations (as Guides: add mandatory curl --fail option #3610)
removes superfluous restart: "no"

MonkeyCanCode · 2026-01-29T18:17:22Z

getting-started/keycloak/docker-compose.yml

-        apk add --no-cache jq && 
-        chmod +x /polaris/create-catalog.sh && 
+        apk add --no-cache jq &&
        token=$$(curl http://keycloak:8080/realms/iceberg/protocol/openid-connect/token --user client1:s3cr3t -d 'grant_type=client_credentials' | jq -r .access_token) && 


should we do implicitly --fail-with-body here too for curl?

My assumption is that the token would just be "invalid".
It's also that the pipe "eliminates" the exit code from curl, but jq would fail then.

That is fair enough. Yeah. Just thought to be consistent. But not a blocker.

MonkeyCanCode · 2026-01-29T18:24:18Z

getting-started/ceph/docker-compose.yml

+        echo Creating Ceph bucket... &&
+        aws s3 mb s3://${S3_POLARIS_BUCKET} &&
+        aws s3 ls &&
+        echo Bucket setup complete.


This will return 0 as well no?

So we have at most 2 setup services for these docker compose and you are adding sleep 120 to pass the CI based on docker-compose (as exit of status code of zero will be treat as failed). Thus, this will also return status code of zero.

Ah, I think I know what you're referring to.
Those setup-bucket services behave in an "interesting" way, which is different than how the "final" polaris-setup services behave (in terms of exit behavior).
An "intermediate" setup-service like setup-bucket, on which for example polaris depends using the service_completed_successfully condition, is fine to exit and does not "break" a docker compose up --detach --wait.
A "final" setup-service like some polaris-setup, a service that no other service depends on, lets docker compose up --detach --wait fail when it exits.

The "crux" with those "final" setup-services is that the behavior isn't immediately obvious and depends on the timing of the relevant services and when exactly docker compose up --detach --wait "thinks" the whole compose thing is ready.

But this ^^ is all from experiments. It's tricky to find good documentation on this behavior. However, the behavior of depends_on (... a setup service) with service_completed_successfully condition makes sense.

MonkeyCanCode · 2026-01-29T18:26:18Z

getting-started/keycloak/docker-compose.yml

        /polaris/create-catalog.sh realm-external $$token && 
-        /polaris/create-catalog.sh realm-mixed $$token
+        /polaris/create-catalog.sh realm-mixed $$token &&
+        sleep 120


So yeah, while I was doing a local prototyping, I ran into this issue as well where --exit-code-from is not happy with this. What I ended up doing is to use tail /dev/null to keep the service up then use health check on service ready instead of completed. Sleep 120 here will work but if for whatever reason these bash scripts took more than 120s (assuming they get very complex later on), this will bite us.

Man, docker-compose isn't a good friend at all :(

Neither tail /dev/null nor sleep 120 is particularly great.
I wanted to stay away from the added complexity of health-checks for the setup-scripts though.
Let me think a bit about this.

Probably best to let the polaris-setup services "tail forever" and have health-checks. Added that

Yes. That is what I ended up doing in my local prototyping with tail forever as status code of zero is not okay for compose service.

The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful. This change updates those scripts to use `&&` for the statement concatenation. "Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services". To avoid merge conflicts, this change also: * updates affected `curl` invocations (as apache#3610) * removes superfluous `restart: "no"`

snazy · 2026-02-03T10:33:26Z

@MonkeyCanCode mind taking another look?

The statements in the shell scripts for the setup services are often concatenated using `;`, which means that a previous' command exit code is _not_ propagated and the service, although it failed, is determined to be successful. This change updates those scripts to use `&&` for the statement concatenation. "Final" setup services (aka "polaris-setup") now have a final `sleep 120`. This is due to the behavior of `docker compose up --detach --wait`, which considers _any_ service (without dependants) that exits with exit code 0 as a failure, leading to that docker-compose command yielding an error code. That would break the guides testing code (apache#3553). That `sleep 120` in "polaris-setup" services does **not** cause a delay of the compose starting up - it is purely a "hack around" Docker Compose not having a notion of "setup services". To avoid merge conflicts, this change also: * updates affected `curl` invocations (as apache#3610) * removes superfluous `restart: "no"`

github-project-automation bot added this to Basic Kanban Board Jan 29, 2026

github-project-automation bot moved this to PRs In Progress in Basic Kanban Board Jan 29, 2026

snazy force-pushed the guides-hc-exit-code branch from 4c23076 to 3645bae Compare January 29, 2026 14:33

MonkeyCanCode reviewed Jan 29, 2026

View reviewed changes

snazy mentioned this pull request Jan 31, 2026

Guides: compose dependencies / long-option #3611

Merged

snazy added 3 commits January 31, 2026 11:04

health-checks

be82d9c

update

974046c

snazy force-pushed the guides-hc-exit-code branch from f2228d6 to 974046c Compare January 31, 2026 10:09

snazy requested a review from MonkeyCanCode February 2, 2026 12:15

MonkeyCanCode approved these changes Feb 3, 2026

View reviewed changes

github-project-automation bot moved this from PRs In Progress to Ready to merge in Basic Kanban Board Feb 3, 2026

snazy merged commit 890b33a into apache:main Feb 3, 2026
15 checks passed

github-project-automation bot moved this from Ready to merge to Done in Basic Kanban Board Feb 3, 2026

snazy deleted the guides-hc-exit-code branch February 3, 2026 13:59

Conversation

snazy commented Jan 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snazy commented Feb 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants