Skip to content

GitHub Action for building docker images #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
derekclair wants to merge 3 commits into
base: main
Choose a base branch
from
+117 −7
Draft

GitHub Action for building docker images #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
24b2bc0
build pipeline
derekclair Feb 17, 2025
bf3da7c
LFC: method for secure OIDC configuration
derekclair Feb 17, 2025
59c15a8
Add QEMU setup
derekclair Feb 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions .github/workflows/docker_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
name: "Appserver: Build & Push Docker Image"
run-name: Build & Push appserver/${{ github.base_ref || github.ref_name }}
on:
push:
branches: [main]
defaults:
run:
shell: bash
jobs:
build-and-push:
runs-on: ubuntu-latest
env:
REGISTRY: harbor.delivery.iqgeo.cloud/${{ vars.harbor_project_name }}
platform_version: 7.2
Copy link

@LucasThelen LucasThelen Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont we want version to be a variable? that would allow you to have one action that can build multiple images.

Copy link
Contributor Author

@derekclair derekclair Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered how that might go, yes (and spent way more time than I'd have liked to). But, in the end, the problem as I see it is; what is the 'source of truth' for that? The GitHub repository? It could work for the workflow, yes, but what about dev environments...? The .iqgeorc file? Yes, probably. Or, at least, that is the status-quo. While the source of truth it's NOT the Action's workflow, it's parameterized here for convenience and ease-of-use for when the time comes that it does need to be changed.
Or, that's my $0.02 on the matter. RFC!

steps:
-
name: Checkout code
uses: actions/checkout@v4
with:
lfs: true

-
name: Authenticate Docker to Harbor
uses: docker/login-action@v3
with:
registry: harbor.delivery.iqgeo.cloud
username: ${{ secrets.container_registry_username }}
password: ${{ secrets.container_registry_password }}

-
name: Set up QEMU
uses: docker/setup-qemu-action@v3

-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

-
name: "Build: Platform"
id: platform
uses: docker/build-push-action@v6
with:
push: false
pull: false
context: .
file: deployment/dockerfile.build
build-args: |
CONTAINER_REGISTRY=${{ env.REGISTRY }}
platforms: linux/amd64,linux/arm64
tags: |
${{ env.REGISTRY }}/platform-build:${{ env.platform_version }}
iqgeo-myproj-build
Comment on lines +51 to +52
Copy link
Contributor Author

@derekclair derekclair Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luiscamachopt @mstrong98 @tomthetommy @ianrobertson-iqgeo so the record shows it, this is NOT a good tagging strategy.

I see this, adding a CI/CD pipeline definition to this project template, as a golden opportunity for us to establish a tagging strategy; let's be sure to capitalize on it, and let's work toward consensus on what our tagging strategy is.

If ever we wanted to get tagging right, here is our chance!

Copy link
Contributor Author

@derekclair derekclair Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm considering taking another pass at this @luiscamachopt to reflect my most recent (and most current iteration of this workflow) which, personally, I feel to be my best iteration to-date. But, I don't want to jump the gun on the conversation here. Will you @luiscamachopt take a look at this implementation too? https://github.com/IQGeo/project-nmti-trials/blob/dba9e106e092009df2678083b66368de944dad89/.github/workflows/sre-image-build.yml

cache-to: type=gha,mode=max
cache-from: type=gha

-
name: Extract Appserver metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/platform-appserver
tags: |
type=raw,value={{branch}}-{{sha}}
type=sha,prefix={{branch}}-,enable={{!is_default_branch}}
Copy link
Contributor Author

@derekclair derekclair Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure that this enable={{!is_default_branch}} syntax will work, it's un-verified but, none the less it expresses the intent. I'll work to verify this before opening the PR.

Copy link

@LucasThelen LucasThelen Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have not adopted github enterprise should we get this working in Jenkins first or also?

Copy link
Contributor Author

@derekclair derekclair Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point @LucasThelen! Glad you mentioned it.

Is this the fish you wish? #45

type=raw,value=${{ env.platform_version }},enable={{is_default_branch}}

-
name: "Build: Appserver"
uses: docker/build-push-action@v6
with:
push: true
pull: true
context: deployment
file: deployment/dockerfile.appserver
# secret-files: |
# oidc=${{ secrets.oidc_config }}
Comment on lines +76 to +77
Copy link
Contributor Author

@derekclair derekclair Feb 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# secret-files: |
# oidc=${{ secrets.oidc_config }}
secret-files: |
oidc=${{ secrets.oidc_config }}

This, along with the included changes recommended to the dockerfile.appserver would facilitate a secure method for providing the OIDC configuration to the build. (An exclusively GitHub Actions flavored method, but it would serve as a guide regardless.)

build-args: |
CONTAINER_REGISTRY=${{ env.REGISTRY }}
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha

-
name: Extract Appserver Tools metadata (tags, labels) for Docker
id: tools_metadata
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/platform-tools
tags: |
type=raw,value={{branch}}-{{sha}}
type=sha,prefix={{branch}}-,enable={{!is_default_branch}}
type=raw,value=${{ env.platform_version }},enable={{is_default_branch}}

-
name: "Build: Appserver Tools"
uses: docker/build-push-action@v6
with:
push: true
pull: true
context: deployment
file: deployment/dockerfile.tools
build-args: |
CONTAINER_REGISTRY=${{ env.REGISTRY }}
platforms: linux/amd64,linux/arm64
tags: ${{ steps.tools_metadata.outputs.tags }}
cache-from: type=gha

5 changes: 3 additions & 2 deletions deployment/dockerfile.appserver
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG CONTAINER_REGISTRY=harbor.delivery.iqgeo.cloud/releases/
ARG CONTAINER_REGISTRY=${CONTAINER_REGISTRY:-harbor.delivery.iqgeo.cloud/releases}

FROM iqgeo-myproj-build AS iqgeo_builder

Expand All @@ -16,7 +16,7 @@ RUN rm -rf ${MODULES}/*/node_modules \
&& rm -rf ${MODULES}/*/native

############################################## project appserver image
FROM ${CONTAINER_REGISTRY}platform-appserver:7.2
FROM ${CONTAINER_REGISTRY}/platform-appserver:7.2

USER root

Expand Down Expand Up @@ -49,3 +49,4 @@ USER www-data
COPY --chown=www-data:www-data entrypoint.d/* /entrypoint.d/

COPY --chown=www-data:www-data appserver_config/ /opt/iqgeo/config/
# RUN --mount=type=secret,id=oidc,target=/opt/iqgeo/config/oidc/conf.json
Comment on lines 51 to +52
Copy link
Contributor Author

@derekclair derekclair Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
COPY --chown=www-data:www-data appserver_config/ /opt/iqgeo/config/
# RUN --mount=type=secret,id=oidc,target=/opt/iqgeo/config/oidc/conf.json
RUN --mount=type=secret,id=oidc,target=/opt/iqgeo/config/oidc/conf.json

This will need some re-touching if it is to account for all 'appserver_config' scenarios, but does illustrate exposing secrets to the build as a best practice.

Copy link

@LucasThelen LucasThelen Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have to add this to the tools image as well?

Copy link
Contributor Author

@derekclair derekclair Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LucasThelen Negative. At least, not in current implementation. It does not COPY anything from the host file system.

Now, that said, if tools image does need the /opt/iqgeo/config/ directory (meaning, it does use the OIDC configuration currently), its unclear to me how that directory/path is getting into that image now, because it is not explicitly doing so in the dockerfile.tools.
Obviously, if that should be a requirement, yes, it can be done there too.

6 changes: 3 additions & 3 deletions deployment/dockerfile.build
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
ARG CONTAINER_REGISTRY=harbor.delivery.iqgeo.cloud/releases/
ARG CONTAINER_REGISTRY=${CONTAINER_REGISTRY:-harbor.delivery.iqgeo.cloud/releases}
# START SECTION Aliases for Injector images
FROM ${CONTAINER_REGISTRY}comms:3.2 AS comms
FROM ${CONTAINER_REGISTRY}/comms:3.2 AS comms
# END SECTION

# Create container for building the project
FROM ${CONTAINER_REGISTRY}platform-build:7.2
FROM ${CONTAINER_REGISTRY}/platform-build:7.2

# START SECTION Copy the modules - if you edit these lines manually note that your change will get lost if you run the IQGeo Project Update tool
COPY --link custom ${MODULES}/custom
Expand Down
4 changes: 2 additions & 2 deletions deployment/dockerfile.tools
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
ARG CONTAINER_REGISTRY=harbor.delivery.iqgeo.cloud/releases/
ARG CONTAINER_REGISTRY=${CONTAINER_REGISTRY:-harbor.delivery.iqgeo.cloud/releases}

FROM iqgeo-myproj-build AS iqgeo_builder

# START SECTION optional dependencies (build) - if you edit these lines manually note that your change will get lost if you run the IQGeo Project Update tool

# END SECTION

FROM ${CONTAINER_REGISTRY}platform-tools:7.2 AS tools_intermediate
FROM ${CONTAINER_REGISTRY}/platform-tools:7.2 AS tools_intermediate

USER root

Expand Down