feat: use git tags to decide on when to build

[AtzeDeVries]

This will change the when to build what.

This will do a diff based on a git tag. When a build has been done on main that builds an oci it will set a tag. Next time on main it will do a diff based on this tag. If the diff actually has changes related to this tag.

Why?

We used to do this based on the diff which came with the PR event. This PR event was also used on the main branch. If the job is canceled or removed from queue this creates a problem. Also in github actions, jobs in a queue don't have a guaranteed order. Also rerunning a job based on a PR event would rerun certain steps based on the event data. This is not the behavior you want if the artifact is already published. Also the PR invent was a bit hard to get insight to.

What will happen here

When a PR is created the, a diff between the main branch and the PR branch is made and based on the files in the diff decisions are made on which part of CI are required to run.
When merged, mage will look for the most recent tag based on a tag pattern and use this tag as a diff target. When this results in a diff, mage will run certain CI (mage ci for golang with OCI build) and when finished write a new tag. So if you rerun this job it won't rerun because the diff between the tag and the head is none.
This, in the end will give a way more predictable CI. We can also merge the job queue on the main branch resulting less action minutes spend and not being exposed to the order in which jobs run on the main branch.

How to use this

If set inputs.tag-based-diff is set to true this behavior is enabled. This is disabled by default to ensure a smooth transition. It is strongly encouraged to enable this.

Updating your image tags in helm

When a release is created we can trigger a job to update the OCI tags in your helm values. This is best done via renovate. See examples below.

Relevant issues #356

[AtzeDeVries]

This works great in combination with renovate. First of all you need a renovate config which is pretty generic

{
  "baseBranchPatterns": [
    "main"
  ],
  "rebaseWhen": "behind-base-branch",
  "labels": [
    "dependencies",
    "renovate",
    "{{depName}}"
  ],
  "automergeStrategy": "squash",
  "enabledManagers": [
    "helmv3",
    "helm-values"
  ],
  "packageRules": [
    {
      "matchDatasources": [
        "helm"
      ],
      "automerge": true
    },
    {
      "branchPrefix": "helm/dev/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-dev.yaml$/"
      ],
      "addLabels": [
        "development"
      ],
      "prHourlyLimit": 0
    },
    {
      "branchPrefix": "helm/staging/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-staging.yaml$/"
      ],
      "addLabels": [
        "staging"
      ],
      "prHourlyLimit": 0
    },
    {
      "branchPrefix": "helm/production/",
      "matchManagers": [
        "helm-values"
      ],
      "matchDatasources": [
        "docker"
      ],
      "automerge": true,
      "matchFileNames": [
        "/(^|/)values-production.yaml$/"
      ],
      "addLabels": [
        "production"
      ],
      "prHourlyLimit": 0
    }
  ],
  "helm-values": {
    "managerFilePatterns": [
      "/(^|/)values(-\\w+)?\\.ya?ml$/"
    ]
  }
}

You also need renovate ci, we trigger this on tag creation, so it will be called after the release is done. Renovate will check GAR for new images, it will create. a changelog, or update an existing PR with the new version (and changelog). In the future we can get rid of this workflow and let renovate server handle this

on:
  push:
    tags:
      - "v*"
  workflow_call:
    inputs:
      logLevel:
        description: "Override default log level"
        required: false
        default: "info"
        type: string
    secrets: {}
  workflow_dispatch:
    inputs:
      logLevel:
        description: "Override default log level"
        required: false
        default: "info"
        type: string
  schedule:
    # At 07:30 AM and 12:30 PM, every day
    - cron: "30 7,12 * * *"
jobs:
  renovate:
    permissions:
      contents: write
      pull-requests: write
      id-token: write
      issues: write
    runs-on: ubuntu-24.04
    steps:
      - name: Checkout
        uses: actions/checkout@v6
      - name: Get token
        id: get_token
        uses: actions/create-github-app-token@v2.2.1
        with:
          app-id: 635546
          private-key: ${{ secrets.RENOVATE_APP_PRIVATE_KEY_PEM }}
          owner: ${{github.repository_owner }}
          repositories: helloworld
          # permission-contents: write
          # permission-pull-requests: write
          # permission-issues: write
      - name: Autenticate with GCP
        id: gcp-auth
        uses: google-github-actions/auth@v3
        with:
          token_format: access_token
          workload_identity_provider: projects/889992792607/locations/global/workloadIdentityPools/github-actions/providers/github-actions-provider
          service_account: gh-ap-helloworld@helloworld-shared-0918.iam.gserviceaccount.com
          create_credentials_file: true
      - name: Run Renovate
        uses: renovatebot/github-action@822441559e94f98b67b82d97ab89fe3003b0a247 # v44.2.0
        env:
          # Repository taken from variable to keep configuration file generic
          RENOVATE_REPOSITORIES: ${{ github.repository }}
          # Onboarding not needed for self-hosted
          RENOVATE_ONBOARDING: "false"
          # Username for GitHub authentication (should match GitHub App name + [bot])
          RENOVATE_USERNAME: "coopnorge-renovate[bot]"
          # Git commit author used, must match GitHub App
          RENOVATE_GIT_AUTHOR: "coopnorge-renovate <121964725+coopnorge-renovate[bot]@users.noreply.github.com>"
          # Use GitHub API to create commits (this allows for signed commits from GitHub App)
          RENOVATE_PLATFORM_COMMIT: "true"
          # Override schedule if set
          RENOVATE_FORCE: "true"
          #RENOVATE_FORCE: ${{ github.event.inputs.overrideSchedule == 'true' && '{''schedule'':null}' || '' }}
          RENOVATE_HOST_RULES: '[{"matchHost":"europe-docker.pkg.dev","token":"${{ steps.gcp-auth.outputs.access_token }}"}]'
          RENOVATE_PR_BODY_TEMPLATE: "{{{header}}}{{{table}}}{{{warnings}}}{{{notes}}}{{{changelogs}}}{{{configDescription}}}{{{controls}}}{{{footer}}}"
          LOG_LEVEL: ${{ inputs.logLevel || 'info' }}
        with:
          configurationFile: .github/renovate.json
          token: ${{ steps.get_token.outputs.token }}

[AtzeDeVries]

Example PR by renovate

https://github.com/coopnorge/helloworld/pull/3091

[bendiknesbo]

I like this approach, as it fixes a bug where several PR's being merged can lead to some changes not being built a docker-image for.

[bendiknesbo]

Incorrect changes to formatting here

[bendiknesbo]

I guess this variable must be set to true from the goapp.yaml, based on tag-based-diff and possibly other things?

[AtzeDeVries]

ah yeah that i think should be deleted, i first though giving the option to just tag instead of release.

[bendiknesbo]

In a repo without any existing tags, this may exit with the error fatal: No names found, cannot describe anything.

[bendiknesbo]

The --always flag can be added to prevent it from failing, but then it's going to fall back to the commit hash of HEAD.

[AtzeDeVries]

Yeah good point, i forget to add. I think that 'failing' on no tag is found is the actual correct behavior, else we might end up some silent fails.

My initial idea to approach this would be that when we start using this a user/owner should set the initial tag. I could fall back to the initial commit in the main branch but then the changelog would be to big and cause issues down the line.
I will look into the --always if that might fix the problem here.