I think it would be reasonable to refactor the GHA to use a better action type. Docker is limited to Ubuntu workers and has a certain amount of runtime overhead that I believe can be addressed with Composite GitHub Actions. I've promptly checked the current setup and I don't believe there's anything that is tightly coupled with the docker runtime and can't be substituted.
Ref: https://docs.github.com/en/actions/creating-actions/creating-a-composite-action