Update to use Spack build for OM3 part of CM3 build.

[chrisb13]

Currently, the CM3 build process is using the deprecated COSIMA build system. The OSIT team has moved to using Spack and this is the preferred method going forward. It would be best if CM3 were also using this build system as there is known issue where the two build systems give different answers, this complicates OM3 / CM3 comparisons.

I think it should be a straightforward swap, we could try it with a prebuilt exe as a starting point? I think it’ll make OM upgrades in CM easier too?

Related to this issue.

[anton-seaice]

A first pass would be to module load the libraries for each model component here:

https://github.com/ACCESS-NRI/dev-access-cm3-build/blob/b72433689b9c860401693de2240f6689eac6467b/build.sh#L23-L26

we'd need to add access3-share, access-mom6, access-cice to the modules created here by spack in a prerelease PR

and then fix any linking/names etc

Longer term is to make the whole thing in spack - see ACCESS-NRI/access-spack-packages#212

[blimlim]

See here for a rough suite example which builds using the deployed libraries worked on with @anton-seaice a little while ago.

One slightly messy thing to note is that we currently still need to a local copy of the access-om3 repo used to build the libraries when building the coupled executable from, since the coupled build builds from /CMEPS/cesm/driver/esm.F90, /CMEPS/cesm/driver/ensemble_driver.F90, and CMEPS/cesm/driver/esm_time_mod.F90.

[anton-seaice]

See here for a rough suite example which builds using the deployed libraries worked on with @anton-seaice a little while ago

Well need to update again for a "component" based build, using different spack packages for each model component

One slightly messy thing to note is that we currently still need to a local copy of the access-om3 repo used to build the libraries when building the coupled executable from, since the coupled build builds from /CMEPS/cesm/driver/esm.F90, /CMEPS/cesm/driver/ensemble_driver.F90, and CMEPS/cesm/driver/esm_time_mod.F90.

When we build through spack, the access3 packages will provide these :). You only need a local copy of CMEPS btw, not the whole access-om3 repo (although this slightly depends where the build files are).

[chrisb13]

Replicating @blimlim handy tips for getting this going here:

1. Build libraries for the OM3 components from Spack in a PR (e.g here)
2. Load prerelease module in CM3 suite, then compile UM library and final coupled model from the CM3 suite.
3. Run from the suite

In the future step 2 will hopefully be merged into step 1, so that there's a single spack build step for the whole model. The AM3 team are working on the spack package for the UM, and so this would happen once that's all ready.

There's a few awkward tweaks I've made to cmake files and spack-packages to get it working (mostly due to my lack of understanding cmake). I think @anton will have good ideas on how to resolve them when he's back. In the mean time I'll put some notes in the github issue with instructions for getting it running and will document the things that we might be able to clean up.

[blimlim]

Thanks @chrisb13 and thanks @kieranricardo for updating the suites and instructions. Just noting down those tweaks and todos:

1. Extra paths: in the spack.yaml I've exported several paths. I remembered that the build previously failed when these paths weren't available, and previously was hardcoding them in the suite, however I should double check if all of them are necessary.

2. CDEPs dependency. The build currently includes cdeps here although we don't specify it as a required component for CM3. However only finds the FoX dependency if CDEPs is specified as a required component. For CM3, this causes a build failure as CDEPs ends up being included but FoX is not searched for.

   Meanwhile CICE and MOM always link to cdeps-common. To get the build working, I've set it to always find FoX regardless of if CDEPS is specified.

   I think we could either: Just use the unmodified access3-share CMake files, and add CDEPs as a required component of CM3 here in the suite, or otherwise we could add some extra logic so that CDEPS is not always required.

3. Added a +cm3 variant to the access3 spack-package which builds gcom when active. Not sure if this is the best way of doing it.

4. I hardcoded the access/cmeps path in place of nuopc/cmeps in the CICE CMakeLists file. Preferably this would be controlled from the spack.yaml, maybe via the +cm3 variant is active?

[anton-seaice]

1. Extra paths: in the spack.yaml I've exported several paths. I remembered that the build previously failed when these paths weren't available, and previously was hardcoding them in the suite, however I should double check if all of them are necessary.

We can chat to the release team about the best way to do this, possible we set them in the spack package for the UM.

It looks like there is a method for doing this already here and we might be able to add the extra dependencies needed

2. CDEPs dependency. The build currently includes cdeps here although we don't specify it as a required component for CM3. However only finds the FoX dependency if CDEPs is specified as a required component. For CM3, this causes a build failure as CDEPs ends up being included but FoX is not searched for.

Is it a linking or a build failure ?

Meanwhile CICE and MOM always link to cdeps-common. To get the build working, I've set it to always find FoX regardless of if CDEPS is specified.
I think we could either: Just use the unmodified access3-share CMake files, and add CDEPs as a required component of CM3 here in the suite, or otherwise we could add some extra logic so that CDEPS is not always required.

CDEPS is required to build CICE with the NUOPC driver. It's related to some data assimiliation / prescribed ice that we don't use, but is in the code.

It looks like the coupled build is not linking to Fox, so maybe its not being exported with one of the packages which the build depends on (i.e. an issue with the dependency rather than UM) ?

3. Added a +cm3 variant to the access3 spack-package which builds gcom when active. Not sure if this is the best way of doing it.

I hope we can include the UM into the "configurations" variant of the access3 spack package, and the related CMAKE variable, instead of adding an extra variant. This will then build the whole UM, which in-turn has gcom as a variant.

A better temporary workaround when building the UM outside of spack would be to just add gcom to the top level spec of the spack environment file.

4. I hardcoded the `access/cmeps` path in place of `nuopc/cmeps` in the [CICE CMakeLists file](https://github.com/ACCESS-NRI/CICE/blob/d4e7ca14826cc2ca652f13e135cc5f41e9fd3d03/configuration/scripts/cmake/CMakeLists.txt#L39). Preferably this would be controlled from the spack.yaml, maybe via the `+cm3` variant is active?

We should be able to set the CICE_DRIVER cmake variable by adding to the spack package

p.s. I think you know this, I am hoping to incorporate https://github.com/ACCESS-NRI/access3-configs/blob/962b4095e8606306b00d573fdc6ab69c937d5029/app/cpl_bld/file/CMakeLists.txt#L53 into https://github.com/ACCESS-NRI/access3-share/blob/main/cmake/Access3BinInstall.cmake by adding some BuildConfigurations which include the UM. But first we need to build the UM through spack, and produce a package conf or cmake config file (or similar), so that CMake can find the UM using find_package.