Random notes on cached esy + docker #1
Description
My goal here is to cache .esy separately from normal Docker layering but still be able to have a universal workflow where Docker works as customary both on CI and locally. I looked at --mount=type=cache - it is built for tools like esy or cargo. I posted some of my findings here:
My biggest concerns are:
- Keeping the cache size constant (modulo change in size of dependency graph)
- Not corrupting Docker internal state
- Reducing constant factors (such as needlessly pruning everything with
docker system prune)
I figured out how to prune everything except for the mount cache with docker builder prune:
DOCKER_BUILDKIT=1 docker builder prune --filter type=source.local --all
DOCKER_BUILDKIT=1 docker builder prune --filter type=regular --all
But it still doesn't tell us how to properly save the state and later restore it.
In the current exploration, we export the whole /var/lib/docker which has large constant factors and probably requires careful pruning but I'm not familiar with how it all works.
Alternatively, there's buildx which is properly supported and probably allows us to achieve the caching we want without resorting to any hacks but alas I'm not familiar with it either.