Flaky macos documentation builds

[nvaytet]

Once in a while (it's happened enough times now that I am opening this issue), the docs PR build fails.
It's something to do with maybe not properly completing a file download.
An example of failed workflow is here.

It only happens on macos. I've searched the Pooch issue list for something but could not see much there.