feat: support nydus image format

[fatelei]

resolve #565

support nydus image format

[packit-as-a-service]

Packit jobs failed. @containers/packit-build please check.

[podmanbot]

✅ A new PR has been created in buildah to vendor these changes: containers/buildah#6610

[mtrmac]

Thanks!

Per https://github.com/containers/container-libs/blob/main/CONTRIBUTING.md#sign-your-prs :

Use a real name (sorry, no anonymous contributions)

(please read on for more nuance to that.)

---

Cc: @giuseppe (Referencing https://github.com/dragonflyoss/nydus/blob/master/docs/nydus-design.md, that says:

• The image format has an artifactType in the index, but not in the manifest itself?!
• It is not really an artifact, it has the usual OCI config type
• … it’s “just” a new structure of the layer blobs.

Reading this PR, I don’t think much of these changes is actually necessary. Do copy operations fail otherwise??

---

Apart from pure unchanged copies, long-term we need to have some idea of what this format is from the end-user workflow perspective, and I think it’s necessary to think about that now, to shape the API / UI appropriately.

Is it a compression format, like zstd:chunked, where each individual layer can be ~losslessly converted between tar and the other compression? (Seems not to be the case, with the bootstrap/blob differentiation.)

Is it a ~different manifest format, really distinct from OCI? (Looking at the metadata format suggests “no”; in particular, returning NonImageArtifactError from OCIConfig seems inappropriate.)

And yet, it is ultimately significantly different, at least in that, right now, podman pull would fail.

How would a conversion from a tar format to Nydus, or back, work? Skopeo is (well, currently) not set up to do conversions that need to download and significantly manipulate large blobs — conversions between v2s1/v2s2/OCI are metadata operations + compression/decompression where we can stream the data with little local storage. Would these conversions ~forever be delegated to specialized tools? Or is the idea that podman build+podman push should be able to somehow directly convert to Nydus on the fly?

[mtrmac]

This function is deprecated and has no known users (at least within the Podman+friends tools), so I don’t think there is any point in changing it.