Enable blake3 for Bazel builds#565
Conversation
aaronmondal
left a comment
aaronmondal
left a comment
There was a problem hiding this comment.
Reviewable status: 0 of 2 LGTMs obtained, and pending CI: Analyze (javascript-typescript), Analyze (python), Bazel Dev / ubuntu-22.04, Cargo Dev / ubuntu-22.04, Local / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), pre-commit-checks, publish-image, ubuntu-20.04, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, windows-2022 / stable (waiting on @adam-singer and @allada)
adam-singer
left a comment
adam-singer
left a comment
There was a problem hiding this comment.
Based on bazelbuild/bazel@0a8380b && bazelbuild/remote-apis#235 should we be considering SHA256TREE? Seems like some existing research has been done, at a high level Blake3 might be better for x86/x64 and slower than SHA256 on Arm, SHA256TREE is mentioned in the thread better on both platforms.. Not sure if its worth evaluating which digest to use based on platform, or just keep it simple, pick one for now and revisit later.
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: 1 of 2 LGTMs obtained, and pending CI: docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04) (waiting on @allada)
aaronmondal
left a comment
aaronmondal
left a comment
There was a problem hiding this comment.
@adam-singer Interesting. Is that used in the same way as the other ones though? If something other than blake3 is faster on arm we could set the digest function to different values for different configurations to get the most out of all CI runners.
cross-ref #566
Reviewable status: 1 of 2 LGTMs obtained, and pending CI: docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04) (waiting on @allada)
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04)
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04)
a discussion (no related file):
verify store does not yet support blake3:
aaronmondal
force-pushed
the
enable-blake3
branch
from
c39ce7f to
a8465e1
Compare
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
aaronmondal
force-pushed
the
enable-blake3
branch
from
a8465e1 to
1f1a3f0
Compare
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04)
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04)
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Remote / large-ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04)
deployment-examples/docker-compose/local-storage-cas.json line 28 at r2 (raw file):
}, "verify_size": true, "hash_verification_function": "blake3"
Hmmm, lets remove verification store here all together. We use it in testing and by default bazel uses sha256, so it'll make this not work "out-of-the-box" unless the user adds cli flags (and has clients that support blake3).
If we remove verification store it should just work.
aaronmondal
force-pushed
the
enable-blake3
branch
from
1f1a3f0 to
297d85e
Compare
aaronmondal
left a comment
aaronmondal
left a comment
There was a problem hiding this comment.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), publish-image, windows-2022 / stable
deployment-examples/docker-compose/local-storage-cas.json line 28 at r2 (raw file):
Previously, allada (Nathan (Blaise) Bruer) wrote…
Hmmm, lets remove verification store here all together. We use it in testing and by default bazel uses sha256, so it'll make this not work "out-of-the-box" unless the user adds cli flags (and has clients that support blake3).
If we remove verification store it should just work.
Done.
allada
left a comment
allada
left a comment
There was a problem hiding this comment.
Reviewed 1 of 1 files at r3.
Reviewable status:complete! 2 of 2 LGTMs obtained
adam-singer
left a comment
adam-singer
left a comment
There was a problem hiding this comment.
Reviewed all commit messages.
Reviewable status:
3 participants
This change is