Skip to content

Conversation

@arrdem
Copy link
Collaborator

@arrdem arrdem commented Dec 18, 2025

As https://peps.python.org/pep-0720/ explains in frustrating detail, there isn't currently a good way to do crossbuilds from one Python platform to another. While many relevant compiled languages (C, Rust, Fortran) can support crossbuilding, the only really sound thing to do is to let python3 -m build do its thing on the target.

So we need to adapt the sdist_build repository rule so that the underlying sdist_build rule is transitioned so that the execution platform matches the target platform.

However we don't want to transition all sdist builds to the target, since most[^1] Python libraries are purelib and doing so would force users of the uv machinery to deploy RBE when they don't strictly need to do so. As a temporary measure, extend the annotations file schema so that individual requirements can be opted into being built on the target.

In the future, we likely want to distribute a standard "database" of such annotations so that users don't have to, and/or adapt the sdist_build repository rule to use a hermetic tar to unzip the downloaded sdist at repo configuration time and check the sdist content for obvious signs of containing C, Cython, Rust or Fortran code.

[^1] Citation needed

Changes are visible to end-users: yes

  • Searched for relevant documentation and updated as needed: yes
  • Breaking change (forces users to change their own code or config): no
  • Suggested release notes appear below: yes

The uv extension now allows for requirements to be opted into platform-dependent builds in support of native extensions.

Test plan

  • Manual testing

TBD

@arrdem arrdem requested a review from thesayyn December 18, 2025 00:54
Copy link
Member

@thesayyn thesayyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@aspect-workflows
Copy link

aspect-workflows bot commented Dec 18, 2025

test-os:linux-bzl:8 (Test)

⚠️ Buildkite build #1711 failed.

@@+uv+sbuild__pypi__default__bravado_core//:whl failed to build

python3 failed: error executing Action command (from target @@+uv+sbuild__pypi__default__bravado_core//:whl) bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3 uv/private/sdist_build/build_helper.py --validate-anyarch ... (remaining 2 arguments skipped)
 
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
 
/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3
/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3: No module named build
Traceback (most recent call last):
  File "/mnt/ephemeral/output/rules_py/__main__/sandbox/linux-sandbox/2046/execroot/_main/uv/private/sdist_build/build_helper.py", line 34, in <module>
    check_call([
  File "/mnt/ephemeral/output/rules_py/__main__/external/rules_python++python+python_3_9_x86_64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3', '-m', 'build', '--wheel', '--no-isolation', '--outdir', '/mnt/ephemeral/output/rules_py/__main__/sandbox/linux-sandbox/2046/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/build']' returned non-zero exit status 1.

💡 To reproduce the build failures, run

bazel build @@+uv+sbuild__pypi__default__bravado_core//:whl

test-os:linux-bzl:latest (Test)

⚠️ Buildkite build #1711 failed.

@@+uv+sbuild__pypi__default__bravado_core//:whl failed to build

python3 failed: error executing Action command (from target @@+uv+sbuild__pypi__default__bravado_core//:whl) bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3 uv/private/sdist_build/build_helper.py --validate-anyarch ... (remaining 2 arguments skipped)
 
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
 
/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3
/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3: No module named build
Traceback (most recent call last):
  File "/mnt/ephemeral/output/rules_py/__main__/sandbox/linux-sandbox/9/execroot/_main/uv/private/sdist_build/build_helper.py", line 34, in <module>
    check_call([
  File "/mnt/ephemeral/output/rules_py/__main__/external/rules_python++python+python_3_9_x86_64-unknown-linux-gnu/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/mnt/ephemeral/output/rules_py/__main__/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/.build_venv/bin/python3', '-m', 'build', '--wheel', '--no-isolation', '--outdir', '/mnt/ephemeral/output/rules_py/__main__/sandbox/linux-sandbox/9/execroot/_main/bazel-out/k8-fastbuild-ST-fd81cbb45820/bin/external/+uv+sbuild__pypi__default__bravado_core/build']' returned non-zero exit status 1.

💡 To reproduce the build failures, run

bazel build @@+uv+sbuild__pypi__default__bravado_core//:whl

test-os:linux-bzl:8 (Test)

e2e

All tests were cache hits

14 tests (100.0%) were fully cached saving 10s.


test-os:linux-bzl:latest (Test)

e2e

All tests were cache hits

14 tests (100.0%) were fully cached saving 11s.


test-os:linux-bzl:8 (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 335ms.


test-os:linux-bzl:latest (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 335ms.

@arrdem arrdem force-pushed the arrdem/uv-sdists-build-on-target branch from f537878 to ba2de0e Compare December 18, 2025 17:16
"_helper": attr.label(allow_single_file = True, default = Label(":build_helper.py")),
},
toolchains = [
# TODO: Py toolchain needs to be in the `host` configuration, not the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why host? That feels like almost always the wrong thing?

def _transition_to_target_impl(settings, attr):
return {
# String conversion is needed to prevent a crash with Bazel 6.x.
"//command_line_option:extra_execution_platforms": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uhhh this is super sketch. Is it possible to use exec_compatible_with constraints to guide it to a platform matching the target instead?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants