pixi-build used dependencies (i.e. backend + host and build envs) should be persisted in pixi.lock

[pavelzw]

This came up in Quantco/pixi-pack#244 (comment) but also in some other internal use cases.

I think it would be nice for full reproducibility if we were able to pin all of the used things to build an environment in pixi.lock as well. Maybe optionally?
Otherwise you might run into the situation in the future that a package is not buildable anymore or produced different results than before.

[ruben-arts]

This is for sure the goal! We've still working on the idea on how to make this without exploding the lockfile. We might initial implement this and accept these mega lockfiles...

One idea that I have is that it would be nice to default to having a solve-group with the environment it's installed in. So that it reuses as much as possible.

[baszalmstra]

I was thinking about at least storing the date of the newest package in the build/host environments in the lockfile and using that with excludenewer to enforce at least some guarantees. Although that will be strictly less reproducible it will probably be much much better than nothing at all without the lockfile blowing up.

[pavelzw]

This just fell on our feet: hatchling 1.28.0 now doesn't ship editables which breaks pip install -e (fix incoming in conda-forge/hatchling-feedstock#67).
Now, pixi install fails without us having changed the lockfile.

[package]
name = "project"
# replaced in CI
version = "0.0.0"

[package.build.backend]
name = "pixi-build-python"
version = "*"

[package.host-dependencies]
python = ">=3.12"
hatchling = "*"

[package.run-dependencies]
python = ">=3.12"
pydantic-ai = ">=1.32.0,<2"
pydantic = ">=2.12.3,<3"

Run pixi run -e default test
  pixi run -e default test
  shell: /usr/bin/bash -e {0}
✨ Pixi task (test in default): pytest

 ╭─ Running build for recipe: project-0.0.0-pyh4616a5c_0
 │
 │ ╭─ Fetching source code
 │ │ No sources to fetch
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 │ ╭─ Externally resolved dependencies recipe: project-0.0.0-pyh4616a5c_0
 │ │ 
 │ │ Resolved build dependencies(project-0.0.0-pyh4616a5c_0):
 │ │ ╭─────────┬──────┬─────────┬───────┬─────────┬──────╮
 │ │ │ Package ┆ Spec ┆ Version ┆ Build ┆ Channel ┆ Size │
 │ │ ╞═════════╪══════╪═════════╪═══════╪═════════╪══════╡
 │ │ ╰─────────┴──────┴─────────┴───────┴─────────┴──────╯
 │ │ Resolved host dependencies(project-0.0.0-pyh4616a5c_0):
 │ │ ╭────────────────────┬───────────────┬──────────────┬──────────────────────┬─────────────┬─────────────╮
 │ │ │ Package            ┆ Spec          ┆ Version      ┆ Build                ┆ Channel     ┆        Size │
 │ │ ╞════════════════════╪═══════════════╪══════════════╪══════════════════════╪═════════════╪═════════════╡
 │ │ │ hatchling          ┆ hatchling     ┆ 1.28.0       ┆ pyhcf101f3_0         ┆ conda-forge ┆   59.48 KiB │
 │ │ │ pip                ┆ pip           ┆ 25.3         ┆ pyh145f28c_0         ┆ conda-forge ┆    1.12 MiB │
 │ │ │ python             ┆ python >=3.12 ┆ 3.14.2       ┆ h32b2ec7_100_cp314   ┆ conda-forge ┆   35.09 MiB │
 │ │ │ _libgcc_mutex      ┆               ┆ 0.1          ┆ conda_forge          ┆ conda-forge ┆    2.50 KiB │
 │ │ │ _openmp_mutex      ┆               ┆ 4.5          ┆ 2_gnu                ┆ conda-forge ┆   23.07 KiB │
 │ │ │ bzip2              ┆               ┆ 1.0.8        ┆ hda65f42_8           ┆ conda-forge ┆  254.24 KiB │
 │ │ │ ca-certificates    ┆               ┆ 2026.1.4     ┆ hbd8a1cb_0           ┆ conda-forge ┆  143.08 KiB │
 │ │ │ icu                ┆               ┆ 78.1         ┆ h33c6efd_0           ┆ conda-forge ┆   12.13 MiB │
 │ │ │ importlib-metadata ┆               ┆ 8.7.0        ┆ pyhe01879c_1         ┆ conda-forge ┆   33.83 KiB │
 │ │ │ ld_impl_linux-64   ┆               ┆ 2.45         ┆ default_hbd61a6d_105 ┆ conda-forge ┆  713.70 KiB │
 │ │ │ libexpat           ┆               ┆ 2.7.3        ┆ hecca717_0           ┆ conda-forge ┆   74.85 KiB │
 │ │ │ libffi             ┆               ┆ 3.5.2        ┆ h9ec8514_0           ┆ conda-forge ┆   56.47 KiB │
 │ │ │ libgcc             ┆               ┆ 15.2.0       ┆ he0feb66_16          ┆ conda-forge ┆ 1018.36 KiB │
 │ │ │ libgomp            ┆               ┆ 15.2.0       ┆ he0feb66_16          ┆ conda-forge ┆  589.14 KiB │
 │ │ │ liblzma            ┆               ┆ 5.8.1        ┆ hb9d3cd8_2           ┆ conda-forge ┆  110.25 KiB │
 │ │ │ libmpdec           ┆               ┆ 4.0.0        ┆ hb9d3cd8_0           ┆ conda-forge ┆   89.05 KiB │
 │ │ │ libsqlite          ┆               ┆ 3.51.1       ┆ hf4e2dac_1           ┆ conda-forge ┆  921.34 KiB │
 │ │ │ libstdcxx          ┆               ┆ 15.2.0       ┆ h934c35e_16          ┆ conda-forge ┆    5.59 MiB │
 │ │ │ libuuid            ┆               ┆ 2.41.3       ┆ h5347b49_0           ┆ conda-forge ┆   39.37 KiB │
 │ │ │ libzlib            ┆               ┆ 1.3.1        ┆ hb9d3cd8_2           ┆ conda-forge ┆   59.53 KiB │
 │ │ │ ncurses            ┆               ┆ 6.5          ┆ h2d0b736_3           ┆ conda-forge ┆  870.74 KiB │
 │ │ │ openssl            ┆               ┆ 3.6.0        ┆ h26f9b46_0           ┆ conda-forge ┆    3.02 MiB │
 │ │ │ packaging          ┆               ┆ 25.0         ┆ pyh29332c3_1         ┆ conda-forge ┆   61.01 KiB │
 │ │ │ pathspec           ┆               ┆ 0.12.1       ┆ pyhd8ed1ab_1         ┆ conda-forge ┆   40.11 KiB │
 │ │ │ pluggy             ┆               ┆ 1.6.0        ┆ pyhf9edf01_1         ┆ conda-forge ┆   25.27 KiB │
 │ │ │ python_abi         ┆               ┆ 3.14         ┆ 8_cp314              ┆ conda-forge ┆    6.83 KiB │
 │ │ │ readline           ┆               ┆ 8.3          ┆ h853b02a_0           ┆ conda-forge ┆  336.99 KiB │
 │ │ │ tk                 ┆               ┆ 8.6.13       ┆ noxft_ha0e22de_103   ┆ conda-forge ┆    3.13 MiB │
 │ │ │ tomli              ┆               ┆ 2.3.0        ┆ pyhcf101f3_0         ┆ conda-forge ┆   20.48 KiB │
 │ │ │ trove-classifiers  ┆               ┆ 2025.12.1.14 ┆ pyhd8ed1ab_0         ┆ conda-forge ┆   19.14 KiB │
 │ │ │ tzdata             ┆               ┆ 2025c        ┆ hc9c84f9_1           ┆ conda-forge ┆  116.34 KiB │
 │ │ │ zipp               ┆               ┆ 3.23.0       ┆ pyhcf101f3_1         ┆ conda-forge ┆   23.63 KiB │
 │ │ │ zstd               ┆               ┆ 1.5.7        ┆ hb78ec9c_6           ┆ conda-forge ┆  587.28 KiB │
 │ │ ╰────────────────────┴───────────────┴──────────────┴──────────────────────┴─────────────┴─────────────╯
 │ │ Resolved run dependencies(project-0.0.0-pyh4616a5c_0):
 │ │ ╭──────────────────┬──────────────────────────╮
 │ │ │ Name             ┆ Spec                     │
 │ │ ╞══════════════════╪══════════════════════════╡
 │ │ │ Run dependencies ┆                          │
 │ │ │ pydantic         ┆ >=2.12.3,<3              │
 │ │ │ pydantic-ai      ┆ >=1.32.0,<2              │
 │ │ │ python           ┆ >=3.12                   │
 │ │ │                  ┆ * (RE of [host: python]) │
 │ │ ╰──────────────────┴──────────────────────────╯
 │ │
 │ ╰─────────────────── (took 0 seconds)
 │
 │ ╭─ Running build script
 │ │ Using pip 25.3 from $PREFIX/lib/python3.14/site-packages/pip (python 3.14)
 │ │ Non-user install because user site-packages disabled
 │ │ Ignoring indexes: https://pypi.org/simple
 │ │ Created temporary directory: /tmp/pip-build-tracker-_junl_k2
 │ │ Initialized build tracking at /tmp/pip-build-tracker-_junl_k2
 │ │ Created build tracker: /tmp/pip-build-tracker-_junl_k2
 │ │ Entered build tracker: /tmp/pip-build-tracker-_junl_k2
 │ │ Created temporary directory: /tmp/pip-install-jh4fbpbx
 │ │ Created temporary directory: /tmp/pip-ephem-wheel-cache-4wr15ma0
 │ │ Obtaining file:///home/runner/work/project/project
 │ │   Added file:///home/runner/work/project/project to build tracker '/tmp/pip-build-tracker-_junl_k2'
 │ │   Checking if build backend supports build_editable: started
 │ │   Running command Checking if build backend supports build_editable
 │ │   Checking if build backend supports build_editable: finished with status 'done'
 │ │   Created temporary directory: /tmp/pip-modern-metadata-ntq1rxx2
 │ │   Preparing editable metadata (pyproject.toml): started
 │ │   Running command Preparing editable metadata (pyproject.toml)
 │ │   Traceback (most recent call last):
 │ │     File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 195, in prepare_metadata_for_build_editable
 │ │       hook = backend.prepare_metadata_for_build_editable
 │ │              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │   AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'
 │ │   During handling of the above exception, another exception occurred:
 │ │   Traceback (most recent call last):
 │ │     File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
 │ │       main()
 │ │       ~~~~^^
 │ │     File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
 │ │       json_out["return_val"] = hook(**hook_input["kwargs"])
 │ │                                ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 204, in prepare_metadata_for_build_editable
 │ │       whl_basename = build_hook(metadata_directory, config_settings)
 │ │     File "$PREFIX/lib/python3.14/site-packages/hatchling/build.py", line 83, in build_editable
 │ │       return os.path.basename(next(builder.build(directory=wheel_directory, versions=["editable"])))
 │ │                               ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     File "$PREFIX/lib/python3.14/site-packages/hatchling/builders/plugin/interface.py", line 157, in build
 │ │       artifact = version_api[version](directory, **build_data)
 │ │     File "$PREFIX/lib/python3.14/site-packages/hatchling/builders/wheel.py", line 522, in build_editable
 │ │       return self.build_editable_detection(directory, **build_data)
 │ │              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     File "$PREFIX/lib/python3.14/site-packages/hatchling/builders/wheel.py", line 525, in build_editable_detection
 │ │       from editables import EditableProject
 │ │   ModuleNotFoundError: No module named 'editables'
 │ │   error: subprocess-exited-with-error
 │ │   
 │ │   × Preparing editable metadata (pyproject.toml) did not run successfully.
 │ │   │ exit code: 1
 │ │   ╰─> No available output.
 │ │   
 │ │   note: This error originates from a subprocess, and is likely not a problem with pip.
 │ │   full command: $PREFIX/bin/python $PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_e
 │ │ ditable /tmp/tmpwg_o57fr
 │ │   cwd: /home/runner/work/project/project
 │ │   Preparing editable metadata (pyproject.toml): finished with status 'error'
 │ │ error: metadata-generation-failed
 │ │ × Encountered error while generating package metadata.
 │ │ ╰─> from file:///home/runner/work/project/project
 │ │ note: This is an issue with the package mentioned above, not pip.
 │ │ hint: See above for details.
 │ │ Exception information:
 │ │ Traceback (most recent call last):
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/operations/build/metadata_editable.py", line 36, in generate_editable_metadata
 │ │     distinfo_dir = backend.prepare_metadata_for_build_editable(metadata_dir)
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/utils/misc.py", line 728, in prepare_metadata_for_build_editable
 │ │     return super().prepare_metadata_for_build_editable(
 │ │            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
 │ │         metadata_directory=metadata_directory,
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │         config_settings=cs,
 │ │         ^^^^^^^^^^^^^^^^^^^
 │ │         _allow_fallback=_allow_fallback,
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 306, in prepare_metadata_for_build_editable
 │ │     return self._call_hook(
 │ │            ~~~~~~~~~~~~~~~^
 │ │         "prepare_metadata_for_build_editable",
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     ...<4 lines>...
 │ │         },
 │ │         ^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 392, in _call_hook
 │ │     self._subprocess_runner(
 │ │     ~~~~~~~~~~~~~~~~~~~~~~~^
 │ │         [python, abspath(str(script)), hook_name, td],
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │         cwd=self.source_dir,
 │ │         ^^^^^^^^^^^^^^^^^^^^
 │ │         extra_environ=extra_environ,
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/utils/subprocess.py", line 240, in runner
 │ │     call_subprocess(
 │ │     ~~~~~~~~~~~~~~~^
 │ │         cmd,
 │ │         ^^^^
 │ │     ...<3 lines>...
 │ │         spinner=spinner,
 │ │         ^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/utils/subprocess.py", line 212, in call_subprocess
 │ │     raise error
 │ │ pip._internal.exceptions.InstallationSubprocessError: Preparing editable metadata (pyproject.toml) exited with 1
 │ │ The above exception was the direct cause of the following exception:
 │ │ Traceback (most recent call last):
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/cli/base_command.py", line 107, in _run_wrapper
 │ │     status = _inner_run()
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/cli/base_command.py", line 98, in _inner_run
 │ │     return self.run(options, args)
 │ │            ~~~~~~~~^^^^^^^^^^^^^^^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/cli/req_command.py", line 85, in wrapper
 │ │     return func(self, options, args)
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/commands/install.py", line 388, in run
 │ │     requirement_set = resolver.resolve(
 │ │         reqs, check_supported_wheels=not options.target_dir
 │ │     )
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 79, in resolve
 │ │     collected = self.factory.collect_root_requirements(root_reqs)
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 538, in collect_root_requirements
 │ │     reqs = list(
 │ │         self._make_requirements_from_install_req(
 │ │     ...<2 lines>...
 │ │         )
 │ │     )
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 494, in _make_requirements_from_install_req
 │ │     cand = self._make_base_candidate_from_link(
 │ │         ireq.link,
 │ │     ...<2 lines>...
 │ │         version=None,
 │ │     )
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 205, in _make_base_candidate_from_link
 │ │     self._editable_candidate_cache[link] = EditableCandidate(
 │ │                                            ~~~~~~~~~~~~~~~~~^
 │ │         link,
 │ │         ^^^^^
 │ │     ...<3 lines>...
 │ │         version=version,
 │ │         ^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 343, in __init__
 │ │     super().__init__(
 │ │     ~~~~~~~~~~~~~~~~^
 │ │         link=link,
 │ │         ^^^^^^^^^^
 │ │     ...<4 lines>...
 │ │         version=version,
 │ │         ^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
 │ │     self.dist = self._prepare()
 │ │                 ~~~~~~~~~~~~~^^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 238, in _prepare
 │ │     dist = self._prepare_distribution()
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 353, in _prepare_distribution
 │ │     return self._factory.preparer.prepare_editable_requirement(self._ireq)
 │ │            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/operations/prepare.py", line 714, in prepare_editable_requirement
 │ │     dist = _get_prepared_distribution(
 │ │         req,
 │ │     ...<3 lines>...
 │ │         self.check_build_deps,
 │ │     )
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/operations/prepare.py", line 77, in _get_prepared_distribution
 │ │     abstract_dist.prepare_distribution_metadata(
 │ │     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
 │ │         build_env_installer, build_isolation, check_build_deps
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/distributions/sdist.py", line 71, in prepare_distribution_metadata
 │ │     self.req.prepare_metadata()
 │ │     ~~~~~~~~~~~~~~~~~~~~~~~~~^^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/req/req_install.py", line 524, in prepare_metadata
 │ │     self.metadata_directory = generate_editable_metadata(
 │ │                               ~~~~~~~~~~~~~~~~~~~~~~~~~~^
 │ │         build_env=self.build_env,
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │         backend=self.pep517_backend,
 │ │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 │ │         details=details,
 │ │         ^^^^^^^^^^^^^^^^
 │ │     )
 │ │     ^
 │ │   File "$PREFIX/lib/python3.14/site-packages/pip/_internal/operations/build/metadata_editable.py", line 38, in generate_editable_metadata
 │ │     raise MetadataGenerationFailed(package_details=details) from error
 │ │ pip._internal.exceptions.MetadataGenerationFailed: metadata generation failed
 │ │ Removed file:///home/runner/work/project/project from build tracker '/tmp/pip-build-tracker-_junl_k2'
 │ │ Removed build tracker: '/tmp/pip-build-tracker-_junl_k2'
 │ │ × error Script failed with status 1
 │ │ × error 
 │ │ × error Script execution failed.
 │ │ × error 
 │ │ × error   Work directory: /home/runner/work/project/project/.pixi/build/work/project-UdWkRD18ix8-TwLmj22Yeao/work
 │ │ × error   Prefix: /home/runner/work/project/project/.pixi/build/work/project-UdWkRD18ix8-TwLmj22Yeao/host_placehold_placehold_placehold_placehold_p
 │ │ × error lacehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol
 │ │ × error   Build prefix: /home/runner/work/project/project/.pixi/build/work/project-UdWkRD18ix8-TwLmj22Yeao/bld
 │ │ × error 
 │ │ × error To run the script manually, use the following command:
 │ │ × error 
 │ │ × error   cd "/home/runner/work/project/project/.pixi/build/work/project-UdWkRD18ix8-TwLmj22Yeao/work" && ./conda_build.sh
 │ │ × error 
 │ │ × error To run commands interactively in the build environment:
 │ │ × error 
 │ │ × error   cd "/home/runner/work/project/project/.pixi/build/work/project-UdWkRD18ix8-TwLmj22Yeao/work" && source build_env.sh
 │ │
 │ ╰─────────────────── (took 3 seconds)
 │
 ╰─────────────────── (took 3 seconds)
Error:   × failed to build 'project' from '.'
  ╰─▶   × Script failed to execute