Use venvstack for portable python runtime

[madclaws]

https://lmstudio.ai/blog/venvstacks

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

✅ Successfully generated implementation plan.

💡 Use @coderabbitai <feedback> to replan

Examples

• Example 1
• Example 2

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

Coding Plan

Approach

Replace the current Python runtime distribution approach with venvstacks to create a portable Python environment that can be distributed with the application, eliminating the need for users to install Python and its dependencies on the target system.

Observations

The Tiles project is a Rust CLI application (built with Cargo) that manages a Python-based FastAPI inference server. The current approach uses uv to create a single virtual environment during installation, requiring Python 3.13 to be available on the target system. The distribution consists of a tarball containing a compiled Rust binary and the Python server source code. The Rust application spawns the Python server using uv run and manages its lifecycle via PID tracking. Build scripts (bundler.sh) create platform-specific archives, while installation scripts (install.sh) handle Python installation, virtual environment creation, and dependency synchronization.

Assumptions

Assumption 1: Should the application layer (server/ code) be part of the venvstacks layers or remain separate?

Options Considered:

• Make server/ an installable package and include it as an application layer in venvstacks
• Keep server/ as a separate directory and only use venvstacks for runtime + dependencies

Chosen Option: 2

Rationale: Keep server/ separate. This minimizes changes to the existing structure while still achieving the portability goal. The venvstacks layers will provide the Python interpreter and all dependencies, while server/ remains a directory that gets copied to the target system and executed using the venvstacks Python interpreter.

Context: The current codebase treats server/ as a directory of scripts rather than an installable Python package. Converting it to a package would require restructuring and adding package metadata.

Assumption 2: Should we use separate runtime and framework layers or combine them into a single layer?

Options Considered:

• Use two layers (runtime + framework) to match venvstacks recommended architecture
• Use a single combined layer since there's no framework sharing benefit

Chosen Option: 2

Rationale: Use a single combined layer. Since Tiles doesn't need to share dependencies across multiple applications, a single layer containing both the Python interpreter and all dependencies is simpler to build, distribute, and maintain. This reduces complexity while still achieving the portability goal.

Context: venvstacks is designed to share large frameworks across multiple applications, but Tiles has only one Python application component.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: venvstacks Stack Definition and Build Integration

Set up the venvstacks specification and integrate it into the build process to create portable Python runtime layers.

Task 1: Create venvstacks Stack Specification

Define the layered Python environment structure that will replace the current single virtual environment approach.

• Create venvstacks.toml at the repository root defining a single "app" layer containing both the Python 3.13 runtime (via python-build-standalone) and all dependencies from server/pyproject.toml
• Configure the layer to use python-build-standalone as the runtime source (managed via pdm) and specify dependencies using requirements format or reference to server/pyproject.toml
• Specify platform targets matching current bundler.sh platforms (aarch64-apple-darwin, x86_64-apple-darwin, aarch64-unknown-linux-gnu, x86_64-unknown-linux-gnu)
• Set output directory for layer archives to align with the dist/ directory structure

Task 2: Install venvstacks and pdm as Build Dependencies

Add build-time tooling required for creating portable Python layers.

• Update scripts/bundler.sh to check for and install venvstacks (via pipx) if not available
• Install pdm as a build dependency since it's required for python-build-standalone integration with venvstacks
• Add error handling for missing dependencies with clear installation instructions

Task 3: Add venvstacks Build Process

Modify the build script to use venvstacks commands for creating portable Python runtime layers.

• Update scripts/bundler.sh to run venvstacks lock to generate locked dependency specifications from the stack definition
• Run venvstacks build to create the layer with locked dependencies and python-build-standalone runtime
• Run venvstacks export to generate a portable archive of the layer for the current platform
• Update the tarball creation to include the exported venvstacks layer archive alongside the Rust binary and server/ directory
• Maintain exclusion of pycache and temporary build artifacts from the distribution

Task 4: Add Platform-Specific Build Support

Handle cross-platform builds since venvstacks requires building on each target platform.

• Document in bundler.sh that platform-specific builds must run on the target platform (or via CI matrix with multiple platform runners)
• Update bundler.sh to detect the current platform and only build the layer for that platform
• Add comments explaining that cross-platform distribution requires running the build on each target OS/architecture combination
• Consider adding a CI workflow file (e.g., .github/workflows/build.yml) that runs bundler.sh on multiple platform runners if cross-platform releases are needed

Task 5: Add Development Workflow Support

Provide commands for developers to work with venvstacks locally during development.

• Add venvstacks-lock and venvstacks-build commands to justfile for local venvstacks testing
• Optionally keep the existing uv-based development workflow (just run) for rapid iteration while developing, as it provides a familiar and fast development experience
• Document the distinction between development workflow (uv) and production distribution (venvstacks) in developer documentation

🤖 Prompt for AI agents

Set up venvstacks specification and integrate it into the build process to
create portable Python runtime layers.

Create venvstacks.toml at repository root:
- Define single "app" layer with Python 3.13 runtime via python-build-standalone
- Include all dependencies from server/pyproject.toml
- Specify platform targets: aarch64-apple-darwin, x86_64-apple-darwin,
aarch64-unknown-linux-gnu, x86_64-unknown-linux-gnu
- Configure output directory to align with dist/ structure

Update scripts/bundler.sh:
- Add checks and installation for venvstacks (via pipx) and pdm
- Add error handling with clear installation instructions
- Run venvstacks lock, build, and export commands in sequence
- Update tarball creation to include exported venvstacks layer archive alongside
Rust binary and server/ directory
- Detect current platform and build layer only for that platform
- Document platform-specific build requirements
- Maintain exclusion of __pycache__ and build artifacts

Update justfile:
- Add venvstacks-lock and venvstacks-build commands for local testing
- Document distinction between uv development workflow and venvstacks production
distribution

Phase 2: Installation Process Adaptation

Update the installation script to unpack and configure the venvstacks layer instead of creating a virtual environment from scratch.

Task 1: Simplify Installation Requirements

Remove Python installation logic since the runtime is now bundled with the application.

• Remove all Python 3 detection and installation code from scripts/install.sh (lines 19-30 that handle Homebrew and apt installation)
• Remove the Python 3.13 availability check since the runtime layer provides it
• Remove the uv installation steps since venvstacks will be used instead

Task 2: Unpack and Activate venvstacks Layer

Replace virtual environment creation with venvstacks layer extraction and activation.

• Extract the venvstacks layer archive from the downloaded tarball to the target installation location (~/.local/share/tiles/python_env/ or similar)
• The extracted layer is a complete virtual environment with sitecustomize.py for layer chaining (though with a single layer, chaining is not used)
• Verify the layer's bin/python executable is present and has correct permissions
• Copy the server/ directory to ~/.local/share/tiles/server as before
• Note: venvstacks layers use absolute paths internally; relinking may be handled automatically by sitecustomize.py, or may require running venvstacks commands if the tool provides them

🤖 Prompt for AI agents

Update installation script to unpack venvstacks layer instead of creating
virtual environment from scratch.

Modify scripts/install.sh:
- Remove Python 3 detection and installation code (Homebrew and apt installation
logic)
- Remove Python 3.13 availability check
- Remove uv installation steps
- Extract venvstacks layer archive from tarball to
~/.local/share/tiles/python_env/
- Verify bin/python executable exists and has correct permissions
- Copy server/ directory to ~/.local/share/tiles/server

Phase 3: Runtime Execution Changes

Modify the Rust application to use the venvstacks Python interpreter instead of relying on uv to manage the environment.

Task 1: Update Python Server Spawning

Change the server process spawning logic to directly execute the venvstacks Python interpreter.

• Modify src/runner/mlx.rs in the start_server_daemon() function to replace Command::new("uv") with direct Python execution
• Update the command to use the path to the venvstacks layer Python interpreter (e.g., ~/.local/share/tiles/python_env/bin/python)
• Change arguments from uv's pattern to direct Python execution: ["-m", "server.main"]
• Set the current directory to the server directory location using .current_dir()
• Remove the --project and related uv-specific arguments

Task 2: Add Path Resolution for venvstacks Python

Create helper functions to locate the venvstacks Python interpreter.

• Add a function get_python_interpreter() in src/runner/mlx.rs (similar to the existing get_server_dir()) that returns the PathBuf to the venvstacks layer Python interpreter
• In debug mode, point to a local venvstacks build output directory (e.g., ./build/app/bin/python or similar)
• In production mode, construct the path based on XDG_DATA_HOME or ~/.local/share/tiles/python_env/bin/python
• Use this function in start_server_daemon() to determine the Python interpreter path
• Add error handling for missing interpreter with helpful error message

Task 3: Remove uv Dependency Check

Update system health checks since uv is no longer required at runtime.

• Modify src/core/health.rs if it checks for uv availability to remove that check
• Keep the Python3 check if desired for debugging purposes, but note that it should always succeed since Python is bundled

🤖 Prompt for AI agents

Modify Rust application to use venvstacks Python interpreter instead of uv.

Update src/runner/mlx.rs:
- Add get_python_interpreter() function that returns PathBuf to venvstacks
Python interpreter
- In debug mode: point to ./build/app/bin/python
- In production mode: use XDG_DATA_HOME or
~/.local/share/tiles/python_env/bin/python
- Add error handling for missing interpreter
- In start_server_daemon(), replace Command::new("uv") with direct Python
execution using get_python_interpreter()
- Change arguments to ["-m", "server.main"]
- Set current_dir() to server directory location
- Remove --project and uv-specific arguments

Update src/core/health.rs:
- Remove uv availability check

Phase 4: Documentation and Cleanup

Update project documentation and remove obsolete configuration related to the previous installation approach.

Task 1: Update Build and Installation Documentation

Document the new venvstacks-based workflow for contributors and users.

• Update README or relevant documentation to explain that the application now bundles its Python runtime
• Document the new build process requirements: venvstacks and pdm must be available during bundling (installable via pipx)
• Explain that platform-specific builds require running the build on each target platform
• Update installation instructions to reflect that Python installation is no longer required
• Add troubleshooting guidance for venvstacks-specific issues (missing dependencies, platform mismatches, etc.)

Task 2: Clean Up Obsolete Dependencies

Remove references to uv in runtime contexts where it's no longer needed.

• Remove uv from any runtime dependency lists or checks, keeping it only as an optional development tool
• Update server/pyproject.toml comments or metadata if they reference uv-specific workflow
• Consider whether server/uv.lock should be removed or kept for development use (can remain for dev workflow consistency)

🤖 Prompt for AI agents

Update documentation and clean up obsolete configuration.

Update README and relevant documentation:
- Explain that application now bundles Python runtime
- Document build requirements: venvstacks and pdm (installable via pipx)
- Explain platform-specific build requirements
- Update installation instructions to remove Python installation requirement
- Add troubleshooting guidance for venvstacks issues

Clean up obsolete dependencies:
- Remove uv from runtime dependency lists and checks
- Update server/pyproject.toml comments if they reference uv-specific workflow

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Set up venvstacks specification and integrate it into the build process to
create portable Python runtime layers.

Create venvstacks.toml at repository root:
- Define single "app" layer with Python 3.13 runtime via python-build-standalone
- Include all dependencies from server/pyproject.toml
- Specify platform targets: aarch64-apple-darwin, x86_64-apple-darwin,
aarch64-unknown-linux-gnu, x86_64-unknown-linux-gnu
- Configure output directory to align with dist/ structure

Update scripts/bundler.sh:
- Add checks and installation for venvstacks (via pipx) and pdm
- Add error handling with clear installation instructions
- Run venvstacks lock, build, and export commands in sequence
- Update tarball creation to include exported venvstacks layer archive alongside
Rust binary and server/ directory
- Detect current platform and build layer only for that platform
- Document platform-specific build requirements
- Maintain exclusion of __pycache__ and build artifacts

Update justfile:
- Add venvstacks-lock and venvstacks-build commands for local testing
- Document distinction between uv development workflow and venvstacks production
distribution
===============================================================================

Task: 2

Update installation script to unpack venvstacks layer instead of creating
virtual environment from scratch.

Modify scripts/install.sh:
- Remove Python 3 detection and installation code (Homebrew and apt installation
logic)
- Remove Python 3.13 availability check
- Remove uv installation steps
- Extract venvstacks layer archive from tarball to
~/.local/share/tiles/python_env/
- Verify bin/python executable exists and has correct permissions
- Copy server/ directory to ~/.local/share/tiles/server
===============================================================================

Task: 3

Modify Rust application to use venvstacks Python interpreter instead of uv.

Update src/runner/mlx.rs:
- Add get_python_interpreter() function that returns PathBuf to venvstacks
Python interpreter
- In debug mode: point to ./build/app/bin/python
- In production mode: use XDG_DATA_HOME or
~/.local/share/tiles/python_env/bin/python
- Add error handling for missing interpreter
- In start_server_daemon(), replace Command::new("uv") with direct Python
execution using get_python_interpreter()
- Change arguments to ["-m", "server.main"]
- Set current_dir() to server directory location
- Remove --project and uv-specific arguments

Update src/core/health.rs:
- Remove uv availability check
===============================================================================

Task: 4

Update documentation and clean up obsolete configuration.

Update README and relevant documentation:
- Explain that application now bundles Python runtime
- Document build requirements: venvstacks and pdm (installable via pipx)
- Explain platform-specific build requirements
- Update installation instructions to remove Python installation requirement
- Add troubleshooting guidance for venvstacks issues

Clean up obsolete dependencies:
- Remove uv from runtime dependency lists and checks
- Update server/pyproject.toml comments if they reference uv-specific workflow

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- @coderabbitai You can skip phase 3. Add a simple unit test case for phase 2.
- @coderabbitai For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!