Skip to content

Comments

Support environment.python.requires option in manifest#648

Merged
amol- merged 24 commits intomainfrom
manifest-with-requirements
Mar 26, 2025
Merged

Support environment.python.requires option in manifest#648
amol- merged 24 commits intomainfrom
manifest-with-requirements

Conversation

@amol-
Copy link
Collaborator

@amol- amol- commented Mar 20, 2025

Intent

Connect supports python interpreter version requirements from the environment.python.requires field in the manifest.
That field should be filled based on the version requirements in the project metadata files:

  • pyproject.toml
  • setup.cfg
  • .python-version

Type of Change

  • New Feature
  • Breaking Change

Approach

  1. Refactor the various deploy commands to generate the Environment information via the same code path
  2. Implement support for detecting the interpreter requirement in that shared entry point
  3. Write the detected constraint to the generated manifest before submitting it to the server.

Automated Tests

  • Existing test suite checks that the support for current deploys is still working as expected
  • New tests check that the manifest.json gets generated with the right environment.python.requires field when detected

Directions for Reviewers

  • --override-python-version still allows overwriting the detected version, but on long term the plan is to deprecate the option and ask users to create a .python-version file to do the same. So that is a more standard approach in the Python ecosystem.

Checklist

  • I have updated CHANGELOG.md to cover notable changes.
  • I have updated all related GitHub issues to reflect their current state.

@github-actions
Copy link

github-actions bot commented Mar 24, 2025

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
4888 3666 75% 0% 🟢

New Files

File Coverage Status
rsconnect/subprocesses/init.py 100% 🟢
rsconnect/subprocesses/inspect_environment.py 84% 🟢
TOTAL 92% 🟢

Modified Files

File Coverage Status
rsconnect/actions.py 44% 🟢
rsconnect/api.py 71% 🟢
rsconnect/bundle.py 79% 🟢
rsconnect/environment.py 84% 🟢
rsconnect/main.py 63% 🟢
rsconnect/pyproject.py 100% 🟢
TOTAL 73% 🟢

updated for commit: cbab772 by action🐍

manifest["files"] = {}

return manifest
manifest: Manifest = Manifest(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All places where we were manually creating ManifestData have been migrated to use the Manifest class. This ensures that the same behaviour is applied independently from the command used, and thus that the python version requirements are always applied to the manifest without having to set them in every single place.

This is also more robust to future changes to the manifest.

return exists(python_path) or exists(win_path)


def list_environment_dirs(directory: str | Path):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything related to environments has been moved into rsconnect.environment

def _replace(self, **kwargs: object):
return replace(self, **kwargs)

class Environment:
Copy link
Collaborator Author

@amol- amol- Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to improve separation of concerns isolating the detection of the environment state in subprocesses.inspect_environment which will only focus on inspecting the environment and return EnvironmentData in JSON.

Then the developer is expected to always only interact with Environment class which invokes inspect_environment itself. The EnvironmentData is considered an internal implementation detail and only acts as the communication protocol between Environment and inspect_environment.

Every time a python environment has to be describe, the developer is expected to build an Environment object, usually by invoking Environment.create_python_environment which is the preferred factory method that takes care of inspecting an existing envrionment and creating an Environment object out of it.

@@ -0,0 +1,235 @@
#!/usr/bin/env python
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't change, just renamed from rsconnect.environment

import toml as tomllib # type: ignore[no-redef]


def detect_python_version_requirement(directory: typing.Union[str, pathlib.Path]) -> typing.Optional[str]:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more or less the entry point for the whole version requirement detectiong process, based on the previously merged metadata parsing functions.

@amol- amol- marked this pull request as ready for review March 24, 2025 13:35
@amol- amol- self-assigned this Mar 24, 2025
@amol- amol- requested review from Lytol and aronatkins March 24, 2025 13:35
@amol-
Copy link
Collaborator Author

amol- commented Mar 24, 2025

There are the two failures that are also happening on main and independent from this PR

Copy link
Contributor

@Lytol Lytol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outstanding! Really appreciate the refactor of both the Environment class as well as extracting the environment inspection into subprocesses. I did not review line-by-line on the refactors that were simply moving things around, so I'm trusting that the test coverage is enough there.

My comments are trivial nits and/or copy edits, so do with them what you'd like.

README.md Outdated
if no `.python-version` file exists.
3. A `setup.cfg` with a `options.python_requirs` field exists.
In such case the requirement specified in the field will be used
if no `.python-version` or `pyproject.toml` files exist.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The requirement might still be used if pyproject.toml exists, but only if it doesn't specify a project.requires-python. This is pretty nuanced though, and I think it's fine to leave it as is.

README.md Outdated
4. If no other source of version requirement was found, then
the interpreter in use is considered the one required to run the content.

On newer Posit Connect versions the requirement detected by `rsconnect` is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we specify what "newer Posit Connect versions" means here? Specifically, I think it'll be versions >=2025.03.0.

Comment on lines +247 to +249
We recommend providing a `pyproject.toml` with a `project.requires-python` field
if the deployed content is an installable package and a `.python-version` file
for plain directories.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ It's really nice to provide specific guidance like this.

amol- and others added 2 commits March 26, 2025 10:24
Co-authored-by: Brian Smith <brian.e.smith@gmail.com>
@amol- amol- merged commit 98923cb into main Mar 26, 2025
11 of 13 checks passed
@amol- amol- deleted the manifest-with-requirements branch March 26, 2025 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants