Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions .github/workflows/build-and-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
name: Build and Publish

on:
push:
tags: [ 'v*' ]
pull_request:
branches: [ main ]
release:
types: [ published ]

env:
FORCE_COLOR: 1

jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, macos-13, macos-14]

steps:
- uses: actions/checkout@v4

- name: Install Linux dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libattr1-dev libfuse3-dev fuse3 pkg-config
- name: Install macOS dependencies
if: runner.os == 'macOS'
run: |
brew install macfuse pkg-config
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install cibuildwheel
run: python -m pip install cibuildwheel==2.16.5

- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
CIBW_BUILD: cp38-* cp39-* cp310-* cp311-* cp312-* cp313-*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove end-of-life distributions.


CIBW_SKIP: "*-win32 *-manylinux_i686 *-musllinux*"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why skip manylinux?


- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

build_sdist:
name: Build source distribution
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Install Linux dependencies
run: |
sudo apt-get update
sudo apt-get install -y libattr1-dev libfuse3-dev fuse3 pkg-config
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'

- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools cython build
- name: Build Cython files
run: python setup.py build_cython

- name: Build sdist
run: python -m build --sdist

- uses: actions/upload-artifact@v4
with:
name: cibw-sdist
path: dist/*.tar.gz

publish_to_pypi:
name: Publish to PyPI
needs: [build_wheels, build_sdist]
runs-on: ubuntu-24.04
if: github.event_name == 'release' && github.event.action == 'published'
environment:
name: pypi
url: https://pypi.org/p/pyfuse3
permissions:
id-token: write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify how the permissioning works? Do we need to get a token from PyPi and store it somewhere where Github can access it?


steps:
- uses: actions/download-artifact@v4
with:
pattern: cibw-*
path: dist
merge-multiple: true

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

publish_to_testpypi:
name: Publish to TestPyPI
needs: [build_wheels, build_sdist]
runs-on: ubuntu-24.04
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
environment:
name: testpypi
url: https://test.pypi.org/p/pyfuse3
permissions:
id-token: write

steps:
- uses: actions/download-artifact@v4
with:
pattern: cibw-*
path: dist
merge-multiple: true

- name: Publish to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
49 changes: 49 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: Create Release

on:
push:
tags:
- 'v*'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to match the current tagging schema...?


permissions:
contents: write

jobs:
create-release:
name: Create GitHub Release
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4

- name: Extract version from tag
id: version
run: echo "version=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT

- name: Extract changelog
id: changelog
run: |
# Extract changelog for this version from Changes.rst
if [ -f "Changes.rst" ]; then
# Try to extract the section for this version
awk '/^pyfuse3 '"${{ steps.version.outputs.version }}"'/{flag=1; next} /^pyfuse3 [0-9]/{flag=0} flag' Changes.rst > changelog.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add some output to the PR that illustrates that this parsing has been tested?

Also, can we make the job fail if parsing fails?

if [ -s changelog.txt ]; then
echo "changelog<<EOF" >> $GITHUB_OUTPUT
cat changelog.txt >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
else
echo "changelog=Release ${{ steps.version.outputs.version }}" >> $GITHUB_OUTPUT
fi
else
echo "changelog=Release ${{ steps.version.outputs.version }}" >> $GITHUB_OUTPUT
fi
- name: Create Release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: pyfuse3 ${{ steps.version.outputs.version }}
body: ${{ steps.changelog.outputs.changelog }}
draft: false
prerelease: ${{ contains(steps.version.outputs.version, 'rc') || contains(steps.version.outputs.version, 'beta') || contains(steps.version.outputs.version, 'alpha') }}
147 changes: 147 additions & 0 deletions RELEASE.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be integrated into developer-notes/release_process.rst instead.

Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Release Process for pyfuse3

This document describes the automated release process for pyfuse3.

## Overview

The project uses GitHub Actions to automatically build wheels and publish to PyPI when releases are created. The process supports:

- **Multi-platform wheel building** (Linux, macOS) using cibuildwheel
- **Automatic PyPI publishing** on GitHub releases
- **TestPyPI publishing** on main branch pushes
- **Source distribution building**

## Release Workflows

### 1. Build and Publish (`build-and-publish.yml`)

This workflow runs on:
- GitHub releases → Publishes to PyPI
- Pull requests → Builds wheels for testing

**Jobs:**
- `build_wheels`: Builds wheels for Linux and macOS using cibuildwheel
- `build_sdist`: Builds source distribution
- `publish_to_pypi`: Publishes to PyPI on releases (requires trusted publishing)
- `publish_to_testpypi`: Publishes to TestPyPI on main branch pushes

### 2. Release Creation (`release.yml`)

Automatically creates GitHub releases when version tags are pushed.

### 3. Testing (`test.yml`)

Runs the existing test suite across multiple Python versions.

## Making a Release

### Option 1: Using the Release Script

```bash
# Create a new release
python scripts/release.py 3.4.1

# Dry run to see what would happen
python scripts/release.py 3.4.1 --dry-run

# Update version without creating tag
python scripts/release.py 3.4.1 --no-tag
```

### Option 2: Manual Process

1. **Update version** in `setup.py`:
```python
PYFUSE3_VERSION = '3.4.1'
```

2. **Commit and tag**:
```bash
git add setup.py
git commit -m "Bump version to 3.4.1"
git tag -a v3.4.1 -m "Release 3.4.1"
```

3. **Push**:
```bash
git push origin main
git push origin v3.4.1
```

## Repository Setup Requirements

### GitHub Repository Settings

1. **Enable GitHub Actions** in repository settings

2. **Configure PyPI Trusted Publishing**:
- Go to PyPI → Account Settings → Publishing
- Add trusted publisher for your GitHub repository
- Set environment name to `release`

3. **Configure TestPyPI Trusted Publishing** (optional):
- Same process for test.pypi.org
- Set environment name to `test-release`

4. **Create GitHub Environments**:
- Go to repository Settings → Environments
- Create `release` environment (for production releases)
- Create `test-release` environment (for test releases)
- Enable "Required reviewers" if desired

### Dependencies

The build process requires these system dependencies:
- **Linux**: `libfuse3-dev`, `pkg-config`
- **macOS**: `macfuse`, `pkg-config` (via Homebrew)

These are automatically installed by the workflow.

## Build Configuration

The build is configured via `pyproject.toml` using cibuildwheel:

- **Python versions**: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13
- **Platforms**: Linux (x86_64), macOS (x86_64, arm64)
- **Skip**: 32-bit builds, musl Linux builds
- **Dependencies**: Automatically installs FUSE development libraries

## Testing

Each built wheel is tested by importing pyfuse3 to ensure basic functionality.

## Troubleshooting

### Build Failures

1. **FUSE dependency issues**: Check that system dependencies are properly installed
2. **Cython compilation**: Ensure Cython files are built before wheel creation
3. **Platform-specific issues**: Check cibuildwheel logs for platform-specific errors

### Publishing Failures

1. **Authentication**: Ensure trusted publishing is configured correctly
2. **Duplicate versions**: PyPI doesn't allow re-uploading the same version
3. **Environment protection**: Check GitHub environment settings

### Local Testing

Test the release process locally:

```bash
# Install cibuildwheel
pip install cibuildwheel

# Build wheels locally
python -m cibuildwheel --output-dir wheelhouse

# Test installation
pip install wheelhouse/*.whl
python -c "import pyfuse3; print('Success!')"
```

## Monitoring

- **GitHub Actions**: Monitor workflow runs in the Actions tab
- **PyPI**: Check package page for successful uploads
- **TestPyPI**: Verify test uploads work correctly
28 changes: 27 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[build-system]
requires = ["setuptools"]
requires = ["setuptools", "cython"]
build-backend = "setuptools.build_meta"

[tool.mypy]
Expand All @@ -8,3 +8,29 @@ packages = ["pyfuse3"]
modules = ["_pyfuse3", "pyfuse3_asyncio"]
namespace_packages = false
strict = true

[tool.cibuildwheel]
build = "cp38-* cp39-* cp310-* cp311-* cp312-* cp313-*"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to repeat code from the Github workflow.

skip = "*-win32 *-manylinux_i686 *-musllinux*"

# Use newer manylinux images to avoid CentOS 7 mirror issues
manylinux-x86_64-image = "manylinux_2_28"
manylinux-aarch64-image = "manylinux_2_28"

before-build = [
"pip install setuptools cython",
"python setup.py build_cython"
]

test-requires = ["pytest", "pytest-trio", "trio"]
test-command = "python -c \"import pyfuse3; print('pyfuse3 imported successfully')\""

[tool.cibuildwheel.linux]
before-all = [
"dnf install -y fuse3-devel pkgconfig"
]
environment = { PKG_CONFIG_PATH = "/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig" }

[tool.cibuildwheel.macos]
before-all = ["brew install macfuse pkg-config"]
environment = { PKG_CONFIG_PATH = "/usr/local/lib/pkgconfig:/opt/homebrew/lib/pkgconfig" }
Loading