Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 26 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,9 @@

A python project template to simplify project setup. Adapted from https://github.com/fmind/cookiecutter-mlops-package

This template copy omits the MLFlow functionality. Use the linked mlops-package template if this is desired

The template provides a robust foundation for building, testing, packaging, and deploying Python packages and Docker Images. Adapt it to your project's needs; the source material is MLOps-focused but is suitable for a wide array of Python projects.

**Source resources**:
**Original resources**:
- **[MLOps Coding Course (Learning)](https://mlops-coding-course.fmind.dev/)**: Learn how to create, develop, and maintain a state-of-the-art MLOps code base.
- **[MLOps Python Package (Example)](https://github.com/fmind/mlops-python-package)**: Kickstart your MLOps initiative with a flexible, robust, and productive Python package.

Expand All @@ -18,9 +16,13 @@ This template equips you with the essentials for creating, testing, and packagin

You have the freedom to structure your `src/` and `tests/` directories according to your preferences. Alternatively, you can draw inspiration from the structure used in the [MLOps Python Package](https://github.com/fmind/mlops-python-package) project for a ready-made implementation.

## Applications

This template includes a few optional application skeletons. See the nested README for details.

## Key Features

(This section was copied into the created project's README so tool info is available to users.)
This section was copied into the created project's README so tool info is available.

* **Streamlined Project Structure:** A well-defined directory layout for source code, tests, documentation, tasks, and Docker configurations.
Uv Integration: Effortless dependency management and packaging with [uv](https://docs.astral.sh/uv/).
Expand All @@ -30,6 +32,20 @@ Uv Integration: Effortless dependency management and packaging with [uv](https:/
* **uv+just Task Automation:** [just](https://github.com/casey/just) commands to simplify development workflows such as cleaning, installing, formatting, checking, building, documenting and running the project.
* **Comprehensive Documentation:** [pdoc](https://pdoc.dev/) generates API documentation, and Markdown files provide clear usage instructions.
* **GitHub Workflow Integration:** Continuous integration and deployment workflows are set up using [GitHub Actions](https://github.com/features/actions), automating testing, checks, and publishing.
* Profiling: Several standard profilers are included for developers to choose from. Two popular call-stack profilers are [pyinstrument](https://github.com/joerick/pyinstrument) and [pyspy](https://github.com/benfred/py-spy). [memray](https://github.com/bloomberg/memray) is included for memory profiling.
* Load testing with [Locust](https://locust.io/).

## Development

### Checks

This will run all checks on this cookiecutter repo (not just the project template) as specified in the `tasks/check.just` command: code quality, test coverage, unit tests, formatting, typing, and security.

```shell
uv run just check
```

### Type checking

## Quick Start

Expand All @@ -43,7 +59,7 @@ uv tool install cookiecutter
cookiecutter gh:irod973/python-project-template
```

You'll be prompted for the following variables:
You'll be prompted for the following variables.

- `user`: Your GitHub username.
- `name`: The name of your project.
Expand All @@ -53,6 +69,10 @@ You'll be prompted for the following variables:
- `version`: The initial version of your project.
- `description`: A brief description of your project.
- `python_version`: The Python version to use (e.g., 3.12).
- `include_fastapi`: Whether to include a sample FastAPI application.
- `include_metaflow`: Whether to include a sample Metaflow application.
- `include_torchvision`: Whether to include a sample Torchvision application.
- `include_package`: Whether to include a sample application for publishing a Python package.

2. **Initialize a git repository:**

Expand All @@ -72,7 +92,7 @@ git init
- `src/{{cookiecutter.package}}`: Your Python package source code.
- `tests/`: Unit tests for your package.
- `tasks/`: `just` commands for automation.
- `Dockerfile`: Configuration for building your Docker image.
- `docker/Dockerfile.python`: Configuration for building your Docker image.
- `docker-compose.yml`: Orchestration file for running your project.

4. **Start developing!**
Expand All @@ -89,16 +109,6 @@ Use the provided `just` commands to manage your development workflow:
- `uv run just package`: Build your Python package.
- `uv run just project`: Run the project in the CLI.

## Example Usage

### Building and Running Your Docker Image

```bash
invoke containers
```

This builds a Docker image based on your [`Dockerfile`](https://github.com/fmind/cookiecutter-mlops-package/blob/main/%7B%7Bcookiecutter.repository%7D%7D/Dockerfile) and runs it.

## License

The source material this is adapted from is licensed under the [MIT License](https://opensource.org/license/mit). See the [`LICENSE.txt`](https://github.com/fmind/cookiecutter-mlops-package/blob/main/LICENSE.txt) file for details.
4 changes: 3 additions & 1 deletion cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@
"package": "{{cookiecutter.repository.replace('-', '_')}}",
"license": "MIT",
"version": "0.1.0",
"description": "TODO",
"description": "TODO: Fill in description.",
"python_version": "3.12",
"include_fastapi": true,
"include_metaflow": true,
"include_package": true,
"include_torchvision": true,
"coverage_threshold": 80,
"__prompts__": {
"user": "GitHub User",
Expand All @@ -22,6 +23,7 @@
"include_fastapi": "Include FastAPI component? (y/n)",
"include_metaflow": "Include Metaflow component? (y/n)",
"include_package": "Include Python package component? (y/n)",
"include_torchvision": "Include TorchVision example component? (y/n)",
"coverage_threshold": "Minimum code coverage percentage (cov-fail-under)"
}
}
33 changes: 15 additions & 18 deletions hooks/post_gen_project.py
Original file line number Diff line number Diff line change
@@ -1,30 +1,27 @@
import shutil
"""Remove any unwanted optional components after project generation."""
import os
import shutil

def is_false(val):
return str(val).strip().lower() in ("false", "n", "no", "0")

def is_false(val: str) -> bool:
return val.strip().lower() in ("false", "n", "no", "0")

# Remove FastAPI component if not included
if is_false('{{cookiecutter.include_fastapi}}'):
shutil.rmtree('src/fastapi_app', ignore_errors=True)
if is_false("{{cookiecutter.include_fastapi}}"):
shutil.rmtree("src/fastapi_app", ignore_errors=True)

# Remove Metaflow component if not included
if is_false('{{cookiecutter.include_metaflow}}'):
shutil.rmtree('src/metaflow_app', ignore_errors=True)
if is_false("{{cookiecutter.include_metaflow}}"):
shutil.rmtree("src/metaflow_app", ignore_errors=True)

# Remove Python package component if not included
if is_false('{{cookiecutter.include_package}}'):
if is_false("{{cookiecutter.include_package}}"):
shutil.rmtree("src/{{cookiecutter.package}}", ignore_errors=True)
# Remove publish workflow if present
workflow_path = os.path.join('.github', 'workflows', 'publish.yml')
workflow_path = os.path.join(".github", "workflows", "publish.yml")
if os.path.exists(workflow_path):
os.remove(workflow_path)
# Remove publish badge from README if present
readme_path = os.path.join('.', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
lines = f.readlines()
with open(readme_path, 'w') as f:
for line in lines:
if 'publish.yml' not in line:
f.write(line)

# Remove torchvision app if not included in template options
if is_false("{{cookiecutter.include_torchvision}}"):
shutil.rmtree("src/torchvision_app", ignore_errors=True)
10 changes: 10 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,13 @@ update_changelog_on_bump = true
[tool.pytest.ini_options]
log_cli = true
log_cli_level = "INFO"

[tool.mypy]
pretty = true
python_version = "3.13"
check_untyped_defs = true
ignore_missing_imports = true
strict = false
disable_error_code = ["import-untyped"]
# See https://mypy.readthedocs.io/en/stable/config_file.html#untyped-definitions-and-calls
disallow_untyped_decorators = false
8 changes: 5 additions & 3 deletions tests/test_cookiecutter.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,12 @@ def test_project_generation(cookies: Cookies) -> None:
"license": "Apache-2.0", # Note: needs to be a "valid SPDX identifier"
"version": "1.0.0",
"description": "A test project.",
Copy link

Copilot AI Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The Python version was changed from 3.13 to 3.12 in tests, but this change is undocumented in the PR description. If this is intentional due to compatibility issues with the new dependencies (torch, etc.), consider adding a comment explaining why 3.13 is not used for testing.

Suggested change
"description": "A test project.",
"description": "A test project.",
# Python 3.12 is used instead of 3.13 due to compatibility issues with some dependencies (e.g., torch).

Copilot uses AI. Check for mistakes.
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignoring, this is simply standardizing on Python 3.12

"python_version": "3.13",
"python_version": "3.12",
"include_fastapi": "y",
"include_metaflow": "y",
"include_package": "y",
"coverage_threshold": "1"
"include_torchvision": "y",
"coverage_threshold": "1",
}
repository = context['name'].lower().replace(' ', '-')
package = repository.replace('-', '_')
Expand All @@ -58,8 +59,9 @@ def test_project_generation(cookies: Cookies) -> None:
"python_version": context["python_version"],
"include_fastapi": context["include_fastapi"],
"include_metaflow": context["include_metaflow"],
"include_torchvision": context["include_torchvision"],
"include_package": context["include_package"],
"coverage_threshold": context["coverage_threshold"]
"coverage_threshold": context["coverage_threshold"],
}
# - commands
shell = Subprocess(cwd=result.project_path)
Expand Down
25 changes: 0 additions & 25 deletions {{cookiecutter.repository}}/Dockerfile

This file was deleted.

40 changes: 30 additions & 10 deletions {{cookiecutter.repository}}/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# {{cookiecutter.name}}

[![check.yml](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/actions/workflows/check.yml/badge.svg)](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/actions/workflows/check.yml)
{% if cookiecutter.include_package %}
[![publish.yml](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/actions/workflows/publish.yml/badge.svg)](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/actions/workflows/publish.yml)
{% endif %}
[![Documentation](https://img.shields.io/badge/documentation-available-brightgreen.svg)](https://{{cookiecutter.user}}.github.io/{{cookiecutter.repository}}/)
[![License](https://img.shields.io/github/license/{{cookiecutter.user}}/{{cookiecutter.repository}})](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/blob/main/LICENCE.txt)
[![Release](https://img.shields.io/github/v/release/{{cookiecutter.user}}/{{cookiecutter.repository}})](https://github.com/{{cookiecutter.user}}/{{cookiecutter.repository}}/releases)
Expand All @@ -10,7 +12,7 @@

{{cookiecutter.description}}.

This README is generated from a cookiecutter template. Delete this comment and modify your README!
TODO: This README is generated from a cookiecutter template. Delete this comment and modify your README!

# Installation

Expand All @@ -21,21 +23,39 @@ uv run just install
```
# Usage

The provided template apps can be executed with the existing Docker templates. **Note:** Dependencies are not yet parametrized and need to be added using `uv add`
TODO: Fill in with your project's details.

{% if cookiecutter.include_torchvision %}
## Torchvision app

This template includes an example torchvision app and associated dependencies.

This has support for installing either CPU-only or CUDA 12.8 torch wheels via the `--extra {cpu, gpu}` uv sync argument.
```shell
# Invoke docker compose
uv run just docker-compose

# Or run with docker compose
docker compose up --build
uv run just docker-compose torchvision_app
```
{% endif %}
{% if cookiecutter.include_metaflow %}
## Metaflow app

# Or run with docker
# Note: specify platform if running on Apple M chip
docker build --platform linux/amd64 -t {{cookiecutter.repository}}-image -f Dockerfile .
docker run -it --platform linux/amd64 --name {{cookiecutter.repository}}-ctr -p 8000:8000 {{cookiecutter.repository}}-image
This template includes an example Metaflow app and associated dependencies.
```shell
# Invoke docker compose
uv run just docker-compose metaflow_app
```
{% endif %}
{% if cookiecutter.include_fastapi %}
## FastAPI

This template includes an example FastAPI app and associated dependencies.

Note that this runs `fastapi dev` which includes auto-reload by default. This should be switched to `fastapi run` in production.
```shell
# Invoke docker compose
uv run just docker-compose fastapi_app
```
{% endif %}
## Development Features

* **Streamlined Project Structure:** A well-defined directory layout for source code, tests, documentation, tasks, and Docker configurations.
Expand Down
62 changes: 49 additions & 13 deletions {{cookiecutter.repository}}/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,26 +1,62 @@
# https://docs.docker.com/compose/compose-file/

services:
{% if cookiecutter.include_fastapi %}
python:
build:
context: .
dockerfile: docker/Dockerfile.python
volumes:
- ./pyproject.toml:/app/pyproject.toml
- ./uv.lock:/app/uv.lock
{% if cookiecutter.include_fastapi %}
fastapi_app:
build: .
command: ["uvicorn", "example_app.main:app", "--host", "0.0.0.0", "--reload"]
build:
context: .
dockerfile: docker/Dockerfile.python
args:
UV_SYNC_OPTIONS: "--group fastapi"
command: ["fastapi", "dev", "fastapi_app/main.py", "--host", "0.0.0.0"]
ports:
- "8000:8000"
volumes:
- ./src:/app/src
{% endif %}
{% if cookiecutter.include_metaflow %}
- ./src/fastapi_app:/app/fastapi_app
- ./pyproject.toml:/app/pyproject.toml
- ./uv.lock:/app/uv.lock
{% endif %}
{% if cookiecutter.include_metaflow %}
metaflow_app:
build: .
build:
context: .
dockerfile: docker/Dockerfile.python
args:
UV_SYNC_OPTIONS: "--group metaflow"
command: ["python", "metaflow_app/spin_prototype.py"]
volumes:
- ./src:/app/src
{% endif %}
{% if cookiecutter.include_package %}
- ./src/metaflow_app:/app/metaflow_app
- ./pyproject.toml:/app/pyproject.toml
- ./uv.lock:/app/uv.lock
{% endif %}
{% if cookiecutter.include_package %}
python_package:
build: .
build:
context: .
dockerfile: docker/Dockerfile.python
command: ["python", "-m", "{{cookiecutter.package}}"]
volumes:
- ./src:/app/src
{% endif %}
- ./src/{{cookiecutter.package}}:/app/{{cookiecutter.package}}
- ./pyproject.toml:/app/pyproject.toml
- ./uv.lock:/app/uv.lock
{% endif %}
{% if cookiecutter.include_torchvision %}
torchvision_app:
build:
context: .
dockerfile: docker/Dockerfile.python
args:
UV_SYNC_OPTIONS: "--extra cpu"
command: ["python", "torchvision_app/torchvision_example.py"]
volumes:
- ./src/torchvision_app:/app/torchvision_app
- ./pyproject.toml:/app/pyproject.toml
- ./uv.lock:/app/uv.lock
{% endif %}
Loading