Skip to content

Commit 4eb49cd

Browse files
committed
docs: improve contributor testing guide and dockerised cookstyle
1 parent e109384 commit 4eb49cd

File tree

4 files changed

+180
-52
lines changed

4 files changed

+180
-52
lines changed

CONTRIBUTING.md

Lines changed: 153 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,77 +1,194 @@
1-
# Contribution Guidelines
1+
# Contributing
2+
3+
This repository contains the Chef code used to configure OpenStreetMap's infrastructure.
4+
This guide walks you through running the same checks as CI locally before opening a pull request.
5+
6+
## What you should run before a pull request
7+
8+
At a minimum, please run:
9+
10+
- **Cookstyle** (lint/style): `bundle exec cookstyle`
11+
- **Test Kitchen** for the cookbooks/roles you changed or added (integration): `bundle exec kitchen test <suite>-<platform>`
12+
13+
CI runs a large matrix; contributors are not expected to run every Kitchen instance locally.
14+
15+
## Quick start (if you already have Ruby + Docker)
16+
17+
From the repository root:
18+
19+
```bash
20+
bundle install
21+
bundle exec cookstyle
22+
bundle exec kitchen list
23+
bundle exec kitchen test dns-ubuntu-2204
24+
```
25+
26+
Replace `dns-ubuntu-2204` with a Kitchen instance relevant to the cookbooks/roles you changed.
227

328
## Workflow
429

5-
We operate the "Fork & Pull" model explained at
30+
We use the "fork and pull request" workflow:
31+
32+
- Fork the repository
33+
- Create a topic branch
34+
- Open a pull request back to `openstreetmap/chef`
35+
36+
GitHub has an overview at https://help.github.com/articles/using-pull-requests
37+
38+
## Local test setup
39+
40+
### Prerequisites
641

7-
https://help.github.com/articles/using-pull-requests
42+
- **Docker**
43+
- macOS: Docker Desktop
44+
- Linux: Docker Engine
45+
- Windows: WSL2 + Docker may work, but is not used by the operations team and may not work in your environment (see below).
46+
- **Ruby** (see [.ruby-version](.ruby-version)) and **Bundler**
47+
- `git`
848

9-
You should fork the project into your own repo, create a topic branch
10-
there and then make one or more pull requests back to the openstreetmap/chef repository.
11-
Your pull requests will then be reviewed and discussed.
49+
We recommend using `rbenv` to manage Ruby versions. On macOS, Homebrew is often the easiest way to install it.
1250

13-
## Running the Infrastructure Tests locally
51+
### Recommended setup (rbenv + Bundler)
1452

15-
- **[Cookstyle](https://docs.chef.io/workstation/cookstyle/)** is used for linting, ensuring that our Chef recipes follow style guidelines and best practices.
16-
- **[Test Kitchen](https://kitchen.ci/)** combined with **InSpec** and [Dokken](https://github.com/test-kitchen/kitchen-dokken) is used to verify the functionality of our Chef code, ensuring it behaves as expected.
53+
The steps below work on macOS and Linux. The macOS-only steps are called out explicitly.
1754

18-
The following guidelines are to help set up and run these checks locally:
55+
1) Install Docker
1956

20-
#### **1. Install Docker**
21-
- Visit [Docker's official site](https://www.docker.com/products/docker-desktop) to download and install Docker.
57+
- macOS: Docker Desktop - https://www.docker.com/products/docker-desktop
58+
- Linux: Docker Engine - https://docs.docker.com/engine/install/
2259

23-
#### **2. Install Homebrew (Apple MacOS only)**
24-
- Install Homebrew by following the instructions [here](https://brew.sh/).
60+
2) macOS only: install Homebrew (if you do not already have it)
2561

26-
#### **3. Install rbenv (recommended)**
27-
- Install rbenv by following the instructions [here](https://github.com/rbenv/rbenv#installation).
62+
- https://brew.sh/
2863

29-
rbenv is a ruby version manager. rbenv allows projects to use a different version of ruby than the version of install with your operating system.
64+
3) Install `rbenv`
3065

31-
> *Note on rbenv: While we recommend using rbenv for managing Ruby versions, it's not strictly necessary. If you have Ruby already installed feel free to use that. If you're not using rbenv, simply omit the `rbenv exec` prefix from the commands below.*
66+
- https://github.com/rbenv/rbenv#installation
3267

33-
#### **4. Increase File Limit (Important for MacOS)**
68+
`rbenv` is a Ruby version manager. It lets you use the Ruby version this repository expects without changing your system Ruby.
3469

35-
To avoid errors when running tests on MacOS, you might need to increase the number of files your system can open at once. Here's how:
70+
If you do not use `rbenv`, omit the `rbenv exec` prefix in the commands below.
71+
72+
4) macOS only: increase the open file limit (often needed)
73+
74+
If you see errors like `Too many open files (Errno::EMFILE)` while running tests:
3675

37-
1. Run the command:
3876
```bash
3977
ulimit -n 1024
4078
```
41-
2. To make the change permanent, add the above line to either `~/.zshrc` or `~/.bash_profile`, depending on your shell.
4279

43-
**Note:** MacOS has a low default limit of just 256 open files. If you exceed this while testing, you'll see an error like: `Too many open files - getcwd (Errno::EMFILE)`. This step helps prevent that.
80+
To make it permanent, add the line above to your shell profile (for example `~/.zshrc`).
81+
82+
5) Install the Ruby version for this repo
83+
84+
From the repository root:
4485

45-
#### **5. Install Required Ruby Version (recommended)**
46-
Navigate to the git checkout of the OpenStreetMap chef repo and run:
4786
```bash
48-
rbenv install
87+
rbenv install --skip-existing
4988
```
50-
This will install the recommended version of ruby for running the tests. The recommended version of ruby is defined in the [.ruby-version](.ruby-version) file.
5189

52-
#### **6. Install Dependencies with Bundler**
90+
6) Install Ruby gems
91+
5392
```bash
5493
rbenv exec gem install bundler
5594
rbenv exec bundle install
5695
```
57-
This will install the [bundler](https://bundler.io/), the ruby gem packages manager, and then uses `bundle` to install the required gem packages for the tests.
5896

59-
#### **7. Run Cookstyle for Linting and Style Checks**
97+
## Running the checks
98+
99+
### 1) Cookstyle (lint and style)
100+
101+
Cookstyle checks Chef/Ruby style and common issues.
102+
60103
```bash
61104
rbenv exec bundle exec cookstyle
62105
```
63-
This will run [cookstyle](https://docs.chef.io/workstation/cookstyle/) a linting tool which reports on any linting issues.
64106

65-
> *Automatically run cookstyle lint: We have a sample [git pre-commit hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) in the `hooks/pre-commit` file which can be copied to the local checkout of this repo to the file `.git/hooks/pre-commit` to ensure the lint passes when running a git commit.*
107+
If you prefer not to install Ruby locally, you can run Cookstyle via Docker:
108+
109+
```bash
110+
docker compose run --rm cookstyle
111+
```
112+
113+
If you only want to lint the files you are committing, there is a sample pre-commit hook at [hooks/pre-commit](hooks/pre-commit).
114+
Copy it to `.git/hooks/pre-commit` and make it executable.
115+
116+
### 2) Test Kitchen (integration tests)
117+
118+
We use Test Kitchen with InSpec and the Dokken driver to converge recipes and verify behaviour inside containers.
119+
120+
1) List available instances:
66121

67-
#### **8. List Available Tests**
68122
```bash
69123
rbenv exec bundle exec kitchen list
70124
```
71-
This lists the [Test Kitchen](https://kitchen.ci/) tests which are available. The list of tests is generated from the definitions in the [.kitchen.yml](.kitchen.yml) file. The individual tests are written in [InSpec](https://docs.chef.io/inspec/) and are stored in the `test/integration/` directory.
72125

73-
#### **9. Run an Example Test**
126+
Kitchen instances are named `<suite>-<platform>`.
127+
The suite is usually the cookbook/role under test (see [.kitchen.yml](.kitchen.yml)), and the platform is the OS image.
128+
129+
The available Kitchen suites and platforms are defined in [.kitchen.yml](.kitchen.yml).
130+
The InSpec integration tests themselves live under [test/integration/](test/integration/).
131+
If your change adds a new cookbook, please add a corresponding suite to [.kitchen.yml](.kitchen.yml) and include appropriate integration tests.
132+
133+
2) Run a specific instance:
134+
74135
```bash
75136
rbenv exec bundle exec kitchen test dns-ubuntu-2204
76137
```
77-
This runs the [Test Kitchen](https://kitchen.ci/) [InSpec](https://docs.chef.io/inspec/) `dns` tests using the `Ubuntu 22.04` platform. The tests are run inside a Docker container using the Test Kitchen [Dokken driver](https://github.com/test-kitchen/kitchen-dokken).
138+
139+
`kitchen test` will create the container, converge, verify, then destroy it.
140+
141+
For faster iteration, you can split the steps:
142+
143+
```bash
144+
rbenv exec bundle exec kitchen converge dns-ubuntu-2204
145+
rbenv exec bundle exec kitchen verify dns-ubuntu-2204
146+
rbenv exec bundle exec kitchen destroy dns-ubuntu-2204
147+
```
148+
149+
#### Debugging a failed converge
150+
151+
If `kitchen converge` fails, it can be useful to inspect the running test container.
152+
153+
1) Find the container name by running `docker ps` (example output):
154+
155+
```text
156+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
157+
3b83c7f96575 9a7e6edc5b-dns-ubuntu-2204:latest "/bin/systemd" 14 minutes ago Up 14 minutes 9a7e6edc5b-dns-ubuntu-2204
158+
```
159+
160+
2) Enter the container:
161+
162+
```bash
163+
docker exec -it 9a7e6edc5b-dns-ubuntu-2204 bash -l
164+
```
165+
166+
Once you are finished debugging, remember to clean up the instance:
167+
168+
```bash
169+
rbenv exec bundle exec kitchen destroy dns-ubuntu-2204
170+
```
171+
172+
## Windows (WSL2)
173+
174+
It may be possible to run the toolchain under WSL2 (for example Ubuntu on WSL2) with Docker.
175+
However, the operations team do not use WSL2, and we cannot guarantee it will work or be able to troubleshoot WSL2-specific issues.
176+
177+
If you try this route, aim to match the Linux instructions as closely as possible and ensure Docker is reachable from within WSL2.
178+
179+
## Troubleshooting
180+
181+
- **Docker is not running**: start Docker Desktop (macOS) or the Docker service (Linux).
182+
- **Open file limit errors on macOS**: increase `ulimit -n` as described above.
183+
- **Kitchen networking issues**: if Docker has IPv6 disabled, Dokken may fail to create its network; enabling IPv6 in Docker's settings can help.
184+
185+
## Pull request checklist
186+
187+
- Run `bundle exec cookstyle` locally
188+
- Run relevant `kitchen test <suite>-<platform>` instances for the cookbooks/roles you changed or added
189+
- Keep changes focused and include context in the pull request description
190+
191+
## Need help?
192+
193+
If you get stuck, the operations team are available in `#osmf-operations` on `irc.oftc.net`.
194+
You can access `#osmf-operations` via https://irc.openstreetmap.org/ or via the Matrix IRC bridge in [#\_oftc_#osmf-operations](https://matrix.to/#/#_oftc_#osmf-operations:matrix.org).

Dockerfile

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,24 @@
1-
# Basic Dockerfile to run cookstyle linting
2-
# run: docker build -t chef-test .
3-
FROM ruby:3.1-alpine as build
1+
# Docker image used for running repo tooling (e.g. Cookstyle) in local containers.
2+
FROM ruby:3.3-trixie AS build
43

54
# Add Gem build requirements
6-
RUN apk add --no-cache build-base
5+
RUN apt-get update \
6+
&& apt-get install -y --no-install-recommends \
7+
build-essential \
8+
pkg-config \
9+
libxml2-dev \
10+
libxslt1-dev \
11+
zlib1g-dev \
12+
&& rm -rf /var/lib/apt/lists/*
713

814
# Create app directory
9-
WORKDIR /app
15+
WORKDIR /usr/src/app
1016

1117
# Add Gemfile and Gemfile.lock
1218
ADD Gemfile* ./
1319

1420
# Install Gems
15-
RUN gem install bundler \
16-
&& bundle config build.nokogiri --use-system-libraries \
17-
&& bundle config --global jobs $(nproc) \
21+
RUN gem install bundler -v 2.6.9 \
22+
&& bundle config set build.nokogiri --use-system-libraries \
23+
&& bundle config set --global jobs $(nproc) \
1824
&& bundle install
19-
20-
# Add repo
21-
ADD . .
22-
23-
# Run linting
24-
RUN bundle exec cookstyle -f fuubar

README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,12 @@ We use the 'Organization Repository' approach, where we have all our cookbooks i
3737

3838
# Contributing
3939

40-
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for more details. The guide also includes details on how to run the tests locally.
40+
Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for more details, including how to run the tests locally.
41+
If you need help, the operations team are available in [#osmf-operations on irc.oftc.net](https://irc.openstreetmap.org/) or via the Matrix IRC bridge in [#\_oftc_#osmf-operations](https://matrix.to/#/#_oftc_#osmf-operations:matrix.org).
4142

4243
# Contact Us
4344

4445
* Mastodon: [@osm_tech](https://en.osm.town/@osm_tech)
4546
* IRC: [#osm-dev on irc.oftc.net](https://irc.openstreetmap.org/) or [#osmf-operations on irc.oftc.net](https://irc.openstreetmap.org/)
46-
* Matrix: [#\_oftc_#osmf-operations](https://matrix.to/#/#_oftc_#osmf-operations:matrix.org)
47+
* Matrix (IRC bridge): [#\_oftc_#osmf-operations](https://matrix.to/#/#_oftc_#osmf-operations:matrix.org)
4748
* Email: [operations@osmfoundation.org](mailto:operations@osmfoundation.org)

docker-compose.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
services:
2+
cookstyle:
3+
build:
4+
context: .
5+
dockerfile: Dockerfile
6+
image: osm-chef-tools
7+
working_dir: /workspace
8+
volumes:
9+
- .:/workspace:ro
10+
command: bundle exec cookstyle -f fuubar

0 commit comments

Comments
 (0)