📝 Update pytest section

Author: veit

docs/test/pytest/command-line-options.rst

@@ -0,0 +1,212 @@
+Command line options
+====================
+
+In :ref:`dynamic-fixture-scope`, we have already seen how the fixture scope can
+be changed using a command line option. Now let’s take a closer look at the
+command line options.
+
+Passing different values to a test function
+-------------------------------------------
+
+Suppose you want to write a test that depends on a command line option. You can
+achieve this using the following pattern:
+
+.. code-block:: python
+   :caption: test_example.py
+
+   def test_db(items_db, db_path, cmdopt):
+       if cmdopt == "json":
+           print("Save as JSON file")
+       elif cmdopt == "sqlite":
+           print("Save in a SQLite database")
+       assert items_db.path() == db_path
+
+For this to work, the command line option must be added and ``cmdopt`` must be
+provided via a fixture function:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import pytest
+
+
+   def pytest_addoption(parser):
+       parser.addoption(
+           "--cmdopt",
+           action="store",
+           default="json",
+           help="Store data as JSON file or in a SQLite database",
+       )
+
+
+   @pytest.fixture
+   def cmdopt(request):
+       return request.config.getoption("--cmdopt")
+
+You can then call up your tests, for example, with:
+
+.. code-block:: console
+
+   $ pytest --sqlite
+
+In addition, you can add a simple validation of the input by listing the
+options:
+
+.. code-block:: python
+   :caption: conftest.py
+   :emphasize-lines: 7
+
+   def pytest_addoption(parser):
+       parser.addoption(
+           "--cmdopt",
+           action="store",
+           default="json",
+           help="Store data as JSON file or in a SQLite database",
+           choices=("json", "sqlite"),
+       )
+
+This is how we receive feedback on an incorrect argument:
+
+.. code-block:: console
+
+   $ pytest --postgresql
+   ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
+   pytest: error: argument --cmdopt: invalid choice: 'postgresql' (choose from json, sqlite)
+
+If you want to provide more detailed error messages, you can use the ``type``
+parameter and raise ``pytest.UsageError``:
+
+.. code-block:: python
+   :caption: conftest.py
+   :emphasize-lines: -6, 15
+
+   def type_checker(value):
+       msg = "cmdopt must specify json or sqlite"
+       if not value.startswith("json" or "sqlite"):
+           raise pytest.UsageError(msg)
+
+       return value
+
+
+   def pytest_addoption(parser):
+       parser.addoption(
+           "--cmdopt",
+           action="store",
+           default="json",
+           help="Store data as JSON file or in a SQLite database",
+           type=type_checker,
+       )
+
+However, command line options often need to be processed outside of the test and
+more complex objects need to be passed.
+
+Adding command line options dynamically
+---------------------------------------
+
+With :ref:`addopts`, you can add static command line options to your project.
+However, you can also change the command line arguments dynamically before they
+are processed:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import sys
+
+
+   def pytest_load_initial_conftests(args):
+       if "xdist" in sys.modules:
+           import multiprocessing
+
+           num = max(multiprocessing.cpu_count() / 2, 1)
+           args[:] = ["-n", str(num)] + args
+
+If you have installed the :ref:`xdist-plugin` plugin, test runs will always be
+performed with a number of subprocesses close to your CPU.
+
+Command line option for skipping tests
+--------------------------------------
+
+Below, we add a :file:`conftest.py` file with a command line option
+``--runslow`` to control the skipping of tests marked with ``pytest.mark.slow``:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import pytest
+
+
+   def pytest_addoption(parser):
+       parser.addoption(
+           "--runslow", action="store_true", default=False, help="run slow tests"
+       )
+
+
+   def pytest_collection_modifyitems(config, items):
+       if config.getoption("--runslow"):
+           # If --runslow is specified on the CLI, slow tests are not skipped.
+           return
+       skip_slow = pytest.mark.skip(reason="need --runslow option to run")
+       for item in items:
+           if "slow" in item.keywords:
+               item.add_marker(skip_slow)
+
+If we now write a test with the ``@pytest.mark.slow`` decorator, a skipped
+‘slow’ test will be displayed when pytest is called:
+
+.. code-block:: pytest
+
+   $ uv run pytest
+   ============================= test session starts ==============================
+   ...
+   test_example.py s.                                                         [100%]
+
+   =========================== short test summary info ============================
+   SKIPPED [1] test_example.py:8: need --runslow option to run
+   ========================= 1 passed, 1 skipped in 0.05s =========================
+
+Extend test report header
+-------------------------
+
+Additional information can be easily provided in a ``pytest -v`` run:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import sys
+
+
+   def pytest_report_header(config):
+       gil = sys._is_gil_enabled()
+       return f"Is GIL enabled? {gil}"
+
+
+.. code-block:: pytest
+   :emphasize-lines: 5
+
+   $ uv run pytest -v
+   ============================= test session starts ==============================
+   platform darwin -- Python 3.14.0b4, pytest-8.4.1, pluggy-1.6.0
+   cachedir: .pytest_cache
+   Is GIL enabled? False
+   rootdir: /Users/veit/sandbox/items
+   configfile: pyproject.toml
+   plugins: anyio-4.9.0, Faker-37.4.0, cov-6.2.1
+   ...
+   ============================== 2 passed in 0.04s ===============================
+
+Determine test duration
+-----------------------
+
+If you have a large test suite that runs slowly, you will probably want to use
+``-vv --durations`` to find out which tests are the slowest.
+
+.. code-block:: pytest
+
+   $ uv run pytest -vv --durations=3
+   ============================= test session starts ==============================
+   ...
+   ============================= slowest 3 durations ==============================
+   0.02s setup    tests/api/test_add.py::test_add_from_empty
+   0.00s call     tests/cli/test_help.py::test_help[add]
+   0.00s call     tests/cli/test_help.py::test_help[update]
+   ============================== 83 passed in 0.17s ==============================

docs/test/pytest/config.rst

@@ -76,7 +76,9 @@ marker per line is permitted.
 This example is a simple :file:`pytest.ini` file that I use in almost all my
 projects. Let’s briefly go through the individual lines:
 
-``addopts =``
+.. _addopts:
+
+``addopts``
     allows you to specify the pytest options that we always want to execute in
     this project.
 ``--strict-markers``

docs/test/pytest/fixtures.rst

@@ -557,6 +557,8 @@ We have seen how different fixture scopes work and how different scopes can be
 used in different fixtures. However, you may need to define a scope at runtime.
 This is possible with dynamic scoping.
 
+.. _dynamic-fixture-scope:
+
 Set fixture scope dynamically
 -----------------------------
 
@@ -660,6 +662,9 @@ scope:
 
 The database is now set up before each test function and then dismantled again.
 
+.. seealso::
+   * :doc:`command-line-options`
+
 ``autouse`` for fixtures that are always used
 ---------------------------------------------
 

docs/test/pytest/index.rst

@@ -60,5 +60,6 @@ You can install pytest in `virtual environments <venv>` with:
    markers
    plugins
    config
+   command-line-options
    debug
    coverage

docs/test/pytest/markers.rst

@@ -873,6 +873,147 @@ Let’s run the tests now to make sure everything is working properly:
 
       ============================== 3 passed in 0.09s ===============================
 
+Generating markers
+------------------
+
+Suppose you have a test suite that marks tests for specific platforms, namely
+``pytest.mark.darwin``, ``pytest.mark.win32``, and so on, and you also have
+tests that run on all platforms and do not have a specific marker. If you are
+looking for a way to run only the tests for your specific platform, you can use
+the following:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import sys
+
+   import pytest
+
+   ALL = {"win32", "darwin", "linux"}
+
+
+   def pytest_setup(item):
+       supported_platforms = ALL.intersection(
+           mark.name for mark in item.iter_markers()
+       )
+       pf = sys.platform
+       if supported_platforms and pf not in supported_platforms:
+           pytest.skip(f"cannot run on platform {pf}")
+
+This means that tests are skipped if they have been specified for another
+platform. Now let's create a small test file to show what this looks like:
+
+.. code-block:: python
+   :caption: test_platform.py
+
+   import pytest
+
+
+   def test_foo_everywhere():
+       pass
+
+
+   @pytest.mark.win32
+   def test_foo_on_win32():
+       pass
+
+
+   @pytest.mark.darwin
+   def test_foo_on_darwin():
+       pass
+
+
+   @pytest.mark.linux
+   def test_foo_on_linux():
+       pass
+
+Now we can run pytest and see the reasons for the skipped tests:
+
+.. code-block:: pytest
+
+   $ uv run pytest -rs tests/test_platform.py
+   ============================= test session starts ==============================
+   platform darwin -- Python 3.14.0b4, pytest-8.4.1, pluggy-1.6.0
+   ...
+   collected 4 items
+
+   tests/test_platform.py ..ss                                              [100%]
+
+   =========================== short test summary info ============================
+   SKIPPED [2] tests/conftest.py:20: cannot run on platform darwin
+   ========================= 2 passed, 2 skipped in 0.03s =========================
+
+or more specifically:
+
+.. code-block:: pytest
+
+   $ uv run pytest -m darwin -rs tests/test_platform.py
+   ============================= test session starts ==============================
+   platform darwin -- Python 3.14.0b4, pytest-8.4.1, pluggy-1.6.0
+   ...
+   collected 4 items / 3 deselected / 1 selected
+
+   tests/test_platform.py .                                                 [100%]
+
+   ======================= 1 passed, 3 deselected in 0.02s ========================
+
+Markers based on test names
+---------------------------
+
+Alternatively, markers can also be specified using the names of the test
+functions by implementing a hook that automatically defines markers:
+
+.. code-block:: python
+   :caption: test_platform.py
+
+   def test_foo_everywhere():
+       pass
+
+
+   def test_foo_on_win32():
+       pass
+
+
+   def test_foo_on_darwin():
+       pass
+
+
+   def test_foo_on_linux():
+       pass
+
+Now we dynamically define three markers in :file:`conftest.py` in
+`pytest_collection_modifyitems
+<https://docs.pytest.org/en/latest/reference/reference.html#pytest.hookspec.pytest_collection_modifyitems>`_:
+
+.. code-block:: python
+   :caption: conftest.py
+
+   import pytest
+
+
+   def pytest_collection_modifyitems(items):
+       for item in items:
+           if "win32" in item.nodeid:
+               item.add_marker(pytest.mark.win32)
+           elif "darwin" in item.nodeid:
+               item.add_marker(pytest.mark.darwin)
+           elif "linux" in item.nodeid:
+               item.add_marker(pytest.mark.linux)
+
+Now we can use the ``-m`` option to select a set:
+
+.. code-block:: pytest
+
+   $ uv run pytest -m darwin
+   ============================= test session starts ==============================
+   platform darwin -- Python 3.14.0, pytest-9.0.1, pluggy-1.6.0
+   ...
+   collected 4 items / 3 deselected / 1 selected
+
+   tests/test_platform.py .                                                 [100%]
+
+   ======================= 1 passed, 3 deselected in 0.00s ========================
+
 List markers
 ------------