Refactor environment variable documentation generation to use sphinx directives

[stevenbal]

Question from @sergei-maertens via slack:

https://odrc.readthedocs.io/en/latest/installation/config.html -> this is auto-generated via the management command from OAF. is there a way to exclude some settings? e.g. we don't use django.contrib.sites so I don't want SITE_ID to be mentioned. We're also not supporting outgoing email for now

We could add something like --exclude-var to the management command (https://github.com/maykinmedia/open-api-framework/blob/main/open_api_framework/management/commands/generate_envvar_docs.py#L32-L36) to exclude individual variables/settings

@sergei-maertens would this work for you?

TODO:

• Add maykin-common to the dependencies of open-api-framework
• Move the config util from open-api-framework to maykin-common
• Add sphinx directives for the following (similar sphinx directives are part of django-setup-configuration, so the setup there could be used as inspiration
  • :config_group:`param_name` to show the docs for a single envvar (should be possible to override the default help text in case a custom default is used for example
  • :config_group:`group_name` to show the envvar docs for a specific group (should have :members: and :exclude: options that function as allow and denylists for specific envvars
  • :config_all_params: to display all envvar groups (similar to https://github.com/maykinmedia/django-setup-configuration/blob/main/django_setup_configuration/documentation/setup_config_usage.py)
  • :config_usage: to describe how to specify envvars
• Add documentation on how to use these sphinx directives
• Replace the existing documentation in the components to use the new directive(s)
  • Open Zaak
  • Open Notificaties Feature/oaf 83 refactor envvar docs open-zaak/open-notificaties#324
  • Objects
  • Open Klant Feature/oaf 83 refactor envvar docs open-klant#498
  • Referentielijsten
  • Open Product
  • Open Werk
  • Open Inkomen
  • Open Inburgering

[sergei-maertens]

I suppose, though it's not ideal since whenever OAF is updated in a project, I then have to check which settings were added/changed and update my shell scripts/commands accordingly.

Personally I'd much rather have a custom Sphinx directive akin to sphinx.ext.autodoc so that I can write my documentation based on an allow list:

Available configuration parameters
==================================

.. entire group with all settings within group
:config_group:`Required settings`
    :members:

.. use a group, but apply allowlist to only document specified members
:config_group:`Cross Origin resource sharing`
    :members: CORS_ALLOW_ALL_ORIGINS,CORS_ALLOWED_ORIGINS

.. use a group, but exclude some members (e.g. it's available but considered dangerous -> treat as private API)
:config_group:`Cross Origin resource sharing`
    :exclude: CORS_ALLOW_ALL_ORIGINS

.. document individual parameters, allowlist basis
Special snowflake parameters
----------------------------

.. use documentation from `config` helper
:config_param:`DISABLE_2FA`

.. override baked in documentation
:config_param:`DISABLE_2FA`: Custom documentation snippet to override whatever is specified in `config`.
    .. ability to override the default, in case it was overridden (e.g. via `os.environ.setdefault("...", ...)`)
    :default: 1  

This closely matches my thoughts about django-setup-configuration setup/docs too, so tagging @pi-sigma for completeness sake.

[stevenbal]

Estimate to refactor to use a sphinx directive: 4 days

django-setup-configuration also has a sphinx directive for documentation, so maybe we could reuse parts of that

[sergei-maertens]

I'd also love if this is something that we can do in maykin-common instead of open-api-framework, because the utility to extract settings from envvars and have documentation is of course more broadly useful than just in API projects.

[stevenbal]

@sergei-maertens agreed, let's use this as an opportunity to move it to maykin-common