test links and coverage missing in docu

[AlexanderLanin]

The docs builds should run pytest and not consumer tests:

docs-as-code/.github/workflows/test_and_docs.yml

Line 42 in 7a600ff

uses: ./.github/workflows/consumer_test.yml

consumer tests should run in parallel, totally independent

[MaximilianSoerenPollak]

The fix for this should just be a small change in the yaml right?
Changing the tests from consumer to the unit testing via pytest?

If so I can take this.

[MaximilianSoerenPollak]

So we have three solutions to this. I'm wondering which path is the one I should choose.

1. Make a new workflow

Create a new workflow that lives in cicd repo that has the bazel test as a pre-execution step before it builds the documentation

2. Adapt docs workflow

Adapt the docs workflow to allow for a pre-run step, or add it as optional that the tests run before it

3. Change docs-as-code workflow to not be based on cicd

Change the workflow here to ensure correct order, but then it would not inherit from the cicd workflows anymore.

I think 1 or 2nd make the most sense to keep it consistent in the score ecosystem, but let me know your thoughts please.

[MaximilianSoerenPollak]

Here is how the fix via number 1 / 2 could look like. This currently works (see link at the bottom)

In the workflow we have to add two more inputs

      bazel-venv-target:
        description: "The target to generate the virtualenv if needed (e.g., //:ide_support)"
        required: false
        type: string
      bazel-test-target:
        description: "The Bazel target to test (e.g., //src/...)"
        required: false
        type: string
        default: "//src/..."

Which we then can use before the doc-build step like this:

      - name: Run Venv
        if: ${{ inputs.bazel-venv-target != '' }}
        run: |
          bazel --output_base="${{ steps.paths.outputs.output_base }}" run \
            --repo_contents_cache="${{ steps.paths.outputs.repo_contents }}" \
            --repository_cache="${{ steps.paths.outputs.repo_cache }}" \
            --disk_cache="${{ steps.paths.outputs.disk_cache }}" \
              ${{ inputs.bazel-venv-target }}

      - name: Run Unit-Tests
        run: |
          bazel --output_base="${{ steps.paths.outputs.output_base }}" test \
            --repo_contents_cache="${{ steps.paths.outputs.repo_contents }}" \
            --repository_cache="${{ steps.paths.outputs.repo_cache }}" \
            --disk_cache="${{ steps.paths.outputs.disk_cache }}" \
              ${{ inputs.bazel-test-target }}

This produces the desired effect, the bazel run //:docs now has access to a bazel-testlog folder that is filled and can therefore create the testcase needs and fill out the statistics page.
Rendered page see here: https://maximiliansoerenpollak.github.io/docs-as-code/main/internals/requirements/test_overview.html

[AlexanderLanin]

The idea was to keep testing completely up to the user. As it could be arbitrarily complex. Matrix runs, etc.

[MaximilianSoerenPollak]

The idea was to keep testing completely up to the user. As it could be arbitrarily complex. Matrix runs, etc.

Ah so, it would be better to make it work via the test-reports artifact in order to allow arbitrary testing setups.
Alright.