CMP-3788: Enhance TestTolerations to cover 33610 logic [WIP]

[Anna-Koudelkova]

DO NOT MERGE BEFORE #994

Test case uses function GetScanExitCodeAndErrorMsg() that gets introduced in PR #994 . Without the function it would not work.

What have changed

• Add second part of the testcase that would create a compliancescan on all worker nodes with following setting:

Scan Tolerations:
    Effect:             NoSchedule
    Key:                 co-e2e
    Operator:       Equal
    Value:              val

• Add check for exit code from configmap == 0, as the result of the scan should be compliant.
• Add check that the scan pod is generated for all the worker nodes for the second part of the test.

---

make e2e-parallel E2E_CONTENT_IMAGE_PATH="quay.io/rh-ee-akoudelk/content_repo:testcontent" E2E_BROKEN_CONTENT_IMAGE_PATH="ghcr.io/complianceascode/test-broken-content-ocp" E2E_GO_TEST_FLAGS="-v -timeout 60m -run TestScanProducesRemediations" was used on OCP 4.20 cluster to trigger this test case and it has passed when implementing also the changes in PR #960 and PR #994

=== RUN   TestTolerations
2025/12/03 21:02:11 tainting node: ip-10-0-13-121.us-east-2.compute.internal
2025/12/03 21:02:17 waiting until suite test-tolerations reaches target status 'DONE'. Current status: LAUNCHING
2025/12/03 21:02:22 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:27 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:32 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:37 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:42 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:47 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:52 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:02:57 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:03:02 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:03:07 waiting until suite test-tolerations reaches target status 'DONE'. Current status: RUNNING
2025/12/03 21:03:12 waiting until suite test-tolerations reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 21:03:17 waiting until suite test-tolerations reaches target status 'DONE'. Current status: AGGREGATING
2025/12/03 21:03:27 ComplianceScan ready (DONE)
2025/12/03 21:03:27 All scans in ComplianceSuite have finished (test-tolerations)
2025/12/03 21:03:32 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:03:37 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:03:42 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:03:47 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:03:52 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:03:57 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:04:02 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:04:07 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:04:12 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:04:17 Waiting for run of test-tolerations-equals compliancescan (RUNNING)
2025/12/03 21:04:22 Waiting for run of test-tolerations-equals compliancescan (AGGREGATING)
2025/12/03 21:04:27 Waiting for run of test-tolerations-equals compliancescan (AGGREGATING)
2025/12/03 21:04:32 ComplianceScan ready (DONE)
2025/12/03 21:04:39 removing taint from node: ip-10-0-13-121.us-east-2.compute.internal
--- PASS: TestTolerations (148.64s)
PASS

[openshift-ci-robot]

@Anna-Koudelkova: This pull request references CMP-3788 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

Details

In response to this:

DO NOT MERGE BEFORE #994

Test case uses function GetScanExitCodeAndErrorMsg() that gets introduced in PR #994 . Without the function it would not work.

What have changed

• Add second part of the testcase that would create a compliancescan on all worker nodes with following setting:

Scan Tolerations:
   Effect:             NoSchedule
   Key:                 co-e2e
   Operator:       Equal
   Value:              val

• Add check for exit code from configmap == 0, as the result of the scan should be compliant.
• Add check that the scan pod is generated for all the worker nodes for the second part of the test.

TO DO: Need to still verify this solution works on a cluster if both PR #994 and PR #960 are implemented

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

Hi @Anna-Koudelkova. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[github-actions]

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1014-08b224fe5ab0c1ee32f921bc6a222e82ce0674a8

[Anna-Koudelkova]

/ok-to-test

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Anna-Koudelkova

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [Anna-Koudelkova]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[github-actions]

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1014-3e374c83af21ce148135ec2c479c55a4226082ef

[openshift-ci]

@Anna-Koudelkova: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-rosa	3e374c8	link	true	/test e2e-rosa
ci/prow/e2e-aws-serial	3e374c8	link	true	/test e2e-aws-serial
ci/prow/e2e-aws-serial-arm	3e374c8	link	true	/test e2e-aws-serial-arm

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[xiaojiey]

The existing test is for a compliancesuite on the taited nodes. Newly added function is a compliancescan for all worker nodes. Is my understanding correct? If so, I think no need to add new check points.

[Anna-Koudelkova]

yes, I kept it on all the worker nodes so that I can verify that the number of scanner pods created equals number of worker nodes, like it was in OCP-33610 https://github.com/openshift/openshift-tests-private/blob/abc442c4736b82b4ae15d6068f929c1121ee7111/test/extended/securityandcompliance/compliance_operator.go#L1092. If it was there just as another way to confirm that the scan would run on all nodes nevermind the taint, I wholeheartedly agree.

[Anna-Koudelkova]

As per discussion, the downstream testcase OCP-33610 does not contain any important checkpoints that are not already covered in the upstream, therefore closing the PR.