CMP-3789: Enhance TestScanProducesRemediations to cover remaining 28949 logic [WIP] #1015
Conversation
|
@Anna-Koudelkova: This pull request references CMP-3789 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Anna-Koudelkova The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @Anna-Koudelkova. Thanks for your PR. I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
🤖 To deploy this PR, run the following command: |
|
@Anna-Koudelkova: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| } | ||
| } | ||
|
|
||
| if !found { |
There was a problem hiding this comment.
I am a little confused why api-checks-pod scanner pod still exists after scan done. With default scansetting, all scanner pods should be deleted after scan done.
There was a problem hiding this comment.
I am joining you in the confusion. The api-checks-pod scanner pod still exists in the end when I run the test case just with the defer lines commented so that the tp and ssb stays there:
$ oc get pods
NAME READY STATUS RESTARTS AGE
aggregator-pod-test-scan-produces-remediations 0/1 Completed 0 3m32s
compliance-operator-6fd5c47-ndlf2 1/1 Running 2 (6m34s ago) 6m37s
ocp4-osdk-e2e-5378635b-6de3-40a1-ae18-e07ac721bd4f-pp-5dfbbhfvb 1/1 Running 0 6m20s
rhcos4-osdk-e2e-5378635b-6de3-40a1-ae18-e07ac721bd4f-pp-5fnffxq 1/1 Running 0 6m20s
test-scan-produces-remediations-api-checks-pod 0/2 Completed 0 3m59s
However, if I just create the tp and ssb using following yaml files, the pod gets deleted after the scan finishes like is expected:
$ cat tp.yaml
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
name: test-scan-produces-remediations
namespace: openshift-compliance
annotations:
compliance.openshift.io/product-type: Platform
spec:
description: TestScanProducesRemediations
extends: ocp4-moderate
title: TestScanProducesRemediations
$ cat ssb.yaml
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
name: test-scan-produces-remediations
namespace: openshift-compliance
profiles:
- apiGroup: compliance.openshift.io/v1alpha1
kind: TailoredProfile
name: test-scan-produces-remediations
settingsRef:
apiGroup: compliance.openshift.io/v1alpha1
kind: ScanSetting
name: default
$ oc get pods
NAME READY STATUS RESTARTS AGE
compliance-operator-6778cc566f-fcx5n 1/1 Running 2 (2m38s ago) 2m42s
ocp4-openshift-compliance-pp-976f8cc94-ctvnt 1/1 Running 0 2m22s
rhcos4-openshift-compliance-pp-7bd49b8974-xt2nr 1/1 Running 0 2m22s
I am still not sure what am I missing here
There was a problem hiding this comment.
Did not reproduce the issue with the latest released CO 1.8.0, the -api-checks-pod gets removed after the scan finishes, as expected.
There was a problem hiding this comment.
I think the below code covers "When Debug: true is set, scanner pods should remain available even after scan completed". Maybe you can close this PR.
https://github.com/ComplianceAsCode/compliance-operator/blob/master/tests/e2e/parallel/main_test.go#L567-L591
Anna-Koudelkova
changed the title
Anna-Koudelkova
changed the title
|
As per discussion, the downstream testcase OCP-28494 does not contain any important checkpoints that are not already covered in the upstream (check for the scanner pod is already involved here https://github.com/ComplianceAsCode/compliance-operator/blob/master/tests/e2e/parallel/main_test.go#L567-L591), therefore closing this PR and the Jira ticket |
4 participants
Changes:
-api-checks-podOriginal Jira ticket calls for enhancement of
TestScheduleSuite. This PR adds the check to a test case TestScanProducesRemediations instead, as the remaining logic of the downstream test case OCP-28949 is already implemented to other upstream test cases and this test case already created platform scan.The only remaining checkpoint of OCP-28949 is introduced in this PR, however it might not be necessary at all - I might be wrong but I think the pod is needed for the openscap scanner and if it does not get created for platform scan, it is not possible for the scan to even reach the status DONE and the test case would fail earlier.
make e2e-parallel E2E_CONTENT_IMAGE_PATH="quay.io/rh-ee-akoudelk/content_repo:testcontent" E2E_BROKEN_CONTENT_IMAGE_PATH="ghcr.io/complianceascode/test-broken-content-ocp" E2E_GO_TEST_FLAGS="-v -timeout 60m -run TestScanProducesRemediations" was used on OCP 4.20 cluster to trigger this test case and it has passed when implementing also the changes in PR #960