Skip to content

STOR-2654: Add storage co check for vSphere hybird env #72553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
radeore wants to merge 1 commit into
base: master
Choose a base branch
from
+35 −3
Open

STOR-2654: Add storage co check for vSphere hybird env #72553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ci-operator/config/openshift/release/openshift-release-master__nightly-4.21.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -976,6 +976,7 @@ tests:
CSI_MANAGEMENT_REMOVED: "true"
FEATURE_GATES: '["VSphereMixedNodeEnv=true"]'
FEATURE_SET: CustomNoUpgrade
STORAGE_CO_DEGRADE_CHECK: "true"
TEST_SKIPS: \[sig-storage\]\|\[sig-arch\]\[Early\] Operators low level operators
workflow: openshift-e2e-vsphere-upi-hybrid-env
- as: e2e-vsphere-ovn-disk-setup-techpreview
Expand Down
1 change: 1 addition & 0 deletions ci-operator/config/openshift/release/openshift-release-master__nightly-4.22.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,7 @@ tests:
CSI_MANAGEMENT_REMOVED: "true"
FEATURE_GATES: '["VSphereMixedNodeEnv=true"]'
FEATURE_SET: CustomNoUpgrade
STORAGE_CO_DEGRADE_CHECK: "true"
TEST_SKIPS: \[sig-storage\]\|\[sig-arch\]\[Early\] Operators low level operators
workflow: openshift-e2e-vsphere-upi-hybrid-env
- as: e2e-vsphere-ovn-disk-setup-techpreview
Expand Down
32 changes: 29 additions & 3 deletions ci-operator/step-registry/ipi/install/vsphere/virt/ipi-install-vsphere-virt-commands.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,20 @@ function wait_for_co_ready() {
fi
}

# Wait for Storage cluster operator to have degraded status
function wait_for_storage_co_degrade() {
echo "$(date -u --rfc-3339=seconds) - Waiting for Storage cluster operator to degrade"
if ! oc wait --for=jsonpath='{.status.conditions[?(@.type=="Degraded")].status}'=True clusteroperator/storage --timeout=60m; then
echo "$(date -u --rfc-3339=seconds) - FAILED: Storage cluster operator did not degrade"
exit 1
fi
}

# Patch CSI driver management state to be Removed
function patch_csi_driver_removed() {
echo "$(date -u --rfc-3339=seconds) - Patching clustercsidriver managementState to be Removed"
oc patch clustercsidriver csi.vsphere.vmware.com --type=merge --patch 'spec: {managementState: "Removed"}'
}

# We are going to apply the 'node.openshift.io/platform-type=vsphere' label to all existing nodes as a workaround while waiting for upstream CCM changes
# When upstream changes are merged downstream, the function will output that no nodes were updated.
Expand Down Expand Up @@ -141,9 +155,9 @@ done

# Update CSI driver to be removed. Note, sometimes this can take a little while to start removing pods, but only on first attempt.
# For now, we'll update and move on. It should be resolved by the time the BM is ready to join.
if [ "${CSI_MANAGEMENT_REMOVED}" == "true" ]; then
echo "$(date -u --rfc-3339=seconds) - Patching clustercsidriver managementState to be Removed"
oc patch clustercsidriver csi.vsphere.vmware.com --type=merge --patch 'spec: {managementState: "Removed"}'
# If STORAGE_CO_DEGRADE_CHECK is true, we will patch clustercsidriver managementState later to make sure the Storage cluster operator degrades.
if [ "${CSI_MANAGEMENT_REMOVED}" == "true" ] && [ "${STORAGE_CO_DEGRADE_CHECK}" == "false" ]; then
patch_csi_driver_removed
fi

# Start VM
Expand All @@ -163,5 +177,17 @@ done
# Wait for node to be ready to prevent any interruption during the e2e tests
wait_for_node_ready

# Verifying Storage cluster operator is degraded for (OCP >=4.21) feature, STOR-2620: vSphere Bare Metal nodes support
# Wait for Storage cluster operator to degrade
if [ "${STORAGE_CO_DEGRADE_CHECK}" == "true" ]; then
wait_for_storage_co_degrade
fi

# Update CSI driver to be removed. Note, sometimes this can take a little while to start removing pods, but only on first attempt.
# We will patch clustercsidriver managementState now if skipped earlier.
if [ "${CSI_MANAGEMENT_REMOVED}" == "true" ] && [ "${STORAGE_CO_DEGRADE_CHECK}" == "true" ]; then
patch_csi_driver_removed
fi

# Wait for all CO to be stable
wait_for_co_ready
4 changes: 4 additions & 0 deletions ci-operator/step-registry/ipi/install/vsphere/virt/ipi-install-vsphere-virt-ref.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ ref:
default: "false"
documentation: |-
Specifies if cluster should modify the csi driver operator's management state to be "Removed".
- name: STORAGE_CO_DEGRADE_CHECK
default: "false"
documentation: |-
Specifies if storage cluster operator degrade check should be performed.
documentation: >-
static IP addresses are allocated and stored in $SHARED_DIR/hosts.txt
for later use in creating an install-config.yaml with static IP address
Expand Down