From 02772e7b073a5da55a8764d2d288fa4648d652d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Chris=20Suszy=C5=84ski?= Date: Sat, 7 Feb 2026 14:11:25 +0100 Subject: [PATCH 1/3] Limit metadata-webhook scope to serving-tests namespace MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add namespaceSelector to the MutatingWebhookConfiguration to limit the webhook's scope to namespaces with the samples.knative.dev/release label. This prevents the webhook from blocking resource deletions in other namespaces when the serving-tests namespace is torn down. The issue occurred during upgrade test cleanup where the Route resource for deployment-upgrade-failure could not be deleted because the webhook service was unavailable after namespace cleanup started. Assisted-by: 🤖 Claude Opus/Sonnet 4.5 --- .../metadata-webhook/config/500-webhook-configuration.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/serving/metadata-webhook/config/500-webhook-configuration.yaml b/serving/metadata-webhook/config/500-webhook-configuration.yaml index a0c5fa5c70..a087f5d15a 100644 --- a/serving/metadata-webhook/config/500-webhook-configuration.yaml +++ b/serving/metadata-webhook/config/500-webhook-configuration.yaml @@ -14,6 +14,12 @@ webhooks: failurePolicy: Fail sideEffects: None name: webhook.metadata-webhook.example.com + # Limit scope to only the serving-tests namespace where the webhook + # service is deployed. This prevents blocking resource deletions in + # other namespaces when the serving-tests namespace is torn down. + namespaceSelector: + matchLabels: + samples.knative.dev/release: devel --- apiVersion: v1 kind: Secret From 217bf543e2542d0a2d821a5bac55ec1f8cb18756 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Chris=20Suszy=C5=84ski?= Date: Mon, 9 Feb 2026 10:25:40 +0100 Subject: [PATCH 2/3] Fix metadata-webhook cleanup race condition Delete webhook resources before namespace deletion to prevent blocking Route finalizer removal when webhook service is unavailable. The issue occurs when: 1. Tests complete and cleanup starts 2. 'kubectl delete ns serving-tests' begins namespace deletion 3. Routes have finalizers that need removal 4. Finalizer removal triggers the MutatingWebhookConfiguration 5. Webhook service (in serving-tests) is already being deleted 6. Webhook call times out, blocking namespace deletion Solution: Delete the webhook resources (including the cluster-scoped MutatingWebhookConfiguration) before deleting the serving-tests namespace. This mirrors the installation order and prevents the race condition. --- .../config/500-webhook-configuration.yaml | 6 ------ test/lib.bash | 8 ++++++++ 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/serving/metadata-webhook/config/500-webhook-configuration.yaml b/serving/metadata-webhook/config/500-webhook-configuration.yaml index a087f5d15a..a0c5fa5c70 100644 --- a/serving/metadata-webhook/config/500-webhook-configuration.yaml +++ b/serving/metadata-webhook/config/500-webhook-configuration.yaml @@ -14,12 +14,6 @@ webhooks: failurePolicy: Fail sideEffects: None name: webhook.metadata-webhook.example.com - # Limit scope to only the serving-tests namespace where the webhook - # service is deployed. This prevents blocking resource deletions in - # other namespaces when the serving-tests namespace is torn down. - namespaceSelector: - matchLabels: - samples.knative.dev/release: devel --- apiVersion: v1 kind: Secret diff --git a/test/lib.bash b/test/lib.bash index 3131b8d1d7..1bf81315d0 100644 --- a/test/lib.bash +++ b/test/lib.bash @@ -584,6 +584,14 @@ EOF --upgradeopenshift fi + # Delete metadata-webhook resources before deleting the namespace. + # This prevents blocking Route finalizer removal when webhook service is unavailable. + if [[ $MESH == "true" ]]; then + local rootdir + rootdir="$(dirname "$(dirname "$(realpath "${BASH_SOURCE[0]}")")")" + oc delete -f "${rootdir}/serving/metadata-webhook/config" --ignore-not-found + fi + # Delete the leftover namespace. oc delete namespace serving-tests From db6ae8b50a55b1315a1cc6769a8cc6a7f3961655 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Chris=20Suszy=C5=84ski?= Date: Mon, 9 Feb 2026 12:42:49 +0100 Subject: [PATCH 3/3] Add --ignore-not-found to namespace deletion The webhook config directory includes 100-namespace.yaml which deletes the serving-tests namespace. Adding --ignore-not-found prevents the error when the namespace is already deleted. --- test/lib.bash | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/lib.bash b/test/lib.bash index 1bf81315d0..87c65c7a92 100644 --- a/test/lib.bash +++ b/test/lib.bash @@ -592,8 +592,8 @@ EOF oc delete -f "${rootdir}/serving/metadata-webhook/config" --ignore-not-found fi - # Delete the leftover namespace. - oc delete namespace serving-tests + # Delete the leftover namespace (may already be deleted by webhook config cleanup). + oc delete namespace serving-tests --ignore-not-found logger.success 'Upgrade tests passed' }