intermittent smoke test failure for kubernetes clusters #7778
Description
ISSUE TYPE
- Failing Test
COMPONENT NAME
Smoke tests
CLOUDSTACK VERSION
CONFIGURATION
OS / ENVIRONMENT
SUMMARY
kubernetes upgrade test fails, with the following output
2023-07-26 10:52:35,673 ERROR [c.c.u.s.SshHelper] (API-Job-Executor-49:ctx-1021343c job-875 ctx-a2c18063) (logid:eb42f24d) SSH execution of command sudo ./upgrade-kubernetes.sh2911844247732526921.sh 1.24.0 true false true has an error status code in return. Result output: Installing binaries from /mnt/k8sdisk/
...
error when applying patch:
{"metadata":{"annotations":{"cloud.weave.works/launcher-info":null,"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"ServiceAccount\",\"metadata\":{\"annotations\":{},\"labels\":{\"name\":\"weave-net\"},\"name\":\"weave-net\",\"namespace\":\"kube-system\"}}\n"}}}
to:
Resource: "/v1, Resource=serviceaccounts", GroupVersionKind: "/v1, Kind=ServiceAccount"
Name: "weave-net", Namespace: "kube-system"
for: "/mnt/k8sdisk//network.yaml": Patch "https://10.0.56.183:6443/api/v1/namespaces/kube-system/serviceaccounts/weave-net?fieldManager=kubectl-client-side-apply&fieldValidation=Strict": unexpected EOF
error when retrieving current configuration of:
Resource: "rbac.authorization.k8s.io/v1, Resource=clusterroles", GroupVersionKind: "rbac.authorization.k8s.io/v1, Kind=ClusterRole"
Name: "weave-net", Namespace: ""
from server for: "/mnt/k8sdisk//network.yaml": Get "https://10.0.56.183:6443/apis/rbac.authorization.k8s.io/v1/clusterroles/weave-net": net/http: TLS handshake timeout - error from a previous attempt: EOF
see for instance #7718 (comment)
STEPS TO REPRODUCE
run the test_kubvernetes_clusters.py smoke test
EXPECTED RESULTS
all pass
ACTUAL RESULTS
test_02_upgrade_kubernetes_cluster fails